Intelligence analysis is not so hard, if you have common sense, but then common sense is rare. One serious problem is that the enemy is seldom obliging enough to supply you with ground truth. If you had it, you could identify your mistakes, and better yet, identify classes of errors, which might allow you do better in the future. You could also get rid of those analysts who were uncommonly wrong, presumably by promoting them to management.
Consider the case of Scott Speicher, a Navy pilot shot down in the Gulf War. His fate was unknown in 1991: we had rumors that he was a captive in Baghdad. Let me steal some relevant lines from Wiki:
[In April 1994, a U.S. satellite photographed apparent human-made symbols on the desert floor near the wreck’s location, which might possibly be Speicher’s E & E (Escape and Evade) sign, suggesting that Speicher might have survived the crash.” Speculative theories were developed as to the circumstances of Speicher’s shoot-down, and assuming he was still alive, why the U.S. military might not want to find him and why Iraq might not want to return him. “U.S. Secretary of the Navy Gordon England said, “While the information available to me now does not prove definitively that Capt. Speicher is alive and in Iraqi custody, I am personally convinced the Iraqis seized him sometime after his plane went down. Further, it is my firm belief that the government of Iraq knows what happened to Capt. Speicher.”
Speicher’s possible situation became a more high-profile issue in the build-up to war. In March 2002, the Washington Times ran five successive front-page articles about it, National Review Online ran a long piece on it, and on September 12, 2002, President George W. Bush mentioned Speicher in a speech to the United Nations General Assembly as part of his case for war against Iraq.
In April 2003, Speicher’s possible initials were discovered in a cell at Hakmiyah prison in Baghdad. But later, in 2009, we found (and identified) the body. He died back in the crash back in 1991, and some Bedouins buried him, which is why the Baathists couldn’t say what happened to him. The rumors were false, the reports of the Escape and Evade sign were wrong, the initials weren’t his, the speculative theories were wrong. The Secretary of the Navy was wrong, but then being wrong was part of the job. National Review was wrong – guess there’s a first time for everything. There was smoke, but no fire – just a mystery that became fuel for a bunch of desperate, silly, and/or dishonest people. It can happen. It did happen. But in this example, we know the answer.
This was a typical Iraq story: somehow, we had developed an approach to intelligence that reliably produced fantastically wrong answers, at vast expense. What so special about Iraq? Nothing, probably – except that we acquired ground truth.