UPDATE 03/17/05 See Crossroads Arabia comments below.
… National Security Agency director Michael Hayden tried to explain to his wife the difference between the black-and-white world of facts and the gray world of intelligence this way: ”If it were a fact, it wouldn’t be intelligence.”
That neatly sums up the challenge I have had assessing the general question of Stratfor credibility. Stratfor is in the business of selling intelligence to subscribers, which is of interest to those of use who don’t work for the CIA. Even if I was officed at Langley, the frenzy we’ve seen over “intelligence failures” implies that infallibility is a scarce commodity.
I’ve been attempting to audit George Friedman’s 10/4/2004 Stratfor book America’s Secret War: Inside the Hidden Worldwide Struggle Between America and Its Enemies. with the goal of soliciting informed comment from those with knowledge of the relevant issues, and from long-term Stratfor subscribers with a view on Stratfor credibility. My most recent posting on this is a guest blog item for the foreign service blog Daily Demarche. Many interesting comments have been submitted, but the central question remains open: “which parts of the book content are valid, and which are known to be invalid?”.
My interest in America’s Secret War is for its provocative current history – which is published with no sourcing or references. If it is good work, Friedman offers a coherent outline of the both US and Al Qaeda strategy. These strategy outlines must be largely based on inductive reasoning – I doubt Osama is emailing his thoughts to Austin. While there may be sources in State or Defense, I doubt we would find Rice or Rumsfeld emails on Stratfor hard drives.
Had I been a Stratfor subscriber since 1996 I would have a good idea of their batting average – but I haven’t. My primary interest is in Stratfor’s track record of analysis and current-history (what you get before the Top Secret classifications lapse), with a secondary interest in short-term forecasting (six to twelve months probabilities – see Stratfor definition of prediction vs forecasting below). Longer-term “predicting” isn’t of great interest. As you can infer from the Seeker Blog epigraph, we don’t have much confidence in predicting.
How do we assess and grade Stratfor credibility? Michael J. Sniffen wrote on the general credibility challenge in this 5/18/2004 AP article Citizens, officials have hard time deciding what is credible.
Inside the FBI, Mefford said, “‘credible intelligence’ is a term taken very seriously. It always referred to information we believed to be reliable.”
Nevertheless “credible intelligence is a very subjective term,” Mefford added. And the analysis of intelligence “is not a science; it’s an art.”
With those caveats I’ve undertaken to search out evidence pro or con on Stratfor’s record since their founding in 1996. It is difficult to extract a one sentence summary from all this, but I’ll attempt this:
- Stratfor is not a “silver bullet” – like other sources, the reader still has to do his own thinking, contrasting and correlating.
- Stratfor is not a member of the journalism herd – that has value.
- When I see a Stratfor bulletin relevant to my issues, I’ll read it carefully and include it as input.
- I’ll continue to file their bulletins so I can look back and assess how they are performing.
What did I find useful on the Web? Numerically, there are certainly more favorable than unfavorables. The unfavorables that I’ve found are mostly of two types: (a) a disagreement with a Stratfor conclusion – on issues I would classify not as fact, but as opinion; and (b) “Hrrrrmph. Stratfor is like the US version of Debka. Occasionally right, usually completely fictional.”
None of unfavorables that I’ve found contain any specific justification, such as Stratfor reported “X”, but we know “Y” to be the case because of references “Z”. More often, like the example just quoted, the author goes on to say something that doesn’t enhance his credibility:
Bush is basically the servant of the Saudis, as is our whole government.
I mean, 15 of 19 hijackers were Saudis. So of course, Saudis exempt from that fingerprinting plan. Can that be explained any other way, than Bush is a pawn of the Saudis? No.
Following is my best effort to assemble what I have found.
Glenn Reynold’s Instapundit archives contain about fifteen unique prior posts on Stratfor bulletins. It isn’t encouraging that Prof. Reynold’s confidence level is declining – as the Instapundit posts do not reveal obvious Stratfor blunders, the declining confidence may be related to unpublished reader email comments – e.g., as of 10/12/2004:
ANOTHER UPDATE: Many readers say that I shouldn’t rely on Stratfor. Fair enough — I haven’t followed their analyses, but those who say they have aren’t impressed.
Instapundit’s most recent post on Stratfor sums up the professor’s concerns:
ANOTHER UPDATE: This post on the big picture is kind of interesting, though I’m not a huge fan of Stratfor.
Bill Quick’s Daily Pundit archive, which has comments enabled, contains about 55 unique posts on Stratfor bulletins. Bill gets a lot of good reader comments. The ones I found of most interest are unfortunately from readers whose email/website information is anonymous (I hoped to be able to contact the writers). E.g., one is from an intelligence analyst (I infer), who appears to have some knowledge of the Stratfor origins and staffing, though I’ve no way to assess Dabney’s knowledge nor whether current staffing profile reflects the startup crew:
Posted by: Wesley Dabney on September 11, 2002
“Stratfor’s always had trouble with domestic stuff.” i’m not sure that they have the training for the domestic stuff. most of the guys that started Stratfor are Foreign Area Officers like me. we concentrate and become experts on whatever area we are assigned to. i wouldn’t totally discount what they are saying just yet because i know what kind of training these folks have. but they may have grown too fast and hired some less qualified staff.
Dabney’s comment was just a year after this Austin Chronicle article (covered in more detail below). This doesn’t look like growing too fast:
Yet the company itself was struggling — and appears to have just staggered again. Despite its own rosy financial forecasts only a few weeks ago, last week Stratfor closed its recently opened Washington, D.C. office and reorganized its Austin staff (see “Stratfor Staggers?” p.26). Six people (of roughly 35 full-time staff) will lose their jobs or be offered other positions in the company.
Another is by a professed long-time Stratfor subscriber:
Posted by: P. Stanley on September 10, 2002: I was an enthusiastic Stratfor supporter for a long time. I was one of the first subscribers, probably. Stratfor was good. Very very good. They made consistently accurate predictions about the way governments were going to move based on open source stuff.
I think they started to slip for a few reasons. First, they expanded very quickly. They expanded so quickly that I don’t think all the new staff got completely acclimated to the Stratfor mindset. That, and they had to churn out enormous amounts of material to justify the subscription fee (or at least they *felt* that they had to.)
Second, they got away from the open source stuff a little. Now they sometimes make refrences to “sources”. Sources? Your sources are supposed to be obscure newspapers, or email from the man in the street. (Email from Belgrade residents was one of Stratfor’s main strengths during the NATO bombing in 1999.)
In addition, Stratfor has always had a blind spot in the United States itself. Stratfor’s always had trouble with domestic stuff. They just don’t seem to have a firm handle on political behavior here. They’ve always been set up to be foriegn-oriented.
They’re most glaring fault since Sept. 11 has been their unfailing dedication to the rational actor model, however. They’ve always had that weakness, but it’s been particularly evident since then. They always believe that every engaged party has a realistic worldview, which simply isn’t the case. Stratfor paints the USA and al Qaida as two chess masters engaged in some global chess match, which totally ignores the basic, unavoidable, salient fact of the entire conflict: Al Qaida is run by a bunch of Fruit Loops.
P. Stanley starts off well, relating a bit of Stratfor history – their original “open source” orientation, transitioning to include unattributed sources. Stanley’s unhappiness with the “rational actor” model is interesting – but my contra is that I don’t see what other framework could be useful unless there is evidence of irrational behavior. My reading does not support the view that Al Qaeda is “run by a bunch of Fruit Loops”, but is closer to Friedman’s exposition that AQ is shrewd and well-informed. I wish I could contact P. Stanley for a Q&A on the particulars of faults he found in Stratfor product.
This one deserves its exclamation mark. STRATFOR, an analysis group whose historic record is of zero preference and dispassionate analysis of the Israeli-Palestinian issue, dropped a shocker today with its assessment of Palestinian strategy:
“From a strategic standpoint, the Palestinian suicide bombings may seem incompatible with the goal of attaining statehood. But when this tactic is viewed as a means toward achieving a transformation of the entire Islamic world, then the rationale behind it comes into sharp relief. And if this is the goal, then the Palestinians and al Qaeda are now serving each other’s interests.”
There’s more. Much more.
Here’s what I could find in the major print media on Stratfor:
2-21-2005: Barron’s, World of Worry by Jonathon R. Laing
[Stratfor] The Austin, Texas firm has prospered since we first publicized it just weeks after the Sept. 11 attacks. Collecting real-time information from the Internet and from informants around the world, Stratfor forecasts and geopolitical analyses have become staples in stories on the war on terror. Likewise, Friedman’s commentary is sought by news shows and Fortune 500 clients — including some of the largest financial institutions. His views are frequently provocative and idiosyncratic, but always stimulating.
His forecast for 2005, conveyed via telephone, proved to be no exception. Our talk spanned the globe, building from progress in the Middle East — despite new violence in Lebanon — through North Korea’s rumblings, on to why investors should be cautious on China and Russia.
Mapping Danger: From nukes to new separatist movements: Headed by George Friedman, Stratfor produces geopolitical analyses valued by investors and leaders across the planet.
As a political scientist and admirer of realpolitik, Friedman feels that the U.S.’s aggressive action and military presence in Iraq has inestimably helped the war on terror by, among other things, motivating reluctant allies like Saudi Arabia, Egypt and Pakistan and erstwhile enemies like Syria and Iran to help the U.S. by cutting off their support of al Qaeda and serving up better intelligence to Western governments.
“I call it the coalition of the coerced, but the tempo of timely arrests of al Qaeda operatives around the world, and the fact that the U.S. suffered no terrorist attacks running up to last year’s election, can in good part be attributed to better intelligence from the Islamic world,” Friedman avers. “Our victory in Afghanistan was insufficient. We had to show the Islamic world that we meant business by putting large numbers of troops into the Mideast, into harm’s way, rather than cutting and running such as the U.S. had done previously in its rapid pull-out from Beirut in the 1980s and Somalia in the ‘1990s.”
11-2-2004: Washington Post, The Real ‘October Surprise‘ by David Ignatius
Indeed, there hasn’t been a terrorist attack on U.S. soil since Sept. 11, 2001, and the intriguing question is: Why not? …
A final theory is that America and its allies have been successful in disrupting bin Laden’s operations. The best account I’ve seen comes from George Friedman, who runs a private intelligence service called Stratfor and has just published a book called “America’s Secret War.”
Friedman argues that all the intelligence alerts and warnings, combined with arrests of suspected al Qaeda operatives around the world, have put bin Laden off balance. Every time the Bush administration issues a warning about a possible plot, bin Laden has to assume the worst. His operatives “could be captured without al Qaeda knowing it,” Friedman writes in a recent analysis. “Worse, they could be captured, turned and released back into the field without al Qaeda knowing it. Even if the latter is unlikely, al Qaeda simply cannot be sure and, not being sure, they must abort the mission.”
4-20-2003: New York Times, Spooky by Matt Bai.
Matt interviewed Friedman in Melbourne, Florida the night the Iraq invasion began:
He set up Stratfor in 1996 and began sending out his free daily briefing on world events to contacts in national defense and business, who then sent it on to others. He predicted that the surging Asian economy would collapse by the end of 1997, while the American economy would reach unprecedented heights. These weren’t exactly Nostradamus-type predictions, but Friedman found a modest following nonetheless.
Eventually, Friedman hired a former Russian colonel, who goes only by the name Victor, as his director of intelligence. Victor won’t say exactly what he did in the old days, but let’s put it this way: he knows 10 languages and spent a fair amount of time operating in countries like Iraq, Iran and Afghanistan. You figure it out.
Between them, Friedman and Victor say they’ve amassed hundreds of contacts around the world who now feed them tips or, in some cases, trade what they know for information about another country. Some of Stratfor’s 150,000 Web subscribers — who include devotees in the military, foreign governments and Fortune 500 companies — have become informants as well. ”I’ll give you a range,” Friedman said. ”A senior commander, the head of one of the services of a NATO member, communicates with us continually. Down at the other end, I just got an e-mail from somebody at Fort Hood who’s an enlisted man. And they are everywhere in between. A great deal of the information they give me is dubious, O.K.? But a lot of it is valuable.”
These sources correspond with Friedman and Victor by e-mail or cellphone at all hours of the day. This business of private-sector spying seems remarkably similar to the daily activities of your average 16-year-old: it entails a lot of sitting around with a computer, instant-messaging everyone you know.
On the morning after the opening strike of the war, I was sitting with Friedman in his hotel room when we were interrupted by Meredith. ”Apaches,” she said.
Friedman snapped upright. ”The Apaches are moving?” He had expected bombing first; now the ground forces were mobilizing. He returned his wire-frame glasses to his nose and swiveled his chair back toward the computer in a single motion.
”O.K. – right now, it gets really, really interesting,” he told me, reaching for the phone. ”You don’t know how badly this violates all the doctrines of joint operations we’ve been taught in the last 10 years.”
Soon a midlevel officer in the Air Force was e-mailing Friedman. ”Is this starting to make sense to you now?” the officer asked, almost tauntingly. ”You had most of it laid out in your diary” — a daily feature on the Web site — ”but the timing is/has been difficult to project, even from where I was/is. I’m available tonight if you have questions.”
Friedman smiled. ”I wrote a piece saying this war would be different from every other war, because it was dynamic and would not have a fixed plan,” he told me. ”But it’s really strange sitting here watching it happen.”
Perhaps Victor isn’t considered management, but he is not included in the latest Stratfor Management Team listing of seven execs. Later Bai get’s Friedman to comment on Iraq post-war – a forecast that may not work out:
I asked about the reconstruction.
”I think Iraq will have a formally independent government that will be in perpetual gridlock and chaos,” he said. ”And essentially, there will be a U.S. military administration utilizing NGO’s that can do a lot of the heavy lifting in the country.”…
To me, Friedman’s following 99% comment is valid. What do you think would be the “optimum” success ratio (NWS would like to know the answer also)?
…”If you’re right 99 percent of the time, you’re not making really interesting calls,” he likes to say …
9-14-2001: Austin Chronicle, Austin-based Stratfor Attempts to Make Smart Money on Global Intelligence – Is Knowledge Power? by Michael Erard. See Also Stratfor Staggers of the same date, for a bit more on the journalist staff of the closed DC office. This is the only in-depth Chronicle article, which gives some feel for the culture and staff:
On November 22, 1999, an Austin company called Stratfor sent a bold e-mail to 15,000 recipients around the globe. Its subject line: “Philippine President’s Days Are Numbered.” In the brisk prose that has become its trademark, Stratfor’s evaluation of the political situation in Manila contained bad news for the fortunes of actor-turned politician Joseph Estrada. “Whether removed by force or by the broad coalition arrayed against him,” the message concluded, “Estrada is unlikely to fulfill his six-year term in office.”…
Nearly two years later, President Estrada is gone — ousted in a January 2001 popular uprising. Stratfor, however, is still here. Long after the glow of the NASDAQ meltdown has subsided, Stratfor retains the sexy glint of money. (Yes, they were right on Estrada — more on that below.) According to CEO Don Kuykendall, a few weeks ago the company was about to turn profitable for the first time, with gross revenues between $2.5 million and $3 million. Was that a prediction, a certain future foretold (something Stratfor says it doesn’t do)? Or was it a forecast, which tells a particular future’s likelihood? …
“Stratfor does not normally talk about itself,” read the GIU in early December of 1999, as controversy in the Philippines raged. “We are constantly deluged with e-mail demanding to know who we are, who funds us. … We usually ignore these demands. Normally, Stratfor doesn’t talk about Stratfor; this week, we will.”
This GIU was broadcast, as all Stratfor products are, without a byline, but everyone knew who wrote it. Employees maintain that no product belongs to any one person, because it’s been hammered out among analysts and editors. (And as chief analyst Matt Baker admits: “You can tell a lot about a company by finding out who works there.”) But in this case, the person taking a stab at Stratfor’s own existence was probably the only person who could. The author was Stratfor’s founder, George Friedman, a former professor at Dickinson College in Pennsylvania and a highly regarded intelligence analyst (a recent New York Times Magazine story on space war quoted him several times). In 1995, he founded Stratfor’s prior incarnation — the Center for Geopolitical Studies — at Louisiana State University, in order to marry objective foreign policy analysis with advanced technologies (an early funder was Sun Microsystems). …
The original idea was to build a brand name as a smart private intelligence firm by disseminating free GIU forecasts — attracting corporate clients who need to know how the calculus of power in Azerbaijan is going to shake out in the next two quarters, whether Euro futures are rising, or what butterflies in the Marañon are up to. The company’s profile rose when it correctly forecast the Asian economic crisis in 1996, and again during the Kosovo conflict in 1999, when Stratfor set up a real-time intelligence center that was scooping CNN and more entrenched intelligence agencies. Part of Stratfor’s advantage was having GIU subscribers on the ground in Kosovo sending reliable e-mail reports back to Austin. “It would have cost the CIA a fortune to put somebody on the ground in Kosovo in the middle of a war,” Friedman says. “But a Hotmail account costs nothing.” (To maintain this resource of discount eyeballs on the ground, Stratfor still provides a free intelligence briefing, even after discontinuing the free GIU in August — which at its most popular went to over 100,000 people.)
What most caught people’s eyes was Stratfor’s style. Its positions were often contrarian, and it persisted in supporting them with evidence. It also persisted in being correct. The analyses didn’t always list their sources, but that didn’t matter. “Journalism has trouble following foreign policy,” Friedman muses, “because they want everything to be sourced. Like in China — where you can’t get a picture of what’s going on there, because you don’t have access to the other side.” But in the best tradition of Kremlin-watching, Stratfor spun inferences from afar; unlike old-school Kremlin watchers, Stratfor had better data via the Internet. Also distinctive was Stratfor’s dispassion. On any ideological chart the analysts flatlined, and their prose was crisp, free of jargon. “No one knows what side we’re on,” Friedman says. “In Belgrade they think we’re working for the CIA, and in Washington they thought we were getting paid by the Serbians. When [Nation columnist] Alexander Cockburn praised the same article that Rush Limbaugh did, I knew: we were in.”
If you have a Stratfor atta-boy or an oops to report, please drop a note to Seeker Blog. And if you wish to become one of those Stratfor sources, just send email to firstname.lastname@example.org!
…I don’t know if this one on Stratfor is of interest. Certainly, those of us on the outside find Stratfor intriguing – the problem of course is so much un-sourced material, it’s a challenge to know what is true.
Is Stratfor Credible?
…Without having yet read the piece on STRATFOR, I’ll give my own opinion based on my dealings with them when they came out to the field to do research:
Basically, they’re very good. But they have a few hobby-horses they ride into the ground. One of those is the imminent downfall of the House of Saud. Absent a rash of assassinations, that’s not going to happen any time soon. And there’s no sign that attempted assassinations in the future would be any more successful than in the past. See what’s happening with the recent Libyan plot, as well as Al-Qaeda’s failed efforts in that direction.
Many thanks for the candid remarks on STRATFOR. RE: their theme on “the imminent downfall of the House of Saud”, how recently has that been expressed? I’ve been reading Stratfor for only 2 years – I don’t recall seeing that, so possibly they’ve taken it out of their kit.
Having themes of any kind strikes me as a poor way to go about critical thinking. I criticize the legacy media for that regularly – I believe it is why they rarely get any of the important stories right (they live within their chosen narrative frame).
Possibly, western intelligence regarding the Saddam regime was influenced by failing to question another theme.
There may be some marketing embedded in their reports to spice them up. Friedman is on record that CIA analysts are too cautious, and that:
“…If you’re right 99 percent of the time, you’re not making really interesting calls”