Last Updated April 2, 2005. The graphic at left [click the thumbnail] is from an innovative study, “A Measure of Media Bias“, by Tim Groseclose/UCLA/Stanford and Jeff Milyo/University of Chicago/University of Missouri. I first read this study when published as a working paper in September 2003. It was released in final form December 2004. I will explain their methodology, but before you jump ship from fear of statistics, have a look at the study conclusion:
Although we expected to find that most media lean left, we were astounded by the degree. A norm among journalists is to present both sides of the issue.Consequently, while we expected members of Congress to cite primarily think tanks that are on the same side of the ideological spectrum as they are, we expected journalists to practice a much more balanced citation practice, even if the journalist’s own ideology opposed the think tanks that he or she is sometimes citing. This was not always the case. Most of the mainstream media outlets that we examined (ie all those besides Drudge Report and Fox News’ Special Report) were closer to the average Democrat in Congress than they were to the median member of the House.
I’ve quoted the original September 2003 version (PDF from the Harvard archives) of their paper because the authors have pulled their verbal punches a bit in the final version. And more from the same 2003 study introduction:
Our results show a very significant liberal bias. All of the news outlets except Fox News’ Special Report received a score to the left of the average member of Congress. Moreover, by one of our measures all but three of these media outlets (Special Report, the Drudge Report, and ABC’s World News Tonight) were closer to the average Democrat in Congress than to the median member of the House of Representatives. One of our measures found that the Drudge Report is the most centrist of all media outlets in our sample. Our other measure found that Fox News’ Special Report is the most centrist. These findings refer strictly to the news stories of the outlets. That is, we omitted editorials, book reviews, and letters to the editor from our sample.
What makes this a unique and compelling study is that it directly measures media output. You’ve no doubt read of the various studies which ask reporters to rate themselves, or examine the newsroom’s voting record, etc.. The problem with all of those studies is that while they do measure political leanings, but they do not measure the actual news output. Consider the Pew Research result that in 2004 reporters voted for Kerry over Bush by 9:1. That doesn’t demonstrate media bias – a valid criticism is “that is meaningless, reporters do not reflect their personal views in their writing“.This is how they did it: Groseclose and Milyo used a creative statistical approach to derive an objective measure of conservative/liberal bias of the straight news coverage (no editorials) of twenty selected “mainstream media” sources, including all the usual suspects. Their insight was to construct a comparison between members of the US Congress and each media outlet – using the ratings of members voting records as published by the liberal Americans for Democratic Action (ADA). These ADA scores distribute the members of both houses on a scale from extremely liberal to extremely conservative.The ground truth that Groseclose/Milyo were after is the score that represents the centrist American voter. To get there they do some small statistical adjustments to arrive at the adjusted ADA score (as displayed in the graphic above). Skipping the messy details, all covered very clearly in the paper, their resulting adjusted ADA scores gives the centrist members of Congress a score of about 50.How do they connect the profile of Congress with the corresponding profile of the media outlets? The next clever step is to derive the adjusted ADA scores for the 200 thinktanks most frequently cited by members. Finally they map the media outlets ADA scores by analyzing the frequency with which each straight-news piece quotes the same set of think tanks. This is a massive analytical task – the researchers used a large team of assistants.
As a simplified example, imagine that there were only two think tanks, one liberal and one conservative. Suppose that the New York Times cited the liberal think tank twice as often as the conservative one. Our method asks: What is the typical ADA score of members of Congress who exhibit the same frequency (2:1) in their speeches? This is the score that we would assign to the New York Times.
If you prefer PDFs for a longish report like this, Harvard has this in their archives. Footnotes: Wall Street Journal left of CBS, New York Times? Some readers have commented “how can this be, everybody knows the Journal is conservative”? Well, yes, the editorial pages of the WSJ are conservative – the Groseclose Milyo study is about straight news pages.It is best to read the study report itself. For a quick overview, here are the study comments on the WSJ result:
One surprise is the Wall Street Journal, which we find as the most liberal of all 20 news outlets. We should first remind readers that this estimate (as well as all other newspaper estimates) refers only to the news of the Wall Street Journal; we omitted all data that came from its editorial page. If we included data from the editorial page, surely it would appear more conservative. Second, some anecdotal evidence agrees with our result. For instance, Reed Irvine and Cliff Kincaid (2001) note that â€œThe Journal has had a long-standing separation between its conservative editorial pages and its liberal news pages.â€ Paul Sperry, in an article titled the â€œMyth of the Conservative Wall Street Journal,â€ notes that the news division of the Journal sometimes calls the editorial division â€œNazis.â€ â€œFact is,â€ Sperry writes, â€œthe Journal’s news and editorial departments are as politically polarized as North and South Korea.â€Third, a recent poll from the Pew Research Center indicates that a greater percentage of Democrats, 29%, say they trust the Journal than do Republicans, 23%. Importantly, the question did not say â€œthe news division at the Wall Street Journal.â€ If it had, Democrats surely would have said they trusted the Journal even more, and Republicans even less.
Finally, and perhaps most important, a scholarly study by Lott and Hasset (2004) gives evidence that is consistent with our result. As far as we are aware this is the only other study that examines the political bias of the news pages of the Wall Street Journal. Of the ten major newspapers that it examines, the study estimates the Wall Street Journal as the second-most liberal. Only Newsday is more liberal, and the Journal is substantially more liberal than the New York Times, Washington Post, L.A. Times, and USA Today.
The Paul Sperry article referenced above can be found here: The Myth of the Conservative Wall Street Journal .