When determining bias, there isn’t any true scientific formula that is 100% objective. There are objective measures that can be calculated, but ultimately there will be some degree of subjective judgment to determine these. On each page, we have put up a scale with a yellow dot that shows the degree of bias for each source. Each page also has a “detailed report” section that gives some details about the source and an explanation of their bias. When calculating bias we are not just looking at political bias, but also how factual the information is and if they provide links to credible, verifiable sources. Therefore, the yellow dot may indicate political bias or how factual a source is, or in many cases, both.
Here is a look at some example scales from different media sources:
For Example, CNN looks like this:
Breitbart looks like this:
MSNBC looks like this:
The method for placement of the yellow dot is determined by ranking bias in four different categories. In each category, the source is rated on a 0-10 scale, with 0 meaning without bias and 10 being the maximum bias(worst). These four numbers are then added up and divided by 4. This 0-10 number is then placed on the line according to their Left or Right bias. Scoring is as follows:
0 – 2 = Least Biased
2 – 5 = Left/Right Center Bias
5 – 8 = Left/Right Bias
8 – 10 = Extreme Bias
The categories are as follows:
- Biased Wording/Headlines- Does the source use loaded words to convey emotion to sway the reader. Do headlines match the story?
- Factual/Sourcing- Does the source report factually and back up claims with well-sourced evidence.
- Story Choices: Does the source report news from both sides or do they only publish one side.
- Political Affiliation: How strongly does the source endorse a particular political ideology? In other words how extreme are their views. (This can be rather subjective)
Here is an example of how CNN scored and why they were placed in the middle of Left Bias:
Biased Wording = 5 (CNN uses moderate biased words that favor liberals and headlines typically match the story)
Factual/Sourcing = 4 (CNN is mostly trustworthy for providing evidence and sources, but sometimes jumps the gun on breaking stories)
Story Choices = 7 (CNN mostly favors pro-liberal stories and publishes negative conservative stories)
Political Affiliation = 7 (CNN mostly favors liberal ideology through content and wording)
Total = 23
Average 23/4 = 5.75
5.75 = Moderate Left Bias
Each source has a “Factual Reporting” rating based on their Factual/Sourcing score above. The ratings are as follows.
Factual Reporting: VERY HIGH = a score of 0, which means the source is always factual, sources to credible information, and makes immediate corrections to incorrect information and has never failed a fact check.
Factual Reporting: HIGH = a score of 1 – 3, which means the source is almost always factual, sources to mostly credible low biased or high factual information and makes immediate corrections to incorrect information, has failed only 1 fact check and uses reasonable language that retains context. A source rated High may fail more than one fact check if they produce a high volume of content.
Factual Reporting: MOSTLY FACTUAL = a score of 3 – 4, which means the source is usually factual but may have failed a fact check or two that was not properly corrected in a timely manner. They may occasionally use biased sources, but still link to factual content the majority of the time. These sources are generally pro-science, but may sometimes be misleading in wording and may infrequently offer alternative viewpoints to the consensus. These sources are reasonably transparent and offer a mission statement and ownership. In general, these sources are trustworthy most of the time, but some caution is needed with these sources.
Factual Reporting: MIXED = a score of 5 – 6, which means the source does not always use proper sourcing or sources to other biased/mixed factual sources. Mixed sources will have failed one or more fact checks and do not correct false or misleading information. Further, any source that does not disclose either a mission statement or ownership information will automatically be deemed Mixed as will a source that utilizes extremely loaded language that alters the context of facts, even if properly sourced and has not failed a fact check. Lastly, any source that does not support the consensus of science on such topics as Climate Change, GMO, Vaccinations, Evolution, or any other will automatically be rated Mixed for factual reporting if one or more Mixed criteria are met above.
Factual Reporting: LOW = a score of 7 – 9, which means the source rarely uses credible sources and is simply not trustworthy for reliable information. These are the sources that need to be fact-checked for intentional fake news, conspiracy, and propaganda.
Factual Reporting: VERY LOW = a score of 10, which means the source almost never uses credible sources and is simply not trustworthy for reliable information at all. These are the sources that always need to be fact-checked for intentional fake news, conspiracy, and propaganda.
For each source, a minimum of 10 headlines are reviewed and a minimum of 5 news stories reviewed. We first review news reporting and follow that up by looking at editorial and opinion pieces. If there is still not clear evidence we will use searching methods for articles on the site such as “Republican”, “Democrat”, “Liberal”, “Conservative”, “Trump”, “Clinton” etc. until we are sure of political affiliation. This process can be time-consuming or very simple depending on the source.
In order to understand the ratings, it is important to understand the terminology. We look at all of these when researching a source:
Bias by Omission: leaving one side out of an article, or a series of articles over a period of time; ignoring facts that tend to disprove liberal or conservative claims, or that support liberal or conservative beliefs.
Bias by Labeling: Bias by labeling comes in two forms. The first is the tagging of conservative politicians and groups with extreme labels while leaving liberal politicians and groups unlabeled or with more mild labels, or vice versa. The second kind of bias by labeling occurs when a reporter not only fails to identify a liberal as a liberal or a conservative as a conservative but describes the person or group with positive labels, such as “an expert” or “independent consumer group.”
Bias by Placement: is where on a website (or newspaper) or in an article a story or event is printed; a pattern of placing news stories so as to downplay information supportive of either conservative views or liberal views.
Bias by Selection of Sources: including more sources that support one view over another.
Bias by Spin: is a reporter’s subjective comments about objective facts; makes one side’s ideological perspective look better than another.
Bias by Story Selection: a pattern of highlighting news stories that coincide with the agenda of either the Left or the Right, while ignoring stories that coincide with the opposing view.
Confirmation Bias: also called confirmatory bias or my side bias, is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s beliefs or hypotheses while giving disproportionately less consideration to alternative possibilities.
Connotation: the emotional and imaginative association surrounding a word that can be either positive or negative.
Denotation: the strict dictionary meaning of the word.
Loaded Language (Words): (also known as loaded terms or emotive language) is wording that attempts to influence an audience by using appeal to emotion or stereotypes. Such wording is also known as high-inference language or language persuasive techniques.
Purr Words: words used to describe something that is favored or loved.
Snarl Words: words used when describing something that a person is against or hates.
Other factors to look for:
Do the headlines and stories match?
Are important stories featured prominently?
Does the story offer an alternative point of view?
Consider the source!
A questionable source exhibits any of the following: extreme bias, overt propaganda, poor or no sourcing to credible information, a complete lack of transparency and/or is fake news. Fake News is the deliberate attempt to publish hoaxes and/or disinformation for the purpose of profit or influence. Sources listed in the Questionable Category may be very untrustworthy and should be fact checked on a per article basis. Typically, the number one criteria for this category is how factual their reporting is. The less factual the more likely to be placed in this category. Further, if a source does not disclose at least one of the following: Mission, Ownership, and Authorship they will automatically be deemed Questionable due to a lack of transparency, despite how factual their content may be. Finally, sources originating from countries that exert extreme government censorship will automatically be placed on the Questionable list. (This is a work in progress).
We may occasionally label a source “fake news”, but this is designated specifically for 100% hoax websites. We also classify “hate groups” in this category.
The Conspiracy/Pseudoscience designation is reserved for sources that publish unverifiable information that relates to known conspiracies such as the New World Order, Illuminati, False Flags, Aliens, anti-vaccination propaganda etc. Pseudoscience is determined by publishing unverified health and scientific claims. For example, sources that promote human-influenced climate change denial or take anti-vaccination positions will be classified as pseudoscience. According to the Stanford Encyclopedia of Philosophy on Science and Pseudoscience “Philosophers and other theoreticians of science differ widely in their views on what science is. Nevertheless, there is virtual unanimity in the community of knowledge disciplines on most particular issues of demarcation. There is a widespread agreement for instance that creationism, astrology, homeopathy, Kirlian photography, dowsing, ufology, ancient astronaut theory, Holocaust denialism, Velikovskian catastrophism, and climate change denialism are pseudosciences.“ In order to be classified in this group, the central theme of the source must revolve around conspiracies or pseudoscience.
Media Bias/Fact Check (MBFC News) adheres to the International Fact-Checking Network Fact-checkers’ Code of Principles. These principles were developed by the Poynter Institute to promote excellence and standardization in Fact Checking.
MBFC News strictly adheres to the following principles for all fact checks:
- A COMMITMENT TO NONPARTISANSHIP AND FAIRNESS
We fact-check claims using the same standard for every fact check. We do not concentrate our fact-checking on any one side. We follow the same process for every fact check and let the evidence dictate our conclusions. We do not advocate or take policy positions on the issues we fact-check.
- A COMMITMENT TO TRANSPARENCY OF SOURCES
We want our readers to be able to verify our findings themselves. We provide all sources in enough detail that readers can replicate our work, except in cases where a source’s personal security could be compromised. In such cases, we provide as much detail as possible.
- A COMMITMENT TO TRANSPARENCY OF FUNDING & ORGANIZATION
We are transparent about our funding sources. If we accept funding from other organizations, we ensure that funders have no influence over the conclusions we reach in our reports. We detail the professional background of all key figures in our organization and explain our organizational structure and legal status. We clearly indicate a way for readers to communicate with us.
- A COMMITMENT TO TRANSPARENCY OF METHODOLOGY
We explain the methodology we use to select, research, write, edit, publish and correct our fact checks. We encourage readers to send us claims to fact-check and are transparent on why and how we fact-check.
- A COMMITMENT TO OPEN AND HONEST CORRECTIONS
We publish our corrections policy and follow it scrupulously. We correct clearly and transparently in line with our corrections policy, seeking so far as possible to ensure that readers see the corrected version.
Alexander Dyck & Natalya Volchkova & Luigi Zingales, 2008. “The Corporate Governance Role of the Media: Evidence from Russia,” Journal of Finance, American Finance Association, vol. 63(3), pages 1093-1135, 06.
Baron, David P. “Persistent media bias.” Journal of Public Economics 90.1 (2006): 1-36.
Bolinger, Dwight. Language-the loaded weapon: the use and abuse of language today. Routledge, 2014.
Chun-Fang Chiang & Brian Knight, 2011. “Media Bias and Influence: Evidence from Newspaper Endorsements,” Review of Economic Studies, Oxford University Press, vol. 78(3), pages 795-820.
DellaVigna, Stefano and Ethan Kaplan. “The Fox News Effect: Media Bias and Voting.” Quarterly Journal of Economics 122 (August 2007): 1187-1234.
Entman, R. M. (2007), Framing Bias: Media in the Distribution of Power. Journal of Communication, 57: 163–173. doi:10.1111/j.1460-2466.2006.00336.x
Eveland, W. P. and Shah, D. V. (2003), The Impact of Individual and Interpersonal Factors on Perceived News Media Bias. Political Psychology, 24: 101–117. doi:10.1111/0162-895X.00318
Farhi, Paul. How Biased is the Media Really. 27 April 2012. 20 November 2012. <http://www.washingtonpost.com/lifestyle/style/how-biased-is-the-media-really/2012/04/27/gIQA9jYLmT_story.html>.
Gentzkow, Matthew and Jesse M. Shapiro. “Media Bias And Reputation,” Journal of Political Economy, 2006, v114(2,Apr), 280-316.
“How To Detect Bias In News Media.” FAIR. N.p., 2012. Web. 15 Feb. 2016.
Matthews, Jack. “The effect of loaded language on audience comprehension of speeches.” Communications Monographs 14.1-2 (1947): 176-186.
Morris, J. S. (2007), Slanted Objectivity? Perceived Media Bias, Cable News Exposure, and Political Attitudes*. Social Science Quarterly, 88: 707–728. doi:10.1111/j.1540-6237.2007.00479.x
Nie, N. H., Miller, III, D. W., Golde, S., Butler, D. M. and Winneg, K. (2010), The World Wide Web and the U.S. Political News Market. American Journal of Political Science, 54: 428–439. doi:10.1111/j.1540-5907.2010.00439.x
Puglisi R, Snyder JM. Newspaper Coverage of Political Scandals. Journal of Politics. 2011;73(3):931-950.
Russell, Nick. “Morals and the Media Ethics in Canadien Journalism: Second Edition.” Vancouver: UBC Press, 2006.
S.I. Hayakawa, Alan Hayakawa. “Language in Thought and Action: Fifth Edition.” New York: Harcourt Brace Jovanovich,
Tewksbury, D., Jensen, J. and Coe, K. (2011), Video News Releases and the Public: The Impact of Source Labeling on the Perceived Credibility of Television News. Journal of Communication, 61: 328–348. doi:10.1111/j.1460-2466.2011.01542.x
The Quarterly Journal of Economics (2005) 120 (4): 1191-1237.doi: 10.1162/003355305775097542
Disclaimer: The methodology used by Media Bias Fact Check is our own. It is not a tested scientific method. It is meant as a simple guide for people to get an idea of a source’s bias. Media Bias Fact Check will always review and change any factual errors when brought to our attention. We make every effort to be as factual as possible. Our goal is to have MBFC rated as least biased by our own criteria.