Objective: To determine the bias of media and information sources and the level of overall factual reporting through a combination of objective measures and subjective analysis through the use of our stated methodology.
When determining bias, there isn’t any true scientific formula that is 100% objective. There are objective measures that can be calculated, but ultimately there will be some subjective judgment to determine these. We have put up a scale with a yellow dot on each page that shows the degree of bias for each source. Each page also has a “detailed report” section that gives some details about the source and explains their bias. When calculating bias, we are looking at political bias, how factual the information is, and links to credible, verifiable sources. Therefore, the yellow dot may indicate political bias or how factual a source is, or in many cases, both. It is important to note that our bias scale is based on the USA political scale, which may differ from other countries. For example, the Democratic Party is considered centrist or even right-center in many countries worldwide; however, in the USA, they are considered Left-Center. Please keep this in mind if our ratings seem off in your native country. (12/10/2019)
Here is a look at some example scales from different media sources:
For example, CNN looks like this:
Fox News looks like this:
Reuters looks like this:
The method for placement of the yellow dot is determined by ranking bias in four different categories. The source is rated on a 0-10 scale in each category, with 0 meaning without bias and 10 being the maximum bias(worst). These four numbers are then added up and divided by 4. This 0-10 number is then placed on the line according to their Left or Right bias. Scoring is as follows:
0 – 2 = Least Biased
2 – 5 = Left/Right Center Bias
5 – 8 = Left/Right Bias
8 – 10 = Extreme Bias
The categories are as follows:
- Biased Wording/Headlines- Does the source use loaded words to convey emotion to sway the reader. Do headlines match the story?
- Factual/Sourcing- Does the source report factually and back up claims with well-sourced evidence.
- Story Choices: Does the source report news from both sides, or do they only publish one side.
- Political Affiliation: How strongly does the source endorse a particular political ideology? In other words, how extreme are their views? (This can be rather subjective)
Here is an example of how CNN scored and why they were placed in the middle of Left Bias:
Biased Wording = 5 (CNN uses moderate biased words that favor liberals and headlines typically match the story)
Factual/Sourcing = 4 (CNN is mostly trustworthy for providing evidence and sources, but sometimes jumps the gun on breaking stories)
Story Choices = 7 (CNN mostly favors pro-liberal stories and publishes negative conservative stories)
Political Affiliation = 7 (CNN mostly favors liberal ideology through content and wording)
Total = 23
Average 23/4 = 5.75
5.75 = Moderate Left Bias
Each source has a “Factual Reporting” rating based on their Factual/Sourcing score above. The ratings are as follows.
Factual Reporting: VERY HIGH = a score of 0, which means the source is always factual, sources to credible information, and makes immediate corrections to incorrect information, and has never failed a fact check in either news reporting or op-eds. (9/30/2020)
Factual Reporting: HIGH = a score of 1 – 2, which means the source is almost always factual, sources from mostly credible low biased or high factual information and makes immediate corrections to incorrect information, has failed only 1 news reporting fact check, 2 or less op-ed fact checks and uses reasonable language that retains context. A high-rated source may fail more than one fact check if they produce a high volume of content or some are based on op-eds. (9/30/2020)
Factual Reporting: MOSTLY FACTUAL = a score of 3 – 4, which means the source is usually factual but may have failed a fact check or two that was not properly corrected promptly. Further, they may fail up to 3 op-ed fact checks (less if a low volume site). They may occasionally use biased sources but still link to factual content the majority of the time. These sources are generally pro-science but may sometimes be misleading in wording and may infrequently offer alternative viewpoints to the consensus. These sources are reasonably transparent and offer a mission statement and ownership. In general, these sources are trustworthy most of the time, but some caution is needed. A source that outright rejects science consensus in any area cannot be considered mostly factual despite clean fact check records. (9/30/2020)
Factual Reporting: MIXED = a score of 5 – 6, which means the source does not always use proper sourcing or sources to other biased/mixed factual sources. Mixed sources will have failed one or many fact checks and do not correct false or misleading information. Further, any source that does not disclose either a mission statement or ownership information will automatically be deemed Mixed, as will a source that utilizes extremely loaded language that alters the context of facts, even if properly sourced and has not failed a fact check. Lastly, any source that does not support science’s consensus on such topics as Climate Change, GMO, Vaccinations, Evolution, or any other will automatically be rated Mixed for factual reporting if one or more Mixed criteria are met above. Further, a source that has failed more than 5 fact checks can remain on the Mixed list so long as they are a high-volume source. (9/30/2020)
Factual Reporting: LOW = a score of 7 – 9, which means the source rarely uses credible sources and is not trustworthy for reliable information. These are the sources that need to be fact-checked for intentional fake news, conspiracy, and propaganda. (12/10/2019)
Factual Reporting: VERY LOW = a score of 10, which means the source rarely uses credible sources and is not trustworthy for reliable information at all. These sources always need to be fact-checked for intentional fake news, conspiracy, and propaganda. (12/10/2019)
For each source, a minimum of 10 headlines are reviewed and a minimum of 5 news stories reviewed. We first review news reporting and follow that up by looking at editorial and opinion pieces. If there is still no clear evidence, we will use searching methods for articles on the site such as “Republican,” “Democrat,” “Liberal,” “Conservative,” “Trump,” “Clinton,” etc., until we are sure of political affiliation. This process can be time-consuming or very simple, depending on the source.
To understand the ratings, it is important to understand the terminology. We look at all of these when researching a source:
Bias by Omission: leaving one side out of an article or a series of articles over a period of time; ignoring facts that tend to disprove liberal or conservative claims or that support liberal or conservative beliefs.
Bias by Labeling: Bias by labeling comes in two forms. The first is the tagging of conservative politicians and groups with extreme labels while leaving liberal politicians and groups unlabeled or with more mild labels, or vice versa. The second kind of bias by labeling occurs when a reporter not only fails to identify a liberal as a liberal or a conservative as a conservative but describes the person or group with positive labels, such as “an expert” or “independent consumer group.”
Bias by Placement: Whether on a website (or newspaper) or in an article, a story or event is printed; a pattern of placing news stories to downplay information supportive of either conservative views or liberal views.
Bias by Selection of Sources: including more sources that support one view over another.
Bias by Spin: is a reporter’s subjective comments about objective facts; makes one side’s ideological perspective look better than another.
Bias by Story Selection: a pattern of highlighting news stories that coincide with the agenda of either the Left or the Right while ignoring stories that coincide with the opposing view.
Confirmation Bias: also called confirmatory bias or my side bias, is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s beliefs or hypotheses while giving disproportionately less consideration to alternative possibilities.
Connotation: the emotional and imaginative association surrounding a word that can be either positive or negative.
Denotation: the strict dictionary meaning of the word.
Loaded Language (Words): (also known as loaded terms or emotive language) is wording that attempts to influence an audience by appealing to emotion or stereotypes. Such wording is also known as high-inference language or language persuasive techniques.
Purr Words: words used to describe something that is favored or loved.
Snarl Words: words used when describing something that a person is against or hates.
Other factors to look for:
Do the headlines and stories match?
Are important stories featured prominently?
Does the story offer an alternative point of view?
Consider the source!
A questionable source exhibits any of the following: extreme bias, overt propaganda, poor or no sourcing to credible information, a complete lack of transparency, and/or is fake news. Fake News is the deliberate attempt to publish hoaxes and/or disinformation for profit or influence. Sources listed in the Questionable Category may be very untrustworthy and should be fact-checked on a per-article basis. Typically, the number one criteria for this category is how factual their reporting is—the less factual, the more likely to be placed in this category. Further, if a source does not disclose at least one of the following: Mission, Ownership, and Authorship, they will automatically be deemed Questionable due to a lack of transparency, despite how factual their content may be. Finally, sources originating from countries that exert extreme government censorship will automatically be placed on the Questionable list. (This is a work in progress). (12/10/2019)
We may occasionally label a source “fake news,” but this is designated specifically for 100% hoax websites. We also classify “hate groups” in this category.
The Conspiracy/Pseudoscience designation is reserved for sources that publish unverifiable information related to known conspiracies such as the New World Order, Illuminati, False Flags, Aliens, anti-vaccination propaganda, etc. Pseudoscience is determined by publishing unverified health and scientific claims. For example, sources that promote human-influenced climate change denial or take anti-vaccination positions will be classified as pseudoscience. According to the Stanford Encyclopedia of Philosophy on Science and Pseudoscience, “Philosophers and other theoreticians of science differ widely in their views on what science is. Nevertheless, there is virtual unanimity in the community of knowledge disciplines on most particular issues of demarcation. There is a widespread agreement, for instance, that creationism, astrology, homeopathy, Kirlian photography, dowsing, ufology, ancient astronaut theory, Holocaust denialism, Velikovskian catastrophism, and climate change denialism are pseudosciences.“ To be classified in this group, the source’s central theme must revolve around conspiracies or pseudoscience. See out Pseudoscience Dictionary for more. (9/30/2020)
Media Bias Fact Check pulls page views data from Similar Web to determine the amount of traffic a source receives. If data is not available, we use the best estimate. Our categories are as follows: (3/1/2021)
Minimal Traffic: Under 250K page views per month.
Medium Traffic: 250K to 5 million page views.
High Traffic: Over 5 million page views.
MBFC Credibility Rating
This rating is calculated mathematically with some subjective input. Ratings are calculated based on factual reporting, traffic, and whether the source falls in the conspiracy-pseudoscience or questionable category. Bias rating is not considered with this rating. (3/1/2021)
High Credibility: Any source that is rated Very-High or High in factual reporting. Any source that is rated Mostly Factual in reporting and has Medium or High Traffic.
Medium Credibility: Any source rated Mostly Factual but has minimal traffic and rated Left or Right. Any factually Mixed source with Medium or High Traffic or Minimal Traffic and very few failed fact checks. A source that lightly promotes pseudoscience, such as fad diets or other more harmless misinformation areas, but otherwise reports factually can attain medium credibility (this is subjectively determined on a source by source basis).
Low Credibility: A mixed factual source with low traffic that fails fact checks or any source rated Low for factual reporting or is rated as Questionable or Conspiracy.
Media Bias/Fact Check rarely conducts original fact checks as many other sources are faster and do a better job. We primarily rely on fact-checkers affiliated with the International Fact-Checking Network ( IFCN). Below is their code of principles. (9/30/2020)
Media Bias/Fact Check (MBFC News) adheres to the International Fact-Checking Network Fact-checkers’ Code of Principles. The Poynter Institute developed these principles to promote excellence and standardization in Fact-Checking.
MBFC News strictly adheres to the following principles for all fact checks:
- A COMMITMENT TO NONPARTISANSHIP AND FAIRNESS
We fact-check claims using the same standard for every fact check. We do not concentrate our fact-checking on any one side. We follow the same process for every fact check and let the evidence dictate our conclusions. We do not advocate or take policy positions on the issues we fact-check.
- A COMMITMENT TO TRANSPARENCY OF SOURCES
We want our readers to be able to verify our findings themselves. We provide all sources in enough detail that readers can replicate our work, except in cases where a source’s personal security could be compromised. In such cases, we provide as much detail as possible.
- A COMMITMENT TO TRANSPARENCY OF FUNDING & ORGANIZATION
We are transparent about our funding sources. If we accept funding from other organizations, we ensure that funders do not influence the conclusions we reach in our reports. We detail all key figures’ professional backgrounds in our organization and explain our organizational structure and legal status. We clearly indicate a way for readers to communicate with us.
- A COMMITMENT TO TRANSPARENCY OF METHODOLOGY
We explain the methodology we use to select, research, write, edit, publish and correct our fact checks. We encourage readers to send us claims to fact-check and are transparent on why and how we fact-check.
- A COMMITMENT TO OPEN AND HONEST CORRECTIONS
We publish our corrections policy and follow it scrupulously. We correct clearly and transparently in line with our corrections policy, seeking so far as possible to ensure that readers see the corrected version.
Alexander Dyck & Natalya Volchkova & Luigi Zingales, 2008. “The Corporate Governance Role of the Media: Evidence from Russia,” Journal of Finance, American Finance Association, vol. 63(3), pages 1093-1135, 06.
Baron, David P. “Persistent media bias.” Journal of Public Economics 90.1 (2006): 1-36.
Bolinger, Dwight. Language-the loaded weapon: the use and abuse of language today. Routledge, 2014.
Chun-Fang Chiang & Brian Knight, 2011. “Media Bias and Influence: Evidence from Newspaper Endorsements,” Review of Economic Studies, Oxford University Press, vol. 78(3), pages 795-820.
DellaVigna, Stefano and Ethan Kaplan. “The Fox News Effect: Media Bias and Voting.” Quarterly Journal of Economics 122 (August 2007): 1187-1234.
Entman, R. M. (2007), Framing Bias: Media in the Distribution of Power. Journal of Communication, 57: 163–173. doi:10.1111/j.1460-2466.2006.00336.x
Eveland, W. P. and Shah, D. V. (2003), The Impact of Individual and Interpersonal Factors on Perceived News Media Bias. Political Psychology, 24: 101–117. doi:10.1111/0162-895X.00318
Farhi, Paul. How Biased is the Media Really. 27 April 2012. 20 November 2012. <http://www.washingtonpost.com/lifestyle/style/how-biased-is-the-media-really/2012/04/27/gIQA9jYLmT_story.html>.
Gentzkow, Matthew and Jesse M. Shapiro. “Media Bias And Reputation,” Journal of Political Economy, 2006, v114(2,Apr), 280-316.
“How To Detect Bias In News Media.” FAIR. N.p., 2012. Web. 15 Feb. 2016.
Matthews, Jack. “The effect of loaded language on audience comprehension of speeches.” Communications Monographs 14.1-2 (1947): 176-186.
Morris, J. S. (2007), Slanted Objectivity? Perceived Media Bias, Cable News Exposure, and Political Attitudes*. Social Science Quarterly, 88: 707–728. doi:10.1111/j.1540-6237.2007.00479.x
Nie, N. H., Miller, III, D. W., Golde, S., Butler, D. M. and Winneg, K. (2010), The World Wide Web and the U.S. Political News Market. American Journal of Political Science, 54: 428–439. doi:10.1111/j.1540-5907.2010.00439.x
Puglisi R, Snyder JM. Newspaper Coverage of Political Scandals. Journal of Politics. 2011;73(3):931-950.
Russell, Nick. “Morals and the Media Ethics in Canadien Journalism: Second Edition.” Vancouver: UBC Press, 2006.
S.I. Hayakawa, Alan Hayakawa. “Language in Thought and Action: Fifth Edition.” New York: Harcourt Brace Jovanovich,
Tewksbury, D., Jensen, J. and Coe, K. (2011), Video News Releases and the Public: The Impact of Source Labeling on the Perceived Credibility of Television News. Journal of Communication, 61: 328–348. doi:10.1111/j.1460-2466.2011.01542.x
The Quarterly Journal of Economics (2005) 120 (4): 1191-1237.doi: 10.1162/003355305775097542
Disclaimer: The methodology used by Media Bias Fact Check is our own. It is not a tested scientific method. It is meant as a simple guide for people to get an idea of a source’s bias. Media Bias Fact Check will always review and change any factual errors when brought to our attention. We make every effort to be as factual as possible. Our goal is to have MBFC rated as least biased by our own criteria.