When determining bias, there isn’t any true scientific formula that is 100% objective. There are objective measures that can be calculated, but ultimately there will be some degree of subjective judgement to determine these. On each page we have put up a scale with a yellow dot that shows the degree of bias for each source. Each page also has a “Notes” section that gives some details about the source and an explanation of their bias. When calculating bias we are not just looking at political bias, but also how factual the information is and if they provide links to credible, verifiable sources. Therefore, the yellow dot may indicate political bias or how factual a source is, or in many cases, both. Further, we have placed a voting poll on each page for readers to vote on the bias of the source. This provides more information for the media consumer when deciding whether to trust a source. This is similar to how movie sites such as IMDB or Rotten Tomatoes rate their movies. They provide the reviewer’s information and also allow the public to rate the movie. This gives two perspectives in which people can decide whom to trust. This voting poll also allows MBFC News to examine if the public is agreeing or disagreeing with our ratings. If there is a large discrepancy we will re-examine that source and adjust accordingly after a thorough review.
Here is a look at some example scales from different media sources:
For Example, CNN looks like this:
Breitbart looks like this:
MSNBC looks like this:
The method for placement of the yellow dot is determined by ranking bias in four different categories. In each category the source is rated on a 0-10 scale, with 0 meaning without bias and 10 being the maximum bias(worst). These four numbers are then added up and divided by 4. This 0-10 number is then placed on the line according to their Left or Right bias. Scoring is as follows:
0 – 2 = Least Biased
2 – 5 = Left/Right Center Bias
5 – 8 = Left/Right Bias
8 – 10 = Extreme Bias
The categories are as follows:
- Biased Wording/Headlines- Does the source use loaded words to convey emotion to sway the reader. Do headlines match the story.
- Factual/Sourcing- Does the source report factually and back up claims with well sourced evidence.
- Story Choices: Does the source report news from both sides or do they only publish one side.
- Political Affiliation: How strongly does the source endorse a particular political ideology? In other words how extreme are their views. (This can be rather subjective)
Here is an example of how CNN scored and why they were placed in the middle of Left Bias:
Biased Wording = 5 (CNN uses moderate biased words that favor liberals and headlines typically match the story)
Factual/Sourcing = 4 (CNN is mostly trustworthy for providing evidence and sources, but sometimes jumps the gun on breaking stories)
Story Choices = 7 (CNN mosly favors pro-liberal stories and publishes negative conservative stories)
Political Affiliation = 7 (CNN mostly favors liberal ideology through content and wording)
Total = 23
Average 23/4 = 5.75
5.75 = Moderate Left Bias
Each source has a “Factual Reporting” rating based on their Factual/Sourcing score above. The ratings are as follows.
Factual Reporting: VERY HIGH = a score of 0, which means the source is always factual, sources to credible information and makes immediate corrections to incorrect information and has never failed a fact check.
Factual Reporting: HIGH = a score of 1 – 3, which means the source is almost always factual, sources to mostly credible low biased or high factual information and makes immediate corrections to incorrect information and has failed only 1 fact check.
Factual Reporting: MIXED = a score of 4 – 6, which means the source does not always use proper sourcing or sources to other biased/mixed sources. They may also report well sourced information as well. Mixed sources will have failed one or more fact checks and does not immediately correct false or misleading information. While the majority of the information may be factual on these sources, they need to be checked. Further, any source that does not disclose a mission or ownership information will automatically be deemed mixed.
Factual Reporting: LOW = a score of 6 – 8, which means the source rarely uses credible sources and is simply not trustworthy for reliable information. These are the sources that need to be fact checked for intentional fake news, conspiracy and propaganda.
Factual Reporting: VERY LOW = a score of 9 – 10, which means the source almost never uses credible sources and is simply not trustworthy for reliable information at all. These are the sources that always need to be fact checked for intentional fake news, conspiracy and propaganda.
For each source a minimum of 10 headlines are reviewed and a minimum of 5 news stories reviewed. We first review news reporting and follow that up by looking at editorial and opinion pieces. If there is still not clear evidence we will use searching methods for articles on the site such as “Republican”, “Democrat”, “Liberal”, “Conservative”, “Trump”, “Clinton” etc. until we are sure of political affiliation. This process can be time consuming or very simple depending on the source.
In order to understand the ratings it is important to understand the terminology. We look at all of these when researching a source:
Bias by Omission: leaving one side out of an article, or a series of articles over a period of time; ignoring facts that tend to disprove liberal or conservative claims, or that support liberal or conservative beliefs.
Bias by Labeling: Bias by labeling comes in two forms. The first is the tagging of conservative politicians and groups with extreme labels while leaving liberal politicians and groups unlabeled or with more mild labels, or vice versa. The second kind of bias by labeling occurs when a reporter not only fails to identify a liberal as a liberal or a conservative as a conservative, but describes the person or group with positive labels, such as “an expert” or “independent consumer group.”
Bias by Placement: is where on a website (or newspaper) or in an article a story or event is printed; a pattern of placing news stories so as to downplay information supportive of either conservative views or liberal views.
Bias by Selection of Sources: including more sources that support one view over another.
Bias by Spin: is a reporter’s subjective comments about objective facts; makes one side’s ideological perspective look better than another.
Bias by Story Selection: a pattern of highlighting news stories that coincide with the agenda of either the Left or the Right, while ignoring stories that coincide with the opposing view.
Confirmation Bias: also called confirmatory bias or my side bias, is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s beliefs or hypotheses, while giving disproportionately less consideration to alternative possibilities.
Connotation: the emotional and imaginative association surrounding a word that can be either positive or negative.
Denotation: the strict dictionary meaning of the word.
Loaded Language (Words): (also known as loaded terms or emotive language) is wording that attempts to influence an audience by using appeal to emotion or stereotypes. Such wording is also known as high-inference language or language persuasive techniques.
Purr Words: words used to describe something that is favored or loved.
Snarl Words: words used when describing something that a person is against or hates.
Other factors to look for:
Do the headlines and stories match?
Are important stories featured prominently?
Does the story offer an alternative point of view?
Consider the source!
A questionable source exhibits any of the following: extreme bias, overt propaganda, poor or no sourcing to credible information and/or is fake news. Fake News is the deliberate attempt to publish hoaxes and/or disinformation for the purpose of profit or influence. Sources listed in the Questionable Category may be very untrustworthy and should be fact checked on a per article basis. Typically, the number one criteria for this category is how factual their reporting is. The less factual the more likely to be placed in this category.
We may occasionally label a source “fake news”, but this is designated specifically for a 100% hoax websites. We also classify “hate groups” in this category.
The Conspiracy/Pseudoscience designation is reserved for sources that publish unverifiable information that relates to known conspiracies such as the New World Order, Illuminati, False Flags, Aliens, etc. Pseudoscience is determined by publishing unverified health and scientific claims. In order to be classified in this group the central theme of the source must revolve around conspiracies or pseudoscience.
Media Bias/Fact Check (MBFC News) adheres to the International Fact-Checking Network Fact-checkers’ Code of Principles. These principles were developed by the Poynter Institute to promote excellence and standardization in Fact Checking.
MBFC News strictly adheres to the following principles for all fact checks:
- A COMMITMENT TO NONPARTISANSHIP AND FAIRNESS
We fact-check claims using the same standard for every fact check. We do not concentrate our fact-checking on any one side. We follow the same process for every fact check and let the evidence dictate our conclusions. We do not advocate or take policy positions on the issues we fact-check.
- A COMMITMENT TO TRANSPARENCY OF SOURCES
We want our readers to be able to verify our findings themselves. We provide all sources in enough detail that readers can replicate our work, except in cases where a source’s personal security could be compromised. In such cases, we provide as much detail as possible.
- A COMMITMENT TO TRANSPARENCY OF FUNDING & ORGANIZATION
We are transparent about our funding sources. If we accept funding from other organizations, we ensure that funders have no influence over the conclusions we reach in our reports. We detail the professional background of all key figures in our organization and explain our organizational structure and legal status. We clearly indicate a way for readers to communicate with us.
- A COMMITMENT TO TRANSPARENCY OF METHODOLOGY
We explain the methodology we use to select, research, write, edit, publish and correct our fact checks. We encourage readers to send us claims to fact-check and are transparent on why and how we fact-check.
- A COMMITMENT TO OPEN AND HONEST CORRECTIONS
We publish our corrections policy and follow it scrupulously. We correct clearly and transparently in line with our corrections policy, seeking so far as possible to ensure that readers see the corrected version.
FACT CHECKING METHODOLOGY
Media Bias/Fact Check (MBFC News) fact checks a wide range of political statements from both elected and non-elected government officials, media pundits, and special interest groups. We also examine claims made on social media in the form of memes and viral images.
MBFC News recognizes that Politifact, Fact Check and Snopes do a fantastic job fact checking major claims. We feel our role is to check claims that perhaps have been bypassed. We will occasionally select our own fact-checks to research, but primarily rely on submissions from users of our website. MBFC News selects what to fact check using the same criteria as Politifact:
- Is the statement rooted in a fact that can be verified? We don’t check opinions.
- Is the statement leaving a particular impression that may be misleading?
- Is the statement significant?
- Is the statement likely to be repeated by others?
- Would a typical person wonder if the statement is true or not?
MBFC’s research generally begins by attempting to contact the source of a claim. If an elected official, for example, makes a claim we reach out to that individual’s office for clarification or request a source to back up that claim.
If it is not possible to contact the source, MBFC will then turn to news articles, journals, and on-record interviews with reporters and experts on the subject. On-record interviews assist our writers and researchers with the interpretation of data. MBFC does not allow off-record sources. In other words, we do not accept “experts claim.”
MBFC always seeks out nonpartisan data sources whenever possible. Such sources often include government agencies like the Bureau of Labor Statistics, the Congressional Budget Office, Tax Policy Center, the Tax Foundation and the Center for Effective Government, etc.
MBFC always publishes a list of its sources along with each article.
MBFC rating system has four ratings:
- TRUE – The statement is accurate and there’s nothing significant missing.
- MOSTLY TRUE – The statement is in general accurate but may need additional information or clarification.
- MOSTLY FALSE – The statement contains an element of truth but ignores critical facts that may give the wrong impression.
- BLATANT LIE – The statement is not accurate at all and makes a ridiculous claim.
MBFC does not use the Half-True claim as we feel it does not have value. If something is Half-True it simply is not worth reporting.
Alexander Dyck & Natalya Volchkova & Luigi Zingales, 2008. “The Corporate Governance Role of the Media: Evidence from Russia,” Journal of Finance, American Finance Association, vol. 63(3), pages 1093-1135, 06.
Baron, David P. “Persistent media bias.” Journal of Public Economics 90.1 (2006): 1-36.
Bolinger, Dwight. Language-the loaded weapon: the use and abuse of language today. Routledge, 2014.
Chun-Fang Chiang & Brian Knight, 2011. “Media Bias and Influence: Evidence from Newspaper Endorsements,” Review of Economic Studies, Oxford University Press, vol. 78(3), pages 795-820.
DellaVigna, Stefano and Ethan Kaplan. “The Fox News Effect: Media Bias and Voting.” Quarterly Journal of Economics 122 (August 2007): 1187-1234.
Entman, R. M. (2007), Framing Bias: Media in the Distribution of Power. Journal of Communication, 57: 163–173. doi:10.1111/j.1460-2466.2006.00336.x
Eveland, W. P. and Shah, D. V. (2003), The Impact of Individual and Interpersonal Factors on Perceived News Media Bias. Political Psychology, 24: 101–117. doi:10.1111/0162-895X.00318
Farhi, Paul. How Biased is the Media Really. 27 April 2012. 20 November 2012. <http://www.washingtonpost.com/lifestyle/style/how-biased-is-the-media-really/2012/04/27/gIQA9jYLmT_story.html>.
Gentzkow, Matthew and Jesse M. Shapiro. “Media Bias And Reputation,” Journal of Political Economy, 2006, v114(2,Apr), 280-316.
“How To Detect Bias In News Media.” FAIR. N.p., 2012. Web. 15 Feb. 2016.
Matthews, Jack. “The effect of loaded language on audience comprehension of speeches.” Communications Monographs 14.1-2 (1947): 176-186.
Morris, J. S. (2007), Slanted Objectivity? Perceived Media Bias, Cable News Exposure, and Political Attitudes*. Social Science Quarterly, 88: 707–728. doi:10.1111/j.1540-6237.2007.00479.x
Nie, N. H., Miller, III, D. W., Golde, S., Butler, D. M. and Winneg, K. (2010), The World Wide Web and the U.S. Political News Market. American Journal of Political Science, 54: 428–439. doi:10.1111/j.1540-5907.2010.00439.x
Puglisi R, Snyder JM. Newspaper Coverage of Political Scandals. Journal of Politics. 2011;73(3):931-950.
Russell, Nick. “Morals and the Media Ethics in Canadien Journalism: Second Edition.” Vancouver: UBC Press, 2006.
S.I. Hayakawa, Alan Hayakawa. “Language in Thought and Action: Fifth Edition.” New York: Harcourt Brace Jovanovich,
Tewksbury, D., Jensen, J. and Coe, K. (2011), Video News Releases and the Public: The Impact of Source Labeling on the Perceived Credibility of Television News. Journal of Communication, 61: 328–348. doi:10.1111/j.1460-2466.2011.01542.x
The Quarterly Journal of Economics (2005) 120 (4): 1191-1237.doi: 10.1162/003355305775097542
Disclaimer: The methodology used by Media Bias Fact Check is our own. It is not a tested scientific method. It is meant as a simple guide for people to get an idea of a source’s bias. Media Bias Fact Check will always review and change any factual errors when brought to our attention. We make every effort to be as factual as possible. Our goal is to have MBFC rated as least biased by our own criteria.
We signed the Pro-Truth Pledge:
please hold us accountable.
Ads do not necessarily reflect the views of MBFC