Snopes is a Least Biased Source despite what you may have read

We receive about 50 or more contacts per day from people submitting sources to add to the website, fact check requests and complaints and/or suggestions to improve the website. Today, I want to address a complaint that we frequently receive. This complaint involves Snopes. Almost every morning I awaken to a person saying we are wrong to list Snopes as “least biased” because they believe they are extremely liberal. This of course leads to accusations that MBFC is liberally biased. I typically don’t respond to these, but I feel now is the time to address this publicly as it happens daily. By the way, we also get mail accusing us of being extreme right. As I have said on many occasions, I see this as doing a good job. If hate mail only came from one group I would reconsider.

I want to clarify why Snopes is least biased (not unbiased), as to be completely unbiased defies human nature. We all have biases and no matter how hard we try most of us will fall victim to confirmation bias (gravitating toward information that feels right to you even though it might be wrong). I believe confirmation bias plays a huge role in how people perceive fact checkers. I also believe that the right wing media campaign to discredit fact checkers plays a significant role. When I ask people to list credible fact checkers, who are not Snopes, Politifact, Factcheck etc. I never get an answer. My thought is that if these sources are not credible there must be other credible sources. Who are they and what  established criteria are they using?

When we evaluate a source we strictly use our methodology that looks at 4 criteria. Every source goes through this process. I want to break down Snopes for you so that you can see how and why it scores least biased by our criteria.

We always evaluate a minimum of 10 articles, or more, if necessary to be accurate. First, we look at wording. This starts by looking at the headlines. Do the headlines have loaded (emotional) words  in them? Yes or no? We then move on to compare that the headlines match the actual content of the article. We score on a 0 – 10 scale, with 0 being perfect and 10 being dreadful. This has a subjective component as what might be dreadful to one reviewer might be more tolerable to another. Hence, why we have multiple reviewers. Back to Snopes. On wording and headlines we score Snopes at 0. Their headlines usually just ask a question and do not convey emotion or opinion. The actual content of the article matches the headline without deception. It asks the question and then answers it using sources to support the claim.

Next, we look at how factual/well sourced the articles are. Do they list sources and are they credible? In other words, are they going directly to the source, such as transcripts of what someone said and/or to low biased news agencies such as Reuters etc. On sourcing, Snopes always lists where the info comes from and when they cannot be certain they list the claim as Unproven or Mixed. On factual sourcing we score them 0 again as they are very thorough.

The third step is to look at reporting choices. Does the source report both liberal and conservative view points and do they cover them equally? Snopes fact checks everything. If you go to their page right now you will see they are covering a diverse collection of claims. On story selection we score Snopes a 2. This means that they tend to fact check more conservative claims than liberal, but not by very much. We don’t have a raw total, but a basic scan reveals it is close.

Lastly, we look at their political affiliation. This is pretty much a summary of everything we have learned through the first 3 steps and then factor in research about the people behind the website and also funding. Snopes was founded by David Mikkelson who is a political independent, and the website is funded through advertising and not corporate/political donations. Therefore, we score Snopes a 2 on political affiliation as they do cover more conservative fact checks by a very small margin and funding does not seem to be a factor.

If you add up our scores 0 + 0 + 2 + 2 = 4. and then divide this score by 4 and we get a score of 1. Any score between 0-2 is listed as least biased. Snopes is listed on the left side of least biased by our criteria, but not enough to be considered Left-Center.

I know this will not convince all that Snopes is least biased, but I hope it sheds light on the methodology and why it scores the way it does. I am certain I will have many complaints regarding why many right wing claims are false and that perhaps is another article and there is an explanation for that too.

I am expecting this article will not go over well with some.

By Dave Van Zandt


Follow Blog via Email

Enter your email address to follow this blog and receive notifications of new posts by email.

26 Comments on Snopes is a Least Biased Source despite what you may have read

  1. Sarah Grantham // February 26, 2017 at 8:20 pm //

    Thanks for the explanation. I Use Snopes first ( for a long time) , then Factcheck and Politifact. I Don’t know any other reliable sources. Love the details of how you evaluate info.

    Sarah Grantham

    Sent from my iPad


  2. Thanks Sarah. Our Methodology is not vetted by science (yet) and I am sure there are many holes. Our aim is only to point in the right direction. I think we do this. We are not trying to be the definitive source, but the means to give people the tools to look further and to question.

  3. They tend to refute strawmen sometimes. They will exaggerate a position just to refute it.

  4. Just Curious // February 27, 2017 at 3:28 pm //

    Jeff Thompson, not disputing your claim (or agreeing with it), but can you provide examples of Snopes setting up strawman positions, exaggerating the position just to refute it?

  5. Susie Clements // February 27, 2017 at 4:22 pm //

    I think people sometimes read the Snopes Facebook page & conflate it with the website. Articles on the FB page aren’t posted by Snopes and can have very suggestive titles etc (and people get a good kicking in the comments).
    I do wonder if fact checking sites see an up turn of one sort of story or another dependent on who is in power? If our friends in Macedonia etc realise that one sort of story will get more clicks (eg Trump outrage stories) presumably they’ll push more of it out, leading to more fact checking and a feeling that one side or another is getting targeted?

  6. Thank you for such a well written explanation for your rating of Snopes. It was very clear and a fair assessment. I do have one question for you which you did not ask yourselves nor did you ask us. You stated, “Therefore, we score Snopes a 2 on political affiliation as they do cover more conservative fact checks by a very small margin and funding does not seem to be a factor.” This is assuming an equal bombardment of statements from both sides that need fact-checking. The question is, does Snopes cover more conservative fact checks because they have an ever so slightly more liberal bias, or is it because conservative assertions/memes simply tend to outnumber the liberal ones? To rephrase, did you “penalize” Snopes simply because the lying is not 50/50?

  7. This is awesome. How do you have money and time to do all this research and run this site?

  8. Linda Turnipseed // February 28, 2017 at 1:26 pm //

    Perhaps another reason that Snopes is asked to research more conservative questions is because more liberals use Snopes. I have heard several conservatives say that Snopes has a liberal bias. I don’t think that is so, but if conservatives feel that way, it would stand to reason that they would use it less than liberals do. And liberals tend to be using Snopes to check on some conservative statement that they are questioning.

  9. Thanks! I don’t have money, but I have great researchers who work for free because we all feel this is important.

  10. I wish this site was more mainstream. Some of the ratings could use a bit more text (more than the sentence that is there, but less than a whole article). I think this site could be seen as having a mildly liberal bias, though, as conservatives are just bloomin liers.

  11. Vivian Russell // February 28, 2017 at 8:18 pm //

    Thank you for your clear detailing of your methodology. As a moderate Independent, I agree Snopes is least biased and would probably have given it an even lower composite score than 4.
    My only suggestion for this and future sources is to use more than 10 articles on which to score their bias. As a professional auditor and analyst, the limit of 10 examples caused me to spontaneously furrow my brow and tilt my head. My colleagues would tell you that is not a good sign for the department being audited and it’s going to be a long night.

  12. Thank you for explaining your process on Snopes. I’m amazed at how many sites claim to be neutral but are so obviously biased.

  13. Vivian, the minimum of 10 is for new sources we have never heard of. For example, it is pretty easy to get a handle on the “Angry Patriot” with less than 10 articles. A source like Snopes, has been read 100’s of times if not more. As I pointed out, 10 is the minimum, sometimes it takes much more to get the rating accurate. It also sometimes requires more than one researcher to get it right. Also, Snopes score was 1 out of 10 for bias and not 4. A 4 would be classified on the high end of Left-Center.

  14. It is, especially given the current attitude of rampant ‘anti-fact’and anti-intellectualism. No one knows what to believe, and as you so elegantly expressed, most people have some degree of confirmation bias. I’d like to believe some sources are just so obviously trash that ANYONE could see, but the proliferation of Breitbart refernces tell me this is not the case. haha

  15. A little more transparency on your methodology would be helpful. Do you have specific criteria for each point on your 0-10 ratings? Or just a few points on the scale (like 0, 5, and 10)?
    For example: you say that on the political affiliation scale, Snopes has a low score because the founder is a political independent and its founding source is general ad revenue rather than politically-related contributions. That sounds about as good as it gets, yet Snopes scored a 2 rather than 0.
    It would be somewhat problematic if within the scale, or even part of the scale, the specific score is based on how it “feels” to the evaluator, rather than an explicitly-defined set of criteria. I understand that you try to account for issues of judgment by having multiple reviewers, but this could be one point where the methodology needlessly allows for the entry of personal opinion.

  16. Snopes is ok as long as the reader actually reads the entire entry text and doesn’t just rely on their headline. For example, I have, on more than one occasion, found the word “False” used in a Snopes headline only to find after reading the entire text that the important fact/s in question is/are actually stated by Snopes to be “True” while some insignificant aspect of an issue is labeled as “False”, giving Snopes what appears to be the excuse for using the word “False” in the headline, instead of the more appropriate “Partially False”, or, “Partially True”. These “discrepancies and inconsistencies have discouraged me from relying on Snopes. However, since It has been awhile since I have used Snopes, maybe things have improved.

  17. I think you guys are doing a great job, and as you mentioned at the beginning of the second paragraph, being 100%unbiased defies human nature so one shouldn’t read your evaluations as the law, but as a tool to make up their own mind, maybe hear some facts they haven’t heard before and then draw their own conclusions.

    I am catching myself more often than I like being emotional about certain things and therefore reacting biased. But that is human nature. As long as we keep that in mind and as long as we take a step back once we recognize these emotional reactions, I think it’s all good.

    As long as we respectfully disagree, we can all still be friends 🙂

    God bless.

  18. How can I check the bias of mediabias/factcheck?

  19. The same way you would any website. Read it objectively, put your own personal biases aside and see what you come up with.

  20. As a general observation snopes does a pretty good job – but sadly it does have its Achilles heel.

    During the election it managed to lose the Clinton logic chip, and substantially ignored evidence for arguments against her (despite people even providing citations to detailed analyses pointing out that snopes was either factually incorrect or just ignoring evidence) – although we can’t be certain if it was blind adherence to Clinton – or fear of Trump.

  21. You do have to read an entire Snopes article because the “True or False” is too black and white. Muslim no-go zones were labeled as false because of the fact that they won’t kill you on site, yet they still mention how it’s life threatening to still go in those areas. So they didn’t exactly refute it, they are just saying that you aren’t going to be immediately murdered if you go in.

  22. Timothy Cook // May 15, 2017 at 5:01 pm //

    In my opinion, there are more fake news sources and inaccurate popularizations on the right than there are on the left. If true, this would explain why Snopes, squarely in the middle, is so often perceived as leftist by people who are viewing them from the perspective of the far right. It’s comparable to the way New Yorkers speak of the sun setting on the western plains every night, a perspective Californians cannot possibly understand.

  23. That seems reasonable Timothy. Someone calculated that 70% of our questionable websites are extreme right. This either means we review more right wing sites or they simply have more questionable sites. Several studies from well respected low biased sources have confirmed that more fake news originates from the right, but this number has been tightening since the election of Trump. As long as Snopes sources to credible information they will remain in our least biased category.

  24. Gulliver Foyle // June 3, 2017 at 12:16 pm //

    “The third step is to look at reporting choices. Does the source report both liberal and conservative view points and do they cover them equally? Snopes fact checks everything. If you go to their page right now you will see they are covering a diverse collection of claims. On story selection we score Snopes a 2. This means that they tend to fact check more conservative claims than liberal, but not by very much. We don’t have a raw total, but a basic scan reveals it is close.”

    This metric concerns me a little bit. Choosing an equal amount of right and left claims to fact check isn’t being unbiased, it is being center-biased. An unbiased outlet should not even entertain the political leaning of a claim as one of the criteria for choosing whether to fact check it or not. They should target claims based on their popularity, based on whether other reputable fact checkers have already looked at them or not, and other utilitarian criteria.

    It could easily turn out that one side or another was circulating a lot more, or a lot more popular, unverified claims. Lets say for the sake of argument that it is the right doing this. In that case a truly unbiased outlet would be fact checking a lot more right wing claims than left wing claims, and your metric would rate them as biased. Meanwhile a center-biased outlet might ignore major/popular right wing claims in order to fact check minor/obscure left wing claims, in pursuit of the appearance of being unbiased. They’d score better on your metric, but obviously aren’t.

  25. To make this even more complicated I did my own fact checking of sorts on what was perceived to be a right-center article on a conservative friendly site. The news piece was reluctantly giving concern about potential descent within the party that needed to be unveiled and addressed. Further research on the author revealed 1) He wrote articles for a couple other sites but they were clearly liberal. 2) He had photos of his recent same sex marriage. 3)He was seeking referrals on Facebook for journalist that are ‘unapologetic’ in their mission. He recieved several replies with names along assurances they right for the job.

    This was only one example but clearly there is a strategy in play to deceive and sabotage.

  26. It is rather natural that there is a tendency to have more conservative fact checking, since it is a conservative government in power right now – they simply do more that matters right now. At the same time the current president have a tendency to not be stellar at telling the truth or basing his factual information on somewhat shady sources and on top of it he tend to spin it emotionally and it may even get blown up in numbers by the comments from his nearest administration, who attempts to back him up. This at large makes conservative fact checking look like it has a liberal bias, because more of the significant originators (president + cabinet + top reps) turns out some fraudulent or semifalse claims more often than it would usually have been due to the mentality of the president currently in the WH and how he communicates. just my 2c :p

Leave a Reply

%d bloggers like this: