Confirmation Bias

In order to do good high quality and fun science, we general have a question we want an answer to, which is more often referred to as a hypothesis. (Not all good science is done this way, some is strictly and/or initially observational.) Now when we ask a question, we may very well have a personal stake in the answer.

An example may be warranted. Let’s say it’s 1940 and you’re a scientist interested in the question does smoking cause lung cancer? Your hypothesis is probably along the lines of “I hypothesize that smoking leads to lung cancer.” If your dearest grandmother and father died of lung cancer, you may already think the hypothesis is correct; if you happen to collect a paycheck from Phillip-Morris and smoke yourself, you may think the hypothesis is wrong.

Now having a pre-established bias is not inherently problematic, it’s inherently human. What you do with this bias is the issue. In the above example, it is fairly clear what the biases are and why they exist and these biases are likely to be identified by a scientist who’s worth her salt. One of our jobs as good scientists is to identify sources of bias. In the above example there are strong personal sources of bias, but in the day-to-day workings of a research laboratory there are more subtle biases. The I-think-this-is-a-cool-idea bias (probably the most common), or the this-result-could-result-in-a-glamour-mag-publication bias, or my-competitor-has-another-idea bias., the list goes on. So, it is important to identify your biases. But this begs the question WHY?

One reason I want to address here, because it goes well beyond the laboratory, is the issue of confirmation bias. Confirmation bias is what happens when you conflate positive data (results that affirm your preconceived notions) and/or diminish negative data (results that contradict your preconceived notions). Confirmation bias helps the casinos make several billion dollars in profit every year. Confirmation bias leads to the retraction of some high profile publications every year. Confirmation bias makes pundits and those that parrot them look like idiots.

So, using our smoking/lung cancer hypothesis above scientist #2 might use a few 2 pack/day smokers without cancer as her sample population. Whereas scientist #1 might recruit her study population in the cancer wards. Although my examples are over the top, the choices made by a scientist that are a product of confirmation bias could be much more subtle and not apparent to other scientists who may be reviewing the work for a publication. This is why good scientists are, or at least try to be, vigilant about their biases and take steps to ensure they aren’t screwing up.

Say for instance you think there is a link between Chronic Fatigue Syndrome and XMRV or that there is a link between the MMR vaccine and autism. Do you find the one poorly designed study that supports your position while simultaneously excluding the plethora of other studies that don’t? If so, you may be a confirmationally biased redneck.

This goes well beyond the sciency stuff. Do you go to Vegas and proclaim the $250 jackpot you won, while failing to point out the 32 $20 bets you lost? The casinos aren’t making billions in profits every year on paying out more than they take in.

Do you have a problem with illegal aliens in Arizona as a recent threat to our well being? Maybe you note whenever an illegal is arrested for a crime as proof of a problem while ignoring fact that crime is lower* in Phoenix in 2009** than in 2002**?

Do you think atheists have split into the old pleasant docile type and some New type that is the root cause of most problems in America? Maybe you’ll take the word of a sockpuppet identity thief because they agree with what you already think over many real voices that disagree.

Hot today...global warming is true. Cold tomorrow...global warming is a myth.

*This is a 10 year comparison for Phoenix specifically.

**Compare pages 21 and 14 in the 2009 and 2002 years for a simplified breakdown.