Grey Literature in the Health Sciences: Bias

Publication bias is a reporting bias. Many researchers have shown that those studies with significant, positive, results have a better chance of being published, are published earlier, are published in journals with higher impact factors, and are easier to find.  Conclusions exclusively based on published studies, therefore, can be misleading. 

Including grey literature has been proposed as one method to reduce publication bias.

References:

• Trespidi C, Barbui C, Cipriani A. Why it is important to include unpublished data in systematic reviews. Epidemiol Psychiatr Sci. 2011;20(2):133-5.

• Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: MR000006. DOI: 10.1002/14651858.MR000006.pub3. Read

• Dubben HH, Beck-Bornholdt HP. Systematic review of publication bias in studies on publication bias. BMJ. 2005 Aug 20;331(7514):433-4.Read

• Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867-72. Read

This video tutorial from Health InfoNet of Alabama demonstrates evaluating 2 representative health websites using the A-B-C approach (authority, currency, and bias). 

Bias a term used to describe a tendency or preference towards a particular perspective, ideology or result.

It is important to evaluate sources of reports and white papers found online.  Some of the "think tanks" and organizations that  publish reports and working papers online have their own political or social agenda you should consider.

Health statistics can be influenced by an organization’s perspective and bias. Whenever possible, read the notes describing the reasons for and methods of data collection. Remember that statistics are collected to meet the needs of the collector. 

Here are some questions to ask to help evaluate for bias:

• What is the purpose of the website? Who is providing funding or paying for the site?
  • Is the site is supported by public funds, donations or by commercial advertising.
  • Are advertisements clearly labeled?  They should say "Advertisement" or "From our Sponsor."
• How objective is the information? 
  • What do you know about who is publishing this information? 
  • Is there a political, social or commercial agenda? 
• Does the information try to inform or persuade? How balanced is the presentation on opposing perspectives? 
• What is the tone of language used (angry, sarcastic, balanced, educated)?

from: Public health information and data: a training manual [Internet]. Bethesda (MD): National Library of Medicine (US); 2004 http://www.phpartners.org/pdf/phmanual.pdf 

Sourcewatch is a collaborative, specialized encyclopedia of the people, organizations, and issues shaping the public agenda.  It is published by the Center for Media and Democracy. It can be one place to look to find information on an organization.