Skip to main content

Reviews: From Systematic to Narrative: Systematic Review

UPDATE

Change in protocol for Systematic Reviews requires the inclusion of unpublished data...

 "This week the Cochrane team explains why their experience with Roche blows a hole in the systematic review enterprise (doi:10.1136/bmj.c7258). ...The two main published trials don’t mention any adverse events, but the partial study reports from Roche listed 10 serious events, three of which were classified as possibly due to oseltamivir. ...

 
From now on, they say, reviewers must have access to all unpublished data, not only from unpublished trials—the usual focus of concern about publication bias—but also from those that have been published in peer reviewed journals. Reviewers must assess entire trial programmes, and so new tools and methods are needed. If the trial reports are incomplete, reviewers should turn to reports from the drug regulators. As Tom Jefferson, the lead author for the Cochrane review, told me, 'it’s goodbye PubMed, goodbye Embase.'”

 BMJ 2011; 342:d212 doi: 10.1136/bmj.d212 (Published 12 January 2011) 

Definition

Generally, a review is an article published after examination of published material on a subject.  It may be comprehensive to various degrees and the time range of material scrutinized may be broad or narrow, but the reviews most often desired are reviews of the current literature.  The textual material examined may be equally broad and can encompass, in medicine specifically, clinical material as well as experimental research or case reports.  State-of-the-art reviews tend to address more current matters. (MeSH National Library of Medicine)

For an in-depth description of the systematic review process, click on the pdf icon in the box below.

Description

A Systematic Review is a specific type of literature review in which a concept is identified and the research which has studied it is analyzed and evaluated. The results of this research are synthesized to present the current state of knowledge regarding the concept.
 

"Systematic reviews exhaustively search for, identify, and summarise the available evidence that addresses a focused clinical question, with particular attention to methodological quality. ...  Clinicians can then apply these results to the wide array of patients who do not differ importantly from those enrolled in the summarised studies. Systematic reviews can also inform investigators about the frontier of current research. Thus, both clinicians and researchers should be able to reliably and quickly find valid systematic reviews of the literature."  (BMJ. 2005 January 8; 330(7482): 68-73)

 

Cook, Mulfow and Haynes explained the differences in reviews in their 1997 article -

 

Systematic reviews: synthesis of best evidence for clinical decisions.  Cook DJ, Mulrow CD, Haynes RB.  Ann Intern Med. 1997 Mar 1;126(5):376-80.PMID: 9054282

"Systematic reviews are scientific investigations in themselves, with pre-planned methods and an assembly of original studies as their “subjects.” They synthesize the results of multiple primary investigations by using strategies that limit bias and random error. These strategies include a comprehensive search of all potentially relevant articles and the use of explicit, reproducible criteria in the selection of articles for review. Primary research designs and study characteristics are appraised, data are synthesized, and results are interpreted.

When the results of primary studies are summarized but not statistically combined, the review may be called a qualitative systematic review. A quantitative systematic review, or meta-analysis, is a systematic review that uses statistical methods to combine the results of two or more studies. The term “overview” is sometimes used to denote a systematic review, whether quantitative or qualitative. Summaries of research that lack explicit descriptions of systematic methods are often called narrative reviews.

Review articles are one type of integrative publication; practice guidelines, economic evaluations, and clinical decision analyses are others. These other types of integrative articles often incorporate the results of systematic reviews. For example, practice guidelines are systematically developed statements intended to assist practitioners and patients with decisions about appropriate health care for specific clinical circumstances. Evidence-based practice guidelines are based on systematic reviews of the literature, appropriately adapted to local circumstances and values. Economic evaluations compare both the costs and the consequences of different courses of action; the knowledge of consequences that are considered in these evaluations is often generated by systematic reviews of primary studies. Decision analyses quantify both the likelihood and the valuation of the expected outcomes associated with competing alternatives. "

 

Systematic Reviews & Public Health:

In clinical medicine, the randomized controlled trial (RCT) is most often viewed as providing the highest level of evidence. In some cases, particularly with public health topics, it is not always feasible or ethical to conduct a RCT.  However, that does not mean that there is no evidence that can be systematically reviewed.  A 2009 editorial in the Bulletin of the World Health Organization has a very nice overview of systematic reviews for public health. ( 2009 March; 87(3): 163)

"After all, public health problems require us to draw on complex sets of qualitative as well as quantitative evidence; and some policy interventions have never been subjected to randomized controlled trials (RCTs) but have been evaluated using other approaches; everything from controlled and uncontrolled before-and-after studies, to time series analyses, to qualitative methods."

 

Systematic Reviews & Evidence-based Practice

 

Systematic reviews grew out of the evidence-based medicine (EBM) movement.  To fully understand why the conclusions of systematic reviews carry more weight than other types of reviews and articles, a little background in evidence-based practice (EBP) would help.  You can look up the definition of EBM online, for our purposes, a simple definition of EBP is "Evidence-based practice is based on evaluation research that highlights interventions that have been found to be effective."

A key pictorial description of the levels of evidence sources is the pyramid.  Of course, the best are at the top and as the picture shows, you have much fewer resources there than a lower level, i.e., Background Info.  The following is one example of an evidence pyramid.

 

(See the Resources page for more information about the Trip Database.) 

 

  For more about Evidence-based Public Health, see the online tutorials:

·         Evidenced-based Practice for Public Health from the Lamar Soutter Library, or

·         Public Health Information & Data tutorial from PHPartners.

 
A Meta Analysis is the quantitative analysis of two or more independent studies to integrate and synthesize the findings and describe the features of the studies that contribute to variation in their results.
 
 
Integrative Review of Research (IRR) pulls together the existing work on a topic and works to understand trends in that body of scholarship. In such a review, the author describes how the issue is conceptualized within the literature, how research methods and theories have shaped the outcomes of scholarship, and what the strengths and weaknesses of the literature are. Meta-analyses are of particular interest when they are accompanied by an interpretive framework that takes the article beyond the reporting of effect sizes and the bibliographic outcome of a computer search. Most IRRs occur when the original meta-analysis does not have sufficient studies for the statistical analysis to be valid.

 The following chart provides a brief comparison of the three major types of reviews.

 

Systematic Review

Meta-analysis

IRR

Definition

“…research article that identifies relevant studies, appraises their quality and summarizes their results using scientific methodology.” (Khan, 2003)

“…a mathematical synthesis of the results….” (Greenhalgh, 1997)

“…a narrative account of information that is already currently available…” (Jesson & Lacey, 2006)

Method of Analysis

Data is abstracted in a standardized format. (Center for Outcomes Research and Education)

Statistical equations are used to synthesize data.

Narrative is used to discuss implications of findings in the current literature.

Searching Technique

Expert and reproducible search techniques applied to databases, print journals and indexes, current studies, experts, and grey literature.

Expert and reproducible search techniques applied to databases, print journals and indexes, current studies, experts, and grey literature.

Expert search techniques applied to applicable databases and search engines.

 

Stage/Time Table

ACTIVITY MONTH TO BEGIN AND END
Preparation of Protocol 1 - 2
Searches for published/unpublished studies 3 - 8
Pilot Test of Eligibility Criteria 2 - 3
Inclusion Assessments 3 - 8   
Pilot test of "Risk of Bias" Assessment 3
Validity Assessments 3 -10
Pilot Test of Data Collection 3
Data Collection 3 - 10
Data Entry 3 - 10
Follow-up of missing information 5 - 11
Analysis 8 - 10
Preparation of Review Report 1 - 11
Keeping the review current 12 -

Systematic Reviews PDF

Web Sites

The links below are web sites that have good introductions and explanations for what a systematic review is and how one is conducted.  If you want to get a beginning handle on the methodology of conducting a systematic review, the Cochrane Methodology Review Group is a good place to start.  Also, if you have access to the Cochrane Library, do a search of systematic review and select the Search All Text option.  That way you will see the Cochrane Review Groups that are involved in the various reviews.  The Methodology Section of that search will give you more articles on the methodology involved.

Introduction to Systematic Reviews

Cochrane Review Groups (CRGs)  Cochrane Methodology Review Group

Systematic Review LibGuide by Angela Hardie at Washington University School of Medicine St. Louis Bernard Becker Medical Library

Software

EPPI-REVIEWER is a web-based programs that manages review data from recording search logs and screening references through to data extraction.  It includes a range of synthesis methods including meta-analysis and qualitative "thematic synthesis.  It is a product of the EPPI-Centre (Evidence for Poplicy and Practice Information and Co-ordinating Centre) at the Social Science Research Unit at the Institute of Education, University of London, UK. You may download the program for a one-month free trial.

Comprehensive Meta-Analysis:   is a tool for conducting MA with a wide range of statistical methods available.  It is owned and developed by a company called Biostat, Inc.  They also have developed a program called Power and Precisions which is a program for statistical power analysis.

RevMan is the software used for preparing and maintaining Cochrane Reviews.  Support for this program and Archie accounts are only available to registered Cochrane authors.

 

© UAB Libraries ι University of Alabama at Birmingham ι About Us ιContact Us ι Disclaimer