Systematic Reviews: Appraisal & Analysis

• RevMan5 is the software used for preparing and maintaining Cochrane Reviews. Free version available.

• EPPI-Reviewer is from the EPPI-Centre, Department of Social Science, University College London. Free trial.

• JBI SUMARI (System for the Unified Management, Assessment and Review of Information) is the Joanna Briggs Institute's suite of software tools used to support all aspects of the systematic review process. Free version available.

• Rayyan is a free web application used to support collaborative screening and other systematic review tasks.

• Parsifal is a free web-based tool which supports multiple stages of a systematic review.

• CADIMA is a free online tool for systematic reviews and evidence/systematic maps. CADIMA guides review authors through the evidence synthesis process, facilitates the coordination of team members, and provides documentation.

• Colandr is an open access machine-learning assisted app for conducting systematic syntheses of evidence from primary and grey literature sources.

• The NIH Study Quality Assessment Tools for various research study types.
• The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has developed an approach to grading quality of evidence and strength of recommendations.

• RoB 2: the Cochrane Collaboration's tool for assessing risk of bias in studies included in Cochrane reviews.

• CEBM: Centre for Evidence Based Medicine, critical appraisal worksheets.

• The JADAD Score is a seven item scale for assessing the quality of clinical trials.

• The Newcastle-Ottawa Scale (NOS) is used for assessing the quality of nonrandomized studies.

• The PEDro Scale is a quality assessment scale for RCTs developed by the Physiotherapy Evidence Database (PEDro).

• Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields. Prepared by Kmet LM, Lee RC & Cook LS, Alberta Heritage Foundation for Medical Research, 2004.

• QUADAS-2 is used in systematic reviews to evaluate the risk of bias and applicability of primary diagnostic accuracy studies.

• Checklists to appraise both quantitative and qualitative studies are available in the Joanna Briggs Institute Reviewers' Manual, in the appendices. Critical appraisal tool downloads are here.

• BestBETs Best Evidence Topics: BETS CA Worksheets are from the Emergency Dept. at Manchester Royal Infirmary, and were originally published in the Journal of Accident & Emergency Medicine.

• Critical Appraisal Skills Programme (CASP): CASP Checklists include tools for systematic reviews, RCTs, cohort studies, case control studies, economic evaluations, diagnostic studies, qualitative studies and clinical prediction rule.

• Scottish Intercollegiate Guidelines Network (SIGN) Critical Appraisal Notes and Checklists can be used to assess the quality of systematic reviews & meta-analyses, RCTs, cohort studies, case-control studies, diagnostic studies, economic studies.

• The CHARMS Checklist, (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies, is designed to help form a review question for and appraisal of all types of primary prediction modelling studies, including, regressions, neural network, genetic programming, and vector machine learning models.

• Critical appraisal checklists from the Specialist Unit for Review Evidence (SURE), Cardiff University.

• Critical Appraisal Worksheets from Duke University Medical Center Library & Archives.

• The CONSORT Statement is an evidence-based minimum set of recommendations for reporting RCTs.

• Critical Appraisal Worksheets from the Dartmouth Biomedical Libraries.

• SYRCLE's (Systematic Review Center for Laboratory Animal Experimentation) risk of bias tool for preclinical animal intervention studies is based on Cochrane's risk of bias tool.

• ROBINS-I (Risk of Bias in Non-randomised Studies - of Interventions) is useful to those undertaking reviews that include non-randomized studies.

The Cochrane Handbook, Section 8.3.3, Quality Scales and Cochrane Reviews, discourages use of scales for assessing quality or risk of bias for Cochrane reviews. According to the Handbook, the use of scales "is not supported by empirical evidence" and "scales have been shown to be unreliable assessments of validity and are less likely to be transparent to users." The Cochrane Handbook recommends using "simple approaches for assessing validity that can be fully reported (i.e. how each trial was rated on each criterion)." Bottom line: perhaps it is advisable to use these scales as support for a thorough assessment of study validity and bias rather than as the sole basis for that assessment.

• Abstrackr is a free online tool for screening systematic review results.

• Center for Research in Evidence-Based Practice (CREBP) Systematic Review Assistant is a free online screening and deduplication tool.

• Covidence is a web-based software platform that streamlines the production of systematic reviews, including Cochrane Reviews. Covidence can be used for citation importing and screening, full-text review, study selection, quality assessment, data extraction, and data exporting. Access to Covidence is provided by UAB Libraries. Please see the Library Services tab for more information.

• DistillerSR is web-based software used for screening references, data extraction and reporting for systematic reviews. Pricing starts at $30/month for students.

• EROS (Early Review Organizing System) is free, web-based software designed to assist with the initial phases of a systematic review: reference management, screening, and quality assessment.

SR Toolbox is a searchable catalog of software tools that can be used to support the systematic review process. The SR Toolbox also includes checklists, guidelines and reporting standards.

Contact us:
Lister Hill Library
2nd Floor
205-934-2231

Sterne Library
HUB Desk
205-934-6364