In step 5, after the screening process, a key step in conducting a systematic review is assessing each included study for both:
Risk of Bias (internal validity)
Quality (reliability, relevance, and applicability)
Choosing the right assessment tools depends on the study design of your included articles. Always use tools that are specifically designed and validated for those designs.
🔍 For example, if your review includes only randomized controlled trials (RCTs), use the Cochrane Risk of Bias 2.0 Tool (RoB 2) to evaluate potential biases, and pair it with a critical appraisal tool like the CASP RCT Appraisal Tool to assess overall quality.
This dual approach helps ensure your synthesis is both rigorous and transparent.
Risk of Bias (RoB) - Evaluates the risk of bias inherent in the methodology of the study.
Quality Assessment (QA) - Used interchangeably with RoB, QA tools evaluate the quality of the methodology of the study.
Critical Appraisal (CA) - For the purposes of our tool repository, we considered anything that was designed to be used by clinicians or students to evaluate the quality of a article for patient care to be a CA tool.
A librarian can advise you on quality assessment for your systematic review, including:
Quality Assessment and Risk of Bias Tool Repository © 2021 by Leila Ledbetter and Stephanie Hendren is licensed under Attribution-NonCommercial 4.0 International. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/
Risk of bias refers to systematic errors or flaws in study design, conduct, or reporting that can distort the findings. Identifying these risks is essential for evaluating the internal validity of included studies in a systematic review.
Does the study design or conduct introduce systematic errors that could affect the results?
The most widely used tool in evidence synthesis is the Cochrane Risk of Bias 2.0 (RoB 2) tool, developed specifically for randomized controlled trials (RCTs).
Bias from the randomization process
(e.g., inadequate allocation concealment)
Bias due to deviations from intended interventions
(e.g., participants not receiving the assigned treatment)
Bias due to missing outcome data
(e.g., high dropout rates without explanation)
Bias in measurement of the outcome
(e.g., lack of blinding or subjective outcome assessment)
Bias in selection of the reported result
(e.g., selective outcome reporting)
Cochrane RoB 2 Tool – The official RoB 2.0 tool, with guidance, examples, and templates.
RoB 2 Excel Tool – A downloadable Excel template to complete assessments.
ROBINS-I Tool – Used for non-randomized studies of interventions.
Common Tools for RoB (Quality) Assessment – Covidence includes more tools available in their blog.
Critical appraisal is the systematic evaluation of a study’s trustworthiness, relevance, and value in a specific clinical or policy context. While risk of bias tools assess internal validity, CA tools go further—helping determine whether findings are meaningful and applicable to your patient population or setting.
Is this study valid, reliable, and relevant to my patient or population?
CASP (Critical Appraisal Skills Programme) – Checklists for appraising different study types (RCTs, qualitative, systematic reviews, etc.)
JBI (Joanna Briggs Institute) Critical Appraisal Tools – Study-specific tools widely used in health sciences and evidence syntheses.
SIGN (Scottish Intercollegiate Guidelines Network) Checklists – Methodological checklists used in guideline development.
Is the study design appropriate for the research question?
Are inclusion/exclusion criteria clearly defined?
Is the effect size clinically meaningful?
Are the results precise (e.g., confidence intervals provided)?
Was the sample size adequate?
Were participants randomized? Were groups comparable?
Were validated tools used to measure outcomes?
Could confounding factors affect the results?
Covidence supports both standardized and custom templates for data extraction and quality assessment. It includes the Cochrane Risk of Bias 2.0 tool by default, but you can also tailor templates to fit your team’s methods and review goals.
Use the resources below to learn how to set up and customize templates:
Covidence Data Extraction 1
Overview of the Extraction 1 workflow, including single or dual reviewer setup and resolving conflicts.
Covidence Data Extraction 2
Guide to the updated Extraction 2.0 interface, including how to build and manage custom extraction fields.
Covidence Quality Assessment 1
Introduction to using built-in tools like Cochrane Risk of Bias 2.0 and customizing assessment criteria.
Covidence Quality Assessment 2
Step-by-step guide to creating and editing custom quality assessment templates for your review.
This page was based on pages from the University of North Carolina Health Sciences Library and Duke University Medical Center Library & Archives guides on screening in systematic reviews. We are grateful for their support and assistance.
The design of this page was adapted in part from Research: By Course, Subject, or Topic, by University of Arizona Libraries, © 2020 The Arizona Board of Regents on behalf of The University of Arizona, licensed under a Creative Commons Attribution 4.0 International License.