Skip to Main Content

Systematic Reviews

Assess Bias & Quality of Included Studies

In step 5, after the screening process, a key step in conducting a systematic review is assessing each included study for both:

  1. Risk of Bias (internal validity)

  2. Quality (reliability, relevance, and applicability)

Choosing the right assessment tools depends on the study design of your included articles. Always use tools that are specifically designed and validated for those designs.

🔍 For example, if your review includes only randomized controlled trials (RCTs), use the Cochrane Risk of Bias 2.0 Tool (RoB 2) to evaluate potential biases, and pair it with a critical appraisal tool like the CASP RCT Appraisal Tool to assess overall quality.

This dual approach helps ensure your synthesis is both rigorous and transparent.

Risk of Bias vs Quality Assessment vs Critical Appraisal

Risk of Bias (RoB) - Evaluates the risk of bias inherent in the methodology of the study.     

Quality Assessment (QA) - Used interchangeably with RoB, QA tools evaluate the quality of the methodology of the study.     

Critical Appraisal (CA) - For the purposes of our tool repository, we considered anything that was designed to be used by clinicians or students to evaluate the quality of a article for patient care to be a CA tool.

How a Librarian can help with Step 5

A librarian can advise you on quality assessment for your systematic review, including: 

  • What the quality assessment or risk of bias stage of the review entails
  • How to choose an appropriate quality assessment tool
  • Best practices for reporting quality assessment results in your review

Quality Assessment and Risk of Bias Tool Repository

Contributors:

Quality Assessment and Risk of Bias Tool Repository © 2021 by Leila Ledbetter and Stephanie Hendren is licensed under Attribution-NonCommercial 4.0 International. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/

Risk of Bias

Risk of Bias (RoB)

Risk of bias refers to systematic errors or flaws in study design, conduct, or reporting that can distort the findings. Identifying these risks is essential for evaluating the internal validity of included studies in a systematic review.

Does the study design or conduct introduce systematic errors that could affect the results?

The most widely used tool in evidence synthesis is the Cochrane Risk of Bias 2.0 (RoB 2) tool, developed specifically for randomized controlled trials (RCTs).

RoB 2.0 assesses five domains of potential bias:

  1. Bias from the randomization process
    (e.g., inadequate allocation concealment)

  2. Bias due to deviations from intended interventions
    (e.g., participants not receiving the assigned treatment)

  3. Bias due to missing outcome data
    (e.g., high dropout rates without explanation)

  4. Bias in measurement of the outcome
    (e.g., lack of blinding or subjective outcome assessment)

  5. Bias in selection of the reported result
    (e.g., selective outcome reporting)


Recommended Risk of Bias Tools & Resources:

Critical Appraisal

Critical Appraisal (CA)

Critical appraisal is the systematic evaluation of a study’s trustworthiness, relevance, and value in a specific clinical or policy context. While risk of bias tools assess internal validity, CA tools go further—helping determine whether findings are meaningful and applicable to your patient population or setting.

Is this study valid, reliable, and relevant to my patient or population?


Popular Critical Appraisal Tools:


Ask questions like:

When you appraise, you should evaluate for relevance, reliability, validity, and applicability

Relevance:  ​

  • Is the study design appropriate for the research question?

  • Are inclusion/exclusion criteria clearly defined?

 

Reliability:  ​

  • Is the effect size clinically meaningful?

  • Are the results precise (e.g., confidence intervals provided)?

 

Validity: ​

  • Was the sample size adequate?

  • Were participants randomized? Were groups comparable?

  • Were validated tools used to measure outcomes?

  • Could confounding factors affect the results?

 

Applicability:  ​

  • Are the findings relevant to your setting or patient?

Covidence Data Extraction & Quality Assessment Templates

Covidence Data Extraction & Quality Assessment Templates

Covidence supports both standardized and custom templates for data extraction and quality assessment. It includes the Cochrane Risk of Bias 2.0 tool by default, but you can also tailor templates to fit your team’s methods and review goals.

Use the resources below to learn how to set up and customize templates:

Data Extraction Resources

Quality Assessment Resources

Credits

This page was based on pages from the University of North Carolina Health Sciences Library and Duke University Medical Center Library & Archives guides on screening in systematic reviews. We are grateful for their support and assistance.