When performing literature searches for a systematic review it's important to use a wide range of resources and searching methods in order to identify all relevant studies. As expert searchers, librarians play an important role in making sure your searches are comprehensive and reproducible. Standard 3.1.1 of the Institute of Medicine's Finding What works in Health Care (2011) recommends working with a librarian trained in performing systematic reviews to plan the search strategy, and Standard 3.1.3 states that an independent librarian should peer review the search strategy.
The following databases are recommended for locating information in support of systematic reviews in health sciences. The databases you choose to include in your systematic review will depend on the topic of your review, but the Cochrane Handbook for Systematic Reviews of Interventions says the three most important databases to consider are MEDLINE (PubMed via LHL), Embase, and the Cochrane Central Register of Controlled Trials (CENTRAL). Remember, no single database can search all the medical literature.
Sources for finding databases:
For more information on choosing and using databases, please see our guide, Getting Started: Finding Scholarly Articles.
Grey literature is information that falls outside the mainstream of published journal and monograph literature, and is not controlled by commercial publishers. It includes conference abstracts and papers, clinical trials, governmental or private sector research, hard to find studies, reports, and dissertations. Including grey literature in your systematic review has been proposed as one method to reduce publication bias. Our guide, Grey Literature in the Health Sciences, includes more information on why it is important to include grey literature in your research, as well as techniques and sources for locating it.
The PRISMA Statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) includes an evidence-based minimum set of reporting items as well as a flow-chart to follow when reporting a systematic review or meta-analysis.
These two items on the checklist relate to the search process:
According to Liberati (2009), for each database searched authors should report, at a minimum, the database, platform, or provider, the start and end dates for the search of each database, and who developed and conducted the searches. Authors should also report the use of supplementary searching techniques, such as handsearching and checking reference lists, and should report if they attempted to acquire any missing information from investigators or sponsors.
With regards to the search, Liberati advises authors to report the full electronic search strategy for at least one major database, indicate how the search took into account other databases searched, and state whether search strategies were peer reviewed. Authors should also report any limitations relevant to the search, such as language and date restrictions.
The Protocol
Section 6.6.2.1 of the Cochrane Handbook, "Reporting the search process in the protocol," says to include the following search information in the review's protocol:
The Abstract
Section 6.6.2.2 of the Cochrane Handbook, "Reporting the search process in the review" says to include the following in the review's abstract:
The Methods
Section 6.6.2.2 of the Cochrane Handbook, "Reporting the search process in the review" says to include the following in the 'Search methods for identification of studies' section:
The Appendix
"Documenting the search process," section 6.6.1 of the Cochrane Handbook for Systematic Reviews of Interventions, says your search process should be documented in enough detail in the review so that all the searches of all the databases are reproducible, and that full search strategies (along with set numbers and the number of records retrieved) should be included in an Appendix to your review. Section 11.2.1 of the Cochrane Handbook encourages review authors to include a study flow diagram as recommended by the PRISMA statement.
Appendix 3: Documenting the search process, from Systematic Reviews: CRD's Guidance for Undertaking Reviews in Health Care, Centre for Reviews and Dissemination, University of York, 2009, says the search should be described briefly in the methods section of the review, and a detailed description can be made available as a web document or as an appendix. The description should include:
More guidance from the CRD:
To describe handsearching: "Provide a list of journal full titles in alphabetical order. State the earliest month and year searched, together with the latest month and year searched, and any missing journal issues which were not searched."
To describe searching websites: "Report the website, the URL, the date searched, any specific sections searched and the search terms used."
Open the text of Appendix 3 for more examples, including: describing searches of conference proceedings, describing the search process, describing backward and forward citation tracking, and more.
The Institute of Medicine's Finding What Works in Health Care: Standards for Systematic Reviews, 2011, Box S-3, "Recommended Standards for Finding and Assessing Individual Studies" includes the following:
Standard 3.4 Document the search
Required elements:
|
3.4.1 |
Provide a line-by-line description of the search strategy, including the date of every search for each database, web browser, etc. |
|
3.4.2 |
Document the disposition of each report identified including reasons for their exclusion if appropriate |
MOOSE: Meta-analysis of Observational Studies in Epidemiology
Table 1: A Proposed Reporting Checklist for Authors, Editors, and Reviewers of Meta-analyses of Observational Studies
Reporting of search strategy should include:
This Excel worksheet was created by Murray Turner at the University of Canberra, and is provided with his permission. You can use this worksheet to document your searches. The worksheet is designed to comply with the IOM Standards for Systematic Reviews and for use with the PRISMA 2009 Flow Diagram.
Contact us:
Lister Hill Library
2nd Floor
205-934-2231
Sterne Library
HUB Desk
205-934-6364
A good search strategy is both sensitive and specific. Sensitivity is the ability to identify all the relevant studies and specificity is the ability to exclude irrelevant studies. As you build your search strategy, you'll need to strike a balance between being comprehensive and maintaining relevance. Increasing the comprehensiveness, or sensitivity, of a search will reduce its precision, but a search that is overly specific may miss important studies. A good search strategy is also a carefully defined strategy that can be repeated by others.
 Formulate a focused question using the PICO format. The Forming a Question section from our Evidence-Based Public Health guide describes the PICO formula and gives you worksheets to help you build your question. |
|
|
|
|
 Identify search terminology. Run broad searches based on the PICO question. Select key articles that answer the question, and analyze those database records for subject headings and keywords (see box below). Break the question into its PICO facets. Start with the "P" concept and make a list of all the subject headings and keywords available for that concept; brainstorm with your team to come up with as many terms as possible for the facet (these terms will be connected with 'OR' to form sets.). Use a Word document or make a grid, like the one under the Search Grid tab in the Tools box, to keep up with your terms. Search each concept separately and then combine (the sets of concepts will be connected with 'AND'.). It isn't always necessary or possible to include all the facets of the PICO format in your search strategy. See "Please note" below. |
|
 Test and adjust the search strategy. You'll need to adjust your search strategy if some aspects of your topic seem to be missing, or if some of the papers you are retrieving suggest additional concepts or search terms that need to be tested. Keep adjusting the strategy until as many articles as possible are found, keeping in mind the balance between sensitivity and specificity. Apply limits, doing so carefully. For instance, limit by date only if it is known that relevant studies could only have been reported during a specific time period. Apply filters/hedges for concepts such as study design. See the Filters/Hedges tab under Tools below. |
|
 Tailor the search strategy for each database. In order to be thorough, it is essential to search more than one database for a systematic review. Syntax, fields, limits will vary depending upon the database, so be sure to review the help files as well as the thesaurus for each database you use. |
|
To perform a comprehensive literature search you'll need to search with subject headings using a controlled vocabulary (or thesaurus) as well as keywords. The controlled vocabulary used by PubMed is called MeSH (Medical Subject Headings) and Embase uses Emtree. Subject searching should provide results that are more consistent and precise; however, it is important to use keywords in addition to subject searching because newer articles may not have been assigned headings, and occasionally headings are not assigned correctly within databases. Always locate the thesaurus or controlled vocabulary for each database you are using, if one is available. Some databases, like Scopus, don't use a controlled vocabulary.
Things to consider when searching with keywords (and to keep in mind while reviewing database help files): synonyms, alternative spellings, spelling errors, local usage, brand or generic names, abbreviations, acronyms, truncation and wildcards, phrases, stopwords. Also, be sure to make note of how (or if) each database handles automatic mapping to subject headings, subject heading explosion, subheadings, use of limits such as language and years, field searching, adjacency searching, boolean operators, combining search sets. This knowledge is crucial to creating a good search strategy.
You should consider excluding from search strategies any concepts that are difficult to express. The difficulty may be due to poor or nonexistent subject headings where keywords may be too broad or imprecise, creating too much scatter in results. Also, you should exclude any search terms that may unintentionally introduce bias. For instance, it might be better to leave out terms for outcomes if it's possible that some studies may not be retrieved because the outcome searched isn't mentioned prominently in the record, even though the study may have measured it. When this is the case, these aspects need to be addressed in the inclusion and exclusion criteria.
Term Discovery:
Search filters, or hedges, are standardized search strategies designed to retrieve relevant articles. Filters may be used to improve the recall of various levels of evidence or to identify clinical concepts. Different databases are searched differently, so filters can only be used in the database they were created for. Since systematic reviews attempt to maximize sensitivity by retrieving all relevant documents, it is preferable not to use filters. However, if the sets retrieved are unmanageable, applying a filter may be helpful.
We've collected some filters for PubMed, as well as some for Embase. In addition to giving you the filters (which you can copy and paste), these links show you how to apply the filters within your PubMed and Embase searches.
A grid like this one will help you keep up with all the terms needed to describe your focused question. The terms are combined using the Boolean operators AND/OR. Concepts to be combined with AND go across the grid and concepts to be combined with OR go down the grid.
Considering the length of time it takes to complete a systematic review, you will more than likely need to rerun searches in order to include recently published information. Many databases allow you to create personal accounts and save search strategies in order to run updates.
You'll need a MyNCBI account to save searches in PubMed. Our guide, Making the Most of MyNCBI, shows you how to create an account and save search strategies and set auto-alerts.
Our Embase guide includes information on how to save searches and create email alerts, as does our Scopus guide.
Backward and forward citation tracking: Reading the reference lists (backwards citations) from selected articles can reveal references to studies that might not have been retrieved via database searching. Grey literature, such as conference proceedings, is often discovered by reading reference lists. The database Scopus includes reference lists for articles. Forward citation searching identifies papers citing the article in hand, and this can lead you to more recent research. Scopus and Google Scholar are sources of cited references.
Handsearching: The Cochrane Handbook for Systematic Reviews of Interventions, section 6.2.2.1, defines handsearching as a
"manual page-by-page examination of the entire contents of a journal issue...to identify all eligible reports of trials. In journals, reports of trials may appear in articles, abstracts, news columns, editorials, letters or other text."
Reasons for handsearching include:
"(1) not all trial reports are included in electronic bibliographic databases, and (2) even when they are included, they may not contain relevant search terms in the titles or abstracts or be indexed with terms that allow them to be easily identified as trials"
Researchers should select for review important journals that cover the topic of interest for a chosen period of time in order to ensure that no important studies have been missed. Use JCR Web (Journal Citation Reports) to look for the names of high impact journal titles in a particular field. JCR is divided into the science edition and the social sciences edition. Another resource for journal information is ULRICHSWEB Global Serials Directory. If ULRICHSWEB indicates that an important journal is not indexed fully in the databases used for your searches, consider searching that journal by hand. The Abstracting & Indexing Databases tab in ULRICHSWEB lists which databases index a specific journal title.
Contacting other researchers: It may be necessary to reach out to investigators, authors, and experts in the field. This may help clarify study eligibility, characteristics, and risk of bias, as well as identify unpublished or soon to be published data. PubMed is a good source for email addresses as they are often included in the author information field shown in the abstract display:
If the topic involves pharmaceuticals or medical devices, consider contacting the individual companies and manufacturers for unpublished studies.
