On this page

In this section, learn how to properly and effectively conduct comprehensive searches of the literature.

• Search
• A few tips for conducting a systematic literature review search
• Comprehensiveness vs. precision (sensitivity-specifity)
• Guidelines and standards

Search

Once the research question has been suitably defined, the next step is the complete and unbiased identification of relevant studies, which will require conducting "a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource limits)" (Cochrane handbook, Ch. 4.2.2). Several standard processes need to be involved, such as searching multiple databases and other supplementary search sources, using a combination of keywords in conjunction with subject headings and reassuring that repeated searches retrieve similar results.

A few tips for conducting a systematic literature review search

• Systematic reviews must be comprehensive (involve serveral sources), transparent, reproducible and unbiased.
• Searches should seek high sensitivity, which may also increase the number of irrelevant results.
• Too many different search concepts should be avoided, but a wide variety of search terms should be combined with OR within each concept.
• Highly sensitive search strategies (filters) can be used to identify types of studies, for example randomized controlled trials.
• Don’t use the Library Catalogue as one of your reportable databases - the content fluctuates.
• Don’t limit yourself by using any ‘full text only’ limits available in databases - staff and research degree students have access to the literature of the world via interlibrary loan.
• Search databases one at a time - e.g. do not search MEDLINE and PsycINFO simultaneously on the EBSCO interface.
• Use a combination of subject headings (where available) and free text terms (‘keywords’) to ensure your search is comprehensive.
• Build your search one concept at a time, then combine these ‘search sets’ together.
• If you search for Intervention AND Comparison and find few (or no!) results, try combining these with an OR. This expands your search to find articles including either or both. You could then undertake the data comparison yourself.
• Ensure you are using the right subject headings for that database - e.g. MeSH for MEDLINE and Cochrane Library, APA Thesaurus terms for PsycINFO.
• Typically few limits are applied in systematic review searches and always with justification e.g. English Language as no resources for translation, date limit from date policy introduced.
• Apply any limits as the final part of the search, after searching for and combining concepts.
• Keep your search as consistent across the different databases as possible.
• Save your search history for each database.
• Save your search as an alert to be notified when new material meeting your search criteria is added to the database.
• For reporting purposes, it’s advisable to run and export you search results from all databases on the same day.
• It’s important to allow enough time to develop your final search.You will need to undertake several cycles of revision, making multiple amendments to achieve the best balance of comprehensiveness and precision..
• Use a ‘test set’ of relevant articles you have already located to test the sensitivity of your search strategy in databases where these articles are indexed.
• Truly comprehensive searches will include grey literature and hand searching of reference lists and/or important journals.

Adapted from the Cochrane Handbook, Ch.4: Searching for and selecting studies

Comprehensiveness vs. precision (sensitivity-specificity)

"Searches for systematic reviews aim to be as extensive as possible in order to ensure that as many of the relevant studies as possible are included in the review. It is, however, necessary to strike a balance between striving for comprehensiveness and maintaining relevance when developing a search strategy". (Cochrane handbook, Ch. 4.4.3)

 Increasing the comprehensiveness (sensitivity) of a search will reduce its precision (specificity) and will retrieve more non-relevant articles. 

• Comprehensiveness (sensitivity)  is defined as the number of relevant results identified divided by the total number of relevant results in existence. Sensitive search finds the relevant studies.
• Precision (specificity) is defined as the number of relevant results identified divided by the total number of results identified. A specific
  search avoids locating irrelevant studies. Highly specific searches avoid finding unrelated studies consequently return a lower number of records to examine; however, carry an increased risk of missing relevant studies.

The following approaches can be used to achieve an ideal balance between both concepts:

• If the specificity is low and the results are not what you were looking for, you need to modify your search query, perhaps try other search terms.
• If the sensitivity is low and you do not have enough results or you suspect that you are missing something, you need to expand your search, perhaps include more search terms to better capture all relevant research within the subject.

See the guide from the Univerity of Toronto on how to develop a precise or sensitive search.

Guidelines and standards