Search

Once the research question has been defined, the next step is the complete and unbiased identification of relevant studies, which will require conducting "a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource limits)" (Cochrane handbook, Ch. 4.2.2). 

Key points on searching for systematic or scoping reviews

1. searches must be done in academic or research databases
   • web based searches can also be done but are considered supplementary to database searches
   • search databases one at a time
2. searches should use a systematic line-by-line approach
3. search each concept separately, then combine
4. searches should be exhaustive, use all relevant terms.
5. subject headings and keyword options should be used
   • use the right subject headings for each database
   • subject headings can be hard to identify. Spend some time exploring them.
   • look at relevant papers for their subject headings
6. search strategies should be reported in your review
   • you can save your search strategy within each database.
   • most databases have an option to export the search strategy
7. do not be put off by irrelevant results. Systematic review searches are expected to be very sensitive.
   • it is usual to have to screen a lot of results. Most results will not make it into the final review.
8. do not apply arbitrary limits to reduce the number of results. Instead refine your question to a manageable topic.
9. test your search strategy by seeing if it finds known relevant papers.
10. supplement your database searches with citation searching and grey literature
    • backwards citation searching means checking the references of relevant articles
    • forwards citation searching means looking for articles have cited relevant articles
    • grey literature means non-journal literature such as reports, dissertations, organisational documents

Comprehensiveness vs. precision (sensitivity-specificity)

"Searches for systematic reviews aim to be as extensive as possible ... It is ... necessary to strike a balance between striving for comprehensiveness and maintaining relevance when developing a search strategy". (Cochrane handbook, Ch. 4.4.3)

It is usual to have a lot of irrelevant articles in your search results.

Here is an example of a PRISMA flow chart. 

• Note how few of the papers that were found are included in the final review. This is fairly typical.
• A search that is too precise is likely to miss potential papers.
• A search that is too sensitive may return too many results and become unmanageable.

There is no definitive answer to how sensitive your search should be, or how  many results you screen.

You will have to make your own decision on how sensitive your search is.

• if you results include no or few irrelevant papers, your search is too precise. You are probably missing relevant studies.
• If your results are way too large check your search logic, and check for terms that are too general.

See this guide from the Univerity of Toronto on how to develop a precise or sensitive search.

Guidelines and standards