The Literature Review: Systematic Reviews

Cochrane - what are systematic reviews? Video

Guides to writing a systematic literature review

Systematic review tools

EndNote - a reference management tool which helps you to save and manage bibliographic references. 

Covidence - an online systematic review program. You can access the free trial version to work out if it's suitable for your project. Otherwise it is free to use for Cochrane authors.

RevMan 5 - the software used for preparing and maintaining Cochrane Reviews.

What are systematic reviews?

"A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making."

Cochrane Library 2017, About Cochrane Reviews, Cochrane Library, viewed 19 October 2017, <http://www.cochranelibrary.com/about/about-cochrane-systematic-reviews.html>

As explained in this guide, a good literature review should be carried out in an organised, systematic way (consistent search terms,documenting your search). However, a well organised literature review does not qualify it as a 'systematic literature review'. A systematic literature review must follow a specific methodology which is explicit, reproducible and transparent. Systematic Review guidelines and handbooks (such as PRISMA Preferred Reporting Items for Systematic Reviews and Meta-AnalysesThe Campbell Collaboration, the  Cochrane handbook for systematic reviews of interventions and the Joanna Briggs Institute reviewer's manual ) will outline a set of discrete steps and processes to follow to satisfy the criteria of a 'systematic review'. The common thread between all of these is that the steps and processes should be identified in advance, in a review protocol. Ideally the review protocol and final review report will be published together for transparency and reproducibility. A typical review protocol will include:

  • Databases to be searched and additional sources (particularly for grey literature)
  • Keywords to be used in the search strategy
  • Screening process or inclusion/exclusion criteria (may be conducted by two or more people)
  • Data to be extracted
  • Synthesis (where the findings of the included studies are integrated to answer the review question)

 

Narrative vs. systematic

The key differences and features of narrative and systematic literature reviews
  Narrative Literature Reviews Systematic Literature Reviews
Focus:

Describe and critically analyse the literature on a specific topic from a theoretical and contextual point of view.

Clear focus on appraising the quality of evidence encountered in the literature, often with a view to informing and improving practice outcomes
Explicitness of search strategy: Often relatively undefined Clearly defined and systematically applied
Comprehensiveness: May offer wide coverage, but does not necessarily aspire to be comprehensive Aims to be as exhaustive as possible
Decisions about inclusion/exclusion of material: Can be selective and piecemeal Clear protocol is devised to guide decision making about what is included and excluded
Potential for bias: Author(s)’ decision-making process is often not explicit so there is scope for bias Collaborative creation of protocol and transparent process tend to minimise potential for bias
Validity and reliability: The lack of transparency about how the review has been carried out and the potential for ad hoc inclusion of material raises questions about the confidence that can be placed in the findings Due to the systematic use of the protocol in all stages of the research, and the transparency with which review decisions are described, findings are likely to be valid and reliable
Key advantages: Narrative reviews can be useful for synthesising ideas, theories and concepts from a broad range of literature The systematic nature of this work means that it is of a high quality and is repeatable
Key disadvantages: Quality can be variable, but in trying to assess the quality of narrative reviews one may be frustrated by a lack of transparency in respect of the review process Although systematic reviews can be good at identifying ‘what’ works, they may not always be the most appropriate tool for identifying ‘why’ something works

Kiteley, R & Stogdon, C 2014, 'What is a literature review?', in R Kiteley & C Stogdon, Literature reviews in social work, SAGE Publications Ltd., London, pp. 5-22. 

 

Before you begin your systematic review

Before you begin your systematic review, it can be helpful to read other reviews related to your topic (and also make sure you are not 're-inventing the wheel'). PROSPERO is an international register of systematic reviews in health and social care, welfare, public health, education, crime, justice, and international development, where there is a health related outcome. 

You can also search a range of subject databases (available at VU Library), such as such as MEDLINE, CINAHL, PsycINFO and limit the search to "systematic review". Also Cochrane Library, which is a database that focuses specifically on indexing evidence-based medicine, systematic reviews, etc. For instructions on how to access Library databases, consult the Searching the Literature section of this guide.

Formulate the research question

The following are frameworks that you can use to help formulate a clear and focused research question, which you can then use to convert into an effective search strategy for your review:

PICO for quantitative studies
P Patient, Population or Problem
I Intervention or Exposure
C Comparison
O Outcome
  ​Lawani, M. A., Valéra, B., Fortier-Brochu, E., Légaré, F., Carmichael, P. H., Côté, L., ... Giguere, A. M. C. (2017). Five shared decision-making tools in 5 months: use of rapid reviews to develop decision boxes for seniors living with dementia and their caregivers. Systematic Reviews, 6(56), 1-12. doi:10.1186/s13643-017-0446-2.

 

 

PICo for qualitative studies
P Population
I Interest
Co Context
 

Rathbone, J., Albarqouni, L., Bakhit, M., Beller, E., Byambasuren, O., Hoffmann, T., … Glasziou, P. (2017). Expediting citation screening using PICo-based title-only screening for identifying studies in scoping searches and rapid reviews. Systematic Reviews, 6(1), 1-7. doi:10.1186/s13643-017-0629-x.

SPIDER for qualitative and mixed methods research studies
S Sample
PI Phenomenon of Interest
D Design
E Evaluation
R Research type
  Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qualitative Health Research, 22(10), 1435–1443. doi:10.1177/1049732312452938.

 

SPICE for qualitative evidence
S Setting
P Perspective
I Intervention
C Comparison
E Evaluation
  Harris, J. L. , Booth, A., Cargo, M., Hannes, K., Harden, A., Flemming, K., ... Noyes, J. (2018). Cochrane Qualitative and Implementation Methods Group guidance series—paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis, Journal of Clinical Epidemiology, 97(4), 39-48. doi:10.1016/j.jclinepi.2017.10.023.

 

The PRISMA Statement

The PRISMA Statement is a great starting point for any systematic literature review as it lays out the steps/tasks involved in an uncomplicated way. The PRISMA Statement consists of a checklist and flow diagram.

If you have used PRISMA in your systematic review, you must cite it. The PRISMA statement has been published in several journal articles, and PRISMA recommend that you cite via one of these, rather than the website directly. For more information on how to cite PRISMA, see here.