Chapter II: Planning a Cochrane Review

This is an archived version. For the current version, please go to training.cochrane.org/handbook/current

Miranda Cumpston, Ella Flemyng

Key Points:

  • In planning a Cochrane Review, there are additional considerations for author teams over and above what is required for other systematic reviews.
  • Proposals for new Cochrane Reviews are submitted by author teams to Cochrane before commencing the review. 
  • Authors should establish a team with appropriate experience and expertise in both the topic of the review and systematic review methods.
  • All named authors should meet the ICMJE criteria for authorship of a publication.
  • Authors are advised to give due consideration to the resources needed to undertake a Cochrane Review, of which the most important resource is time.
  • Cochrane authors are expected to follow Cochrane policy for avoiding and declaring potential conflicts of interest.

This chapter should be cited as: Cumpston M, Chandler J. Chapter II: Planning a Cochrane Review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook.

II.1 Undertaking a Cochrane Review

The process of preparing and publishing a Cochrane Review is different from that of other journals, in that there is a closer working relationship between Cochrane and the review authors. In addition, Cochrane Reviews follow a highly structured format so that they can be published within the Cochrane Database of Systematic Reviews, and their preparation follows a structured process. This chapter overviews the procedures and practicalities of starting a Cochrane Review. It aims to supplement the more general and methodological guidance available in Chapter 1 and Chapter 2.

II.1.1 Working with Cochrane

Some author teams working on Cochrane Reviews may work with one of the Cochrane Review Groups (CRGs) or Thematic Groups, which focus on a specific area of health.

All authors, whether working with a Cochrane group or independently, should follow pre-specified minimum expectations (See Chapter 1, Section 1.4). In some cases, Cochrane can provide specific services to support the review, such as translation, statistical advice, and expert information specialists to assist in developing search strategies and to conduct searches of electronic databases. For the latest details of support available to review authors please contact the Cochrane Support team (support@cochrane.org).

II.1.2 Proposing a new Cochrane Review

Instructions for authors on proposing new Cochrane Reviews are available at the Cochrane Library (https://www.cochranelibrary.com/about/author-information). Proposals for new reviews are assessed by topic experts to ensure that the scope of the proposed review is appropriate and to avoid duplication between Cochrane Reviews. Before proposing a new review, authors are advised to conduct an advanced search on the Cochrane Library to ensure the review question will provide new evidence.

If the review proposal is accepted, review authors will be asked to develop and submit a protocol to the Central Editorial Service (see Section II.1.4 and Chapter 1, Section 1.5). Cochrane's editorial processes, including peer review, are similar to those of other journals. Cochrane may reject manuscripts that are not of a sufficient standard for publication.

Cochrane Reviews may be updated based on need (see Chapter IV). When proposing a new review, author teams should be aware of the commitment needed to prepare a review, and to the responsibility of responding to post-publication feedback (e.g. comments from readers) and to updating as required.

II.1.3 Structure of a review title

The title should state succinctly the intervention(s) to be reviewed and the problem at which the intervention is directed. In some cases, the title may also state a specific population or setting. The titles of Cochrane Reviews on the effects of interventions should follow the structure shown in Table II.1.a.

Table II.1.a Structure of titles for Cochrane Reviews of interventions

Scenario

Structure

Example

Basic structure

[Intervention] for [health problem]

Antibiotics for acute bronchitis

Comparing two active interventions

[Intervention A] versus [intervention B] for [health problem]

Immediate versus delayed treatment for cervical intraepithelial neoplasia

Type of people being studied or location of intervention mentioned explicitly

[Intervention] for [health problem] in [participant group/location]

Inhaled nitric oxide for respiratory failure in preterm infants

Not specifying a particular health problem, or if the intervention intends to influence a variety of problems

[Intervention] in OR for [participant group/location]

Planned hospital birth versus planned home birth

Prophylactic synthetic surfactant in preterm infants

Sometimes it is necessary to specify that the intervention is for preventing, treating, or preventing and treating the health problem(s).

[Intervention] for preventing AND/OR treating [health problem]

Pool fencing for preventing drowning in children

Amodiaquine for treating malaria

Vitamin C for preventing and treating the common cold

II.1.4 Cochrane protocols

Preparing a Cochrane Review is complex and involves many judgements. All Cochrane Reviews must have a written protocol, specifying in advance the scope and methods to be used by the review, to assist in planning and reduce the risk of bias in the review process (see Chapter 1, Section 1.5 for a detailed discussion of the rationale for protocols). Cochrane provides a highly structured format for both its protocols and reviews to guide authors on the information they should report (see Box II.1.a).

As with any study, if the methods proposed in the protocol are changed during the course of conducting the review, these changes should be documented and reported. In a Cochrane Review, changes should be described in the Methods section of the completed review. Sensitivity analyses (see Chapter 10, Section 10.14) exploring the impact of deviations from the protocol should be undertaken, when possible.

Cochrane recommends that all protocols be registered on the PROSPERO register of systematic reviews (https://www.crd.york.ac.uk/prospero), enhancing transparency and accountability and reducing duplication of effort.

Box II.1.a Sections of a protocol for a Cochrane Review

Title

Protocol

  • Background
  • Objectives
  • Methods
    • Criteria for selecting studies for this review
    • Outcome measures
    • Search methods for identification of studies
    • Data collection and analysis
    • Consumer involvement

Other supplementary materials

Information

  • Authors
  • Acknowledgements
  • Contributions of authors
  • Declarations of interest
  • Sources of support
  • Registration and protocol
  • Data, code and other materials

References

Figures and Tables

II.2 The author team

II.2.1 Setting up a review team

Cochrane Reviews should be undertaken by more than one person. In putting together a team, authors should consider the need for clinical and methodological expertise for the review, as well as the perspectives of stakeholders. Cochrane author teams are encouraged to seek and incorporate the views of users, including consumers, clinicians and those from varying regions and settings to develop protocols and reviews. Author teams for reviews relevant to particular settings (e.g. neglected tropical diseases) should involve contributors experienced in those settings.

First-time review authors are encouraged to work with others who are experienced in the process of conducting Cochrane Reviews and to make use of Cochrane training and guidance resources (see Section II.3.3). See Chapter 1, Section 1.3, on the experience required of the review team and guidance on involving consumers and other stakeholders.

II.2.2 Criteria for authorship

Cochrane follows the International Committee of Medical Journal Editors (ICMJE) criteria for authorship (ICMJE 2018). When deciding who should appear in the byline of a Cochrane Review, only those individuals who have made a substantial contribution to the review (and therefore are listed as authors) can be listed. People who have helped in other ways can be listed in the Acknowledgements section with written permission. The specific contributions of each author should be listed in the ‘Contributions of authors’ section. Including authors who do not meet these criteria, or failing to name significant contributors as authors, is not appropriate. Methodological specialists such as statisticians and information specialists should be included as authors where they meet the ICMJE criteria, particularly where they have been substantively involved in the design and execution of the review’s methods.

For further information, see the Cochrane Library editorial policy on authorship.

II.3 Resources

II.3.1 Identifying resources and support

The main resource required by authors is their own time. Many authors will contribute their time free of charge because it will be viewed as part of their existing research or their efforts to keep up to date with an area of interest.

The amount of time required will vary, depending on the topic of the review, the number of included studies, the methods used (e.g. the extent of efforts to obtain unpublished information), and the experience of the authors. The workload associated with undertaking a review is thus very variable. However, consideration of the tasks involved and the time required for each of these might help authors to estimate the amount of time that will be required. These tasks include training, meetings, protocol development, searching for studies, assessing citations and full-text reports of studies for eligibility, assessing the risk of bias of included studies, collecting data, pursuing missing data and unpublished studies, analyzing the data, interpreting the results, writing the review, and keeping the review up to date.

A time chart with target dates for accomplishing key tasks can help with scheduling the time needed to complete a review.

Resources that might be required for these tasks, in addition to the authors’ time, include:

  • searching assistance from an information specialist or librarian is strongly recommended, although all authors share responsibility for planning and executing elements of the search (see Chapter 4, Section 4.2.1);
  • library resources, including access to electronic databases for searching, subscriptions to journals to access papers and interlibrary loans;
  • statistical support for synthesizing (if appropriate) the results of the included studies;
  • equipment (e.g. computing hardware and software);
  • supplies and services (internet connection, printing, telephone charges);
  • office space for staff; and
  • travel funds to attend author or stakeholder meetings or present the results of the review (e.g. at a conference).

II.3.2 Funding and conflict of interest

Many organizations currently provide funding for systematic reviews. These include research funding agencies, organizations that provide or fund healthcare services, those responsible for health technology assessment and those involved in the development of clinical practice guidelines. Author teams may wish to identify and seek funding from such organizations operating in their region or field of health care.

Conflict of interest in the funding and authorship of research gives rise to serious issues, which Cochrane takes very seriously (Bero 2018, Tovey et al 2018). Under Cochrane’s policy on conflict of interest, a Cochrane Review cannot be funded or conducted by commercial sponsors or commercial sources with a real or potential vested interest in the findings of a specific review.

All Cochrane authors will be required to complete a declaration of interest form on submission of a draft protocol or review Individuals who are employed by a company that has a real or potential financial interest in the outcome of the Cochrane Review (including, but not limited to, drug companies or medical device manufacturers), or who hold or have applied for a patent related to the Cochrane Review, are prohibited from being Cochrane Review authors. Any other possible conflicts of interest will be reviewed by Cochrane’s Funding Arbiter. If any conflicts change over time authors should contact Cochrane editorial staff with full details, and ask to redisclose their interests in relation to the review in question.

Further discussion of the issues around conflict of interest in research, and in particular how they apply to the studies included within a review, is in Chapter 7, Section 7.8.

II.3.3 Learning and support services for authors

While many Cochrane authors have a great deal of knowledge and experience in conducting systematic reviews, others are new to the process or to Cochrane methods, or wish to expand their skills. To assist those contributing to Cochrane Reviews in gaining the knowledge, skills and support they need to meet the conduct and reporting standards, Cochrane provides a range of support services to facilitate learning.

In addition to written guidance, Cochrane provides a range of online training resources, as well as face-to-face events in locations around the world. Details of current resources and events are available at https://training.cochrane.org.

II.3.4 Software resources

Cochrane Reviews are supported by an ecosystem of software tools to assist with different aspects of the review process (Elliott et al 2014). The primary piece of software is Cochrane’s own Review Manager (RevMan). RevMan is a mandatory tool and free to use for authors preparing a Cochrane Review. The software is developed through a continuing process of consultation with its users and Cochrane methodologists to support good practice for Cochrane Reviews, and provides structured text drafting, standard tables and reference formats, meta-analysis, online help and error checking mechanisms. RevMan is also used to submit review manuscripts for editorial assessment and publication.

Authors may wish to consider other software resources to assist them with different aspects of the review process. A register of tools designed for use in systematic reviews is maintained in the Systematic Review Toolbox. Developments take place rapidly in tools aiming to increase efficiency and reduce the time required to complete a review or update, with an increasing number of tools able to automate (or semi-automate) elements of the process (O'Connor et al 2020, Scott et al 2021, Khalil et al 2022, Marshall et al 2023, Scott et al 2023).

Software to manage references is usually required during the searching process, and systematic review tools including Covidence and EPPI-Reviewer can be used to assist in the process of screening search results. Advanced processes such as text mining and machine learning to assist in selection are increasingly common, and web-based processes such as crowdsourcing are also available (see Chapter 4, Section 4.6). Cochrane is able to facilitate access to some valuable resources. The Cochrane Central Register of Controlled Trials is populated using a process that incorporates a centralized search for randomized trials, machine learning technology and Cochrane Crowd, a crowd-based system to screen search results. The Screen4Me service, available to authors of Cochrane Reviews, uses the same tools to screen and identify relevant studies for inclusion in reviews (for example, by filtering to identify randomized trials), particularly for reviews with complex searches and large record sets, and has been demonstrated to produce highly accurate results (Noel-Storr et al 2021a, Noel-Storr et al 2021b, Thomas et al 2021). Authors should contact their Cochrane Community Support (support@cochrane.org) for further information.

Software such as Covidence and EPPI-Reviewer can also be used for data collection and other elements of the review process (see Chapter 5, Section 5.5), and both are recommended and freely available to Cochrane authors. Statistical software such as Stata, R and Comprehensive Meta-Analysis may be used to perform statistical analyses that are not available in RevMan. Later in the review process, GRADEpro GDT can be used to create ‘Summary of findings’ tables (see Chapter 14, Section 14.1.4).

The choice of software tools may depend on the authors’ preference, the availability of a stable internet connection, the cost and the extent to which novel methods have been validated for accuracy. Authors are advised to seek methodological advice before incorporating new technologies into their reviews.

II.4 Chapter information

Authors: Miranda Cumpston and Ella Flemyng

Acknowledgements: We thank previous chapter authors and editors Ginny Brunton, Jacqueline Chandler, Sally Green, Julian Higgins, Nicki Jackson, Monica Kjeldstrøm, Harriet MacLehose and Sandy Oliver, as well as contributing authors to earlier editions of the Handbook. We thank Toby Lasserson, Chris Cates, Carol Lefebvre, Philippa Middleton, Denise O’Connor and Lesley Stewart for their helpful comments on current and previous versions.

II.5 References

Bero L. More journals should have conflict of interest policies as strict as Cochrane: BMJ Opinion; 2018. https://blogs.bmj.com/bmj/2018/11/12/lisa-bero-more-journals-should-have-conflict-of-interest-policies-as-strict-as-cochrane/.

Elliott J, Sim I, Thomas J, Owens N, Dooley G, Riis J, Wallace B, Thomas J, Noel-Storr A, Rada G, Struthers C, Howe T, MacLehose H, Brandt L, Kunnamo I, Mavergames C. #CochraneTech: technology and the future of systematic reviews. Cochrane Database of Systematic Reviews 2014; 9: ED000091.

ICMJE. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals: International Committee of Medical Journal Editors; 2018. http://www.icmje.org/recommendations/.

Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. Journal of Clinical Epidemiology 2022; 144: 22-42.

Marshall IJ, Trikalinos TA, Soboczenski F, Yun HS, Kell G, Marshall R, Wallace BC. In a pilot study, automated real-time systematic review updates were feasible, accurate, and work-saving. Journal of Clinical Epidemiology 2023; 153: 26-33.

Noel-Storr A, Dooley G, Affengruber L, Gartlehner G. Citation screening using crowdsourcing and machine learning produced accurate results: Evaluation of Cochrane's modified Screen4Me service. Journal of Clinical Epidemiology 2021a; 130: 23-31.

Noel-Storr A, Dooley G, Elliott J, Steele E, Shemilt I, Mavergames C, Wisniewski S, McDonald S, Murano M, Glanville J, Foxlee R, Beecher D, Ware J, Thomas J. An evaluation of Cochrane Crowd found that crowdsourcing produced accurate results in identifying randomized trials. Journal of Clinical Epidemiology 2021b; 133: 130-139.

O'Connor AM, Glasziou P, Taylor M, Thomas J, Spijker R, Wolfe MS. A focus on cross-purpose tools, automated recognition of study design in multiple disciplines, and evaluation of automation tools: a summary of significant discussions at the fourth meeting of the International Collaboration for Automation of Systematic Reviews (ICASR). Systematic Reviews 2020; 9: 100.

Scott AM, Forbes C, Clark J, Carter M, Glasziou P, Munn Z. Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. Journal of Clinical Epidemiology 2021; 138: 80-94.

Scott AM, Glasziou P, Clark J. We extended the 2-week systematic review (2weekSR) methodology to larger, more complex systematic reviews: A case series. J Clin Epidemiol 2023; 157: 112-119.

Thomas J, McDonald S, Noel-Storr A, Shemilt I, Elliott J, Mavergames C, Marshall IJ. Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane Reviews. Journal of Clinical Epidemiology 2021; 133: 140-151.

Tovey D, Macbeth F, Webster A, Foxlee R. Cochrane's plans to update its COI policy: truth and fiction. Cochrane Database of Systematic Reviews 2018; 11: ED000131.