• Interesting
  • Scholarships
  • UGC-CARE Journals
  • iLovePhD Web Stories

How to Write a Best Review Paper to Get More Citation

Review Paper Writing Guide

Dr. Sowndarya Somasundaram

How to write review paper

Table of contents

What is a review paper, difference between a review paper and a research paper.

  • 6 Types of Review Papers

Purpose of Review Paper

Criteria for good review paper, step-by-step systematic procedure to write a review paper, title, abstract, keywords, introduction, various topics to discuss the critical issues, conclusion and future perspectives, acknowledgment       .

Are you new to academia? Do you want to learn how to write a good review paper to get in-depth knowledge about your domain? you are at the right place. In this article, you will learn how to write the best review paper in a step-by-step systematic procedure with a sample review article format to get more Citation.

A review paper, or a literature review , is a thorough, analytical examination of previously published literature. It also provides an overview of current research works on a particular topic in chronological order.

  • The main objective of writing a review paper is to evaluate the existing data or results, which can be done through analysis, modeling, classification, comparison, and summary.  
  • Review papers can help to identify the research gaps, to explore potential areas in a particular field.
  • It helps to come out with new conclusions from already published works.
  • Any scholar or researcher or scientist who wants to carry out research on a specific theme, first read the review articles relevant to that research area to understand the research gap for arriving at the problem statement.
  • Writing a review article provides clarity, novelty, and contribution to the area of research and it demands a great level of in-depth understanding of the subject and a well-structured arrangement of discussions and arguments.
  • Some journals publish only review papers, and they do not accept research articles. It is important to check the journal submission guidelines.

The difference between a review paper and a research paper is presented below.

6 T ypes of Review Papers

The review papers are classified in to six main categories based on the theme and it is presented in the figure below.

Types of Review Papers

The purpose of a review paper is to assess a particular research question, theoretical or practical approach which provides readers with in-depth knowledge and state-of-the-art understanding of the research area.

The purpose of the review paper can vary based on their specific type and research needs.

  • Provide a unified, collective overview of the current state of knowledge on a specific research topic and provide an inclusive foundation on a research theme.
  • Identify ambiguity, and contradictions in existing results or data.
  • Highlight the existing methodological approaches, research techniques, and unique perceptions.
  • Develop theoretical outlines to resolve and work on published research.
  • Discuss research gaps and future perspectives.

A good review paper needs to achieve three important criteria. ( Palmatier et al 2017 ).

  • First, the area of research should be suitable for writing a review paper so that the author finds sufficient published literature.
  • The review paper should be written with suitable literature, detailed discussion, sufficient data/results to support the interpretation, and persuasive language style.
  • A completed review paper should provide substantial new innovative ideas to the readers based on the comparison of published works.

Review papers are widely read by many researchers and it helps to get more citations for author. So, it is important to learn how to write a review paper and find a journal to publish .

Time needed:  20 days and 7 hours

The systematic procedural steps to write the best review paper are as follows:

Select a suitable area in your research field formulate clear objectives, and prepare the specific research hypotheses that are to be explored.

Designing your research work is an important step for any researcher. Based on the objectives, develop a clear methodology or protocol to review a review paper.

Thorough analysis and understanding of different published works help the author to identify suitable and relevant data/results that will be used to write the paper.

The degree of analysis to evaluate the collected data varies by extensive review. The examination of trends, patterns, ideas, comparisons, and relationships in the study provides deeper knowledge on that area of research .

Interpretation of results is very important for a good review paper. The author should present the discussion systematically without any ambiguity. The results can be presented in descriptive form, tables, and figures. The new insights should have an in-depth discussion of the topic in line with fundamentals. Finally, the author is expected to present the limitations of the existing study with future perspectives.

Steps to Write a Review Paper

Sample Review Article Format

Write an effective and suitable title, abstract, and keywords relevant to your review paper. This will maximize the visibility of your paper online for the readers to find your work. Your title and abstract should be clear, concise, appropriate, and informative.

Present a detailed introduction to your research which is published in chronological order in your own words. Don’t summarize the published literature. The introduction should encourage the readers to read your paper.

Make sure you present a critical discussion, not a descriptive summary of the topic. If there is contradictory research in your area of research, verify to include an element of debate and present both sides of the argument. A good review paper can resolve the conflict between contradictory works.

The written review paper should achieve your objectives. Hence, the review paper should leave the reader with a clear understanding of the following questions:

What they can understand from the review paper?

What still remains a requirement of further investigation in the research area?

This can include making suggestions for future scope on the theme as part of your conclusion.

The authors can submit a brief acknowledgment of any financial, instrumentation, and academic support received about research work.

Citing references at appropriate places in the article is necessary and important to avoid plagiarism. Each journal has its referencing style. Therefore, the references need to be listed at the end of the manuscript. The number of references in the review paper is usually higher than in a research paper .

I hope this article will give you a clear idea of how to write a review paper. Please give your valuable comments.

iLovePhD Learn to Research Find Journals to Publish

research for review paper

  • Academic Citations
  • Academic Writing
  • Citation Boost
  • citation management
  • Citation Strategies
  • Citation Techniques
  • Citations Tips
  • effective writing
  • Literature Review
  • Paper Impact
  • Paper Writing
  • Research Paper
  • research strategies
  • review article
  • review paper
  • Sample Review Paper
  • scholarly impact
  • Scholarly writing
  • Scientific Review
  • Types of Review Paper

Dr. Sowndarya Somasundaram

List of UGC Care Journals Discontinued from Jan 2024

7 types of journal peer review process, 100 work motivational quotes.

Nice thank you for your clarification

How to write a review paper on the relevant of science to education

This blog is very informative. Is it true that an increase in the number of citations improves the quality and impact of a review paper?

Thanks for your ideas. Really helpful

it was very well information

LEAVE A REPLY Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Notify me of follow-up comments by email.

Notify me of new posts by email.

Email Subscription

ilovephd logo

iLovePhD is a research education website to know updated research-related information. It helps researchers to find top journals for publishing research articles and get an easy manual for research tools. The main aim of this website is to help Ph.D. scholars who are working in various domains to get more valuable ideas to carry out their research. Learn the current groundbreaking research activities around the world, love the process of getting a Ph.D.

WhatsApp Channel

Join iLovePhD WhatsApp Channel Now!

Contact us: [email protected]

Copyright © 2019-2024 - iLovePhD

  • Artificial intelligence

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

  • PLOS Biology
  • PLOS Climate
  • PLOS Complex Systems
  • PLOS Computational Biology
  • PLOS Digital Health
  • PLOS Genetics
  • PLOS Global Public Health
  • PLOS Medicine
  • PLOS Mental Health
  • PLOS Neglected Tropical Diseases
  • PLOS Pathogens
  • PLOS Sustainability and Transformation
  • PLOS Collections

How to Write a Peer Review

research for review paper

When you write a peer review for a manuscript, what should you include in your comments? What should you leave out? And how should the review be formatted?

This guide provides quick tips for writing and organizing your reviewer report.

Review Outline

Use an outline for your reviewer report so it’s easy for the editors and author to follow. This will also help you keep your comments organized.

Think about structuring your review like an inverted pyramid. Put the most important information at the top, followed by details and examples in the center, and any additional points at the very bottom.

research for review paper

Here’s how your outline might look:

1. Summary of the research and your overall impression

In your own words, summarize what the manuscript claims to report. This shows the editor how you interpreted the manuscript and will highlight any major differences in perspective between you and the other reviewers. Give an overview of the manuscript’s strengths and weaknesses. Think about this as your “take-home” message for the editors. End this section with your recommended course of action.

2. Discussion of specific areas for improvement

It’s helpful to divide this section into two parts: one for major issues and one for minor issues. Within each section, you can talk about the biggest issues first or go systematically figure-by-figure or claim-by-claim. Number each item so that your points are easy to follow (this will also make it easier for the authors to respond to each point). Refer to specific lines, pages, sections, or figure and table numbers so the authors (and editors) know exactly what you’re talking about.

Major vs. minor issues

What’s the difference between a major and minor issue? Major issues should consist of the essential points the authors need to address before the manuscript can proceed. Make sure you focus on what is  fundamental for the current study . In other words, it’s not helpful to recommend additional work that would be considered the “next step” in the study. Minor issues are still important but typically will not affect the overall conclusions of the manuscript. Here are some examples of what would might go in the “minor” category:

  • Missing references (but depending on what is missing, this could also be a major issue)
  • Technical clarifications (e.g., the authors should clarify how a reagent works)
  • Data presentation (e.g., the authors should present p-values differently)
  • Typos, spelling, grammar, and phrasing issues

3. Any other points

Confidential comments for the editors.

Some journals have a space for reviewers to enter confidential comments about the manuscript. Use this space to mention concerns about the submission that you’d want the editors to consider before sharing your feedback with the authors, such as concerns about ethical guidelines or language quality. Any serious issues should be raised directly and immediately with the journal as well.

This section is also where you will disclose any potentially competing interests, and mention whether you’re willing to look at a revised version of the manuscript.

Do not use this space to critique the manuscript, since comments entered here will not be passed along to the authors.  If you’re not sure what should go in the confidential comments, read the reviewer instructions or check with the journal first before submitting your review. If you are reviewing for a journal that does not offer a space for confidential comments, consider writing to the editorial office directly with your concerns.

Get this outline in a template

Giving Feedback

Giving feedback is hard. Giving effective feedback can be even more challenging. Remember that your ultimate goal is to discuss what the authors would need to do in order to qualify for publication. The point is not to nitpick every piece of the manuscript. Your focus should be on providing constructive and critical feedback that the authors can use to improve their study.

If you’ve ever had your own work reviewed, you already know that it’s not always easy to receive feedback. Follow the golden rule: Write the type of review you’d want to receive if you were the author. Even if you decide not to identify yourself in the review, you should write comments that you would be comfortable signing your name to.

In your comments, use phrases like “ the authors’ discussion of X” instead of “ your discussion of X .” This will depersonalize the feedback and keep the focus on the manuscript instead of the authors.

General guidelines for effective feedback

research for review paper

  • Justify your recommendation with concrete evidence and specific examples.
  • Be specific so the authors know what they need to do to improve.
  • Be thorough. This might be the only time you read the manuscript.
  • Be professional and respectful. The authors will be reading these comments too.
  • Remember to say what you liked about the manuscript!

research for review paper

Don’t

  • Recommend additional experiments or  unnecessary elements that are out of scope for the study or for the journal criteria.
  • Tell the authors exactly how to revise their manuscript—you don’t need to do their work for them.
  • Use the review to promote your own research or hypotheses.
  • Focus on typos and grammar. If the manuscript needs significant editing for language and writing quality, just mention this in your comments.
  • Submit your review without proofreading it and checking everything one more time.

Before and After: Sample Reviewer Comments

Keeping in mind the guidelines above, how do you put your thoughts into words? Here are some sample “before” and “after” reviewer comments

✗ Before

“The authors appear to have no idea what they are talking about. I don’t think they have read any of the literature on this topic.”

✓ After

“The study fails to address how the findings relate to previous research in this area. The authors should rewrite their Introduction and Discussion to reference the related literature, especially recently published work such as Darwin et al.”

“The writing is so bad, it is practically unreadable. I could barely bring myself to finish it.”

“While the study appears to be sound, the language is unclear, making it difficult to follow. I advise the authors work with a writing coach or copyeditor to improve the flow and readability of the text.”

“It’s obvious that this type of experiment should have been included. I have no idea why the authors didn’t use it. This is a big mistake.”

“The authors are off to a good start, however, this study requires additional experiments, particularly [type of experiment]. Alternatively, the authors should include more information that clarifies and justifies their choice of methods.”

Suggested Language for Tricky Situations

You might find yourself in a situation where you’re not sure how to explain the problem or provide feedback in a constructive and respectful way. Here is some suggested language for common issues you might experience.

What you think : The manuscript is fatally flawed. What you could say: “The study does not appear to be sound” or “the authors have missed something crucial”.

What you think : You don’t completely understand the manuscript. What you could say : “The authors should clarify the following sections to avoid confusion…”

What you think : The technical details don’t make sense. What you could say : “The technical details should be expanded and clarified to ensure that readers understand exactly what the researchers studied.”

What you think: The writing is terrible. What you could say : “The authors should revise the language to improve readability.”

What you think : The authors have over-interpreted the findings. What you could say : “The authors aim to demonstrate [XYZ], however, the data does not fully support this conclusion. Specifically…”

What does a good review look like?

Check out the peer review examples at F1000 Research to see how other reviewers write up their reports and give constructive feedback to authors.

Time to Submit the Review!

Be sure you turn in your report on time. Need an extension? Tell the journal so that they know what to expect. If you need a lot of extra time, the journal might need to contact other reviewers or notify the author about the delay.

Tip: Building a relationship with an editor

You’ll be more likely to be asked to review again if you provide high-quality feedback and if you turn in the review on time. Especially if it’s your first review for a journal, it’s important to show that you are reliable. Prove yourself once and you’ll get asked to review again!

  • Getting started as a reviewer
  • Responding to an invitation
  • Reading a manuscript
  • Writing a peer review

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • CAREER FEATURE
  • 04 December 2020
  • Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

research for review paper

  • Research management

What’s the sign for ‘centrifuge’? How we added scientific terms to Indian Sign Language

What’s the sign for ‘centrifuge’? How we added scientific terms to Indian Sign Language

Career Feature 23 FEB 24

How to boost your research: take a sabbatical in policy

How to boost your research: take a sabbatical in policy

World View 21 FEB 24

Structural biology for researchers with low vision

Structural biology for researchers with low vision

Career Column 19 FEB 24

China conducts first nationwide review of retractions and research misconduct

China conducts first nationwide review of retractions and research misconduct

News 12 FEB 24

Could roving researchers help address the challenge of taking parental leave?

Could roving researchers help address the challenge of taking parental leave?

Career Feature 07 FEB 24

Best practice for LGBTQ+ data collection by STEM organizations

Correspondence 06 FEB 24

Open-access publishing: citation advantage is unproven

Correspondence 13 FEB 24

How journals are fighting back against a wave of questionable images

How journals are fighting back against a wave of questionable images

News Explainer 12 FEB 24

Scientific Officer

ABOUT US The Human Frontier Science Program Organization (HFSPO) is a unique organization, supporting international collaboration to undertake inno...

Strasbourg-Ville, Bas-Rhin (FR)

HUMAN FRONTIER SCIENCE PROGRAM ORGANIZATION

research for review paper

Tenure Track Assistant Professor towards Associate Professor in the field of biomedical sciences

UNIL is a leading international teaching and research institution, with over 5,000 employees and 17,000 students split between its Dorigny campus, ...

Lausanne, Canton of Vaud (CH)

University of Lausanne (UNIL)

research for review paper

Faculty Positions at City University of Hong Kong (Dongguan)

CityU (Dongguan) warmly invites individuals from diverse backgrounds to apply for various faculty positions available at the levels of Professor...

Dongguan, Guangdong, China

City University of Hong Kong (Dongguan)

research for review paper

Principal Clinical Investigator in Immuno-Oncology

A new wave of Immunotherapeutics drugs is coming and its development requires specific expertise in the field of clinical research, clinical immuno...

Villejuif (Ville), L'Haÿ-les-Roses (FR)

GUSTAVE ROUSSY

research for review paper

Recruitment of Global Talent at the Institute of Zoology, Chinese Academy of Sciences (IOZ, CAS)

The Institute of Zoology (IOZ), Chinese Academy of Sciences (CAS), is seeking global talents around the world.

Beijing, China

Institute of Zoology, Chinese Academy of Sciences (IOZ, CAS)

research for review paper

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Review articles: purpose, process, and structure

  • Published: 02 October 2017
  • Volume 46 , pages 1–5, ( 2018 )

Cite this article

  • Robert W. Palmatier 1 ,
  • Mark B. Houston 2 &
  • John Hulland 3  

224k Accesses

420 Citations

62 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

Many research disciplines feature high-impact journals that are dedicated outlets for review papers (or review–conceptual combinations) (e.g., Academy of Management Review , Psychology Bulletin , Medicinal Research Reviews ). The rationale for such outlets is the premise that research integration and synthesis provides an important, and possibly even a required, step in the scientific process. Review papers tend to include both quantitative (i.e., meta-analytic, systematic reviews) and narrative or more qualitative components; together, they provide platforms for new conceptual frameworks, reveal inconsistencies in the extant body of research, synthesize diverse results, and generally give other scholars a “state-of-the-art” snapshot of a domain, often written by topic experts (Bem 1995 ). Many premier marketing journals publish meta-analytic review papers too, though authors often must overcome reviewers’ concerns that their contributions are limited due to the absence of “new data.” Furthermore, relatively few non-meta-analysis review papers appear in marketing journals, probably due to researchers’ perceptions that such papers have limited publication opportunities or their beliefs that the field lacks a research tradition or “respect” for such papers. In many cases, an editor must provide strong support to help such review papers navigate the review process. Yet, once published, such papers tend to be widely cited, suggesting that members of the field find them useful (see Bettencourt and Houston 2001 ).

In this editorial, we seek to address three topics relevant to review papers. First, we outline a case for their importance to the scientific process, by describing the purpose of review papers . Second, we detail the review paper editorial initiative conducted over the past two years by the Journal of the Academy of Marketing Science ( JAMS ), focused on increasing the prevalence of review papers. Third, we describe a process and structure for systematic ( i.e. , non-meta-analytic) review papers , referring to Grewal et al. ( 2018 ) insights into parallel meta-analytic (effects estimation) review papers. (For some strong recent examples of marketing-related meta-analyses, see Knoll and Matthes 2017 ; Verma et al. 2016 ).

Purpose of review papers

In their most general form, review papers “are critical evaluations of material that has already been published,” some that include quantitative effects estimation (i.e., meta-analyses) and some that do not (i.e., systematic reviews) (Bem 1995 , p. 172). They carefully identify and synthesize relevant literature to evaluate a specific research question, substantive domain, theoretical approach, or methodology and thereby provide readers with a state-of-the-art understanding of the research topic. Many of these benefits are highlighted in Hanssens’ ( 2018 ) paper titled “The Value of Empirical Generalizations in Marketing,” published in this same issue of JAMS.

The purpose of and contributions associated with review papers can vary depending on their specific type and research question, but in general, they aim to

Resolve definitional ambiguities and outline the scope of the topic.

Provide an integrated, synthesized overview of the current state of knowledge.

Identify inconsistencies in prior results and potential explanations (e.g., moderators, mediators, measures, approaches).

Evaluate existing methodological approaches and unique insights.

Develop conceptual frameworks to reconcile and extend past research.

Describe research insights, existing gaps, and future research directions.

Not every review paper can offer all of these benefits, but this list represents their key contributions. To provide a sufficient contribution, a review paper needs to achieve three key standards. First, the research domain needs to be well suited for a review paper, such that a sufficient body of past research exists to make the integration and synthesis valuable—especially if extant research reveals theoretical inconsistences or heterogeneity in its effects. Second, the review paper must be well executed, with an appropriate literature collection and analysis techniques, sufficient breadth and depth of literature coverage, and a compelling writing style. Third, the manuscript must offer significant new insights based on its systematic comparison of multiple studies, rather than simply a “book report” that describes past research. This third, most critical standard is often the most difficult, especially for authors who have not “lived” with the research domain for many years, because achieving it requires drawing some non-obvious connections and insights from multiple studies and their many different aspects (e.g., context, method, measures). Typically, after the “review” portion of the paper has been completed, the authors must spend many more months identifying the connections to uncover incremental insights, each of which takes time to detail and explicate.

The increasing methodological rigor and technical sophistication of many marketing studies also means that they often focus on smaller problems with fewer constructs. By synthesizing these piecemeal findings, reconciling conflicting evidence, and drawing a “big picture,” meta-analyses and systematic review papers become indispensable to our comprehensive understanding of a phenomenon, among both academic and practitioner communities. Thus, good review papers provide a solid platform for future research, in the reviewed domain but also in other areas, in that researchers can use a good review paper to learn about and extend key insights to new areas.

This domain extension, outside of the core area being reviewed, is one of the key benefits of review papers that often gets overlooked. Yet it also is becoming ever more important with the expanding breadth of marketing (e.g., econometric modeling, finance, strategic management, applied psychology, sociology) and the increasing velocity in the accumulation of marketing knowledge (e.g., digital marketing, social media, big data). Against this backdrop, systematic review papers and meta-analyses help academics and interested managers keep track of research findings that fall outside their main area of specialization.

JAMS’ review paper editorial initiative

With a strong belief in the importance of review papers, the editorial team of JAMS has purposely sought out leading scholars to provide substantive review papers, both meta-analysis and systematic, for publication in JAMS . Many of the scholars approached have voiced concerns about the risk of such endeavors, due to the lack of alternative outlets for these types of papers. Therefore, we have instituted a unique process, in which the authors develop a detailed outline of their paper, key tables and figures, and a description of their literature review process. On the basis of this outline, we grant assurances that the contribution hurdle will not be an issue for publication in JAMS , as long as the authors execute the proposed outline as written. Each paper still goes through the normal review process and must meet all publication quality standards, of course. In many cases, an Area Editor takes an active role to help ensure that each paper provides sufficient insights, as required for a high-quality review paper. This process gives the author team confidence to invest effort in the process. An analysis of the marketing journals in the Financial Times (FT 50) journal list for the past five years (2012–2016) shows that JAMS has become the most common outlet for these papers, publishing 31% of all review papers that appeared in the top six marketing journals.

As a next step in positioning JAMS as a receptive marketing outlet for review papers, we are conducting a Thought Leaders Conference on Generalizations in Marketing: Systematic Reviews and Meta-Analyses , with a corresponding special issue (see www.springer.com/jams ). We will continue our process of seeking out review papers as an editorial strategy in areas that could be advanced by the integration and synthesis of extant research. We expect that, ultimately, such efforts will become unnecessary, as authors initiate review papers on topics of their own choosing to submit them to JAMS . In the past two years, JAMS already has increased the number of papers it publishes annually, from just over 40 to around 60 papers per year; this growth has provided “space” for 8–10 review papers per year, reflecting our editorial target.

Consistent with JAMS ’ overall focus on managerially relevant and strategy-focused topics, all review papers should reflect this emphasis. For example, the domains, theories, and methods reviewed need to have some application to past or emerging managerial research. A good rule of thumb is that the substantive domain, theory, or method should attract the attention of readers of JAMS .

The efforts of multiple editors and Area Editors in turn have generated a body of review papers that can serve as useful examples of the different types and approaches that JAMS has published.

Domain-based review papers

Domain-based review papers review, synthetize, and extend a body of literature in the same substantive domain. For example, in “The Role of Privacy in Marketing” (Martin and Murphy 2017 ), the authors identify and define various privacy-related constructs that have appeared in recent literature. Then they examine the different theoretical perspectives brought to bear on privacy topics related to consumers and organizations, including ethical and legal perspectives. These foundations lead in to their systematic review of privacy-related articles over a clearly defined date range, from which they extract key insights from each study. This exercise of synthesizing diverse perspectives allows these authors to describe state-of-the-art knowledge regarding privacy in marketing and identify useful paths for research. Similarly, a new paper by Cleeren et al. ( 2017 ), “Marketing Research on Product-Harm Crises: A Review, Managerial Implications, and an Agenda for Future Research,” provides a rich systematic review, synthesizes extant research, and points the way forward for scholars who are interested in issues related to defective or dangerous market offerings.

Theory-based review papers

Theory-based review papers review, synthetize, and extend a body of literature that uses the same underlying theory. For example, Rindfleisch and Heide’s ( 1997 ) classic review of research in marketing using transaction cost economics has been cited more than 2200 times, with a significant impact on applications of the theory to the discipline in the past 20 years. A recent paper in JAMS with similar intent, which could serve as a helpful model, focuses on “Resource-Based Theory in Marketing” (Kozlenkova et al. 2014 ). The article dives deeply into a description of the theory and its underlying assumptions, then organizes a systematic review of relevant literature according to various perspectives through which the theory has been applied in marketing. The authors conclude by identifying topical domains in marketing that might benefit from additional applications of the theory (e.g., marketing exchange), as well as related theories that could be integrated meaningfully with insights from the resource-based theory.

Method-based review papers

Method-based review papers review, synthetize, and extend a body of literature that uses the same underlying method. For example, in “Event Study Methodology in the Marketing Literature: An Overview” (Sorescu et al. 2017 ), the authors identify published studies in marketing that use an event study methodology. After a brief review of the theoretical foundations of event studies, they describe in detail the key design considerations associated with this method. The article then provides a roadmap for conducting event studies and compares this approach with a stock market returns analysis. The authors finish with a summary of the strengths and weaknesses of the event study method, which in turn suggests three main areas for further research. Similarly, “Discriminant Validity Testing in Marketing: An Analysis, Causes for Concern, and Proposed Remedies” (Voorhies et al. 2016 ) systematically reviews existing approaches for assessing discriminant validity in marketing contexts, then uses Monte Carlo simulation to determine which tests are most effective.

Our long-term editorial strategy is to make sure JAMS becomes and remains a well-recognized outlet for both meta-analysis and systematic managerial review papers in marketing. Ideally, review papers would come to represent 10%–20% of the papers published by the journal.

Process and structure for review papers

In this section, we review the process and typical structure of a systematic review paper, which lacks any long or established tradition in marketing research. The article by Grewal et al. ( 2018 ) provides a summary of effects-focused review papers (i.e., meta-analyses), so we do not discuss them in detail here.

Systematic literature review process

Some review papers submitted to journals take a “narrative” approach. They discuss current knowledge about a research domain, yet they often are flawed, in that they lack criteria for article inclusion (or, more accurately, article exclusion), fail to discuss the methodology used to evaluate included articles, and avoid critical assessment of the field (Barczak 2017 ). Such reviews tend to be purely descriptive, with little lasting impact.

In contrast, a systematic literature review aims to “comprehensively locate and synthesize research that bears on a particular question, using organized, transparent, and replicable procedures at each step in the process” (Littell et al. 2008 , p. 1). Littell et al. describe six key steps in the systematic review process. The extent to which each step is emphasized varies by paper, but all are important components of the review.

Topic formulation . The author sets out clear objectives for the review and articulates the specific research questions or hypotheses that will be investigated.

Study design . The author specifies relevant problems, populations, constructs, and settings of interest. The aim is to define explicit criteria that can be used to assess whether any particular study should be included in or excluded from the review. Furthermore, it is important to develop a protocol in advance that describes the procedures and methods to be used to evaluate published work.

Sampling . The aim in this third step is to identify all potentially relevant studies, including both published and unpublished research. To this end, the author must first define the sampling unit to be used in the review (e.g., individual, strategic business unit) and then develop an appropriate sampling plan.

Data collection . By retrieving the potentially relevant studies identified in the third step, the author can determine whether each study meets the eligibility requirements set out in the second step. For studies deemed acceptable, the data are extracted from each study and entered into standardized templates. These templates should be based on the protocols established in step 2.

Data analysis . The degree and nature of the analyses used to describe and examine the collected data vary widely by review. Purely descriptive analysis is useful as a starting point but rarely is sufficient on its own. The examination of trends, clusters of ideas, and multivariate relationships among constructs helps flesh out a deeper understanding of the domain. For example, both Hult ( 2015 ) and Huber et al. ( 2014 ) use bibliometric approaches (e.g., examine citation data using multidimensional scaling and cluster analysis techniques) to identify emerging versus declining themes in the broad field of marketing.

Reporting . Three key aspects of this final step are common across systematic reviews. First, the results from the fifth step need to be presented, clearly and compellingly, using narratives, tables, and figures. Second, core results that emerge from the review must be interpreted and discussed by the author. These revelatory insights should reflect a deeper understanding of the topic being investigated, not simply a regurgitation of well-established knowledge. Third, the author needs to describe the implications of these unique insights for both future research and managerial practice.

A new paper by Watson et al. ( 2017 ), “Harnessing Difference: A Capability-Based Framework for Stakeholder Engagement in Environmental Innovation,” provides a good example of a systematic review, starting with a cohesive conceptual framework that helps establish the boundaries of the review while also identifying core constructs and their relationships. The article then explicitly describes the procedures used to search for potentially relevant papers and clearly sets out criteria for study inclusion or exclusion. Next, a detailed discussion of core elements in the framework weaves published research findings into the exposition. The paper ends with a presentation of key implications and suggestions for the next steps. Similarly, “Marketing Survey Research Best Practices: Evidence and Recommendations from a Review of JAMS Articles” (Hulland et al. 2017 ) systematically reviews published marketing studies that use survey techniques, describes recent trends, and suggests best practices. In their review, Hulland et al. examine the entire population of survey papers published in JAMS over a ten-year span, relying on an extensive standardized data template to facilitate their subsequent data analysis.

Structure of systematic review papers

There is no cookie-cutter recipe for the exact structure of a useful systematic review paper; the final structure depends on the authors’ insights and intended points of emphasis. However, several key components are likely integral to a paper’s ability to contribute.

Depth and rigor

Systematic review papers must avoid falling in to two potential “ditches.” The first ditch threatens when the paper fails to demonstrate that a systematic approach was used for selecting articles for inclusion and capturing their insights. If a reader gets the impression that the author has cherry-picked only articles that fit some preset notion or failed to be thorough enough, without including articles that make significant contributions to the field, the paper will be consigned to the proverbial side of the road when it comes to the discipline’s attention.

Authors that fall into the other ditch present a thorough, complete overview that offers only a mind-numbing recitation, without evident organization, synthesis, or critical evaluation. Although comprehensive, such a paper is more of an index than a useful review. The reviewed articles must be grouped in a meaningful way to guide the reader toward a better understanding of the focal phenomenon and provide a foundation for insights about future research directions. Some scholars organize research by scholarly perspectives (e.g., the psychology of privacy, the economics of privacy; Martin and Murphy 2017 ); others classify the chosen articles by objective research aspects (e.g., empirical setting, research design, conceptual frameworks; Cleeren et al. 2017 ). The method of organization chosen must allow the author to capture the complexity of the underlying phenomenon (e.g., including temporal or evolutionary aspects, if relevant).

Replicability

Processes for the identification and inclusion of research articles should be described in sufficient detail, such that an interested reader could replicate the procedure. The procedures used to analyze chosen articles and extract their empirical findings and/or key takeaways should be described with similar specificity and detail.

We already have noted the potential usefulness of well-done review papers. Some scholars always are new to the field or domain in question, so review papers also need to help them gain foundational knowledge. Key constructs, definitions, assumptions, and theories should be laid out clearly (for which purpose summary tables are extremely helpful). An integrated conceptual model can be useful to organize cited works. Most scholars integrate the knowledge they gain from reading the review paper into their plans for future research, so it is also critical that review papers clearly lay out implications (and specific directions) for research. Ideally, readers will come away from a review article filled with enthusiasm about ways they might contribute to the ongoing development of the field.

Helpful format

Because such a large body of research is being synthesized in most review papers, simply reading through the list of included studies can be exhausting for readers. We cannot overstate the importance of tables and figures in review papers, used in conjunction with meaningful headings and subheadings. Vast literature review tables often are essential, but they must be organized in a way that makes their insights digestible to the reader; in some cases, a sequence of more focused tables may be better than a single, comprehensive table.

In summary, articles that review extant research in a domain (topic, theory, or method) can be incredibly useful to the scientific progress of our field. Whether integrating the insights from extant research through a meta-analysis or synthesizing them through a systematic assessment, the promised benefits are similar. Both formats provide readers with a useful overview of knowledge about the focal phenomenon, as well as insights on key dilemmas and conflicting findings that suggest future research directions. Thus, the editorial team at JAMS encourages scholars to continue to invest the time and effort to construct thoughtful review papers.

Barczak, G. (2017). From the editor: writing a review article. Journal of Product Innovation Management, 34 (2), 120–121.

Article   Google Scholar  

Bem, D. J. (1995). Writing a review article for psychological bulletin. Psychological Bulletin, 118 (2), 172–177.

Bettencourt, L. A., & Houston, M. B. (2001). Assessing the impact of article method type and subject area on citation frequency and reference diversity. Marketing Letters, 12 (4), 327–340.

Cleeren, K., Dekimpe, M. G., & van Heerde, H. J. (2017). Marketing research on product-harm crises: a review, managerial implications. Journal of the Academy of Marketing Science, 45 (5), 593–615.

Grewal, D., Puccinelli, N. M., & Monroe, K. B. (2018). Meta-analysis: error cancels and truth accrues. Journal of the Academy of Marketing Science, 46 (1).

Hanssens, D. M. (2018). The value of empirical generalizations in marketing. Journal of the Academy of Marketing Science, 46 (1).

Huber, J., Kamakura, W., & Mela, C. F. (2014). A topical history of JMR . Journal of Marketing Research, 51 (1), 84–91.

Hulland, J., Baumgartner, H., & Smith, K. M. (2017). Marketing survey research best practices: evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science. https://doi.org/10.1007/s11747-017-0532-y .

Hult, G. T. M. (2015). JAMS 2010—2015: literature themes and intellectual structure. Journal of the Academy of Marketing Science, 43 (6), 663–669.

Knoll, J., & Matthes, J. (2017). The effectiveness of celebrity endorsements: a meta-analysis. Journal of the Academy of Marketing Science, 45 (1), 55–75.

Kozlenkova, I. V., Samaha, S. A., & Palmatier, R. W. (2014). Resource-based theory in marketing. Journal of the Academy of Marketing Science, 42 (1), 1–21.

Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis . New York: Oxford University Press.

Book   Google Scholar  

Martin, K. D., & Murphy, P. E. (2017). The role of data privacy in marketing. Journal of the Academy of Marketing Science, 45 (2), 135–155.

Rindfleisch, A., & Heide, J. B. (1997). Transaction cost analysis: past, present, and future applications. Journal of Marketing, 61 (4), 30–54.

Sorescu, A., Warren, N. L., & Ertekin, L. (2017). Event study methodology in the marketing literature: an overview. Journal of the Academy of Marketing Science, 45 (2), 186–207.

Verma, V., Sharma, D., & Sheth, J. (2016). Does relationship marketing matter in online retailing? A meta-analytic approach. Journal of the Academy of Marketing Science, 44 (2), 206–217.

Voorhies, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science, 44 (1), 119–134.

Watson, R., Wilson, H. N., Smart, P., & Macdonald, E. K. (2017). Harnessing difference: a capability-based framework for stakeholder engagement in environmental innovation. Journal of Product Innovation Management. https://doi.org/10.1111/jpim.12394 .

Download references

Author information

Authors and affiliations.

Foster School of Business, University of Washington, Box: 353226, Seattle, WA, 98195-3226, USA

Robert W. Palmatier

Neeley School of Business, Texas Christian University, Fort Worth, TX, USA

Mark B. Houston

Terry College of Business, University of Georgia, Athens, GA, USA

John Hulland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Robert W. Palmatier .

Rights and permissions

Reprints and permissions

About this article

Palmatier, R.W., Houston, M.B. & Hulland, J. Review articles: purpose, process, and structure. J. of the Acad. Mark. Sci. 46 , 1–5 (2018). https://doi.org/10.1007/s11747-017-0563-4

Download citation

Published : 02 October 2017

Issue Date : January 2018

DOI : https://doi.org/10.1007/s11747-017-0563-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Find a journal
  • Publish with us
  • Track your research

How to write a good scientific review article

Affiliation.

  • 1 The FEBS Journal Editorial Office, Cambridge, UK.
  • PMID: 35792782
  • DOI: 10.1111/febs.16565

Literature reviews are valuable resources for the scientific community. With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and highlights key gaps and challenges to address with future research. Writing a review article also helps to expand the writer's knowledge of their specialist area and to develop their analytical and communication skills, amongst other benefits. Thus, the importance of building review-writing into a scientific career cannot be overstated. In this instalment of The FEBS Journal's Words of Advice series, I provide detailed guidance on planning and writing an informative and engaging literature review.

© 2022 Federation of European Biochemical Societies.

Publication types

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

  • Formulate a research question
  • Develop a protocol
  • Search for all relevant studies
  • Apply the selection criteria
  • Extract the data
  • Synthesize the data
  • Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

  • A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
  • Access to databases and journal archives. Often, your educational institution provides you with access.
  • Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
  • Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

  • They minimize research bias by considering all available evidence and evaluating each study for bias.
  • Their methods are transparent , so they can be scrutinized by others.
  • They’re thorough : they summarize all available evidence.
  • They can be replicated and updated by others.

Systematic reviews also have a few cons .

  • They’re time-consuming .
  • They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

  • Allow you to more effectively communicate your research to other researchers and practitioners
  • Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

  • Population(s) or problem(s)
  • Intervention(s)
  • Comparison(s)

You can rearrange these four components to write your research question:

  • What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

  • Type of study design(s)
  • The population of patients with eczema
  • The intervention of probiotics
  • In comparison to no treatment, placebo , or non-probiotic treatment
  • The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
  • Randomized control trials, a type of study design

Their research question was:

  • What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

  • Background information : Provide the context of the research question, including why it’s important.
  • Research objective (s) : Rephrase your research question as an objective.
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

  • Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
  • Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
  • Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
  • Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

  • Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
  • Handsearch: Conference proceedings and reference lists of articles
  • Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
  • Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

  • Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
  • Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

  • Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
  • Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

  • Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
  • Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

  • Abstract : A summary of the review
  • Introduction : Including the rationale and objectives
  • Methods : Including the selection criteria, search method, data extraction method, and synthesis method
  • Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
  • Discussion : Including interpretation of the results and limitations of the review
  • Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

  • Student’s  t -distribution
  • Normal distribution
  • Null and Alternative Hypotheses
  • Chi square tests
  • Confidence interval
  • Quartiles & Quantiles
  • Cluster sampling
  • Stratified sampling
  • Data cleansing
  • Reproducibility vs Replicability
  • Peer review
  • Prospective cohort study

Research bias

  • Implicit bias
  • Cognitive bias
  • Placebo effect
  • Hawthorne effect
  • Hindsight bias
  • Affect heuristic
  • Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved February 22, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

Page Content

Overview of the review report format, the first read-through, first read considerations, spotting potential major flaws, concluding the first reading, rejection after the first reading, before starting the second read-through, doing the second read-through, the second read-through: section by section guidance, how to structure your report, on presentation and style, criticisms & confidential comments to editors, the recommendation, when recommending rejection, additional resources, step by step guide to reviewing a manuscript.

When you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. Try to respond to invitations promptly - it will prevent delays. It is also important at this stage to declare any potential Conflict of Interest.

The structure of the review report varies between journals. Some follow an informal structure, while others have a more formal approach.

" Number your comments!!! " (Jonathon Halbesleben, former Editor of Journal of Occupational and Organizational Psychology)

Informal Structure

Many journals don't provide criteria for reviews beyond asking for your 'analysis of merits'. In this case, you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, as you gain experience, rely on your own evolving style.

Formal Structure

Other journals require a more formal approach. Sometimes they will ask you to address specific questions in your review via a questionnaire. Or they might want you to rate the manuscript on various attributes using a scorecard. Often you can't see these until you log in to submit your review. So when you agree to the work, it's worth checking for any journal-specific guidelines and requirements. If there are formal guidelines, let them direct the structure of your review.

In Both Cases

Whether specifically required by the reporting format or not, you should expect to compile comments to authors and possibly confidential ones to editors only.

Reviewing with Empathy

Following the invitation to review, when you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you need to feedback on how to improve those sections.

The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

Keep a pen and paper handy when skim-reading.

Try to bear in mind the following questions - they'll help you form your overall impression:

  • What is the main question addressed by the research? Is it relevant and interesting?
  • How original is the topic? What does it add to the subject area compared with other published material?
  • Is the paper well written? Is the text clear and easy to read?
  • Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed?
  • If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible?
  • If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, " Specific recommendations for remedying flaws are VERY welcome ."

Examples of possibly major flaws include:

  • Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence
  • The use of a discredited method
  • Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

  • The sampling in analytical papers
  • The sufficient use of control experiments
  • The precision of process data
  • The regularity of sampling in time-dependent studies
  • The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
  • That qualitative research extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. If there are critical flaws in this, it's very likely the manuscript will need to be rejected. Such issues include:

  • Insufficient data
  • Unclear data tables
  • Contradictory data that either are not self-consistent or disagree with the conclusions
  • Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

  • Help the editor properly contextualize the research and add weight to your judgement
  • Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
  • Focus on successful aspects of the paper so the author gets a sense of what they've done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

  • Is the paper's premise interesting and important?
  • Are the methods used appropriate?
  • Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section "When recommending rejection."

Once the paper has passed your first read and you've decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to recommend rejection following a second reading.

" Offer clear suggestions for how the authors can address the concerns raised. In other words, if you're going to raise a problem, provide a solution ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

  • Don't rely solely upon inserting comments on the manuscript document - make separate notes
  • Try to group similar concerns or praise together
  • If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later
  • Note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

Now that you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

As you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

  • Any places where the meaning is unclear or ambiguous
  • Any factual errors
  • Any invalid arguments

You may also wish to consider:

  • Does the title properly reflect the subject of the paper?
  • Does the abstract provide an accessible summary of the paper?
  • Do the keywords accurately reflect the content?
  • Is the paper an appropriate length?
  • Are the key messages short, accurate and clear?

Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, " If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited ."

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

  • Are there certain aspects that could be communicated better, such as parts of the discussion?
  • Should the authors consider resubmitting to the same journal after language improvements?
  • Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don't spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it's important to highlight these. Expect to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

1. The Introduction

A well-written introduction:

  • Sets out the argument
  • Summarizes recent research related to the topic
  • Highlights gaps in current understanding or conflicts in current knowledge
  • Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
  • Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

Originality and Topicality

Originality and topicality can only be established in the light of recent authoritative research. For example, it's impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

Authors may make the case that a topic hasn't been investigated in several years and that new research is required. This point is only valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Clearly, authors can only do this by referencing recent literature. Obviously, where older research is seminal or where aspects of the methodology rely upon it, then it is perfectly appropriate for authors to cite some older papers.

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes ?"

It's common for the introduction to end by stating the research aims. By this point you should already have a good impression of them - if the explicit aims come as a surprise, then the introduction needs improvement.

2. Materials and Methods

Academic research should be replicable, repeatable and robust - and follow best practice.

Replicable Research

This makes sufficient use of:

  • Control experiments
  • Repeated analyses
  • Repeated experiments

These are used to make sure observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. Statistical analyses will not be sound if methods are not replicable. Where research is not replicable, the paper should be recommended for rejection.

Repeatable Methods

These give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.

Robust Research

This has enough data points to make sure the data are reliable. If there are insufficient data, it might be appropriate to recommend revision. You should also consider whether there is any in-built bias not nullified by the control experiments.

Best Practice

During these checks you should keep in mind best practice:

  • Standard guidelines were followed (e.g. the CONSORT Statement for reporting randomized trials)
  • The health and safety of all participants in the study was not compromised
  • Ethical standards were maintained

If the research fails to reach relevant best practice standards, it's usual to recommend rejection. What's more, you don't then need to read any further.

3. Results and Discussion

This section should tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting need to be followed by the author:

  • They should start by describing in simple terms what the data show
  • They should make reference to statistical analyses, such as significance or goodness of fit
  • Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research
  • The outcome should be a critical analysis of the data collected

Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.

4. Conclusions

This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.

5. Information Gathered: Images, Graphs and Data Tables

If you find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. This could be an issue with titles, labels, statistical notation or image quality.

Where information is clear, you should check that:

  • The results seem plausible, in case there is an error in data gathering
  • The trends you can see support the paper's discussion and conclusions
  • There are sufficient data. For example, in studies carried out over time are there sufficient data points to support the trends described by the author?

You should also check whether images have been edited or manipulated to emphasize the story they tell. This may be appropriate but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). Where you feel that an image has been edited or manipulated without explanation, you should highlight this in a confidential comment to the editor in your report.

6. List of References

You will need to check referencing for accuracy, adequacy and balance.

Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. Otherwise, it's the editor’s role to exhaustively check the reference section for accuracy and format.

You should consider if the referencing is adequate:

  • Are important parts of the argument poorly supported?
  • Are there published studies that show similar or dissimilar trends that should be discussed?
  • If a manuscript only uses half the citations typical in its field, this may be an indicator that referencing should be improved - but don't be guided solely by quantity
  • References should be relevant, recent and readily retrievable

Check for a well-balanced list of references that is:

  • Helpful to the reader
  • Fair to competing authors
  • Not over-reliant on self-citation
  • Gives due recognition to the initial discoveries and related work that led to the work under assessment

You should be able to evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

By now you will have a deep understanding of the paper's content - and you may have some concerns about plagiarism.

Identified Concern

If you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or it may be because it is very recent or published in a journal slightly outside their usual field.

You may feel you can advise the author how to emphasize the novel aspects of their own study, so as to better differentiate it from similar research. If so, you may ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. Of course, the research similarities may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

"It's very helpful when a reviewer can point out recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere ." (Editor feedback)

Suspected Concern

If you suspect plagiarism, including self-plagiarism, but cannot recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

Most editors have access to software that can check for plagiarism.

Editors are not out to police every paper, but when plagiarism is discovered during peer review it can be properly addressed ahead of publication. If plagiarism is discovered only after publication, the consequences are worse for both authors and readers, because a retraction may be necessary.

For detailed guidelines see COPE's Ethical guidelines for reviewers and Wiley's Best Practice Guidelines on Publishing Ethics .

8. Search Engine Optimization (SEO)

After the detailed read-through, you will be in a position to advise whether the title, abstract and key words are optimized for search purposes. In order to be effective, good SEO terms will reflect the aims of the research.

A clear title and abstract will improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. The title should contain the relevant SEO terms early on. This has a major effect on the impact of a paper, since it helps it appear in search results. A poor abstract can then lose the reader's interest and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may go no further.

So ask yourself, while the abstract may have seemed adequate during earlier checks, does it:

  • Do justice to the manuscript in this context?
  • Highlight important findings sufficiently?
  • Present the most interesting data?

Editors say, " Does the Abstract highlight the important findings of the study ?"

If there is a formal report format, remember to follow it. This will often comprise a range of questions followed by comment sections. Try to answer all the questions. They are there because the editor felt that they are important. If you're following an informal report format you could structure your report in three sections: summary, major issues, minor issues.

  • Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection
  • Briefly summarize what the paper is about and what the findings are
  • Try to put the findings of the paper into the context of the existing literature and current knowledge
  • Indicate the significance of the work and if it is novel or mainly confirmatory
  • Indicate the work's strengths, its quality and completeness
  • State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

  • Are there any major flaws? State what they are and what the severity of their impact is on the paper
  • Has similar work already been published without the authors acknowledging this?
  • Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
  • If major revisions are required, try to indicate clearly what they are
  • Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
  • Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

  • Are there places where meaning is ambiguous? How can this be corrected?
  • Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?
  • Are there any factual, numerical or unit errors? If so, what are they?
  • Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

Your review should ultimately help the author improve their article. So be polite, honest and clear. You should also try to be objective and constructive, not subjective and destructive.

You should also:

  • Write clearly and so you can be understood by people whose first language is not English
  • Avoid complex or unusual words, especially ones that would even confuse native speakers
  • Number your points and refer to page and line numbers in the manuscript when making specific comments
  • If you have been asked to only comment on specific parts or aspects of the manuscript, you should indicate clearly which these are
  • Treat the author's work the way you would like your own to be treated

Most journals give reviewers the option to provide some confidential comments to editors. Often this is where editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice such as suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

However, this doesn't give reviewers permission to 'backstab' the author. Authors can't see this feedback and are unable to give their side of the story unless the editor asks them to. So in the spirit of fairness, write comments to editors as though authors might read them too.

Reviewers should check the preferences of individual journals as to where they want review decisions to be stated. In particular, bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

You will normally be asked to indicate your recommendation (e.g. accept, reject, revise and resubmit, etc.) from a fixed-choice list and then to enter your comments into a separate text box.

Recommending Acceptance

If you're recommending acceptance, give details outlining why, and if there are any areas that could be improved. Don't just give a short, cursory remark such as 'great, accept'. See Improving the Manuscript

Recommending Revision

Where improvements are needed, a recommendation for major or minor revision is typical. You may also choose to state whether you opt in or out of the post-revision review too. If recommending revision, state specific changes you feel need to be made. The author can then reply to each point in turn.

Some journals offer the option to recommend rejection with the possibility of resubmission – this is most relevant where substantial, major revision is necessary.

What can reviewers do to help? " Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revisio n." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Recommending Rejection

If recommending rejection or major revision, state this clearly in your review (and see the next section, 'When recommending rejection').

Where manuscripts have serious flaws you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

Editors say, " If a reviewer suggests a rejection, but her/his comments are not detailed or helpful, it does not help the editor in making a decision ."

In your recommendations for the author, you should:

  • Give constructive feedback describing ways that they could improve the research
  • Keep the focus on the research and not the author. This is an extremely important part of your job as a reviewer
  • Avoid making critical confidential comments to the editor while being polite and encouraging to the author - the latter may not understand why their manuscript has been rejected. Also, they won't get feedback on how to improve their research and it could trigger an appeal

Remember to give constructive criticism even if recommending rejection. This helps developing researchers improve their work and explains to the editor why you felt the manuscript should not be published.

" When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Visit our Wiley Author Learning and Training Channel for expert advice on peer review.

Watch the video, Ethical considerations of Peer Review

COMMUNICATION IN THE BIOLOGICAL SCIENCES Department of Biology

LITERATURE REVIEW PAPER

WHAT IS A REVIEW PAPER?

CHOOSING A TOPIC

RESEARCHING A TOPIC

HOW TO WRITE THE PAPER    

The purpose of a review paper is to succinctly review recent progress in a particular topic. Overall, the paper summarizes the current state of knowledge of the topic. It creates an understanding of the topic for the reader by discussing the findings presented in recent research papers .

A review paper is not a "term paper" or book report . It is not merely a report on some references you found. Instead, a review paper synthesizes the results from several primary literature papers to produce a coherent argument about a topic or focused description of a field.

Examples of scientific reviews can be found in:

                Current Opinion in Cell Biology

                Current Opinion in Genetics & Development

                Annual Review of Plant Physiology and Plant Molecular Biology

                Annual Review of Physiology

                Trends in Ecology & Evolution

You should read articles from one or more of these sources to get examples of how your paper should be organized.

Scientists commonly use reviews to communicate with each other and the general public. There are a wide variety of review styles from ones aimed at a general audience (e.g., Scientific American ) to those directed at biologists within a particular subdiscipline (e.g., Annual Review of Physiology ).

A key aspect of a review paper is that it provides the evidence for a particular point of view in a field. Thus, a large focus of your paper should be a description of the data that support or refute that point of view. In addition, you should inform the reader of the experimental techniques that were used to generate the data.

The emphasis of a review paper is interpreting the primary literature on the subject.  You need to read several original research articles on the same topic and make your own conclusions about the meanings of those papers.

Click here for advice on choosing a topic.  

Click here for advice on doing research on your topic.  

HOW TO WRITE THE PAPER

Overview of the Paper: Your paper should consist of four general sections:

Review articles contain neither a materials and methods section nor an abstract.

Organizing the Paper: Use topic headings. Do not use a topic heading that reads, "Body of the paper." Instead the topic headings should refer to the actual concepts or ideas covered in that section.

Example  

What Goes into Each Section:

Home  

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

  • PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
  • EDIT Edit this Article
  • EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
  • Browse Articles
  • Learn Something New
  • Quizzes Hot
  • This Or That Game New
  • Train Your Brain
  • Explore More
  • Support wikiHow
  • About wikiHow
  • Log in / Sign up
  • Education and Communications
  • Critical Reviews

How to Write an Article Review

Last Updated: September 8, 2023 Fact Checked

This article was co-authored by Jake Adams . Jake Adams is an academic tutor and the owner of Simplifi EDU, a Santa Monica, California based online tutoring business offering learning resources and online tutors for academic subjects K-College, SAT & ACT prep, and college admissions applications. With over 14 years of professional tutoring experience, Jake is dedicated to providing his clients the very best online tutoring experience and access to a network of excellent undergraduate and graduate-level tutors from top colleges all over the nation. Jake holds a BS in International Business and Marketing from Pepperdine University. There are 13 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 3,068,205 times.

An article review is both a summary and an evaluation of another writer's article. Teachers often assign article reviews to introduce students to the work of experts in the field. Experts also are often asked to review the work of other professionals. Understanding the main points and arguments of the article is essential for an accurate summation. Logical evaluation of the article's main theme, supporting arguments, and implications for further research is an important element of a review . Here are a few guidelines for writing an article review.

Education specialist Alexander Peterman recommends: "In the case of a review, your objective should be to reflect on the effectiveness of what has already been written, rather than writing to inform your audience about a subject."

Things You Should Know

  • Read the article very closely, and then take time to reflect on your evaluation. Consider whether the article effectively achieves what it set out to.
  • Write out a full article review by completing your intro, summary, evaluation, and conclusion. Don't forget to add a title, too!
  • Proofread your review for mistakes (like grammar and usage), while also cutting down on needless information. [1] X Research source

Preparing to Write Your Review

Step 1 Understand what an article review is.

  • Article reviews present more than just an opinion. You will engage with the text to create a response to the scholarly writer's ideas. You will respond to and use ideas, theories, and research from your studies. Your critique of the article will be based on proof and your own thoughtful reasoning.
  • An article review only responds to the author's research. It typically does not provide any new research. However, if you are correcting misleading or otherwise incorrect points, some new data may be presented.
  • An article review both summarizes and evaluates the article.

Step 2 Think about the organization of the review article.

  • Summarize the article. Focus on the important points, claims, and information.
  • Discuss the positive aspects of the article. Think about what the author does well, good points she makes, and insightful observations.
  • Identify contradictions, gaps, and inconsistencies in the text. Determine if there is enough data or research included to support the author's claims. Find any unanswered questions left in the article.

Step 3 Preview the article.

  • Make note of words or issues you don't understand and questions you have.
  • Look up terms or concepts you are unfamiliar with, so you can fully understand the article. Read about concepts in-depth to make sure you understand their full context.

Step 4 Read the article closely.

  • Pay careful attention to the meaning of the article. Make sure you fully understand the article. The only way to write a good article review is to understand the article.

Step 5 Put the article into your words.

  • With either method, make an outline of the main points made in the article and the supporting research or arguments. It is strictly a restatement of the main points of the article and does not include your opinions.
  • After putting the article in your own words, decide which parts of the article you want to discuss in your review. You can focus on the theoretical approach, the content, the presentation or interpretation of evidence, or the style. You will always discuss the main issues of the article, but you can sometimes also focus on certain aspects. This comes in handy if you want to focus the review towards the content of a course.
  • Review the summary outline to eliminate unnecessary items. Erase or cross out the less important arguments or supplemental information. Your revised summary can serve as the basis for the summary you provide at the beginning of your review.

Step 6 Write an outline of your evaluation.

  • What does the article set out to do?
  • What is the theoretical framework or assumptions?
  • Are the central concepts clearly defined?
  • How adequate is the evidence?
  • How does the article fit into the literature and field?
  • Does it advance the knowledge of the subject?
  • How clear is the author's writing? Don't: include superficial opinions or your personal reaction. Do: pay attention to your biases, so you can overcome them.

Writing the Article Review

Step 1 Come up with...

  • For example, in MLA , a citation may look like: Duvall, John N. "The (Super)Marketplace of Images: Television as Unmediated Mediation in DeLillo's White Noise ." Arizona Quarterly 50.3 (1994): 127-53. Print. [10] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

Step 3 Identify the article.

  • For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest.

Step 4 Write the introduction....

  • Your introduction should only be 10-25% of your review.
  • End the introduction with your thesis. Your thesis should address the above issues. For example: Although the author has some good points, his article is biased and contains some misinterpretation of data from others’ analysis of the effectiveness of the condom.

Step 5 Summarize the article.

  • Use direct quotes from the author sparingly.
  • Review the summary you have written. Read over your summary many times to ensure that your words are an accurate description of the author's article.

Step 6 Write your critique.

  • Support your critique with evidence from the article or other texts.
  • The summary portion is very important for your critique. You must make the author's argument clear in the summary section for your evaluation to make sense.
  • Remember, this is not where you say if you liked the article or not. You are assessing the significance and relevance of the article.
  • Use a topic sentence and supportive arguments for each opinion. For example, you might address a particular strength in the first sentence of the opinion section, followed by several sentences elaborating on the significance of the point.

Step 7 Conclude the article review.

  • This should only be about 10% of your overall essay.
  • For example: This critical review has evaluated the article "Condom use will increase the spread of AIDS" by Anthony Zimmerman. The arguments in the article show the presence of bias, prejudice, argumentative writing without supporting details, and misinformation. These points weaken the author’s arguments and reduce his credibility.

Step 8 Proofread.

  • Make sure you have identified and discussed the 3-4 key issues in the article.

Sample Article Reviews

research for review paper

Expert Q&A

Jake Adams

You Might Also Like

Write Articles

  • ↑ https://writing.wisc.edu/handbook/grammarpunct/proofreading/
  • ↑ https://libguides.cmich.edu/writinghelp/articlereview
  • ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/
  • ↑ Jake Adams. Academic Tutor & Test Prep Specialist. Expert Interview. 24 July 2020.
  • ↑ https://guides.library.queensu.ca/introduction-research/writing/critical
  • ↑ https://www.iup.edu/writingcenter/writing-resources/organization-and-structure/creating-an-outline.html
  • ↑ https://writing.umn.edu/sws/assets/pdf/quicktips/titles.pdf
  • ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_periodicals.html
  • ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548565/
  • ↑ https://writingcenter.uconn.edu/wp-content/uploads/sites/593/2014/06/How_to_Summarize_a_Research_Article1.pdf
  • ↑ https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/how-to-review-a-journal-article
  • ↑ https://writingcenter.unc.edu/tips-and-tools/editing-and-proofreading/

About This Article

Jake Adams

If you have to write an article review, read through the original article closely, taking notes and highlighting important sections as you read. Next, rewrite the article in your own words, either in a long paragraph or as an outline. Open your article review by citing the article, then write an introduction which states the article’s thesis. Next, summarize the article, followed by your opinion about whether the article was clear, thorough, and useful. Finish with a paragraph that summarizes the main points of the article and your opinions. To learn more about what to include in your personal critique of the article, keep reading the article! Did this summary help you? Yes No

  • Send fan mail to authors

Reader Success Stories

Prince Asiedu-Gyan

Prince Asiedu-Gyan

Apr 22, 2022

Did this article help you?

research for review paper

Sammy James

Sep 12, 2017

Juabin Matey

Juabin Matey

Aug 30, 2017

Kristi N.

Oct 25, 2023

Vanita Meghrajani

Vanita Meghrajani

Jul 21, 2016

Am I a Narcissist or an Empath Quiz

Featured Articles

Start a Text Conversation with a Girl

Trending Articles

How to Take the Perfect Thirst Trap

Watch Articles

Wrap a Round Gift

  • Terms of Use
  • Privacy Policy
  • Do Not Sell or Share My Info
  • Not Selling Info

Get all the best how-tos!

Sign up for wikiHow's weekly email newsletter

  • UWF Libraries

Literature Review: Conducting & Writing

  • Sample Literature Reviews
  • Steps for Conducting a Lit Review
  • Finding "The Literature"
  • Organizing/Writing
  • Chicago: Notes Bibliography

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

  • Literature Review Sample 1
  • Literature Review Sample 2
  • Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

  • << Previous: MLA Style
  • Next: Get Help! >>
  • Last Updated: Jan 18, 2024 9:21 AM
  • URL: https://libguides.uwf.edu/litreview
  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

990 Accesses

55 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

research for review paper

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Lippincott Williams & Wilkins Open Access

Psychiatric Symptoms Across the Menstrual Cycle in Adult Women: A Comprehensive Review

Learning objective.

After participating in this activity, learners should be better able to:

• Discuss and outline the general and overlapping effects of the menstrual cycle on women’s mental health

A growing body of research demonstrates menstrual cycle–dependent fluctuations in psychiatric symptoms; these fluctuations can therefore be considered as prevalent phenomena. Possible mechanisms underlying these fluctuations posit behavioral, psychological, and neuroendocrine influences. Recent reviews document cyclic exacerbation of symptoms and explore these mechanisms in the context of specific and often single disorders. The question remains, however, as to whether there are general and overlapping effects of the menstrual cycle on women’s mental health. To address this gap, we synthesized the literature examining the exacerbation of a variety of psychiatric symptoms across the menstrual cycle in adult women. Results show that the premenstrual and menstrual phases are most consistently implicated in transdiagnostic symptom exacerbation. Specifically, strong evidence indicates increases in psychosis, mania, depression, suicide/suicide attempts, and alcohol use during these phases. Anxiety, stress, and binge eating appear to be elevated more generally throughout the luteal phase. The subjective effects of smoking and cocaine use are reduced during the luteal phase, but fewer data are available for other substances. Less consistent patterns are demonstrated for panic disorder, symptoms of posttraumatic stress disorder, and borderline personality disorder, and it is difficult to draw conclusions for symptoms of generalized anxiety disorder, social anxiety disorder, obsessive-compulsive disorder, and trichotillomania because of the limited data. Future research should focus on developing standardized approaches to identifying menstrual cycle phases and adapting pharmacological and behavioral interventions for managing fluctuations in psychiatric symptoms across the menstrual cycle.

The menstrual cycle is characterized by predictable and recurrent fluctuations in hormones—namely, the ovarian hormones estrogen and progesterone. The cycle is separated into two distinct phases: the follicular phase, which consists of the first part of the cycle lasting from menstruation to ovulation and which varies in length but typically lasts 14 days; and the luteal phase, which is the second half of the cycle following ovulation and leading up to menstruation, and consistently lasts 14 days (see Mihm et al. 1 for an overview). The days immediately prior to menstruation are often termed the premenstrual phase.

During menstruation, estrogen and progesterone levels are relatively low (see Figure ​ Figure1 1 ). 2 As the cycle advances through the follicular phase, estrogen levels spike, causing the pituitary gland to release a surge of follicle-stimulating hormone and luteinizing hormone—which facilitates the maturing of eggs within the ovaries. 3 When the most mature egg is released, the follicle transforms into a corpus luteum, which produces gradually increasing amounts of progesterone; a moderate amount of estrogen is also produced. 3 If the egg is not fertilized, progesterone and estrogen levels fall, the uterine lining breaks down, and the menstrual cycle resumes with menstruation, which typically lasts between 1 and 7 days. 4

An external file that holds a picture, illustration, etc.
Object name is hrp-30-100-g001.jpg

Estrogen and progesterone levels across a typical 28-day menstrual cycle (adapted from Glover et al. (2013). 2

This cyclic experience may influence women’s mental health through a variety of mechanisms. For example, many women experience physical discomfort (e.g., dysmenorrhea, breast tenderness, joint pain 5 ) around menstruation. This physical discomfort can be associated with increases in psychological distress and irritability, and decreased self-esteem. 6 Many women additionally report increased interpersonal conflicts and reduced social engagement premenstrually and during menstruation 7 , 8 —which may contribute to depression and isolation. 8 Negative affect is linked with increased impulsivity, 9 substance use, 10 and nonsuicidal self-injury. 11 As such, it is unsurprising that systematic and meta-analytic reviews find exacerbations of psychiatric symptoms across the menstrual cycle (e.g., Carroll et al. 12 ).

In addition to affective and behavioral impacts of the menstrual cycle, there are also several direct biological effects on mental health. For example, estrogen downregulates dopamine transmission, which mimics the antidopaminergic action of many antipsychotic medications. 13 Higher estrogen levels are hypothesized to protect against psychiatric symptoms, such as psychosis, thereby increasing vulnerability to psychosis when estrogen is low (e.g., menstruation, postpartum 14 , 15 ). Estrogen also assists in memory consolidation through increased hippocampal activation, 16 , 17 which, in the context of treatment for posttraumatic stress disorder (PTSD), has been shown to facilitate fear extinction recall. 18 Progesterone can have anxiolytic effects 19 through increases in allopregnanolone and, subsequently, increased GABA potentiation. 20 – 22 Other progesterone metabolites, however, are not anxiolytic. In the presence of stress, progesterone is converted into cortisol, increasing stress responses and impairing emotional processing. 23 To this end, it has been suggested that progesterone may underlie menstrual-related mood symptoms. 23

Recently, several elegant reviews documented menstrual exacerbations of numerous psychiatric symptoms, 24 including addictive behaviors, 25 psychosis, 15 suicidality, 26 anxiety, and posttraumatic stress disorder. 27 These reviews increased the scientific understanding of the effects of the menstrual cycle on women’s mental health, including the more pointed effects of estrogen and progesterone. Yet, no review to date comprehensively evaluated the impact of the menstrual cycle on psychiatric symptoms. To further this growing body of research, we conducted a comprehensive review synthesizing the literature on fluctuations in a broad spectrum of psychiatric disorders and symptoms across the menstrual cycle. Summarizing these data in such a way will allow us to potentially identify patterns and draw conclusions beyond what previous reviews examined.

Search Strategy

We conducted a comprehensive search using the PubMed database for articles focusing on psychiatric symptoms across the menstrual cycle. We used combinations of the following search terms to identify potentially relevant articles: menstrual cycle, psychosis, bipolar, mania, depression, suicide, anxiety, obsessive-compulsive disorder (OCD), body dysmorphic disorder, trichotillomania, excoriation, hoarding, impulse control, kleptomania, PTSD, eating disorder, anorexia, bulimia, binge eating, borderline personality disorder (BPD), intermittent explosive disorder, conduct disorder, pyromania, substance use, alcohol, and smoking. We compiled the results, removed duplicates, and reviewed the titles and abstracts of the remaining articles. We then read and assessed for eligibility the full texts of the remaining articles using the following criteria: studies (1) were published in English, (2) presented original findings, (3) included premenopausal women at least 18 years old, and (4) assessed relevant psychiatric symptoms during at least two menstrual cycle phases. We did not include articles focusing on symptoms in women with premenstrual syndrome (PMS) or premenstrual dysphoric disorder, as those conditions inherently vary across the menstrual cycle. See inclusion diagram (Supplemental Figure 1, http://links.lww.com/HRP/A187 ), and Supplemental Tables 1–10, http://links.lww.com/HRP/A188 , for full descriptions of the articles included.

Definition of Menstrual Cycle Phases

Most menstrual cycle research uses a “count” method to estimate cycle phases based on the first day of menstruation (Day 1). Estimates are often based on a 28-day cycle, although a substantial body of evidence suggests significant variation in cycle length in healthy women. 196 Indeed, the follicular phase has been found to vary by as much as 12 days. 197 The luteal phase, by contrast, lasts a relatively consistent amount of time—approximately 14 days. Some exceptions to this pattern include women with luteal phase disorders (e.g., short luteal phase). 198 , 199

Many articles included in this review have varying terminology and definitions of menstrual cycle phases. For example, occasionally the “premenstrual” phase is referred to as the mid- or late-luteal phase. “Perimenstrual” can include both premenstrual and menstrual phases, whereas “early follicular” can include menstruation and the initial days following menstruation. We report the findings using the language described in the articles reviewed; however, our synthesis of the data focuses on overall patterns during specific phases, regardless of terminology.

Quality Rating

To strengthen our synthesis of study results, we developed a quality rating based on study design. We considered studies to be of high quality if they included at least one biological indicator of the menstrual cycle (e.g., basal body temperature, hormone levels), sample size was ≥30 women per group, and, when applicable, clinical diagnoses were made.

Most reports on the exacerbation of psychotic symptoms across the menstrual cycle are single case studies. These studies retrospectively rely on a patient’s self-reported or clinician-observed experiences over several years or decades. In all but one 37 of the identified case studies, the authors reported recurrent psychosis during the premenstrual phase. 28 , 29 , 32 – 34 , 36 , 38 Often, symptom onset occurred at menarche. 34 Though many authors reported that psychotic or manic symptoms occurred during the week or days before menstruation, several noted that psychotic symptoms remitted upon menstruation. 28 , 38

Findings become less clear when examining larger populations. Ray and colleagues 39 followed 40 women with schizophrenia in an inpatient unit in India. Clinicians rated patients’ symptoms weekly for two consecutive menstrual cycles and found that positive symptoms of schizophrenia—namely, excitability and hostility—were highest during the premenstrual phase. Negative symptoms, such as withdrawal and difficulty with abstract thinking, were highest during menstruation. These findings complement earlier work by Harris, 30 who found that, in a sample of 39 inpatient women with schizophrenia, most women experienced affective changes, rather than overt psychosis, during the premenstrual phase. Conversely, in a large, community-based study of 278 healthy women from the United Kingdom, persecutorial thoughts increased during the paramenstrual phase (day 1 of menstruation ± 3 days) compared to midcycle (11–17 days prior to menstruation). 6 It is possible that certain menstrual fluctuations in paranoia exist in nonclinical samples and that affective fluctuations may be more common in clinical samples, where paranoia is more consistently pronounced. For women who also experience menstrual exacerbation of overt psychosis, the exacerbation appears to occur predominantly during the premenstrual phase. 31

Results from high-quality studies

The one high-quality study found that 32.4% of a sample of women with schizophrenia had cyclical worsening of psychotic symptoms. 35

Bipolar Disorders

Several case studies document menstrual fluctuations in symptoms of bipolar disorder. All but one study 43 identified in the present review reported instances of hypomania, 47 mania, 41 or psychosis 50 during the premenstrual and menstrual phases. Kukopoulos and colleagues 40 reported that a 28-year-old Sardinian woman regularly experienced hypomania two weeks and depression two days prior to menstruation, with gradual improvement throughout menstruation. Subsequent reports similarly note the onset of mania and hypomania days prior to menstruation, with some symptoms ceasing upon menstruation 41 or days after menstruation began. 47 The sole case study to report incongruent findings described one woman’s experience with menstrual onset of depression and luteal-phase onset of hypomania. 43

Results from larger clinical and community samples speak to these individual differences, as many studies report no overwhelming effect of the menstrual cycle on bipolar symptoms. 42 , 45 , 46 , 48 Cyclic effects were found, however, in numerous subgroups of study samples. In one study of 41 women, increases in depression and mania were observed during the luteal phase for some women (n’s of 8 and 5, respectively). 42 , 46 Similarly, Leibenluft and colleagues 42 found that, in a sample of 25 women with rapid-cycling bipolar disorder, 6 showed an increase in depression and 5 an increase in hypomania in the days following menstruation. Sit and colleagues 48 posit that the use of mood stabilizers and antipsychotics in treating bipolar disorders could mask patterns in symptom expression across the menstrual cycle. Indeed, one study examining menstrual cycle effects on mood in 34 Turkish women taking lithium or valproate for bipolar disorder and 35 healthy controls found greater mood variability across the cycle in healthy controls. 44 Properly treated bipolar disorder can stabilize naturally occurring menstrual effects on mood, 49 which may explain why large studies often find little or no effects of the menstrual cycle on symptom expression. Alternatively, it is possible that, in the study by Leibenluft and colleagues, 42 mood changes among women with rapid-cycling bipolar disorder occurred too quickly to demonstrate any effect of the menstrual cycle.

Only one study met criteria for being considered high quality. This study found greater mood variability across the menstrual cycle in healthy controls compared to those taking lithium or valproate.

Depression is a profound and often debilitating disorder that disproportionately affects women. 200 Research suggests that interactions among emotional and behavioral sensitivities to fluctuations in ovarian hormones across the menstrual cycle may be, for some women (e.g., see Schmidt et al. 201 ), primary factors leading to depressive symptoms. The concept of premenstrual mood worsening has been a focus of investigation for decades, with many, 52 – 54 , 57 , 58 , 68 but not all, 7 , 55 , 56 , 156 studies finding some evidence of symptom exacerbation (i.e., mood worsening) in this phase. Indeed, women have been treated with hormonal therapy for improving premenstrual symptoms, 51 , 202 with some success. Ongoing work seeks to better characterize the nature, timing, and mechanisms of changes in depression across the menstrual cycle to help develop and evaluate additional therapies.

Epidemiological and self-report studies assessing depressive symptoms in healthy women produced inconsistent findings. In a large study of 248 adult, premenopausal women (60% White), depressive symptoms and hormone levels were measured across at least two menstrual cycles. 71 The authors found no relationship between depressive symptoms and absolute levels of hormonal changes across the menstrual cycle, although women with more depressive symptoms also had worse premenstrual mood changes. Premenstrual mood worsening was also found in an earlier study, 59 and healthy women have reported higher symptoms of depression in both early and late follicular phases compared to the mid-luteal phase. 70 Another study, however, showed no relationship between menstrual cycle phase and negative affect. 74 It is possible that premenstrual mood worsening is related to decreases in reward responsivity. In general, reward responsivity (a possible biomarker for depression, with low reward sensitivity correlated with depression), appears to be highest in the follicular phase and lowest in the luteal phase. 76 , 77 , 191 , 192 , 203 Women in the late luteal phase of the menstrual cycle may also have difficulty with emotion perception as evidenced by neural differences in brain activation compared to men, 77 which could make them vulnerable to depression at this time. Another possibility is that individual women experience different patterns of mood changes throughout the cycle, with the consequence that analyzing data as a group may obfuscate individual differences. 69

An early study found that women with a history of treated depression reported greater symptoms of depression across the menstrual cycle compared to women without a history of depression. There was no phase-specific pattern in this sample; however, these data were collected retrospectively, which may have introduced recall bias. 61 In women with major depression, one study compared menstrual cycle phases at the time of psychiatric admissions, but these data were inconsistent 54 and did not reveal a clear pattern. In a large, community-based sample of 900 girls and women ages 13–54 years (n = 111 Black; n = 121 Hispanic), participants underwent diagnostic interviews and tracked their moods for two cycles. 65 Fifty-two participants had clinical (i.e., major depressive disorder or dysthymia) or subclinical depression. Women with clinical and subclinical depression reported greater symptom exacerbation in the premenstrual phase than did nondepressed women, though premenstrual symptom exacerbation was observed in all participants. Symptom worsening during the follicular phase was highest for clinically depressed, moderate for subclinically depressed, and low for non-depressed women. These data suggest that women with depression are at increased risk of premenstrual symptom worsening, 60 possibly due to impaired estrogen-related modulation of stress reactivity; 72 however, the pattern of increased mood symptoms premenstrually is also generally true across girls and women with and without depression. 78 Similar findings were demonstrated in a sample of Chinese women with depressive disorders. 66 Luteal phase decrease in positive affect has also been shown in women with fibromyalgia and rheumatoid arthritis, 64 and women with epilepsy show lower moods and increased seizures during menstruation. 78

The use of oral contraceptives further complicates the identification of clear patterns of how the menstrual cycle affects mood. In Sweden, a randomized, controlled trial of combined oral contraception (n = 84) compared to placebo (n = 94) examined the effect on mood during three consecutive menstrual cycles. 75 Cycle phases were menstrual (days 1 to 4), premenstrual (days −7 to −1), and intermenstrual (all remaining days). The study found that oral contraceptive users reported a small, significant worsening of mood symptoms during the intermenstrual phase; however, additional analyses revealed that this effect was primarily driven by a subgroup of women with previous significant mood symptoms associated with oral contraceptive use. Other studies reported similar findings, 62 regardless of the type of oral contraceptive used. 63 Interestingly, another randomized, controlled trial in Germany found the opposite pattern: healthy, PMS-free women taking oral contraceptives reported slightly better mood across the cycle compared to naturally cycling women. 73

Data from the high-quality studies do not offer much clarity. Several found that symptoms of depression were not related to absolute hormone levels, although one study focusing on women with clinical symptoms of depression and one study in healthy women both noted a premenstrual worsening of symptoms. Others found no relationship of depression to the menstrual cycle.

Most research in this domain employs psychiatric hospitalization admission data or interviews immediately following hospitalization for suicide attempts. Research consistently demonstrates that rates of self-harm, suicide, and suicide attempts are significantly elevated during the premenstrual and menstrual phases. 81 – 87 , 92 , 93 Histopathological reports suggest rates of completed suicides during the menstrual phase to range from 25% 89 to 54%. 88 Via autopsy, Leenaars and colleagues 89 and Dogra and colleagues 88 compared menstrual cycle phase at time of death in women who died by suicide versus other causes (e.g., motor vehicle accident). Despite the large difference in rates of suicides occurring during menstruation noted above, both studies reported similar rates of death by other causes during menstruation: 4.5% and 6.75%, respectively. Nonetheless, a disproportionate number of suicides occur during the menstrual phase compared to other phases of the menstrual cycle and other causes of death.

These rates are largely consistent with those seen in suicide attempts, with reported rates of attempts occurring during the menstrual phase ranging between 26% 84 and 42%. 79 , 84 Research grouping the premenstrual and menstrual phases reported rates of 47%, 80 and one study reported luteal-phase attempt rates as high as 67%. 91 Baca-García and colleagues 83 posit that women with histories of diagnosed psychiatric disorders are five times more likely to attempt suicide during the menstrual phase than those with no such history. Variations in rates of suicide attempts across the menstrual cycle appear to be unique to naturally cycling women. Fourestié and colleagues 79 found that, in a sample of 108 French women (35 naturally cycling) who attempted suicide, 42% of naturally cycling women attempted suicide while menstruating and 12% attempted during the premenstrual phase. They did not find associations with cycle phase and suicide among women using hormonal contraceptives.

Suicide is highly heritable, with rates ranging from 17%–55%. 204 – 208 Furthermore, serotonergic function and the serotonin transport gene 5-HTT are highly related to suicidal behavior (as reviewed in Kenna et al. 209 ). Researchers examined the possibility of gene × hormone interactions in rates of suicide and suicide attempts across the menstrual cycle. Baca-García and colleagues 84 assessed the role of allele variants in rates of suicide attempts among 104 naturally cycling, White women. Serum assays indicated that, of these women, 17 had two long alleles, 38 had two short alleles, and 49 had one long and one short allele. Among women with two long alleles, a significant proportion of suicide attempts occurred during the menstrual phase (41%). Furthermore, estradiol levels were significantly lower in women with long rather than short alleles. No significant phasic differences emerged for women with two short alleles. As such, it is possible that genetic vulnerabilities may underlie menstrual cycle exacerbations of suicidal behavior.

Eleven studies met criteria for being considered high quality, with nine studies indicating that rates of completed or attempted suicide were highest during menstruation and two studies indicating these rates were highest premenstrually.

Anxiety and Anxiety Disorders I: Anxiety and Stress

Symptoms of anxiety and stress have been examined in healthy women’s menstrual cycles across a variety of laboratory protocols. Research focusing on daily symptoms of anxiety, as well as anxiety in response to stressors, has found clear premenstrual exacerbations of anxiety, 60 , 67 , 94 – 96 , 103 , 115 although six studies (three of which had very small sample sizes) found no significant changes in anxiety related to the menstrual cycle. 55 , 56 , 98 , 99 , 102 , 113 Some research found a divergence between self-reported stress and cortisol responses to stress in healthy women, 98 , 107 while high levels of trait anxiety in women are associated with cortisol only in the follicular phase. 105 State and trait anxiety were related to daily reports of anxiety during the luteal phase compared to the follicular phase in a healthy sample of 203 women, 104 although another study showed that women high in state and trait anxiety do not show changes in symptoms across their menstrual cycles. 100 In the one study that examined the role of ovulation during the menstrual cycle, there were no differences in symptoms, regardless of whether the cycle was ovulatory or anovulatory. 68 A separate study assessing acoustic startle responses in women (as a proxy for anxiety) found larger startle magnitudes during ovulation and the late luteal phase, suggesting a potential vulnerability to anxiety during these phases. 109 Other research has also shown higher levels of anxiety in response to stressors, including exercise, 119 during the luteal phase. 108

Other psychological characteristics representative of anxiety have been explored as factors contributing to menstrual cycle symptom severity. Lower perceived levels of control over anxiety have been correlated with higher levels of menstrual severity, 111 which may be related to overall difficulty regulating emotions. 112 Similarly, higher levels of health anxiety are associated with increased perceived stress, but only during the late luteal phase of the menstrual cycle. 118 One recent study found that calmness was highest during the late luteal and menstrual phases; however, anxiety moderated the relationship between irritability and cycle phase, such that highly anxious women were more irritable during the late luteal and menstrual phases, when estrogen and progesterone levels are low. 114 Chronic anxiety may therefore be a risk factor for more severe premenstrual and menstrual symptoms. 210 In fact, high levels of estrogen may serve as a protective factor against psychosocial stress, as evidenced by changes in brain activation 110 and cardiovascular responses to stress. 106 Evidence also suggests that anxiety may be directly related to progesterone levels across the menstrual cycle, 116 although one recent study found no relationship of anxious jealousy to progesterone levels across the menstrual cycle. 117

Despite relatively consistent evidence of premenstrual anxiety exacerbation across the menstrual cycle, data also suggest that some women experience symptom exacerbation at mid-cycle and decreased symptoms premenstrually. 52 , 69 In a sample of 213 young women attending college in Italy, participant responses were separated into four groups using cluster analysis, with two of the groups suggesting a “classic” PMS pattern, one group revealing a non-cyclic pattern, and the last group suggesting the mid-cycle pattern described earlier. 69 This study represents an important step to examining individual differences across the menstrual cycle, rather than just assuming women experience similar changes across cycle phases—an approach demonstrated by earlier research comparing community volunteers with women who reported high levels of premenstrual symptoms. 101

Although most studies have excluded women using oral contraceptives, several studies have compared anxiety in women who were and were not using exogenous hormones. Generally, naturally cycling women demonstrate the expected pattern of anxiety (higher during menstrual and premenstrual phases), whereas women using oral contraceptives showed no change in anxiety. 97 , 98 One small study showed no cycle-phase or group differences in anxiety in women who were and were not taking oral contraceptives. 56

When examining high-quality studies only, the majority found no effect of menstrual cycle phase on anxiety, although women with high baseline anxiety seemed to experience more symptoms in the luteal phase.

Anxiety and Anxiety Disorders II: Generalized Anxiety Disorder

A recent study compared women with and without generalized anxiety disorder (GAD) on measures of mental and physical fatigue during the early follicular and mid-luteal phases. 121 The only difference to emerge was that women with GAD had higher mental fatigue in the early follicular phase. Furthermore, salivary estradiol and progesterone were not associated with measures of fatigue during any cycle phase. This is consistent with earlier research demonstrating increased symptoms of depression, anxiety, and hostility in women with GAD; however, symptom exacerbation was even greater for women with GAD and PMS, particularly in the premenstrual phase. 120

Neither study met our criteria for being considered high quality.

Anxiety and Anxiety Disorders III: Social Anxiety Disorder

In women with social anxiety disorder, higher social anxiety and avoidance were reported in the premenstrual phase of the menstrual cycle (week 4) compared to the three previous weeks. 122 In a separate study of Chinese women, salivary progesterone was positively correlated with self-reported social feedback sensitivity, regardless of menstrual cycle phase (late follicular or mid-luteal). 124 This finding is supported by a previous study in which progesterone levels were associated with increased attention to social stimuli. 123

Analysis of high-quality studies revealed no cycle effect of interpersonal sensitivity in healthy women, although the luteal phase was associated with greater attention to social stimuli and higher interpersonal anxiety.

Anxiety and Anxiety Disorders IV: Panic Disorder

One of the earliest studies compared retrospective versus prospective reports of anxiety and panic in a small sample of adult women with panic disorder. 127 Interestingly, most women (79%; n = 15) retrospectively reported worsening anxiety symptoms premenstrually, but prospective self-reported anxiety and daily frequency of panic attacks were similar pre- to postmenstrually. These data mirror two earlier published reports, 125 , 126 although another study found that prospective report of panic and anxiety clearly demonstrated premenstrual exacerbation of symptoms. 128

Research on mechanisms of panic disorder, such as anxiety sensitivity, has produced some important findings. 211 In a 1996 study, 337 college women were screened for anxiety sensitivity, and the lower and upper quartiles of respondents participated during either the intermenstrual (days 8 to 22) or premenstrual (days 24 to 28) phase of their cycles. 130 Women in the high anxiety-sensitivity group demonstrated elevated skin conductance reactivity to anxiety-provoking scenes in the premenstrual phase compared to those with low anxiety sensitivity or those in other phases of the menstrual cycle. The authors propose that these data were the first to link both state (menstrual cycle phase) and trait (high anxiety sensitivity) factors that may contribute to vulnerability to panic in women. Similar patterns were found in more recent studies in healthy women with high/low anxiety sensitivity. 134 Additionally, women with high anxiety sensitivity report more menstrual-related symptoms, 131 and women with asthma with or without panic disorder report more state anxiety, 133 regardless of cycle phase. 135

Laboratory-based studies often use what is known as a “CO2 challenge” to evaluate reactivity to the sensation of difficulty breathing. 212 , 213 In this approach, participants inhale a full lung capacity of a gas mixture (typically 35% CO 2 /65% O 2 ) and rate their levels of anxiety. One of the first studies to explore menstrual cycle effects in this paradigm found that women with panic disorder experience significantly more reactivity during the early follicular phase (day 4 of the menstrual cycle) than during the mid-luteal phase (8 days prior to menstruation), whereas healthy women show no differences across the menstrual cycle. 129 A separate laboratory study demonstrated elevated skin conductance responses to anxiety-provoking stimuli during the premenstrual phase in women with panic disorder compared to women without. 132

Two studies of healthy women were considered to be of high quality. They both found no menstrual cycle effects for anxiety generally, although high anxiety sensitivity was associated with higher cognitive panic symptoms in the premenstrual phase.

Obsessive-Compulsive and Related Disorders I: Obsessive-Compulsive Disorder

Early retrospective studies suggested a link between exacerbation of OCD symptoms and the premenstrual phase, specifically. 136 , 137 The first study to evaluate this relationship prospectively included 101 women who met diagnostic criteria for OCD. 138 Approximately half of the women reported premenstrual worsening of OCD symptoms, as demonstrated by significantly higher scores on a self-report measure of OCD symptoms. In a laboratory-based study designed to measure OCD-related checking symptoms, no menstrual cycle phase differences (comparing mid-luteal and mid-follicular) were identified. 139

Only one study meeting criteria for high quality found no differences in checking behaviors.

Obsessive-Compulsive and Related Disorders II: Trichotillomania (Hairpulling)

A single published study explored the relationships among menstrual cycle phases and trichotillomania in 59 adult women. 140 Participants were retrospectively asked whether they believed whether their menstrual cycles and hairpulling were related, and 53.3% indicated that they were. Participants also reported a clear effect of menstrual phase when asked to indicate symptoms premenstrually, during menstruation, and postmenstrually, such that greater frequency and intensity of urges, greater frequency of hairpulling, and decreased ability to control hairpulling were all significantly higher in the premenstrual phase than in the other phases.

The one identified study did not meet our criteria for being considered high quality.

Posttraumatic Stress Disorder

Assessment of trauma and PTSD symptoms and their relationship to the menstrual cycle is complicated by differences in when the trauma occurred and when it was assessed . One study looked at this specific issue in a large sample of women (n = 147) with various types of trauma, including motor vehicle accidents, falls, and nonsexual assaults. 141 Based on retrospective self-reports of their last menstrual periods, women who were in the mid-luteal phase at the time of the trauma (20% of the sample) or at the time of assessment (16% of the sample) reported experiencing significantly more frequent and severe flashbacks than women who experienced trauma in other cycle phases. This finding remained even after controlling for number of days in the hospital, injury severity, age, trauma type, and mild traumatic brain injury. In healthy populations, 142 women exposed to a distressing film during the early luteal phase were more likely to experience intrusive memories of the film in the days following than women who watched the film in the mid-follicular or late luteal phases (when progesterone levels are low). 143 Furthermore, the frequency of intrusions was negatively correlated with the estrogen-to-progesterone ratio, suggesting that estrogen may have protective effects and that both hormones may be important for encoding distressing memories. Similar results have been demonstrated in other intrusive-memory paradigms. 144 In the luteal phase, when the estrogen-to-progesterone ratio is lower than in the follicular phase, women may experience more intrusive memories and impaired fear inhibition. 2

Laboratory assessment of fear also sheds light on biomarkers of PTSD. Prepulse inhibition is a neurobiological process typically assessed through a startle-response paradigm. A weaker version of the stimulus (prepulse) is administered prior to a startle stimulus (pulse), which results in a decreased startle response compared to when no prepulse is administered. This paradigm aims to measure the brain’s ability to effectively filter interruptions (i.e., the startle stimulus) from ongoing processing of the prepulse stimulus. 214 Pineles and colleagues 146 examined prepulse inhibition in women with PTSD and trauma-exposed women without PTSD in the early follicular and mid-luteal phases of the menstrual cycle. Although group differences were found, there were no main effects or interactions with menstrual cycle phase, estradiol, or progesterone levels, suggesting that menstrual cycle phase was not associated with prepulse inhibition. The authors suggest that these null results may indicate that prepulse inhibition evokes early stages of information processing that may not be influenced by the menstrual cycle. Other studies comparing women with PTSD to trauma-exposed women without PTSD found that deficits in extinction learning (i.e., learning such that a stimulus previously associated with a shock is no longer associated with the shock) were present in the mid-luteal phase but only for women with PTSD. 147 , 150 A possible explanation for this deficit is that estrogen, which is lower in the luteal than late follicular phase, may be important for higher-order processes such as extinction learning. Relatedly, it is possible that women with PTSD have deficits in the conversion of progesterone, which is typically higher in the luteal phase, to the GABAergic neurosteroid allopregnanolone, which affects differential fear conditioning and extinction. 148 Although GABA plasma levels appear to be positively correlated with PTSD symptoms in women with PTSD compared to trauma-exposed healthy controls, menstrual cycle phase was not related to GABA levels in either group, according to a recent study. 151

The course of other symptoms of PTSD across the menstrual cycle is not entirely clear. Anxiety sensitivity (i.e., fear of the physical symptoms of anxiety), for example, appears to be stable across the menstrual cycle in women with and without PTSD. 149 However, interpersonal sensitivity, depression, anxiety, hostility, and phobic anxiety are significantly higher in women with PTSD than in those without. 145 Women with PTSD report more phobic anxiety in the early follicular phase compared to the mid-luteal phase, and women without PTSD report no changes across the menstrual cycle. 145

Three studies met our established criteria for high-quality evidence, and all of these explored intrusive memories in healthy women. The data suggest that intrusive memories are more frequent in the luteal phase and when estradiol levels are low.

Eating Disorders

Much research examining eating disorders across the menstrual cycle focuses on binge eating rather than caloric restriction or compensatory behaviors (e.g., purging). As such, results presented in this review surround emotional and binge eating. Research examining emotional or binge eating appears to indicate consistent cyclic effects (see Fowler et al. 164 and Leon et al. 152 ). In both clinical and community-based studies of women diagnosed with bulimia nervosa, significant increases in binge eating were reported during the mid-luteal and premenstrual phases. 153 – 155 , 159 Similar results were noted in community samples of women without diagnosed eating disorders. 157 , 158 , 160 – 162 In a convenience sample of 148 women (84% White), naturally cycling women (n = 67) reported increased hunger during the menstrual phase and increased food cravings and amount of food eaten during both the premenstrual and menstrual phases. 158 Women using hormonal contraceptives (n = 81) demonstrated the same pattern with the addition of increased hunger during the premenstrual phase.

To better understand the biological underpinnings of these fluctuations, researchers investigated associations among progesterone, estradiol, and eating behaviors. In a sample of nine women with bulimia nervosa and eight healthy controls (82.4% White), Edler and colleagues 155 found significant negative associations between binge eating and estradiol, and significant positive associations between binge eating and progesterone. Similarly, Baker and colleagues 163 reported that, when women had low progesterone levels, an inverse relationship between estradiol and body dissatisfaction emerged. When progesterone levels were high, however, positive relationships among estradiol, body dissatisfaction, and binge eating emerged.

Data from the three high-quality studies do not reflect a consistent pattern, with two studies reporting no direct hormonal associations, and one study reporting positive associations, among emotional eating, progesterone, and estradiol.

Borderline Personality Disorder

BPD is characterized by intense and frequent emotional dysregulation, often resulting in anger and aggressive behavior toward others. Individuals with BPD are highly sensitive to criticism and may experience intense mood fluctuations throughout a day. However, few studies examined the role of the menstrual cycle or ovarian hormones in BPD symptoms. In a convenience sample of 226 undergraduate women, researchers found that women using oral contraceptives endorsed significantly more BPD symptoms on a self-report questionnaire. 165 Moreover, the phase of the menstrual cycle when estrogen was rising (days 5 to 10; mid- to late-follicular phase) was associated with more symptoms than those in a low-estrogen phase (days 0 to 3 and 26 to 29). This association was confirmed in a second study reported in the same article that measured salivary estradiol and found a significant positive relationship between rising, but not absolute, estrogen levels and BPD symptoms.

Subsequent studies sought to better understand the relationship between estrogen-to-progesterone ratios and key symptoms. One study found that within-person higher-than-average progesterone levels and lower-than-average estrogen levels predicted increased symptoms for women with high baseline BPD symptoms. 166 In another study of women with BPD, symptoms were generally worse in the perimenstrual phase than mid-luteal, ovulatory, and follicular phases. High-arousal symptoms (e.g., anger) returned to baseline, however, in the early follicular phase (i.e., when estrogen levels are low), whereas low-arousal symptoms (e.g., depression) persisted until ovulation (i.e., when estrogen levels are high). 167 In this same sample, anger/irritability was highest in the perimenstrual phase, with reactive aggression highest in the mid-luteal phase and proactive aggression highest during ovulation and lowest perimenstrually. 168

Two high-quality studies of BPD symptoms in healthy women suggest that BPD symptoms may change as a function of variability in estradiol and progesterone, as opposed to being associated with absolute ovarian hormone levels.

Substance Use Disorders I: Alcohol Use

The literature assessing the relationship between alcohol use and the menstrual cycle is mixed. 169 – 171 In a 2015 meta-analysis, Carroll and colleagues 12 found that 7 of the 13 identified articles reported increased drinking during the premenstrual phase, one reported decreased drinking during the premenstrual phase, and five reported no significant menstrual cycle effects. More recent research suggests that drinking may indeed fluctuate across the menstrual cycle, and this may be linked with progesterone-to-estradiol ratios. In a study by Joyce and colleagues, 173 94 naturally cycling women (76.6% White) documented the quantity of alcohol consumed and reasons for drinking across a full menstrual cycle. Women reported slight increases in drinking during the premenstrual and menstrual phases, and motivations related to coping were significantly associated with these increases (see also Hayaki et al. 174 ). Similarly, social motivations (e.g., “because [drinking] makes social gatherings more fun”) were associated with alcohol consumption around ovulation. 173 These findings align with Martel and colleagues’ work 172 demonstrating increases in drinking and binge drinking during the premenstrual phase and ovulation. High levels of estradiol predicted alcohol consumption, and these effects increased when progesterone was low and decreased when progesterone was high. Mood may also moderate these effects. Research shows that, when progesterone is low, women are more likely to drink when their mood is negative and that, when progesterone is high, women are more likely to drink when their mood is positive. 175 Taken together, recent research has elucidated that menstrual cycle effects may be moderated by positive or negative affect.

The two high-quality studies reported relatively consistent findings. When progesterone is low in the premenstrual and menstrual phases, alcohol consumption appears to be associated with negative mood. Around ovulation and when progesterone rises, alcohol consumption appears to be associated with positive mood.

Substance Use Disorders II: Smoking

Studies have examined menstrual cycle effects on ad lib smoking (i.e., smoking at will), subjective effects of nicotine, cravings, withdrawal symptoms, and smoking cessation. We found no consistent pattern for ad lib smoking, with studies documenting no cyclic effect, 187 increased smoking during the luteal phase, 179 , 184 or increased smoking during menstruation. 176 In a study by Schiller and colleagues, 183 98 female smokers (79% White) attended two laboratory sessions spaced two weeks apart, during which they smoked ad lib for one hour. Researchers found that women’s progesterone-to-estradiol ratios were negatively associated with smoking behavior; women with lower levels of progesterone compared to estradiol smoked more. The authors proposed that these relative levels may partly explain inconsistencies in the extant literature, as relative amounts of these hormones may be an important factor in menstrual cycle–related smoking behavior.

Progesterone may also diminish subjective effects of nicotine. 185 In a study by Goletiani and colleagues, 186 23 naturally cycling female smokers rated the subjective effects of cigarettes throughout two-hour ad lib smoking sessions twice during their menstrual cycles. No phasic effects were found on subjective effects of cigarettes. However, when data collected during the luteal phase were grouped based on progesterone levels, researchers found that women with high levels of progesterone reported significantly lower subjective effects. Similar findings show the effects of progesterone on reducing cravings. 181 , 188 , 189 , 215

Conversely, symptoms of nicotine withdrawal appear to be highest during the luteal phase. 177 , 178 This increase may be related to premenstrual symptoms, which are greater during the luteal phase and include symptoms like those of nicotine withdrawal (e.g., fatigue, headache, anxiety; see Weinberger et al. 216 ). It is unclear whether this is related to the effectiveness of smoking cessation (e.g., quit attempts); however, while the research is limited, there are studies that indicate superior outcomes for smoking cessation initiated during both the luteal 182 and follicular 180 phases. These studies are also limited as they vary regarding whether cessation is assisted by pharmacotherapies such as nicotine replacement, bupropion, or varenicline. Given the negative correlation between progesterone-to-estradiol ratios and smoking behavior, 183 quit attempts made during the follicular phase may be more successful.

Seven studies assessing smoking behavior across the menstrual cycle were considered high quality. Overall, these studies suggest that cravings and affective responses to nicotine are lower in the luteal phase, when progesterone levels are relatively high, compared to the follicular phase, when progesterone levels are relatively low.

Substance Use Disorders III: Cocaine Use

Like menstrual effects on nicotine use, research indicates an attenuation of subjective effects of smoked cocaine when progesterone levels are high (see Collins et al. 193 and Reed et al. 194 ). Sofuoglu and colleagues 190 reported that women (n = 21) had lower ratings of feeling “high” and “stimulated” during the luteal phase than the follicular phase. Evans and colleagues 191 similarly reported that, although women (n = 11; 91% African American) reported greater desire for cocaine during the luteal phase, their ratings of drug effects such as feeling “high,” “stimulated,” “alert,” and “self-confident” were significantly reduced compared to ratings in the follicular phase. Evans and colleagues 192 later examined subjective responses to smoked cocaine in 11 naturally cycling women (91% African American) during the follicular phase, luteal phase, and follicular phase with exogenous progesterone administration. Ten men served as a control group. Subjective effects of cocaine were significantly lower during the luteal phase and when the follicular phase was supplemented with exogenous progesterone compared to the follicular phase and men’s responses. No differences were found between follicular phase responses and men’s responses, which continues to suggest that progesterone modulates subjective responses to smoked cocaine.

To assess the potential role of allopregnanolone, a progesterone metabolite, on cocaine cravings, Milivojevic and associates 195 randomized 46 cocaine-dependent men and women (n’s of 29 and 17, respectively; 73.9% African American) to receive either a progesterone supplement or placebo, and measured blood concentrations of allopregnanolone and self-reported cravings. They found that, through increases in allopregnanolone, those who received progesterone supplementation reported significantly lower cravings than those who received placebo.

No study in this section met our criteria for being of high quality.

Miscellaneous Disorders

We found only one study relating to symptom fluctuation in a woman with kleptomania. 217 The authors did not formally test her symptoms during different cycle phases, but the patient reported experiencing intensified urges to steal during the luteal phase. We felt that the data from this single case study were not sufficient to justify inclusion in the review. There were no studies available for body dysmorphic disorder, excoriation, hoarding, intermittent explosive disorder, conduct disorder, or pyromania.

The aim of this comprehensive review is to describe the findings of previous research examining psychiatric symptom variability across the menstrual cycle. Each study included (1) a comparison of at least two menstrual cycle phases, (2) data not derived from evaluation of an intervention, and (3) premenopausal women, age 18 years or older. Across psychiatric diagnoses, we saw evidence of symptom exacerbation primarily in the luteal, premenstrual, and menstrual phases.

Evidence of Potential Mechanisms Involved in Symptom Fluctuation

Several studies included in this review examined possible mechanisms that may underlie menstrual-related changes in symptoms. Regarding depression, for example, the literature indicates that both healthy women and women with a depressive disorder experience perimenstrual increase of depressive symptoms. However, data exploring potential mechanisms are not as clear. fMRI studies examining functional brain changes show inconsistent results. 218 , 219 Similarly, estradiol may help regulate stress for healthy women but not for women who have experienced clinical depression, 72 suggesting that even a history of depression could make women vulnerable to increased perimenstrual mood changes. 61 These data highlight the complexities of determining how depression changes over the course of the menstrual cycle and whether these processes may be different between healthy and clinical populations, perhaps suggesting the need for different treatment approaches. 220 , 221

Ovarian Hormone Mechanisms

Several studies hypothesized a specific link between ovarian hormones and symptom fluctuation. Regarding substance use, the decrease in cravings in the luteal phase may be due, in part, to increases in allopregnanolone, a progesterone-derived neuroactive steroid. Allopregnanolone produces anxiolytic and hypnotic effects via increased GABA potentiation (see Lambert et al. 21 for a review), which could lessen the subjective effects (e.g., reportedly feeling “high,” “stimulated”) of substances such as cocaine. High levels of progesterone in the mid-luteal phase are also associated with release of glucocorticoids, which help consolidate memories, potentially increasing susceptibility to developing PTSD. 18 , 141 Once symptoms of PTSD have developed, reductions in the conversion of progesterone to allopregnanolone and pregnanolone can further impair learning of new, non-fearful associations. 150 These and other studies have led to the hypothesis that allopregnanolone-to-progesterone ratios may be better biomarkers for psychiatric symptoms, as decreases in this ratio from the follicular to luteal phases are evident despite increases in absolute levels across the menstrual cycle. 222 Impairments in allopregnanolone synthesis may further impair GABAergic function and leave some women at risk for psychiatric disorders, 20 , 22 although this causal link has not yet been clearly demonstrated.

Additionally, according to the estrogen hypothesis, estrogen is protective against psychosis (see Reilly et al. 15 for a review). Reductions in estrogen can facilitate or exacerbate psychosis—which is exemplified by the increased risk of psychosis in postmenopausal and postpartum periods. It is therefore consistent with these data that increases in psychotic experiences tend to occur as estrogen levels decline throughout the premenstrual phase.

Regarding cyclic effects on emotional and binge eating, there are likely both hormonal and genetic underpinnings of these behaviors. The pattern of decreased food intake during the first half of the menstrual cycle and increased food intake during the second half of the menstrual cycle is observed in many mammalian species (see Schneider et al. 223 for a review). From an evolutionary standpoint, it is theorized that this pattern allows for a shift in motivational priorities from reproduction to eating. During the first half of the menstrual cycle, motivational priorities surround increasing sexual desire as ovulation approaches. As the likelihood of conception decreases the further from ovulation a woman is in her cycle, motivational priorities shift toward eating. 223 One hypothesis is that a gene × hormone effect could exaggerate this process in women who have binge-eating behavior. 161 For example, an individual who is genetically vulnerable to binge eating may experience increased activation by certain concentrations of estradiol and progesterone during the luteal phase of the menstrual cycle compared to someone without this genetic predisposition.

Finally, the ratio between progesterone and estradiol appears to play an important role in symptom expression, though not in a consistent direction. Lower levels of progesterone coupled with higher levels of estradiol, for example, have been associated with increased smoking, 183 alcohol consumption, 172 and body dissatisfaction. 163 Conversely, higher levels of progesterone compared to estradiol has been associated with increases in intrusive memories in the context of PTSD. 143

Although we did not examine reproductive mood disorders (e.g., premenstrual dysphoric disorder, postpartum depression, perimenopausal depression) specifically, the literature in these areas suggests that there is a large amount of individual variability in mood sensitivity to ovarian hormones. 201 , 224 – 226 Indeed, it is likely that a minority of women exhibit psychological sensitivity to ovarian hormones across the menstrual cycle and that collapsing participant data into groups may mask this variability. As such, researchers are encouraged to assess for subgroups when analyzing menstrual cycle data.

It is also possible that psychiatric symptoms may be a delayed response to hormonal changes and may therefore not reflect the hormonal phase when the symptoms arise. For example, symptoms that have onset in the luteal phase may be in response to increases in estradiol or progesterone, and symptoms in the early follicular phase may be a response to hormone withdrawal. Indeed, research suggests that some symptoms may peak several weeks following exogenous hormonal manipulation. 201 Schmidt and colleagues 226 argue that menstrual cycle studies may not be able to accurately tease apart the effect of hormonal changes versus absolute hormone levels on psychiatric symptoms, and they encourages the use of hormonal manipulation to address this possible limitation.

Results from High-Quality Studies

Overall, a paucity of studies met our criteria to be considered of high quality. Of the 16 areas examined, three (GAD, trichotillomania, and cocaine use) had no high-quality studies, and three (psychosis, bipolar disorders, and OCD) each had one high-quality study. As such, no strong conclusions can be made for these psychiatric disorders/symptoms. It is recommended that future research continue to explore the presentation of these psychiatric disorders/symptoms across the menstrual cycle. Furthermore, researchers are encouraged to use the flexible design recommendations made by Schmalenberger and colleagues 227 (e.g., using within-subjects designs, incorporating ovulation predictor testing) to enhance study quality and validity of results.

Social anxiety disorder, PTSD, alcohol use, and smoking each had two to three high-quality studies demonstrating relatively consistent results. For social anxiety disorder and PTSD, it appears as though symptoms may worsen in the luteal phase (e.g., greater interpersonal anxiety, more frequent intrusive memories). Alcohol use across the menstrual cycle appears to be influenced by mood such that alcohol use is associated with negative mood in the premenstrual and menstrual phases and is associated with positive mood around ovulation. Regarding smoking, cravings appear to be lower in the luteal phase than in the follicular phase. This may help to explain why quit attempts appear to be more successful when made during the follicular phase: if women experience a reduction in cravings in the luteal phase following quit attempts made during the follicular phase, women may be able to sustain these attempts for a longer period. 178 Cumulatively, these results provide a preliminary understanding of the effect of the menstrual cycle on these symptoms, and more high-quality research within each area is needed.

Results from the two to three high-quality studies for each of the following diagnoses—panic disorder, eating disorders, and BPD—were unclear or inconsistent, and more research is needed to determine the effect of the menstrual cycle on these disorders. We identified 7 high-quality studies examining depression and 14 examining anxiety/stress. Similarly, results from these areas were unclear or inconsistent. Given that the studies in these two areas yielded inconsistent results, it may be that the menstrual cycle has no consistent effects on symptom expression in these areas, although a history of depression or higher levels of baseline depression or anxiety may be a risk factor for menstrual cycle–related exacerbation of symptoms. Alternatively, it may be that the ways in which any menstrual cycle effect is expressed are nuanced and possibly masked by the varying populations and study designs in these studies. Future research aimed at replicating study designs used in these high-quality studies would help elucidate any true menstrual cycle effects.

Research on suicide/suicide attempts was the only area sufficiently studied, with 11 studies meeting criteria to be considered high-quality and yielding consistent results. Overall, results from these studies indicate that rates of suicide/suicide attempts are highest during menstruation.

Limitations

Overall, the data on psychiatric symptoms across the menstrual cycle are limited because of the lack of prospective studies of women with a range of psychiatric disorders in which standardized assessment of the menstrual cycle is collected. This limits our understanding and knowledge of these phenomena, including the implications for specific disorders and the investigation of underlying mechanisms. Varied definitions and assessment of menstrual cycle phases, as well as a lack of standardized assessments of the menstrual cycle—including biological assays of estrogen, estradiol, allopregnanolone, and progesterone, along with their relative ratios, on and off oral contraceptives—all limit existing information. Inconsistencies in the literature may also result from individual variation (see Kiesner 69 for a review). In addition, studies vary in standardization of symptoms for specific psychiatric disorders, further limiting assessment of the existing data.

Varied definitions and assessment of menstrual cycle phases

Menstrual cycle phases are termed and calculated differently across research groups, which may mask or inflate true symptom variability. For example, some researchers define ovulation as a distinct phase or window of days, 174 and others include ovulation as a part of the luteal phase. 164 Assessment of cycle phase is further complicated when comparing studies using hormonal measures and those employing self-reports. Assessing menstrual phases through self-reported days since menstruation is complicated by the known variability in the length of menstrual cycle phases. 196 As such, 16 days since menstruation could, for example, fall during the luteal phase for one woman, coincide with ovulation for another, and, in the case of short luteal phase disorder, fall during the follicular phase for yet another woman.

Lack of standardized assessment of symptoms

Across multiple disorders, symptom assessment is inconsistent. In some studies, researchers employ self-reports, others use behavioral tasks, and still others use observations. These inconsistencies could complicate findings as research indicates there are discrepancies across some forms of symptom assessment. 228 Additionally, psychiatric disorders are comprised of constellations of symptoms, all of which may change independently throughout the menstrual cycle. Laboratory paradigms use tasks as proxies for stressors or psychophysiological measures. Taken together, these factors create a highly complex and nuanced picture that can be hard to interpret. Just as one example, in BPD, high- and low-arousal symptoms are each affected separately by the menstrual cycle; teasing apart arousal symptoms may shed additional light on cycle-related and hormonal relationships. 229 Across disorders, better understanding these relationships may help inform treatment options or guidance. 230 , 231

Sampling biases

With few exceptions, 191 , 192 , 195 most studies reviewed either included samples of primarily White women or did not provide information on participants’ racial and ethnic backgrounds. Given the failure to include this information and the general lack of representation of racial and ethnic minority groups, the generalizability of these findings is an open question. Women from underrepresented groups may experience symptom changes differently, either because of the experience of different acute or chronic stressors (e.g., racial trauma) or because of varying cultural interpretations of symptoms or menstruation.

CONCLUSIONS AND FUTURE DIRECTIONS

The existing literature demonstrates that menstrual-related exacerbation of psychiatric symptoms occurs most commonly during the premenstrual and menstrual phases, and that, for some symptoms, progesterone-to-estradiol ratios play important roles in this relationship. Effective treatment for women with psychiatric disorders will require an understanding of the role of ovarian hormones and other neuroactive steroids such as allopregnanolone but perhaps others as well. 13 To further elucidate the role of ovarian hormones in psychiatric symptom expression, researchers are encouraged to employ prospective designs and incorporate hormone assays in their relevant research, as research has found that retrospective 140 and self-reported 232 assessments of the menstrual cycle are less accurate, which may obfuscate potential findings. Given the mixed findings on the influence of oral contraceptives on mood symptoms, 73 , 75 as well as research indicating the likely importance of progesterone-to-estrogen ratios, researchers are also encouraged to further assess the effects of various types of oral contraceptives (e.g., androgenic vs. antiandrogenic; high vs. low doses of ethinylestradiol) on psychiatric symptoms. These findings would better highlight the roles of progesterone and estrogen in women’s mental health, and also possibly identify oral contraceptives that may assist in symptom stabilization. Furthermore, given the lack of diversity in the included samples, future research should focus on women from racially and ethnically diverse backgrounds to assess the generalizability of these results. Clinicians should also be routinely assessing symptom variability across the menstrual cycle in their patients. The ability to predict worsening of symptoms allows clients to better prepare and utilize effective coping strategies to help manage emotional changes. Clinicians are further encouraged to assess other factors that influence ovarian hormone expression, such as pregnancy status and hormonal contraceptive use.

Declaration of interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Original manuscript received 9 August 2021; revised manuscript received 19 November 2021, accepted for publication 12 December 2021.

Harvard Review of Psychiatry offers CME for readers who complete questions about featured articles. Questions can be accessed from the Harvard Review of Psychiatry website ( www.harvardreviewofpsychiatry.org ) by clicking the CME tab. Please read the featured article and then log into the website for this educational offering. If you are already online, click here to go directly to the CME page for further information.

  • - Google Chrome

Intended for healthcare professionals

  • Access provided by Google Indexer
  • My email alerts
  • BMA member login
  • Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Home

Search form

  • Advanced search
  • Search responses
  • Search blogs
  • Effect of exercise for...

Effect of exercise for depression: systematic review and network meta-analysis of randomised controlled trials

Linked editorial.

Exercise for the treatment of depression

  • Related content
  • Peer review
  • Michael Noetel , senior lecturer 1 ,
  • Taren Sanders , senior research fellow 2 ,
  • Daniel Gallardo-Gómez , doctoral student 3 ,
  • Paul Taylor , deputy head of school 4 ,
  • Borja del Pozo Cruz , associate professor 5 6 ,
  • Daniel van den Hoek , senior lecturer 7 ,
  • Jordan J Smith , senior lecturer 8 ,
  • John Mahoney , senior lecturer 9 ,
  • Jemima Spathis , senior lecturer 9 ,
  • Mark Moresi , lecturer 4 ,
  • Rebecca Pagano , senior lecturer 10 ,
  • Lisa Pagano , postdoctoral fellow 11 ,
  • Roberta Vasconcellos , doctoral student 2 ,
  • Hugh Arnott , masters student 2 ,
  • Benjamin Varley , doctoral student 12 ,
  • Philip Parker , pro vice chancellor research 13 ,
  • Stuart Biddle , professor 14 15 ,
  • Chris Lonsdale , deputy provost 13
  • 1 School of Psychology, University of Queensland, St Lucia, QLD 4072, Australia
  • 2 Institute for Positive Psychology and Education, Australian Catholic University, North Sydney, NSW, Australia
  • 3 Department of Physical Education and Sport, University of Seville, Seville, Spain
  • 4 School of Health and Behavioural Sciences, Australian Catholic University, Strathfield, NSW, Australia
  • 5 Department of Clinical Biomechanics and Sports Science, University of Southern Denmark, Odense, Denmark
  • 6 Biomedical Research and Innovation Institute of Cádiz (INiBICA) Research Unit, University of Cádiz, Spain
  • 7 School of Health and Behavioural Sciences, University of the Sunshine Coast, Petrie, QLD, Australia
  • 8 School of Education, University of Newcastle, Callaghan, NSW, Australia
  • 9 School of Health and Behavioural Sciences, Australian Catholic University, Banyo, QLD, Australia
  • 10 School of Education, Australian Catholic University, Strathfield, NSW, Australia
  • 11 Australian Institute of Health Innovation, Macquarie University, Macquarie Park, NSW, Australia
  • 12 Children’s Hospital Westmead Clinical School, University of Sydney, Westmead, NSW, Australia
  • 13 Australian Catholic University, North Sydney, NSW, Australia
  • 14 Centre for Health Research, University of Southern Queensland, Springfield, QLD, Australia
  • 15 Faculty of Sport and Health Science, University of Jyvaskyla, Jyvaskyla, Finland
  • Correspondence to: M Noetel m.noetel{at}uq.edu.au (or @mnoetel on Twitter)
  • Accepted 15 January 2024

Objective To identify the optimal dose and modality of exercise for treating major depressive disorder, compared with psychotherapy, antidepressants, and control conditions.

Design Systematic review and network meta-analysis.

Methods Screening, data extraction, coding, and risk of bias assessment were performed independently and in duplicate. Bayesian arm based, multilevel network meta-analyses were performed for the primary analyses. Quality of the evidence for each arm was graded using the confidence in network meta-analysis (CINeMA) online tool.

Data sources Cochrane Library, Medline, Embase, SPORTDiscus, and PsycINFO databases.

Eligibility criteria for selecting studies Any randomised trial with exercise arms for participants meeting clinical cut-offs for major depression.

Results 218 unique studies with a total of 495 arms and 14 170 participants were included. Compared with active controls (eg, usual care, placebo tablet), moderate reductions in depression were found for walking or jogging (n=1210, κ=51, Hedges’ g −0.62, 95% credible interval −0.80 to −0.45), yoga (n=1047, κ=33, g −0.55, −0.73 to −0.36), strength training (n=643, κ=22, g −0.49, −0.69 to −0.29), mixed aerobic exercises (n=1286, κ=51, g −0.43, −0.61 to −0.24), and tai chi or qigong (n=343, κ=12, g −0.42, −0.65 to −0.21). The effects of exercise were proportional to the intensity prescribed. Strength training and yoga appeared to be the most acceptable modalities. Results appeared robust to publication bias, but only one study met the Cochrane criteria for low risk of bias. As a result, confidence in accordance with CINeMA was low for walking or jogging and very low for other treatments.

Conclusions Exercise is an effective treatment for depression, with walking or jogging, yoga, and strength training more effective than other exercises, particularly when intense. Yoga and strength training were well tolerated compared with other treatments. Exercise appeared equally effective for people with and without comorbidities and with different baseline levels of depression. To mitigate expectancy effects, future studies could aim to blind participants and staff. These forms of exercise could be considered alongside psychotherapy and antidepressants as core treatments for depression.

Systematic review registration PROSPERO CRD42018118040.

Figure1

  • Download figure
  • Open in new tab
  • Download powerpoint

Introduction

Major depressive disorder is a leading cause of disability worldwide 1 and has been found to lower life satisfaction more than debt, divorce, and diabetes 2 and to exacerbate comorbidities, including heart disease, 3 anxiety, 4 and cancer. 5 Although people with major depressive disorder often respond well to drug treatments and psychotherapy, 6 7 many are resistant to treatment. 8 In addition, access to treatment for many people with depression is limited, with only 51% treatment coverage for high income countries and 20% for low and lower-middle income countries. 9 More evidence based treatments are therefore needed.

Exercise may be an effective complement or alternative to drugs and psychotherapy. 10 11 12 13 14 In addition to mental health benefits, exercise also improves a range of physical and cognitive outcomes. 15 16 17 Clinical practice guidelines in the US, UK, and Australia recommend physical activity as part of treatment for depression. 18 19 20 21 But these guidelines do not provide clear, consistent recommendations about dose or exercise modality. British guidelines recommend group exercise programmes 20 21 and offer general recommendations to increase any form of physical activity, 21 the American Psychiatric Association recommends any dose of aerobic exercise or resistance training, 20 and Australian and New Zealand guidelines suggest a combination of strength and vigorous aerobic exercises, with at least two or three bouts weekly. 19

Authors of guidelines may find it hard to provide consistent recommendations on the basis of existing mainly pairwise meta-analyses—that is, assessing a specific modality versus a specific comparator in a distinct group of participants. 12 13 22 These meta-analyses have come under scrutiny for pooling heterogeneous treatments and heterogenous comparisons leading to ambiguous effect estimates. 23 Reviews also face the opposite problem, excluding exercise treatments such as yoga, tai chi, and qigong because grouping them with strength training might be inappropriate. 23 Overviews of reviews have tried to deal with this problem by combining pairwise meta-analyses on individual treatments. A recent such overview found no differences between exercise modalities. 13 Comparing effect sizes between different pairwise meta-analyses can also lead to confusion because of differences in analytical methods used between meta-analysis, such as choice of a control to use as the referent. Network meta-analyses are a better way to precisely quantify differences between interventions as they simultaneously model the direct and indirect comparisons between interventions. 24

Network meta-analyses have been used to compare different types of psychotherapy and pharmacotherapy for depression. 6 25 26 For exercise, they have shown that dose and modality influence outcomes for cognition, 16 back pain, 15 and blood pressure. 17 Two network meta-analyses explored the effects of exercise on depression: one among older adults 27 and the other for mental health conditions. 28 Because of the inclusion criteria and search strategies used, these reviews might have been under-powered to explore moderators such as dose and modality (κ=15 and κ=71, respectively). To resolve conflicting findings in existing reviews, we comprehensively searched randomised trials on exercise for depression to ensure our review was adequately powered to identify the optimal dose and modality of exercise. For example, a large overview of reviews found effects on depression to be proportional to intensity, with vigorous exercise appearing to be better, 13 but a later meta-analysis found no such effects. 22 We explored whether recommendations differ based on participants’ sex, age, and baseline level of depression.

Given the challenges presented by behaviour change in people with depression, 29 we also identified autonomy support or behaviour change techniques that might improve the effects of intervention. 30 Behaviour change techniques such as self-monitoring and action planning have been shown to influence the effects of physical activity interventions in adults (>18 years) 31 and older adults (>60 years) 32 with differing effectiveness of techniques in different populations. We therefore tested whether any intervention components from the behaviour change technique taxonomy were associated with higher or lower intervention effects. 30 Other meta-analyses found that physical activity interventions work better when they provide people with autonomy (eg, choices, invitational language). 33 Autonomy is not well captured in the taxonomy for behaviour change technique. We therefore tested whether effects were stronger in studies that provided more autonomy support to patients. Finally, to understand the mechanism of intervention effects, such as self-confidence, affect, and physical fitness, we collated all studies that conducted formal mediation analyses.

Our findings are presented according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Network Meta-analyses (PRISMA-NMA) guidelines (see supplementary file, section S0; all supplementary files, data, and code are also available at https://osf.io/nzw6u/ ). 34 We amended our analysis strategy after registering our review; these changes were to better align with new norms established by the Cochrane Comparing Multiple Interventions Methods Group. 35 These norms were introduced between the publication of our protocol and the preparation of this manuscript. The largest change was using the confidence in network meta-analysis (CINeMA) 35 online tool instead of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) guidelines and adopting methods to facilitate assessments—for example, instead of using an omnibus test for all treatments, we assessed publication bias for each treatment compared with active controls. We also modelled acceptability (through dropout rate), which was not predefined but was adopted in response to a reviewer’s comment.

Eligibility criteria

To be eligible for inclusion, studies had to be randomised controlled trials that included exercise as a treatment for depression and included participants who met the criteria for major depressive disorder, either clinician diagnosed or identified through participant self-report as exceeding established clinical thresholds (eg, scored >13 on the Beck depression inventory-II). 36 Studies could meet these criteria when all the participants had depression or when the study reported depression outcomes for a subgroup of participants with depression at the start of the study.

We defined exercise as “planned, structured and repetitive bodily movement done to improve or maintain one or more components of physical fitness.” 37 Unlike recent reviews, 12 22 we included studies with more than one exercise arm and multifaceted interventions (eg, health and exercise counselling) as long as they contained a substantial exercise component. These trials could be included because network meta-analysis methods allows for the grouping of those interventions into homogenous nodes. Unlike the most recent Cochrane review, 12 we also included participants with physical comorbidities such as arthritis and participants with postpartum depression because the Diagnostic Statistical Manual of Mental Health Disorders , fifth edition, removed the postpartum onset specifier after that analysis was completed. 23 Studies were excluded if interventions were shorter than one week, depression was not reported as an outcome, and data were insufficient to calculate an effect size for each arm. Any comparison condition was included, allowing us to quantify the effects against established treatments (eg, selective serotonin reuptake inhibitors (SSRIs), cognitive behavioural therapy), active control conditions (usual care, placebo tablet, stretching, educational control, and social support), or waitlist control conditions. Published and unpublished studies were included, with no restrictions on language applied.

Information sources

We adapted the search strategy from the most recent Cochrane review, 12 adding keywords for yoga, tai chi, and qigong, as they met our definition for exercise. We conducted database searches, without filters or date limits, in The Cochrane Library via CENTRAL, SPORTDiscus via Embase, and Medline, Embase, and PsycINFO via Ovid. Searches of the databases were conducted on 17 December 2018 and 7 August 2020 and last updated on 3 June 2023 (see supplementary file section S1 for full search strategies). We assessed full texts of all included studies from two systematic reviews of exercise for depression. 12 22

Study selection and data collection

To select studies, we removed duplicate records in Covidence 38 and then screened each title and abstract independently and in duplicate. Conflicts were resolved through discussion or consultation with a third reviewer. The same methods were used for full text screening.

We used the Extraction 1.0 randomised controlled trial data extraction forms in Covidence. 38 Data were extracted independently and in duplicate, with conflicts resolved through discussion with a third reviewer.

For each study, we extracted a description of the interventions, including frequency, intensity, and type and time of each exercise intervention. Using the Compendium of Physical Activities, 39 we calculated the energy expenditure dose of exercise for each arm as metabolic equivalents of task (METs) min/week. Two authors evaluated each exercise intervention using the Behaviour Change Taxonomy version 1 30 for behaviour change techniques explicitly described in each exercise arm. They also rated the level of autonomy offered to participants, on a scale from 1 (no choice) to 10 (full autonomy). We also extracted descriptions of the other arms within the randomised trials, including other treatment or control conditions; participants’ age, sex, comorbidities, and baseline severity of depressive symptoms; and each trial’s location and whether or not the trial was funded.

Risk of bias in individual studies

We used Cochrane’s risk of bias tool for randomised controlled trials. 40 Risk of bias was rated independently and in duplicate, with conflicts resolved through discussion with a third reviewer.

Summary measures and synthesis

For main and moderation analyses, we used bayesian arm based multilevel network meta-analysis models. 41 All network meta-analytical approaches allow users to assess the effects of treatments against a range of comparisons. The bayesian arm based models allowed us to also assess the influence of hypothesised moderators, such as intensity, dose, age, and sex. Many network meta-analyses use contrast based methods, comparing post-test scores between study arms. 41 Arm based meta-analyses instead describe the population-averaged absolute effect size for each treatment arm (ie, each arm’s change score). 41 As a result, the summary measure we used was the standardised mean change from baseline, calculated as standardised mean differences with correction for small studies (Hedges’ g). In keeping with the norms from the included studies, effect sizes describe treatment effects on depression, such that larger negative numbers represent stronger effects on symptoms. Using National Institute for Health and Care Excellence guidelines, 42 we standardised change scores for different depression scales (eg, Beck depression inventory, Hamilton depression rating scale) using an internal reference standard for each scale (for each scale, the average of pooled standard deviations at baseline) reported in our meta-analysis. Because depression scores generally show regression to the mean, even in control conditions, we present effect sizes as improvements beyond active control conditions. This convention makes our results comparable to existing, contrast based meta-analyses.

Active control conditions (usual care, placebo tablet, stretching, educational control, and social support) were grouped to increase power for moderation analyses, for parsimony in the network graph, and because they all showed similar arm based pooled effect sizes (Hedges’ g between −0.93 and −1.00 for all, with no statistically significant differences). We separated waitlist control from these active control conditions because it typically shows poorer effects in treatment for depression. 43

Bayesian meta-analyses were conducted in R 44 using the brms package. 45 We preregistered informative priors based on the distributional parameters of our meta-analytical model. 46 We nested effects within arms to manage dependency between multiple effect sizes from the same participants. 46 For example, if one study reported two self-reported measures of depression, or reported both self-report and clinician rated depression, we nested these effect sizes within the arm to account for both pieces of information while controlling for dependency between effects. 46 Finally, we compared absolute effect sizes against a standardised minimum clinically important difference, 0.5 standard deviations of the change score. 47 From our data, this corresponded to a large change in before and after scores (Hedges’ g −1.16), a moderate change compared with waitlist control (g −0.55), or a small benefit when compared with active controls (g −0.20). For credibility assessments comparing exercise modalities, we used the netmeta package 48 and CINeMA. 49 We also used netmeta to model acceptability, comparing the odds ratio for drop-out rate in each arm.

Additional analyses

All prespecified moderation and sensitivity analyses were performed. We moderated for participant characteristics, including participants’ sex, age, baseline symptom severity, and presence or absence of comorbidities; duration of the intervention (weeks); weekly dose of the intervention; duration between completion of treatment and measurement, to test robustness to remission (in response to a reviewer’s suggestion); amount of autonomy provided in the exercise prescription; and presence of each behaviour change technique. As preregistered, we moderated for behaviour change techniques in three ways: through meta-regression, including all behaviour change techniques simultaneously for primary analysis; including one behaviour change technique at a time (using 99% credible intervals to somewhat control for multiple comparisons) in exploratory analyses; and through meta-analytical classification and regression trees (metaCART), which allowed for interactions between moderating variables (eg, if goal setting combined with feedback had synergistic effects). 50 We conducted sensitivity analyses for risk of bias, assessing whether studies with low versus unclear or high risk of bias on each domain showed statistically significant differences in effect sizes.

Credibility assessment

To assess the credibility of each comparison against active control, we used CINeMA. 35 49 This online tool was designed by the Cochrane Comparing Multiple Interventions Methods Group as an adaptation of GRADE for network meta-analyses. 35 In line with recommended guidelines, for each comparison we made judgements for within study bias, reporting bias, indirectness, imprecision, heterogeneity, and incoherence. Similar to GRADE, we considered the evidence for comparisons to show high confidence then downgraded on the basis of concerns in each domain, as follows:

Within study bias —Comparisons were downgraded when most of the studies providing direct evidence for comparisons were unclear or high risk.

Reporting bias —Publication bias was assessed in three ways. For each comparison with at least 10 studies 51 we created funnel plots, including estimates of effect sizes after removing studies with statistically significant findings (ie, worst case estimates) 52 ; calculated an s value, representing how strong publication bias would need to be to nullify meta-analytical effects 52 ; and conducted a multilevel Egger’s regression test, indicative of small study bias. Given these tests are not recommended for comparisons with fewer than 10 studies, 51 those comparisons were considered to show “some concerns.”

Indirectness — Our primary population of interest was adults with major depression. Studies were considered to be indirect if they focused on one sex only (>90% male or female), participants with comorbidities (eg, heart disease), adolescents and young adults (14-20 years), or older adults (>60 years). We flagged these studies as showing some concerns if one of these factors was present, and as “major concerns” if two of these factors were present. Evidence from comparisons was classified as some concerns or major concerns using majority rating for studies directly informing the comparison.

Imprecision — As per CINeMA, we used the clinically important difference of Hedges’ g=0.2 to ascribe a zone of equivalence, where differences were not considered clinically significant (−0.2<g<0.2). Studies were flagged as some concerns for imprecision if the bounds of the 95% credible interval extended across that zone, and they were flagged as major concerns if the bounds extended to the other side of the zone of equivalence (such that effects could be harmful).

Heterogeneity — Prediction intervals account for heterogeneity differently from credible intervals. 35 As a result, CINeMA accounts for heterogeneity by assessing whether the prediction intervals and the credible intervals lead to different conclusions about clinical significance (using the same zone of equivalence from imprecision). Comparisons are flagged as some concerns if the prediction interval crosses into, or out of, the zone of equivalence once (eg, from helpful to no meaningful effect), and as major concerns if the prediction interval crosses the zone twice (eg, from helpful and harmful).

Incoherence — Incoherence assesses whether the network meta-analysis provides similar estimates when using direct evidence (eg, randomised controlled trials on strength training versus SSRI) compared with indirect evidence (eg, randomised controlled trials where either strength training or SSRI uses waitlist control). Incoherence provides some evidence the network may violate the assumption of transitivity: that the only systematic difference between arms is the treatment, not other confounders. We assessed incoherence using two methods: Firstly, a global design-by-treatment interaction to assess for incoherence across the whole network, 35 49 and, secondly, separating indirect and direct evidence (SIDE method) for each comparison through netsplitting to see whether differences between those effect estimates were statistically significant. We flagged comparisons as some concerns if either no direct comparisons were available or direct and indirect evidence gave different conclusions about clinical significance (eg, from helpful to no meaningful effect, as per imprecision and heterogeneity). Again, we classified comparisons as major concerns if the direct and indirect evidence changed the sign of the effect or changed both limits of the credible interval. 35 49

Patient and public involvement

We discussed the aims and design of this study with members of the public, including those who had experienced depression. Several of our authors have experienced major depressive episodes, but beyond that we did not include patients in the conduct of this review.

Study selection

The PRISMA flow diagram outlines the study selection process ( fig 1 ). We used two previous reviews to identify potentially eligible studies for inclusion. 12 22 Database searches identified 18 658 possible studies. After 5505 duplicates had been removed, two reviewers independently screened 13 115 titles and abstracts. After screening, two reviewers independently reviewed 1738 full text articles. Supplementary file section S2 shows the consensus reasons for exclusion. A total of 218 unique studies described in 246 reports were included, totalling 495 arms and 14 170 participants. Supplementary file section S3 lists the references and characteristics of the included studies.

Fig 1

Flow of studies through review

Network geometry

As preregistered, we removed nodes with fewer than 100 participants. Using this filter, most interventions contained comparisons with at least four other nodes in the network geometry ( fig 2 ). The results of the global test design-by-treatment interaction model were not statistically significant, supporting the assumption of transitivity (χ 2 =94.92, df=75, P=0.06). When net-splitting was used on all possible combinations in the network, for two out of the 120 comparisons we found statistically significant incoherence between direct and indirect evidence (SSRI v waitlist control; cognitive behavioural therapy v tai chi or qigong). Overall, we found little statistical evidence that the model violated the assumption of transitivity. Qualitative differences were, however, found for participant characteristics between different arms (see supplementary file, section S4). For example, some interventions appeared to be prescribed more frequently among people with severe depression (eg, 7/16 studies using SSRIs) compared with other interventions (eg, 1/15 studies using aerobic exercise combined with therapy). Similarly, some interventions appeared more likely to be prescribed for older adults (eg, mean age, tai chi=59 v dance=31) or women (eg, per cent female: dance=88% v cycling=53%). Given that plausible mechanisms exist for these systematic differences (eg, the popularity of tai chi among older adults), 53 there are reasons to believe that allocation to treatment arms would be less than perfectly random. We have factored these biases in our certainty estimates through indirectness ratings.

Fig 2

Network geometry indicating number of participants in each arm (size of points) and number of comparisons between arms (thickness of lines). SSRI=selective serotonin reuptake inhibitor

Risk of bias within studies

Supplementary file section S5 provides the risk of bias ratings for each study. Few studies explicitly blinded participants and staff ( fig 3 ). As a result, overall risk of bias for most studies was unclear or high, and effect sizes could include expectancy effects, among other biases. However, sensitivity analyses suggested that effect sizes were not influenced by any risk of bias criteria owing to wide credible intervals (see supplementary file, section S6). Nevertheless, certainty ratings for all treatments arms were downgraded owing to high risk of bias in the studies informing the comparison.

Fig 3

Risk of bias summary plot showing percentage of included studies judged to be low, unclear, or high risk across Cochrane criteria for randomised trials

Synthesis of results

Supplementary file section S7 presents a forest plot of Hedges’ g values for each study. Figure 4 shows the predicted effects of each treatment compared with active controls. Compared with active controls, large reductions in depression were found for dance (n=107, κ=5, Hedges’ g −0.96, 95% credible interval −1.36 to −0.56) and moderate reductions for walking or jogging (n=1210, κ=51, g −0.63, −0.80 to −0.46), yoga (n=1047, κ=33, g=−0.55, −0.73 to −0.36), strength training (n=643, κ=22, g=−0.49, −0.69 to −0.29), mixed aerobic exercises (n=1286, κ=51, g=−0.43, −0.61 to −0.25), and tai chi or qigong (n=343, κ=12, g=−0.42, −0.65 to −0.21). Moderate, clinically meaningful effects were also present when exercise was combined with SSRIs (n=268, κ=11, g=−0.55, −0.86 to −0.23) or aerobic exercise was combined with psychotherapy (n=404, κ=15, g=−0.54, −0.76 to −0.32). All these treatments were significantly stronger than the standardised minimum clinically important difference compared with active control (g=−0.20), equating to an absolute g value of −1.16. Dance, exercise combined with SSRIs, and walking or jogging were the treatments most likely to perform best when modelling the surface under the cumulative ranking curve ( fig 4 ). For acceptability, the odds of participants dropping out of the study were lower for strength training (n=247, direct evidence κ=6, odds ratio 0.55, 95% credible interval 0.31 to 0.99) and yoga (n=264, κ=5, 0.57, 0.35 to 0.94) than for active control. The rate of dropouts was not significantly different from active control in any other arms (see supplementary file, section S8).

Fig 4

Predicted effects of different exercise modalities on major depression compared with active controls (eg, usual care), with 95% credible intervals. The estimate of effects for the active control condition was a before and after change of Hedges’ g of −0.95 (95% credible interval −1.10 to −0.79), n=3554, κ =113. Colour represents SUCRA from most likely to be helpful (dark purple) to least likely to be helpful (light purple). SSRI=selective serotonin reuptake inhibitor; SUCRA=surface under the cumulative ranking curve

Consistent with other meta-analyses, effects were moderate for cognitive behaviour therapy alone (n=712, κ=20, g=−0.55, −0.75 to −0.37) and small for SSRIs (n=432, κ=16, g=−0.26, −0.50 to −0.01) compared with active controls ( fig 4 ). These estimates are comparable to those of reviews that focused directly on psychotherapy (g=−0.67, −0.79 to −0.56) 7 or pharmacotherapy (g=−0.30, –0.34 to −0.26). 25 However, our review was not designed to find all studies of these treatments, so these estimates should not usurp these directly focused systematic reviews.

Despite the large number of studies in the network, confidence in the effects were low ( fig 5 ). This was largely due to the high within study bias described in the risk of bias summary plot. Reporting bias was also difficult to robustly assess because direct comparison with active control was often only provided in fewer than 10 studies. Many studies focused on one sex only, older adults, or those with comorbidities, so most arms had some concerns about indirect comparisons. Credible intervals were seldom wide enough to change decision making, so concerns about imprecision were few. Heterogeneity did plausibly change some conclusions around clinical significance. Few studies showed problematic incoherence, meaning direct and indirect evidence usually agreed. Overall, walking or jogging had low confidence, with other modalities being very low.

Fig 5

Summary table for credibility assessment using confidence in network meta-analysis (CINeMA). SSRI=selective serotonin reuptake inhibitor

Moderation by participant characteristics

The optimal modality appeared to be moderated by age and sex. Compared with models that only included exercise modality (R 2 =0.65), R 2 was higher for models that included interactions with sex (R 2 =0.71) and age (R 2 =0.69). R 2 showed no substantial increase for models including baseline depression (R 2 =0.67) or comorbidities (R 2 =0.66; see supplementary file, section S9).

Effects appeared larger for women than men for strength training and cycling ( fig 6 ). Effects appeared to be larger for men than women when prescribing yoga, tai chi, and aerobic exercise alongside psychotherapy. Yoga and aerobic exercise alongside psychotherapy appeared more effective for older participants than younger people ( fig 7 ). Strength training appeared more effective when prescribed to younger participants than older participants. Some estimates were associated with substantial uncertainty because some modalities were not well studied in some groups (eg, tai chi for younger adults), and mean age of the sample was only available for 71% of the studies.

Fig 6

Effects of interventions versus active control on depression (lower is better) by sex. Shading represents 95% credible intervals

Fig 7

Effects of interventions versus active control on depression (lower is better) by age. Shading represents 95% credible intervals

Moderation by intervention and design characteristics

Across modalities, a clear dose-response curve was observed for intensity of exercise prescribed ( fig 8 ). Although light physical activity (eg, walking, hatha yoga) still provided clinically meaningful effects (g=−0.58, −0.82 to −0.33), expected effects were stronger for vigorous exercise (eg, running, interval training; g=−0.74, −1.10 to −0.38). This finding did not appear to be due to increased weekly energy expenditure: credible intervals were wide, which meant that the dose-response curve for METs/min prescribed per week was unclear (see supplementary file, section S10). Weak evidence suggested that shorter interventions (eg, 10 weeks: g=−0.53, −0.71 to −0.35) worked somewhat better than longer ones (eg, 30 weeks: g=−0.37, −0.79 to 0.03), with wide credible intervals again indicating high uncertainty (see supplementary file, section S11). We also moderated for the lag between the end of treatment and the measurement of the outcome. We found no indication that participants were likely to relapse within the measurement period (see supplementary file, section S12); effects remained steady when measured either directly after the intervention (g=−0.59, −0.80 to −0.39) or up to six months later (g=−0.63, −0.87 to −0.40).

Fig 8

Dose-response curve for intensity (METs) across exercise modalities compared with active control. METs=metabolic equivalents of task

Supplementary file section S13 provides coding for the behaviour change techniques and autonomy for each exercise arm. None of the behaviour change techniques significantly moderated overall effects. Contrary to expectations, studies describing a level of participant autonomy (ie, choice over frequency, intensity, type, or time) tended to show weaker effects (g=−0.28, −0.78 to 0.23) than those that did not (g=−0.75, −1.17 to −0.33; see supplementary file, section S14). This effect was consistent whether or not we included studies that used physical activity counselling (usually high autonomy).

Use of group exercise appeared to moderate the effects: although the overall effects were similar for individual (g=−1.10, −1.57 to −0.64) and group exercise (g=−1.16, −1.61 to −0.73), some interventions were better delivered in groups (yoga) and some were better delivered individually (strength training, mixed aerobic exercise; see supplementary file, section S15).

As preregistered, we tested whether study funding moderated effects. Models that included whether a study was funded did explain more variance (R 2 =0.70) compared with models that included treatment alone (R 2 =0.65). Funded studies showed stronger effects (g=−1.01, −1.19 to −0.82) than unfunded studies (g=−0.77, −1.09 to −0.46). We also moderated for the type of measure (self-report v clinician report). This did not explain a substantial amount of variance in the outcome (R 2 =0.66).

Sensitivity analyses

Evidence of publication bias was found for overall estimates of exercise on depression compared with active controls, although not enough to nullify effects. The multilevel Egger’s test showed significance (F 1,98 =23.93, P<0.001). Funnel plots showed asymmetry, but the result of pooled effects remained statistically significant when only including non-significant studies (see supplementary file, section S16). No amount of publication bias would be sufficient to shrink effects to zero (s value=not possible). To reduce effects below clinical significance thresholds, studies with statistically significant results would need to be reported 58 times more frequently than studies with non-significant results.

Qualitative synthesis of mediation effects

Only a few of the studies used explicit mediation analyses to test hypothesised mechanisms of action. 54 55 56 57 58 59 One study found that both aerobic exercise and yoga led to decreased depression because participants ruminated less. 54 The study found that the effects of aerobic exercise (but not yoga) were mediated by increased acceptance. 54 “Perceived hassles” and awareness were not statistically significant mediators. 54 Another study found that the effects of yoga were mediated by increased self-compassion, but not rumination, self-criticism, tolerance of uncertainty, body awareness, body trust, mindfulness, and attentional biases. 55 One study found that the effects from an aerobic exercise intervention were not mediated by long term physical activity, but instead were mediated by exercise specific affect regulation (eg, self-control for exercise). 57 Another study found that neither exercise self-efficacy nor depression coping self-efficacy mediated effects of aerobic exercise. 56 Effects of aerobic exercise were not mediated by the N2 amplitude from electroencephalography, hypothesised as a neuro-correlate of cognitive control deficits. 58 Increased physical activity did not appear to mediate the effects of physical activity counselling on depression. 59 It is difficult to infer strong conclusions about mechanisms on the basis of this small number of studies with low power.

Summary of evidence

In this systematic review and meta-analysis of randomised controlled trials, exercise showed moderate effects on depression compared with active controls, either alone or in combination with other established treatments such as cognitive behaviour therapy. In isolation, the most effective exercise modalities were walking or jogging, yoga, strength training, and dancing. Although walking or jogging were effective for both men and women, strength training was more effective for women, and yoga or qigong was more effective for men. Yoga was somewhat more effective among older adults, and strength training was more effective among younger people. The benefits from exercise tended to be proportional to the intensity prescribed, with vigorous activity being better. Benefits were equally effective for different weekly doses, for people with different comorbidities, or for different baseline levels of depression. Although confidence in many of the results was low, treatment guidelines may be overly conservative by conditionally recommending exercise as complementary or alternative treatment for patients in whom psychotherapy or pharmacotherapy is either ineffective or unacceptable. 60 Instead, guidelines for depression ought to include prescriptions for exercise and consider adapting the modality to participants’ characteristics and recommending more vigorous intensity exercises.

Our review did not uncover clear causal mechanisms, but the trends in the data are useful for generating hypotheses. It is unlikely that any single causal mechanism explains all the findings in the review. Instead, we hypothesise that a combination of social interaction, 61 mindfulness or experiential acceptance, 62 increased self-efficacy, 33 immersion in green spaces, 63 neurobiological mechanisms, 64 and acute positive affect 65 combine to generate outcomes. Meta-analyses have found each of these factors to be associated with decreases in depressive symptoms, but no single treatment covers all mechanisms. Some may more directly promote mindfulness (eg, yoga), be more social (eg, group exercise), be conducted in green spaces (eg, walking), provide a more positive affect (eg, “runner’s high”’), or be more conducive to acute adaptations that may increase self-efficacy (eg, strength). 66 Exercise modalities such as running may satisfy many of the mechanisms, but they are unlikely to directly promote the mindful self-awareness provided by yoga and qigong. Both these forms of exercise are often practised in groups with explicit mindfulness but seldom have fast and objective feedback loops that improve self-efficacy. Adequately powered studies testing multiple mediators may help to focus more on understanding why exercise helps depression and less on whether exercise helps. We argue that understanding these mechanisms of action is important for personalising prescriptions and better understanding effective treatments.

Our review included more studies than many existing reviews on exercise for depression. 13 22 27 28 As a result, we were able to combine the strengths of various approaches to exercise and to make more nuanced and precise conclusions. For example, even taking conservative estimates (ie, the least favourable end of the credible interval), practitioners can expect patients to experience clinically significant effects from walking, running, yoga, qigong, strength training, and mixed aerobic exercise. Because we simultaneously assessed more than 200 studies, credible intervals were narrower than those in most existing meta-analyses. 13 We were also able to explore non-linear relationships between outcomes and moderators, such as frequency, intensity, and time. These analyses supported some existing findings—for example, our study and the study by Heissel et al 22 found that shorter interventions had stronger effects, at least for six months; our study and the study by Singh et al 13 both found that effects were stronger with vigorous intensity exercise compared with light and moderate exercise. However, most existing reviews found various treatment modalities to be equally effective. 13 27 In our review, some types of exercise had stronger effect sizes than others. We attribute this to the study level data available in a network meta-analysis compared with an overview of reviews 24 and higher power compared with meta-analyses with smaller numbers of included studies. 22 28 Overviews of reviews have the ability to more easily cover a wider range of participants, interventions, and outcomes, but also risk double counting randomised trials that are included in separate meta-analyses. They often include heterogeneous studies without having as much control over moderation analyses (eg, Singh et al included studies covering both prevention and treatment 13 ). Some of those reviews grouped interventions such as yoga with heterogeneous interventions such as stretching and qigong. 13 This practise of combining different interventions makes it harder to interpret meta-analytical estimates. We used methods that enabled us to separately analyse the effects of these treatment modalities. In so doing, we found that these interventions do have different effects, with yoga being an intervention with strong effects and stretching being better described as an active control condition. Network meta-analyses revealed the same phenomenon with psychotherapy: researchers once concluded there was a dodo bird verdict, whereby “everybody has won, and all must have prizes,” 67 until network meta-analyses showed some interventions were robustly more effective than others. 6 26

Predictors of acceptability and outcomes

We found evidence to suggest good acceptability of yoga and strength training; although the measurement of study drop-out is an imperfect proxy of adherence. Participants may complete the study without doing any exercise or may continue exercising and drop out of the study for other reasons. Nevertheless, these are useful data when considering adherence.

Behaviour change techniques, which are designed to increase adherence, did not meaningfully moderate the effect sizes from exercise. This may be due to several factors. It may be that the modality explains most of the variance between effects, such that behaviour change techniques (eg, presence or absence of feedback) did not provide a meaningful contribution. Many forms of exercise potentially contain therapeutic benefits beyond just energy expenditure. These characteristics of a modality may be more influential than coexisting behaviour change techniques. Alternatively, researchers may have used behaviour change techniques such as feedback or goal setting without explicitly reporting them in the study methods. Given the inherent challenges of behaviour change among people with depression, 29 and the difficulty in forecasting which strategies are likely to be effective, 68 we see the identification of effective techniques as important.

We did find that autonomy, as provided in the methods of included studies, predicted effects, but in the opposite direction to our hypotheses: more autonomy was associated with weaker effects. Physical activity counselling, which usually provides a great deal of patient autonomy, was among the lowest effect sizes in our meta-analysis. Higher autonomy judgements were associated with weaker outcomes regardless of whether physical activity counselling was included in the model. One explanation for these data is that people with depression benefit from the clear direction and accountability of a standardised prescription. When provided with more freedom, the low self-efficacy that is symptomatic of depression may stop patients from setting an appropriate level of challenge (eg, they may be less likely to choose vigorous exercise). Alternatively, participants were likely autonomous when self-selecting into trials with exercise modalities they enjoyed, or those that fit their social circumstances. After choosing something value aligned, autonomy within the trial may not have helpful. Either way, data should be interpreted with caution. Our judgement of the autonomy provided in the methods may not reflect how much autonomy support patients actually felt. The patient’s perceived autonomy is likely determined by a range of factors not described in the methods (eg, the social environment created by those delivering the programme, or their social identity), so other studies that rely on patient reports of the motivational climate are likely to be more reliable. 33 Our findings reiterate the importance of considering these patient reports in future research of exercise for depression.

Our findings suggest that practitioners could advocate for most patients to engage in exercise. Those patients may benefit from guidance on intensity (ie, vigorous) and types of exercise that appear to work well (eg, walking, running, mixed aerobic exercise, strength training, yoga, tai chi, qigong) and be well tolerated (eg, strength training and yoga). If social determinants permit, 66 engaging in group exercise or structured programmes could provide support and guidance to achieve better outcomes. Health services may consider offering these programmes as an alternative or adjuvant treatment for major depression. Specifically, although the confidence in the evidence for exercise is less strong than for cognitive behavioural therapy, the effect sizes seem comparable, so it may be an alternative for patients who prefer not to engage in psychotherapy. Previous reviews on those with mild-moderate depression have found similar effects for exercise or SSRIs, or the two combined. 13 14 In contrast, we found some forms of exercise to have stronger effects than SSRIs alone. Our findings are likely related to the larger power in our review (n=14 170) compared with previous reviews (eg, n=2551), 14 and our ability to better account for heterogeneity in exercise prescriptions. Exercise may therefore be considered a viable alternative to drug treatment. We also found evidence that exercise increases the effects of SSRIs, so offering exercise may act as an adjuvant for those already taking drugs. We agree with consensus statements that professionals should still account for patients’ values, preferences, and constraints, ensuring there is shared decision making around what best suits the patient. 66 Our review provides data to help inform that decision.

Strengths, limitations, and future directions

Based on our findings, dance appears to be a promising treatment for depression, with large effects found compared with other interventions in our review. But the small number of studies, low number of participants, and biases in the study designs prohibits us from recommending dance more strongly. Given most research for the intervention has been in young women (88% female participants, mean age 31 years), it is also important for future research to assess the generalisability of the effects to different populations, using robust experimental designs.

The studies we found may be subject to a range of experimental biases. In particular, researchers seldom blinded participants or staff delivering the intervention to the study’s hypotheses. Blinding for exercise interventions may be harder than for drugs 23 ; however, future studies could attempt to blind participants and staff to the study’s hypotheses to avoid expectancy effects. 69 Some of our ratings are for studies published before the proliferation of reporting checklists, so the ratings might be too critical. 23 For example, before CONSORT, few authors explicitly described how they generated a random sequence. 23 Therefore, our risk of bias judgements may be too conservative. Similarly, we planned to use the Cochrane risk of bias (RoB) 1 tool 40 so we could use the most recent Cochrane review of exercise and depression 12 to calibrate our raters, and because RoB 2 had not yet been published. 70 Although assessments of bias between the two tools are generally comparable, 71 the RoB 1 tool can be more conservative when assessing open label studies with subjective assessments (eg, unblinded studies with self-reported measures for depression). 71 As a result, future reviews should consider using the latest risk of bias tool, which may lead to different assessments of bias in included studies.

Most of the main findings in this review appear robust to risks from publication bias. Specifically, pooled effect sizes decreased when accounting for risk of publication bias, but no degree of publication bias could nullify effects. We did not exclude grey literature, but our search strategy was not designed to systematically search grey literature or trial registries. Doing so can detect additional eligible studies 72 and reveal the numbers of completed studies that remain unpublished. 73 Future reviews should consider more systematic searches for this kind of literature to better quantify and mitigate risk of publication bias.

Similarly, our review was able to integrate evidence that directly compared exercise with other treatment modalities such as SSRIs or psychotherapy, while also informing estimates using indirect evidence (eg, comparing the relative effects of strength training and SSRIs when tested against a waitlist control). Our review did not, however, include all possible sources of indirect evidence. Network meta-analyses exist that directly focus on psychotherapy 7 and pharmacotherapy, 25 and these combined for treating depression. 6 Those reviews include more than 500 studies comparing psychological or drug interventions with controls. Harmonising the findings of those reviews with ours would provide stronger data on indirect effects.

Our review found some interesting moderators by age and sex, but these were at the study level rather than individual level—that is, rather than being able to determine whether women engaging in a strength intervention benefit more than men, we could only conclude that studies with more women showed larger effects than studies with fewer women. These studies may have been tailored towards women, so effects may be subject to confounding, as both sex and intervention may have changed. The same finding applied to age, where studies on older adults were likely adapted specifically to this age group. These between study differences may explain the heterogeneity in the effects of interventions, and confounding means our moderators for age and sex should be interpreted cautiously. Future reviews should consider individual patient meta-analyses to allow for more detailed assessments of participant level moderators.

Finally, for many modalities, the evidence is derived from small trials (eg, the median number of walking or jogging arms was 17). In addition to reducing risks from bias, primary research may benefit from deconstruction designs or from larger, head-to-head analyses of exercise modalities to better identify what works best for each candidate.

Clinical and policy implications

Our findings support the inclusion of exercise as part of clinical practice guidelines for depression, particularly vigorous intensity exercise. Doing so may help bridge the gap in treatment coverage by increasing the range of first line options for patients and health systems. 9 Globally there has been an attempt to reduce stigma associated with seeking treatment for depression. 74 Exercise may support this effort by providing patients with treatment options that carry less stigma. In low resource or funding constrained settings, group exercise interventions may provide relatively low cost alternatives for patients with depression and for health systems. When possible, ideal treatment may involve individualised care with a multidisciplinary team, where exercise professionals could take responsibility for ensuring the prescription is safe, personalised, challenging, and supported. In addition, those delivering psychotherapy may want to direct some time towards tackling cognitive and behavioural barriers to exercise. Exercise professionals might need to be trained in the management of depression (eg, managing risk) and to be mindful of the scope of their practice while providing support to deal with this major cause of disability.

Conclusions

Depression imposes a considerable global burden. Many exercise modalities appear to be effective treatments, particularly walking or jogging, strength training, and yoga, but confidence in many of the findings was low. We found preliminary data that may help practitioners tailor interventions to individuals (eg, yoga for older men, strength training for younger women). The World Health Organization recommends physical activity for everyone, including those with chronic conditions and disabilities, 75 but not everyone can access treatment easily. Many patients may have physical, psychological, or social barriers to participation. Still, some interventions with few costs, side effects, or pragmatic barriers, such as walking and jogging, are effective across people with different personal characteristics, severity of depression, and comorbidities. Those who are able may want to choose more intense exercise in a structured environment to further decrease depression symptoms. Health systems may want to provide these treatments as alternatives or adjuvants to other established interventions (cognitive behaviour therapy, SSRIs), while also attenuating risks to physical health associated with depression. 3 Therefore, effective exercise modalities could be considered alongside those intervention as core treatments for depression.

What is already known on this topic

Depression is a leading cause of disability, and exercise is often recommended alongside first line treatments such as pharmacotherapy and psychotherapy

Treatment guidelines and previous reviews disagree on how to prescribe exercise to best treat depression

What this study adds

Various exercise modalities are effective (walking, jogging, mixed aerobic exercise, strength training, yoga, tai chi, qigong) and well tolerated (especially strength training and yoga)

Effects appeared proportional to the intensity of exercise prescribed and were stronger for group exercise and interventions with clear prescriptions

Preliminary evidence suggests interactions between types of exercise and patients’ personal characteristics

Ethics statements

Ethical approval.

Not required.

Acknowledgments

We thank Lachlan McKee for his assistance with data extraction. We also thank Juliette Grosvenor and another librarian (anonymous) for their review of our search strategy.

Contributors: MN led the project, drafted the manuscript, and is the guarantor. MN, TS, PT, MM, BdPC, PP, SB, and CL drafted the initial study protocol. MN, TS, PT, BdPC, DvdH, JS, MM, RP, LP, RV, HA, and BV conducted screening, extraction, and risk of bias assessment. MN, JS, and JM coded methods for behaviour change techniques. MN and DGG conducted statistical analyses. PP, SB, and CL provided supervision and mentorship. All authors reviewed and approved the final manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: None received.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Data sharing Data and code for reproducing analyses are available on the Open Science Framework ( https://osf.io/nzw6u/ ).

The lead author (MN) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Dissemination to participants and related patient and public communities: We plan to disseminate the findings of this study to lay audiences through mainstream and social media.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

  • ↵ World Health Organization. Depression. 2020 [cited 2020 Mar 12]. https://www.who.int/news-room/fact-sheets/detail/depression
  • ↵ Birkjær M, Kaats M, Rubio A. Wellbeing adjusted life years: A universal metric to quantify the happiness return on investment. Happiness Research Institute; 2020. https://www.happinessresearchinstitute.com/waly-report
  • Jacobson NC ,
  • Pinquart M ,
  • Duberstein PR
  • Cuijpers P ,
  • Karyotaki E ,
  • Vinkers CH ,
  • Cipriani A ,
  • Furukawa TA
  • Strawbridge R ,
  • Marwood L ,
  • Santomauro D ,
  • Collins PY ,
  • Generaal E ,
  • Lawlor DA ,
  • Cooney GM ,
  • Recchia F ,
  • Miller CT ,
  • Mundell NL ,
  • Gallardo-Gómez D ,
  • Del Pozo-Cruz J ,
  • Álvarez-Barbosa F ,
  • Alfonso-Rosa RM ,
  • Del Pozo Cruz B
  • Salcher-Konrad M ,
  • ↵ National Collaborating Centre for Mental Health (UK). Depression: The Treatment and Management of Depression in Adults (Updated Edition). Leicester (UK): British Psychological Society; https://www.ncbi.nlm.nih.gov/pubmed/22132433
  • Bassett D ,
  • ↵ American Psychiatric Association. Practice Guideline for the Treatment of Patients with Major Depressive Disorder. Third Edition. Washington, DC: American Psychiatric Association; 2010. 87 p. https://psychiatryonline.org/pb/assets/raw/sitewide/practice_guidelines/guidelines/mdd-1410197717630.pdf
  • ↵ NICE. Depression in adults: treatment and management. [cited 2023 Mar 13]. National Institute for Health and Care Excellence; 2022 https://www.nice.org.uk/guidance/ng222/resources
  • Heissel A ,
  • Brokmeier LL ,
  • Ekkekakis P
  • ↵ Chaimani A, Caldwell DM, Li T, Higgins JPT, Salanti G. Undertaking network meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane; 2022. www.training.cochrane.org/handbook
  • Furukawa TA ,
  • Salanti G ,
  • Miller KJ ,
  • Gonçalves-Bradley DC ,
  • Areerob P ,
  • Hennessy D ,
  • Mesagno C ,
  • Glowacki K ,
  • Duncan MJ ,
  • Gainforth H ,
  • Richardson M ,
  • Johnston M ,
  • Abraham C ,
  • Whittington C ,
  • McAteer J ,
  • French DP ,
  • Olander EK ,
  • Chisholm A ,
  • Mc Sharry J
  • Ntoumanis N ,
  • Prestwich A ,
  • Caldwell DM ,
  • Nikolakopoulou A ,
  • Higgins JPT ,
  • Papakonstantinou T ,
  • Caspersen CJ ,
  • Powell KE ,
  • Christenson GM
  • ↵ Veritas Health Innovation. Covidence systematic review software. Melbourne, Australia; 2023. www.covidence.org
  • Ainsworth BE ,
  • Haskell WL ,
  • Herrmann SD ,
  • Altman DG ,
  • Gøtzsche PC ,
  • Cochrane Bias Methods Group ,
  • Cochrane Statistical Methods Group
  • Hodges JS ,
  • ↵ Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU technical support document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. In: National Institute for Health and Care Excellence (NICE), editor. NICE Decision Support Unit Technical Support Documents. London: Citeseer; 2011. https://www.ncbi.nlm.nih.gov/books/NBK310366/
  • Faltinsen E ,
  • Todorovac A ,
  • Staxen Bruun L ,
  • ↵ R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2022. https://www.R-project.org/
  • Hengartner MP ,
  • Balduzzi S ,
  • Dusseldorp E ,
  • Sterne JAC ,
  • Sutton AJ ,
  • Ioannidis JPA ,
  • Mathur MB ,
  • VanderWeele TJ
  • Leung LYL ,
  • La Rocque CL ,
  • Mazurka R ,
  • Stuckless TJR ,
  • Harkness KL
  • Vollbehr NK ,
  • Hoenders HJR ,
  • Bartels-Velthuis AA ,
  • Zeibig JM ,
  • Seiffer B ,
  • Ehmann PJ ,
  • Alderman BL
  • Bombardier CH ,
  • Gibbons LE ,
  • ↵ American Psychological Association. Clinical practice guideline for the treatment of depression across three age cohorts. American Psychological Association; 2019. https://www.apa.org/depression-guideline/
  • van Straten A ,
  • Reynolds CF 3rd .
  • Johannsen M ,
  • Nissen ER ,
  • Lundorff M ,
  • Coventry PA ,
  • Schuch FB ,
  • Deslandes AC ,
  • Gosmann NP ,
  • Fleck MP de A
  • Saunders DH ,
  • Phillips SM
  • Teychenne M ,
  • Hunsley J ,
  • Di Giulio G
  • Milkman KL ,
  • Hecksteden A ,
  • Savović J ,
  • ↵ Richter B, Hemmingsen B. Comparison of the Cochrane risk of bias tool 1 (RoB 1) with the updated Cochrane risk of bias tool 2 (RoB 2). Cochrane; 2021. Report No.: 1. https://community.cochrane.org/sites/default/files/uploads/inline-files/RoB1_2_project_220529_BR%20KK%20formatted.pdf
  • Chandler J ,
  • Lefebvre C ,
  • Glanville J ,
  • Briscoe S ,
  • Coronado-Montoya S ,
  • Kwakkenbos L ,
  • Steele RJ ,
  • Turner EH ,
  • Angermeyer MC ,
  • van der Auwera S ,
  • Schomerus G
  • Al-Ansari SS ,

research for review paper

Help | Advanced Search

Computer Science > Artificial Intelligence

Title: an interactive agent foundation model.

Abstract: The development of artificial intelligence systems is transitioning from creating static, task-specific models to dynamic, agent-based systems capable of performing well in a wide range of applications. We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents across a wide range of domains, datasets, and tasks. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction, enabling a versatile and adaptable AI framework. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare. Our model demonstrates its ability to generate meaningful and contextually relevant outputs in each area. The strength of our approach lies in its generality, leveraging a variety of data sources such as robotics sequences, gameplay data, large-scale video datasets, and textual information for effective multimodal and multi-task learning. Our approach provides a promising avenue for developing generalist, action-taking, multimodal systems.

Submission history

Access paper:.

  • Download PDF
  • Other Formats

license icon

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

BibSonomy logo

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

  • Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

IMAGES

  1. review of literature format apa

    research for review paper

  2. 👍 Research article analysis paper. How to Write a Journal Critique

    research for review paper

  3. Journal Article Review Template

    research for review paper

  4. (PDF) Writing a Literature Review Research Paper: A step-by-step approach

    research for review paper

  5. Research

    research for review paper

  6. Example Of Literature Review For Fyp

    research for review paper

VIDEO

  1. How to Write a Research Paper Publication

  2. Lecture No. 5, How to Write a Research Paper

  3. Secret To Writing A Research Paper

  4. Step-by-step approach to starting and completing a good research paper

  5. Publication Tips & Tricks

  6. How to Write a Scientific Review Paper

COMMENTS

  1. How to write a review paper

    Steps for Writing a Review Paper Before You Begin to Search or Write Clearly define the topic. Typically, a review writer works in the related field and already has a good knowledge of the topic, but not neces-sarily.

  2. How to review a paper

    Writing a good review requires expertise in the field, an intimate knowledge of research methods, a critical mind, the ability to give fair and constructive feedback, and sensitivity to the feelings of authors on the receiving end.

  3. PDF Format for a review paper

    Abstract: An abstract should be of approximately 200-300 words. Provide a brief summary of the review question being addressed or rationale for the review, the major studies reviewed, and conclusions drawn. Please do not cite references in the Abstract.

  4. How to Write a Best Review Paper to Get More Citation

    A review paper, or a literature review, is a thorough, analytical examination of previously published literature. It also provides an overview of current research works on a particular topic in chronological order.

  5. What is a review article?

    A review article can also be called a literature review, or a review of literature. It is a survey of previously published research on a topic. It should give an overview of current thinking on the topic. And, unlike an original research article, it will not present new experimental results. Writing a review of literature is to provide a ...

  6. How to Write a Peer Review

    Here's how your outline might look: 1. Summary of the research and your overall impression. In your own words, summarize what the manuscript claims to report. This shows the editor how you interpreted the manuscript and will highlight any major differences in perspective between you and the other reviewers. Give an overview of the manuscript ...

  7. How to write a superb literature review

    Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for...

  8. How to Write a Literature Review

    What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic.

  9. Review articles: purpose, process, and structure

    Review papers tend to include both quantitative (i.e., meta-analytic, systematic reviews) and narrative or more qualitative components; together, they provide platforms for new conceptual frameworks, reveal inconsistencies in the extant body of research, synthesize diverse results, and generally give other scholars a "state-of-the-art" snapshot ...

  10. Ten Simple Rules for Writing a Literature Review

    Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7].

  11. How to write a good scientific review article

    Review MeSH terms Writing* Literature reviews are valuable resources for the scientific community. With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a part …

  12. Systematic Review

    A review is an overview of the research that's already been completed on a topic. What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias. The methods are repeatable, and the approach is formal and systematic: Formulate a research question Develop a protocol

  13. Writing a Literature Review Research Paper: A step-by-step approach

    Writing a literature review in the pre or post-qualification, will be required to undertake a literature review, either as part of a course of study, as a key step in the research process. A ...

  14. Step by Step Guide to Reviewing a Manuscript

    Briefly summarize what the paper is about and what the findings are. Try to put the findings of the paper into the context of the existing literature and current knowledge. Indicate the significance of the work and if it is novel or mainly confirmatory. Indicate the work's strengths, its quality and completeness.

  15. What is the difference between a research paper and a review paper

    The research paper will be based on the analysis and interpretation of this data. A review article or review paper is based on other published articles. It does not report original research. Review articles generally summarize the existing literature on a topic in an attempt to explain the current state of understanding on the topic.

  16. How to write the literature review of your research paper

    The main purpose of the review is to introduce the readers to the need for conducting the said research. A literature review should begin with a thorough literature search using the main keywords in relevant online databases such as Google Scholar, PubMed, etc. Once all the relevant literature has been gathered, it should be organized as ...

  17. Writing Review Papers

    The purpose of a review paper is to succinctly review recent progress in a particular topic. Overall, the paper summarizes the current state of knowledge of the topic. It creates an understanding of the topic for the reader by discussing the findings presented in recent research papers. A review paper is not a "term paper" or book report.

  18. Writing a Literature Review

    A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research ...

  19. How to Write an Article Review (with Sample Reviews)

    Identify the article. Start your review by referring to the title and author of the article, the title of the journal, and the year of publication in the first paragraph. For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest. 4.

  20. Literature review as a research methodology: An ...

    The paper has several contributions. First, this paper separates between different types of review methodologies; systematic, semi-systematic and integrative approaches and argues that depending on purpose and the quality of execution, each type of approach can be very effective.

  21. 5 Differences between a research paper and a review paper

    Scholarly literature can be of different types; some of which require that researchers conduct an original study, whereas others can be based on existing research. One of the most popular Q&As led us to conclude that of all the types of scholarly literature, researchers are most confused by the differences between a research paper and a review paper. This infographic explains the five main ...

  22. Literature Review: Conducting & Writing

    This guide will provide research and writing tips to help students complete a literature review assignment.

  23. 'It depends': what 86 systematic reviews tell us about what strategies

    This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and ...

  24. PDF The Impact of Infrastructure on Development Outcomes

    papers written in other languages were also reviewed, if they were cited in the selected literature review papers. Finally, to ensure that more recent (as yet unpublished) research was captured by the review, a global call for new papers on this theme was conducted in preparation for the 2022

  25. Psychiatric Symptoms Across the Menstrual Cycle in Adult Women: A

    The aim of this comprehensive review is to describe the findings of previous research examining psychiatric symptom variability across the menstrual cycle. Each study included (1) a comparison of at least two menstrual cycle phases, (2) data not derived from evaluation of an intervention, and (3) premenopausal women, age 18 years or older.

  26. MM-LLMs: Recent Advances in MultiModal Large Language Models

    In this paper, we provide a comprehensive survey aimed at facilitating further research of MM-LLMs. Initially, we outline general design formulations for model architecture and training pipeline. ... Furthermore, we review the performance of selected MM-LLMs on mainstream benchmarks and summarize key training recipes to enhance the potency of ...

  27. Effect of exercise for depression: systematic review and network meta

    Objective To identify the optimal dose and modality of exercise for treating major depressive disorder, compared with psychotherapy, antidepressants, and control conditions. Design Systematic review and network meta-analysis. Methods Screening, data extraction, coding, and risk of bias assessment were performed independently and in duplicate. Bayesian arm based, multilevel network meta ...

  28. [2402.05929] An Interactive Agent Foundation Model

    The development of artificial intelligence systems is transitioning from creating static, task-specific models to dynamic, agent-based systems capable of performing well in a wide range of applications. We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents across a wide range of domains, datasets, and tasks. Our training ...