In this post, we explain what you can gain from providing peer reviews and tips on how to write constructive reports.

Peer review is an integral part of the academic writing process. In our last two posts, we demystified the peer review process and how to handle revisions as an author.

At some point, you will be on the other side of the desk and asked to serve as a peer reviewer. In this post, we explain what you can gain from providing peer reviews and tips on how to write constructive reports.

Why is it important to be a peer reviewer?

First and foremost, peer review is the backbone to ensuring scientific integrity in published manuscripts. As we describe in our post about how the peer review process works, the managing editor does an initial determination of a manuscript’s fit for the journal and worthiness for peer review, but thereafter relies on the reports from peer reviewers. The reviewers are critical to assessing whether the research has solid methodology, provides sound conclusions, is important to the field, is relevant to the journal’s audience, and meets ethical standards. The peer reviews can also enhance the quality of the work.

Second, journals cannot function without peer reviewers. At some point, all authors have experienced frustrating delays in the peer review process. In some instances, the fault is with editors; for example, we have experienced occasional issues with our own articles being “lost” in a journal’s system or editors taking months to make determinations after receiving the peer review reports. But in most instances, delays are due to editors’ chronic challenges with soliciting peer reviews that are timely and of sufficient quality to make a determination. Delays in providing manuscript determinations can in turn damage a journal’s reputation. There are some journals that we will not submit to anymore following our own and our colleagues’ bad experiences. (JPHMP is not on that list!) If you want to participate in academic publishing and have your own work reviewed, you have a professional obligation to serve as a peer reviewer.

Finally, there are several direct benefits to you as a peer reviewer. Doing reports can help you learn about new developments in the field. Reading manuscripts and critiquing the research and write-up can improve your own writing as you reflect on what is clear (or not) to you as a reader and the strengths and limitations of the research. Most journals will allow you to see other peer reviewers’ reports and the editors’ decision letters sent to authors. If the article received a major revise & resubmit and you are asked to review another draft, you will have the opportunity to see the authors’ response letter and consider how well they responded to the comments including what is clear (or not) about their rebuttal. This can provide you with valuable information about the peer review process, how to improve the clarity of your own writing and response memos, and common critiques from other reviewers.

Doing peer reviews is also a way to help enhance your own visibility in the field. Regularly contributing high-quality and timely peer reviews will get noticed by editors. Regularly accepting review assignments and submitting excellent reviews on time will often leave a favorable impression with editors. If you are an academic on the tenure path, a journal editor might serve as a future external reviewer for your promotion case. Serving as a peer reviewer for multiple journals is a signal that you are gaining visibility as an expert in your area.

How do you become a good peer reviewer?

Below are 10 tips to help you write constructive peer reviews.

Tip #1: Say yes… after careful reflection

As long as the request comes from a reputable journal, and you have the time to commit, you should accept doing peer review reports. This is especially true for journals that have reviewed your prior work, where you have published, and where you would like to publish in the future. However, if the article is outside your area of expertise or you have a conflict of interest, decline the invitation. (If you are unsure whether a journal is “reputable,” consult your mentors.)

In making the decision, be thoughtful about your time availability. It is important to meet the editor’s deadlines. A careful review will take an hour or more; new reviewers may take longer until they are accustomed to doing peer reviews. It is not helpful for editors to receive peer review reports late or with insufficient detail – that adds to their workload and delays the entire process especially if the editor needs to solicit additional reviews to replace yours.

Tip #2: Consider multiple dimensions of the manuscript

Some journals will provide you with tailored instructions on what to discuss in your review. In general, consider the following dimensions:

• Fit within the journal’s scope and likely appeal to the journal’s audience
• Novelty of findings and importance of the topic to the field
• Clearly stated objectives, research questions, and/or hypotheses
• Quality of the research methodology
• Objective, comprehensive, and comprehensible presentation of study findings
• Writing mechanics such as issues with the flow and organization, or major errors
• Ethical violations such as plagiarism, biased reporting, or issues with human subjects

A few notes.

Regarding the “novelty and importance,” there is a general publication bias against null findings. However, null findings are equally important as we can learn valuable information about circumstances where relationships do not hold or programs do not work. In evaluating this dimension, consider whether the work provides new evidence or is a replication of past studies or what is already known. Also think about whether the authors are answering a question that is fundamentally important to the field and has real-world implications, versus being on a topic that is too narrow to be applicable outside their study context.

In evaluating the research methodology, consider the strength of the overall study design (eg, for an intervention study, is there a reasonable control group?), the quality of the data (eg, population represented, response rates, potential nonresponse or coverage bias, use of valid measures, and the reliability of the data), the adequacy of the analytic methods, transparency in how the analyses were conducted and reported, and whether the study design is consistent with the research question. The study limitations should be clearly stated, and not just be limited to platitudes (eg, “our data are cross sectional, so causality cannot be established”).

Regarding the writing mechanics, it is not your job to be a proofreader although it can help to point out some minor issues. However, do highlight if you are unable to follow the authors’ arguments, general “flow,” or presentation of findings.

Tip #3: Be detail-oriented… but fixate on the right details

Do not spend too much time correcting spelling and grammar. If the manuscript needs additional proofreading, simply state that in the review and perhaps give 1-2 examples. Do not make assumptions about whether the authors are “native English speakers” and whether one should review the manuscript. It is acceptable and preferable to highlight outdated or offensive terminology such as Caucasian (outdated), subjects (outdated), obese people (offensive/outdated), or “data is” (grammatically incorrect and like nails on a chalkboard to some readers).

However, it is helpful to look carefully at the tables as that may uncover errors in calculations or inconsistencies that may raise larger concerns about the rigor of the research.

Tip #4: Put comments in an organized list

The format of a review may differ across fields, with some peer review reports being in more of an essay format with citations. If you are doing a peer review for a health journal (most likely, if you are reading this post!) then do a favor to yourself, the authors, and the editors by putting comments in a bulleted or numbered list. Include one item per comment. Organize your comments into “major” versus “minor” comments; where appropriate, you might subdivide your comments into “methodological” versus “editorial” comments.

Creating an organized list saves you time, compared to writing a book report-style review. Clear constructive feedback helps authors identify what you are asking in the revision and create their response memo (see our post on preparing a revision). It can also help editors make determinations about the scope of requested revisions (thus helping with a “revise & resubmit” or “reject” decision) and whether authors were responsive to changes.

Tip #5: Remove ambiguity

Do not mention your recommendation in the peer review report – you will provide a confidential score for the editor in the online editorial management system. However, be explicit about which comments are minor suggestions versus major problems. If something is unclear in the text, ultimately the onus is on the authors to revise it. However, it can be helpful to explain what, exactly, does not make sense to you (eg, use of inconsistent terminology, a specific paragraph that is uninterpretable, a missing element of the study design description, or a disorganized table). If authors do not understand what you are requesting and where you are confused as a reader, their revision may not be fully responsive, leading to frustration among all parties and an avoidable additional round of revisions.

Also be clear about the positive aspects of the manuscript. Is it well-written? Is it on a topic that is important and of interest to the journal’s audience? Does it have strong empirical work? It can be helpful to start your review with a one-paragraph summary of the paper and an explicit statement about its contribution. One of us (EM) is still traumatized from having a dissertation paper rejected with one glowingly positive review and a second review with critiques on the statistical analysis that could have been easily addressed in a revision. In the rejection letter, the editor stated the paper was being rejected because the first reviewer did not state explicitly why the paper was so seminal. (*Head banging against the wall.)

Tip #6: The golden rule: do unto others…

Everyone has had an experience with a vindictive reviewer or editor. Be kind, not a jerk.  

This does not mean that you need to provide a glowing review on a fundamentally flawed manuscript. Rather, think about the type of review that you would like to receive as an author. Use polite language. Avoid statements like, “This manuscript has no merit.” Rather than simply providing a laundry list of criticisms, try to make some concrete suggestions where appropriate. For example, instead of stating the tables and figures are confusing, offer a potential suggestion for the authors’ consideration. If the manuscript’s organization cannot be followed in its present form, you might offer some concrete suggestions on places to rearrange the text or how to make the message clearer. Tips #3, #4, and #5 above are designed with the golden rule in mind.

Tip #7: Use the confidential comments box

The editorial management system will allow you to send confidential comments to the editor that, in theory, do not go to the authors. Use this box to provide less filtered comments directly to the editor. Don’t be caustic (we have seen editors who share a barely modified version of these comments with the authors), but you can be more direct. For example, you may tell the authors, “I have concerns that the sampling strategy has produced a biased analytical sample” but tell the editor, “The sample is terminally flawed and would strongly advise against publishing this study.”

Tip #8: Maintain confidentiality

As a reviewer, you have an ethical obligation to keep information confidential. Do not share the draft or information about the study with a colleague. If the journal is using a single- or double-blinded review process (whereby the reviewers remain anonymous), do not share that you reviewed the article.

Tip #9: Be kind to editors

Have some empathy. While it’s easy to complain about editors being slow or making poor decisions, in reality they are human beings and managing multiple articles and peer reviewers. If you are not an empathetic person, be strategic – the same editors who are handling your peer review report may manage your own manuscript in the future.

Channel Tip #6. Respond to emails from editors. If you decide to decline a review, click on the invitation letter and select the “decline to review” link. If you have a colleague who would be a better fit for the review, please recommend them to the editor. If you accept an invitation, submit your peer review report on time! If you are unable to meet the deadline, contact the editor and request an extension; that is far better than making the editor hound you to submit the review. If you start reviewing and decide you cannot complete it, contact the editor so they can start the process of soliciting an additional reviewer.

Tip #10: Manage your own workload

While doing peer review reports is critical to the field, they will take time. In deciding whether to accept the invitation, think about your other commitments (see Tip #1). While it’s important to provide a thorough and careful review (see Tip #2), you don’t have to be exhaustive. If the paper has major flaws and is fundamentally unacceptable for publication even in a revised form, list a few major items in the report (for authors) and provide a clear explanation of the problem in the confidential comments to the editor. See Tip #3 on items you can simplify (eg, “The manuscript has numerous grammatical errors and needs proofreading.”). If there are numerous major issues, try to focus on the 10 most important items rather than enumerating a comprehensive list. Most likely, the other reviewer(s) will have comments and the manuscript will receive a revise & resubmit (in which case you may see it again).

Finally, it can be helpful to write your review in a Word document. That will make it easy for you to copy and paste relevant sections into the editorial management system.

Wrapping up

If you made it to the end of our three-part series, you should now have a solid foundation for understanding how the peer review process works, how to convert your revise & resubmit into an acceptance, and how to be a good peer reviewer. You will encounter occasional bumps in the publication road throughout your career. However, we hope these tips can help you feel more confident navigating the peer review process and providing constructive reviews.

---

Erika Martin, PhD, MPH, is Professor and PhD Director at Rockefeller College of Public Affairs and Policy, University at Albany-SUNY. As an applied health policy researcher, she uses mixed methods to evaluate issues related to the allocation of scarce public health resources, the adoption and impact of public health policies, with a focus on domestic HIV and related syndemics. She also studies ways to improve the sustainability and impact of open data platforms. Articles she’s written have appeared in an array of leading health and public policy journals such as the New England Journal of Medicine, Journal of the American Medical Association, Health Affairs, American Journal of Public Health, Public Administration Review, and Journal of Policy Analysis and Management.

Justin B. Moore, PhD, MS, FACSM is an Associate Professor in the Department of Implementation Science in the Division of Public Health Sciences at the Wake Forest School of Medicine in Winston-Salem, NC, USA. He holds joint appointments in the departments of Family & Community Medicine and Epidemiology & Prevention. Dr. Moore also serves as the Director of the Implementation Science Affinity Group within the Clinical and Translational Science Institute at Wake Forest University. He conducts community-engaged research focused on the dissemination and implementation of evidence-based strategies for the promotion of healthy behaviors in underserved populations. He also conducts epidemiological research examining the determinants of health behaviors and related comorbidities across the lifespan. He serves as the Associate Editor of the Journal of Public Health Management and Practice as an Associate Editor-in-Chief for the Translational Journal of the American College of Sports Medicine. He previously served at the chair of the editorial board of the American Journal of Public Health. Full bio