How to peer review a scientific or scholarly article

 Amy Keenum, DO, PharmD, Lincoln Memori- al-DeBusk College of Osteopathic Medicine, Harrogate TN 37752.

E-mail address: akeenummsu@gmail.com.

This article was designed to be viewed and distributed as a PDF. Please download the PDF for easiest reading.

‌Why be a reviewer?

The purpose of this article is to help physicians peer review scientific or scholarly articles before publication. Physicians are “reviewers” daily in their clinical practice. They rou- tinely interpret journal articles, television and radio medical newscasts, and newspaper articles. However, this review is often completed unconsciously. Physicians can spend as little as 1 minute per patient to assimilate new information at any given visit. Further, physicians spend as little as 30 minutes per day reading medical journals,1 so physicians must be savvy readers to use the medical literature effectively.

‌Overview of the process

There are multiple steps a scientific or scholarly article must go through before being published in a medical jour- nal. Of course, research is the first step followed by manu- script development. The process of writing a scientific paper is covered in a previous review article that can be found online at http://www2.acofp.org/education/SF_10/handouts/ pm_3.00_Wed_10.27.10_Shubrook_Jay_How_to_Write_a_

When specifics are needed, the authors will use the Osteopathic Family Physician as an example.

‌Overview

When a manuscript is submitted to a medical journal, it is usually first received by the editorial staff. The first review is then completed by the managing editor to make sure the submission is complete and meets the basic stan- dards for submission. If a manuscript passes this review, it is forwarded to the Editor. The Associate Editor’s job is to determine whether the paper would be of interest to the journal’s readership and whether it fits into the upcoming editorial calendar (plan of topics). If the Editor sees merit in the paper, it will be recommended for external review. Most journals have an Editorial Board that contributes to the direction of the journal. Many members of this board also serve as active reviewers for submitted articles. In addition to review by the Editorial Board, subject specialty reviewers are often sought. These reviewers are often content experts in their field. Many journals request that the author suggest reviewers on the topic of the paper. These suggested re- viewers are sometimes but not always approached to review the manuscript.

If the editors agree the submission meets their criteria and the topic would interest readers because the research presents new or different information, the article then moves to step three: peer review. If the submission passes review, chances are good it will be printed in a future issue.

The peer review step is pivotal because it is what distinguishes scientific and scholarly articles from other industry- related articles, such as news features or profiles of accomplished individuals. A peer reviewer carries out more duties than simply giving the paper a cursory read.

This article describes the various factors involved in a peer review based on information provided by Elsevier, a global publisher of journals, books, and major reference works. Elsevier is the publisher of the Osteopathic Family Physician. As a service to our reviewers, Elsevier provides a tutorial on how to review a manuscript.

When a manuscript goes out for external review, the Editor will send an invitation to both Editorial committee– appointed reviewers and a content expert. Reviewers are volunteering their time and can choose to accept or reject the review of a given manuscript. If they accept the review assignment, they will typically receive a blinded manuscript to read with a deadline to provide a recommendation and comments. The guidelines for a reviewer vary widely and can be a simple vote to accept as well as a text box for comments, or it can be a checklist and reporting of all of things done by the reviewer. The authors suggest that before reviewing a manuscript you know what the guidelines are for that journal.

‌Key points

• The importance of a peer review

• Who are peer reviewers?

• Why become a peer reviewer?

• Peer review process

• Submitting peer review

  ‌The importance of a peer review

  Peer review of scientific and scholarly articles is a longstanding tradition and a widely accepted standard of quality. The purpose of a peer review is two-fold. First, it serves as a filter through which the research is assessed and verified. Reviewers are to read as if they would be able to repeat the research process. They also assess whether the authors report what they state they are going to report, that the methods are sound, and that the results and conclusion are compatible. Second, it can elevate the research through professional critiques and suggestions about how to fine tune the presentation or correct inadvertent errors. The re- viewer determines whether the same conclusions can be reached from the data presented. Also the interpretation of the results may lead to additional lines of research. Often authors will overstate the impact or breadth of the results. The reviewer here serves as a referee. This is critically important because often many readers will read only the abstract of an article and if the conclusions are overstated,

  this can be carried forward and used to make practice decisions. The peer review process is critically involved in making sure scientifically rigorous articles are added to the literature. A poor-quality article can hurt the authors and journals alike.
  

  ‌Who are peer reviewers?

  Individuals selected to conduct a peer review are just that: peers. They could be fellow physicians for medical journals, scholars, or editors, depending on the publication and its policies. There are three typical approaches to the process. The most common is the single, blind review, which withholds reviewers’ names from the authors, but reviewers know who wrote the article. Some argue that a single-blinded review allows the reviewer to interpret the article in the context of the expertise of the author. Others would argue that a single-blinded review may be favorable to well-known authors, in which their name would carry an article but could be unfavorable for relatively unknown authors who are not already recognized for their knowledge on this topic.

  A double-blind review keeps both reviewers’ and au- thors’ names private. This is the method used by the Os- teopathic Family Physician. This limits any bias for or against an author. However, in the highest specialized fields, double-blinding may be difficult because the opinions and “voice” of authors in a small, specialized field may be known even without including the name.

  The third is an open review for which all participants are known. In some open reviews the authors may suggest individuals conduct the review, and in other cases the edi- tors choose the reviewers.

  Many journals will ask authors to select potential review- ers. This is usually to make sure content experts in the field who know the related body of work have a chance to review. This is particularly important in journals where the breadth of medical topics covered is wide, as is the case in many primary care journals. This does not ensure that these reviewers will be invited or that they will accept. It does give the editorial team an opportunity to select appropriate reviewers. However, even with reviewer recommendations, the review process can be single- or double-blinded.
  

  ‌Why become a peer reviewer?

  The peer review system obviously benefits authors and researchers, but at the same time it can also work in favor of reviewers. Not only does a reviewer help maintain the integrity of the scientific and scholarly publication process, but on a more personal level it broadens the network among others practicing in the same field and helps the osteopathic physician develop a reputation as an expert on a subject matter. In addition, the experience can build rapport with colleagues, so if in the event the reviewer becomes the researcher and author, the professional courtesy of a peer

  
  

  ‌review can be “repaid.” Many academic institutions also recognize the service of peer review. Further, being selected as a peer reviewer may be an honor that is acknowledged. Many journals provide incentives to the peer reviewers including free access to the journal and databases. Some will also provide continuing medical education (CME) credit for peer review. Finally, a peer review is an oppor- tunity to keep abreast of research findings in a specialty area and how they might affect the industry or current standards‌

  of practice.

  The Osteopathic Family Physician compensates review- ers for their time by providing 1 hour of AOA category 1B CME credit for each article reviewed. These hours are submitted on reviewers’ behalf to the AOA. In addition, Osteopathic Family Physician reviewers are given 30 days’ access to Scopus per each paper reviewed.
  

  ‌Before committing to a peer review‌

  As flattering as it is to be asked to peer review a sub- mission, there are a few factors to weigh before agreeing. First, are you qualified? Does the research topic fall within your scope of practice or specialty? Do you know enough about the topic to provide a qualified critique?

  In addition, be cognizant of any apparent conflicts of interest. Have you previously collaborated with the au- thor(s)? Do you have any financial connections to the sub- ject matter or products under examination? Although a conflict can excuse you as a reviewer, it is not automatic. The best course of action is to notify the editors as soon as the potential conflict comes to light and let the journal’s staff make the decision to either move forward or find a substitute reviewer.

  Do you have the time?3 Journals and publications work against deadlines and may have a specific time frame in which they need the review completed. It is important to only commit to review the manuscript if you feel confident you will be able to complete it. The journal typically will have a set life span of a manuscript and will try to be sensitive to presenting information in print in a timely way. The average article should take approximately 3 to 5 hours for a thorough review. Some suggest the review be con- ducted in one block rather than broken into multiple ses- sions to allow it your undivided attention.

  
  

  ‌The review process

  Before starting the process, it is important to realize that authors and editors hold the expectation of confidentiality.4 Information contained within the submission should not be disclosed to any third party or used for your own profes- sional endeavors. If you think comments from colleagues or students are warranted, check with the editorial staff before- hand.

  The specifics of what to examine might vary from pub- lication to publication, and editors may supply reviewers

  with a guide or form that lists exactly what is to be re- viewed. However, most will cover these main points:

• Originality: Does the article or paper present original information? Does it add new knowledge to the field of study, and is it pertinent information? This is not neces- sarily simply a matter of opinion. A literature search with a tool such as Scopus, a database of research literature and web sources, will help determine how much information on the topic has been published previously. If a search turns up a vast library of published pieces, then the question becomes what new data do the new submission present. Conversely, if a literature search reveals very little, then is the subject matter irrelevant or groundbreak- ing? In addition, if you discover references pertaining to the research that were not included or cited by the au- thor(s), make note of them and turn in your notes along with your other comments.

• Structure: This criterion has more to do with the article format than the integrity of the content. Is there a logical flow, including title, abstract, introduction, methodology, results, and conclusions? Each element serves a purpose.

  The abstract should reflect the article’s full content.

  The introduction should summarize research relevance and address whether previous findings are being chal- lenged or have undergone further examination.

  Methodology explains the research process used. Re- viewers should gauge whether the methods were appro- priate to the subject matter. Were the methods standard procedures? If not, were the procedures thoroughly ex- plained? Is there enough explanation that the research could be replicated?

  The results section is the opportunity to reveal the data, and a reviewer’s responsibility is to evaluate the results. Is the data clearly communicated and does it add up? Does the analysis fit the purpose of the paper?

  The discussion and/or conclusion section is where researchers express their interpretations and how it can affect current practices or revise or add to the knowledge base. Reviewers should examine whether the results pre- viously discussed back up the conclusions drawn.

• Language: A reviewer does not have to be a grammar expert, but if there are numerous or glaring grammatical errors, you can make a notation to the editorial staff. What is the overall tone of the manuscript? It should be neither pretentious nor overly simplistic. Rather, the lan- guage should be professional and straightforward. In terms of figures, charts, or tables: are they clearly la- beled? Is the graphic the best choice to illustrate the data? Remember, the reviewer’s job is to critique and suggest how the article can be honed or the message improved. There will be instances when the grammatical structure of the paper disrupts the ability of the reader to absorb the article. In these scenarios, it may be best to reject the article and suggest the authors seek additional writing resources. It is not the job of a reviewer or journal to “REWRITE” a poorly written article.

  
  

• ‌Ethical issues: If the article cites previous research, are those sources correctly notated according to the journal’s guidelines? If you are unsure, check previous issues or make a note for the editors to follow up. Again, if the literature search turns up other pieces not included by the author(s), keep track of those resources and cite them in your report. We expect authors to use the original source for all materials and we recommend that the reviewer check a few selected references to confirm their accuracy. Also, if the review stirs up questions about plagiarism, fraud, or other ethical concerns, jot down thorough com- ments, and wherever possible, attach sources with full citation as supporting material. Often a reviewer or jour- nal can copy a sentence or phrase and then do a MEDLINE search to determine whether the sentence was copied from other sources. Further, engines such as Sco- pus can allow you to search other works by the authors and other similar papers, as well as look for overlap of your paper with others. This is critical and protects the author and the journal.

‌Submitting the peer review

If the journal or publication supplied a specific review form, make sure each category is filled in. In addition, it is customary to turn in a summary of the article, which serves as both a reminder to editors of the key points and as evidence that you did your due diligence.

If there is no formal review worksheet, make sure your report covers all the main points discussed in this paper and any other information you deem important. Avoid personal comments; however, recommendations as to how the article can be improved generally are encouraged. Suggestions for

changes should be supported by references and/or clear explanations so both editors and author(s) understand your intentions. Also, indicate whether your input is an opinion or based on the data presented.

Typically, most publications classify articles into one of three categories: (1) Rejected because of poor quality or not within the journal’s scope (in fact, many of these papers do not even make it to the peer review step); (2) accepted without revisions; and (3) accepted with revisions. For pa- pers in the last category, indicate whether you would be available to review the revised article.

If the submission passes review, then chances are good it will be printed in a future issue.

‌Follow-up

If the journal or Editor allows, we suggest the reviewer speak with the Editor regarding the quality and content of the review. This ensures the reviews are appropriate to both the author and the journal.

‌Reference

1. ‌Stange KC, Woolf SH, Gjeltema K: One minute for prevention: the power of leveraging to fulfill the promise of health behavior counseling. Am J Prev Med 22:320-323, 2002‌

2. Shubrook JH, Kase JM, Norris M: 2010, How to Write a Scientific Article. Available at: http://www2.acofp.org/education/SF_10/handouts/ pm_3.00_Wed_10.27.10_Shubrook_Jay_How_to_Write_a_Paper.pdf. Ac- cessed July 23, 2012

3. ‌Hoppin FG: How I Review an Original Scientific Article, Am J Respir Crit Care Med 166:1019-1023, 2002

4. Benos DJ, Kirk KL, and Hall JE: How to Review a Paper, Advances in Physiology Education 27:47-52, 2003