So Now I Are
Herbert Jack Rotfeld
editor, Journal of Consumer Affairs
Professor of Marketing
Departing as editor of Journal of Communication, Mark Levy's last editorial reported that he once told his predecessor that his tough choices were causing him to lose friends at a rate of five percent a year. "Only five percent?" George Gerbner replied. "You need to raise your standards." Thomas Sowell once wrote that: "The fact that I have never murdered an editor is proof that the death penalty deters."
After years of writing my own criticisms of editors and the review process, I am now the editor of a national academic journal. I hope my submitting authors let me live.
Since publishing journal articles impact many professors' tenure, prestige, promotions and pay, editors' decisions are guaranteed to alienate many people along the way. Decisions to reject manuscripts are painful enough, but the process itself should not be a pain. Yet at many journals, manuscript reviews have evolved from scientific quality control into a gauntlet to be endured. It is not just an issue of standards. The experience has become as lengthy and involved as writing a doctoral thesis, with delays and trivial revisions ad infinitum, or so it seems.
Of course, the editors all have a problem of finding people willing to review manuscripts. It is a thankless task. More vexing are the members of many editorial boards who like having their name on the cover but won't do the work. Holding the position by virtue of politics or prestige, they ignore the editor's requests and cause the entire process to be delayed.
But even when reviewers are found and responses made, the editor is still hostage to their willingness to provide meaningful feedback that helps make a decision. Marketing research covers many disciplines; Journal of Consumer Affairs is pan disciplinary. Even in the narrowest of marketing journals, no one can seriously claim to be an expert in all aspects of every manuscript that comes across the desk. The alternative to the review process is a table of contents that will be derided as "the editor's friends."
In theory, reviewers are to assess the quality of research and make editorial recommendations on the contribution to the literature. And based on those reviews, the editor decides whether to accept or reject the manuscript. Sometimes a revision is needed because points are not clear or some parts need more explanation to fully assess how well the research was conducted.
Unfortunately, some reviewers are just lazy, telling the editor they didn't like the paper while failing to say why it shouldn't be accepted and not giving any useful feedback to the authors. But even when they do the job, too often the review comments include all sorts of strange and irrelevant notes, making authors wonder if they read or understood the paper. And like an overbearing doctoral advisor, some referees seem compelled to discuss a different study the reviewer felt should have been conducted.
They criticize "failures" to address points that were addressed. They dislike the conclusions, while not stating why they might be in error. They note places where additional papers should be gratuitously cited, though it is not clear how that paper would have influenced the research conceptualization. And sometimes they "don't like" the numbers, or that the researcher didn't use the statistical methods they'd prefer. They seem driven to make comments that are a virtual brochure on their favored research method or approach to data analysis.
This might just seem like a problem of callous reviewers, but too many editors confuse the process with that of a doctoral committee. As the number of reviewers per manuscript has increased to four at many journals, some editors indicate all of them are to "sign off," in effect voting on the final version like a committee. Each revision, no matter how trivial, goes out for another set of reviews, and reviewers are increasingly involved with rewriting of pedantic details.
A thesis or dissertation is a learning experience for which a student gets course credit toward a degree. In that situation, rewrites or new requirements for additional citations and data analysis are to help the student learn new materials. Committee members might raise totally new issues on third or fourth readings as part of their role as teachers.
On the other hand, the journal's double-blind referee's job is more narrowly constrained. They can help improve a paper, but they are not to be redirecting a research project along the lines they like to see. They should not be continually raising new lessons for the author to "learn." If the author is told to revise and resubmit a rejected manuscript, they should address the comments or tell the editor why the suggestion was not used. And then the editor needs to do his or her job, making editorial decisions instead of seeing the reviewers as "voters."
Not every change needs to go back to the reviewers; final decisions are not a simple vote counting of determining the "majority." Different reviewers bring a variety of perspectives and focus to the single manuscript, and sometimes a minority view makes a better case for the eventual editorial decision. As J. Scott Armstrong has pointed out, scholarship is harmed and new ideas are suppressed if the journals require publication to be based on a voting process with full consent of all referees.
In the end, a manuscript that goes through the review process is deemed a worthy contribution to the literature. The research must be solid, clear and honest, which is not the same as saying it is something with which the editor or reviewers philosophically agree. It does not need a disclaimer saying "The editor does not endorse this subject" just because the project was funded by a cigarette manufacturer or a pornography publisher.
Yet the papers should not go through four or more editorial revisions extending over years. That is not quality control of the publication. Instead, that is acceptance by attrition as the referees tire of seeing the same material time and again. Email allows for faster deadlines for reviews and authors should be able to answer the valid questions without being redirected time and again.
A colleague often tells me that there are only two types of papers, published and unpublished, and he is good at getting his work in the former category. But the process should not be political. Getting it accepted should not be a deal. The reviews ask for points to be better explained, for more data to be presented and for the theory to have a stronger foundation in the literature. Authors then write, revise, alter and rewrite -- though the research itself is long done (and not to be altered), authors must try to meet the editorial demands of the referees so the paper will be published.
After it is accepted, the authors then list it in their vita and annual reports, and some might also hope someone will read it when it is published. For my part, I hope people will be reading the whole journal. After all, my editorials will be at the end.