Moving to the Double-Blind Review System

The Journal of Intelligence presently has a single-blind peer review system, unlike most other journals in the social sciences, where the double-blind peer review prevails. The choice between the two systems is not easy. Both systems have pros and cons, and a double-blind review can in fact be a single-blind review, depending on whether or not the authors can be identified after all. For an overview of the advantages and disadvantages of both systems, see, among other sources published by Codina [1] and Lee et al. [2].

The reasons to prefer one system over the other rely partly on hypothesized biases and empirical studies of these hypothesized biases. Unfortunately, the empirical studies are based on assumptions that are hard to verify, and it is unclear whether the results generalize from one context to another.

It may seem evident that biases can play a role if the authors are known to the reviewers, but in fact, biases can play a role in both systems, as illustrated by the following two anecdotes. (1) Single-blind review: For the analysis of his or her data, an author applies an established method in an unusual way. The reviewer rejects the manuscript because he or she does not trust the new way of applying the method, especially because the author does not seem to have any credentials. He or she recommends that the author get in touch with a colleague from another university to learn about the method in question. In fact, the author had applied the method in a new and valid way, but unexperienced as he or she was, he or she had not realized this could be perceived as problematic. The application looked quite straightforward in his or her eyes. The paper was eventually accepted after a discussion between the author and the editor. (2) Double-blind review: An author of a conference proposal criticizes his or her own earlier work in his or her proposal. One of the author’s friends is a reviewer of the proposal and rejects because the proposal contradicts the work of his or her friend. The reviewer trusted the earlier work of his or her friend and was unaware of the fact that his or her friend now has improved on his or her earlier conclusions.

Although generally speaking, double-blind review is believed to be less biased, it is a belief and not a proven fact, and there are problems of another kind with this type of review. For example: it is more difficult to spot conflicts of interest; it is more difficult to find out whether the manuscript describes new work; and it implies extra careful work to make the manuscript anonymous.

Anecdotes do not prove anything, and pros and cons remain mostly hypothetical, even after empirical studies. Therefore, the choice between the two systems is a matter of opinion in the first place. Our opinion and the choice to move to the double-blind review system are based on the following three arguments. First, using a double-blind review system is the more common system in the social sciences, and there are no definite arguments in favor of single-blind review. Second, the implicit social contract between peers implies a symmetrical relationship. Although evaluation is always asymmetrical, single-blind review makes the relationship between authors and reviewers even more asymmetrical. For example, anonymous reviewers criticize the authors without taking the risk that their disagreement and their mistakes in the review are revealed to the authors, while non-anonymous authors do reveal their views and mistakes to the reviewers. Being a reviewer does not make a peer more qualified than the author is. Third, strictly speaking, who the authors are is not relevant information for the evaluation of a manuscript. The situation is different for a research proposal, because the feasibility of a proposal depends on the credentials of the investigators.

Of course, a double-blind review system is only double-blind in theory and not necessarily in practice. One can think of ways to counter the chances of identification, but it requires extra work from the part of the authors and the journal. We want to avoid the extra work and leave it to the authors how far they go to make their submission anonymous to the reviewers. The minimum is to remove explicit identification information.

Although it is not always possible to avoid or reduce asymmetry between peers, it is worth trying to respect the symmetrical nature of the peer relationship as much as possible. We believe it is possible for the review system of our journal to reduce the asymmetry by moving to a double-blind review system. The alternative would be a non-blind review system.

In conclusion, we have decided to switch the Journal of Intelligence to use a double-blind review system. All articles submitted on or after 1 December 2015 will be reviewed in this way.