Peer Review in the Google Age Morrison

Screencast

Peer Review, Heather Morrison

Heather Morrison: This is Heather Morrison, author of the blog "The Imaginary Journal of Poetic Economics," talking about open peer review and collaboration. Peer review is a tradition that has been around for a very long time. As Peggy and Jay have told us, there has been a very great deal of research on peer review, hundreds of thousands of articles. We learn by building on traditions like peer review, but we also learn by challenging some of our basic assumptions.


So let's look at some of the basic assumptions behind peer review. Is it necessary at all? Is the answer to this question the same in every research area? If it is necessary, do we need to continue to do it in the traditional way, or are there new approaches which would allow us to do a lot more?


The expert might not need peer review at all. After all, they can review the article and decide for themselves. For most of us, however, everyone from the general public to the researcher reading outside their own area of specialty, peer review does provide some assurance of quality control.


Ah, but how much assurance of quality control? From my experience, and here I'm not talking about the rare exceptions, but the regular everyday work of peer review, the range of quality is all the way from extremely poor to extremely thorough. For example, I once found an article in which the researcher concluded that the reason human subjects could not memorize nonsense syllables because they could not extract the meaning. How did this pass peer review? On the other hand, I have interviewed a researcher in the area of pharmacology who was extremely thorough, checking all of the article, checking all of the references.


Does peer review have the same meaning and significance in every research area? In my opinion, no. In the area of pharmacology and toxicology the difference between a drive in poison can be the dose, a matter of life and death. Here we would want to be very, very careful about messing around with peer review. We'd want to be very sure that we have another method of quality control that was just as good, if not better. But this is not true in every research area. Consider, for example, literary opinion. If something has passed that probably should not have passed peer review, this is not good, but it's certainly not a matter of life and death. What I'm trying to say here is that in some areas of research there's probably a lot more scope to experiment with new methods of peer review than there is with others.


There are many ways to experiment with peer review. In my blog, "The Imaginary Journal of Poetic Economics," I talk about one theoretical model for a new way of doing peer review. The idea is to have a process that is completely open and transparent. Peer reviewers sign their work and the peer reviews, along with the articles, are openly accessible to anyone.


As much as the processes as possible are automated. For example, there could be a profile of the reviewer up on the web, which ties to a calendaring system, which would know when the reviewer is available. This model could potentially accomplish a number of goals. The openness and transparency would facilitate research to improve peer review. Science literacy could be enhanced. This is because high school students, undergrads and people in developing countries could all see the peer review process in action. The automation aspects would introduce some cost efficiencies into the scholarly publishing system.


But what is most exciting to me is the potential of this model to facilitate new means of collaboration that would work well across the global. Imagine, for example, that I put my profile up on the web. A researcher in a country on another continent could potentially find my peer reviewer profile and realize that I am available. Both of us could learn from this process. I, from learning about research in another country, and the other person from having a reviewer from a completely different background.


So far, we've talked about one new approach to peer review, after the article is written. But why wait? Why not talk to your peers about the research that you're thinking about doing. If there's a better research method, why not find out about it before you do the research, not after it's all done and you feel it's ready for publication. Research funders review projects before they even fund them. Why not share your data as soon as it is ready? Perhaps one of your colleagues could use it for a different research question.


As a final thought, we advance our knowledge by working together, by sharing ideas and critique. Peer review has been a good method for doing this, and in some areas we need to be very careful to ensure that quality control is followed. But other areas, a research is not life and death, and here we can inform to experiment to less.