New Models of PeerReview Implications for the Different

New Models of Peer-Review: Implications for the Different Roles Journals Play in Scholarly Communities David J Solomon, Ph. D College of Human Medicine Michigan State University

Alicia Chang AP article http: //www. webcitation. org/5 OC 7 DQBz. Q

There were 1, 326 manuscripts published by the 3 rd week of June 2007

Developed by Paul Ginsparg at Cornell University in 1991

Peer review has become an integral part of scholarship • What value does peer review add? • What are its limitations? • How can new technology be used to improve traditional models of peer review?

The roles of journals in scholarly communities • • • Building a collective knowledge base Communicating Information Validating the Quality of Research Distributing Rewards Building scientific communities Schafner, A. C. 1994. The future of scientific journals: Lessons from the past. Information Technology and Libraries 13: 239 -47.

Peer Review in relation to journal roles • • • Building a collective knowledge base (Essential) Communicating Information (Detrimental) Validating the Quality of Research (Essential) Distributing Rewards (Essential) Building scientific communities (Irrelevant)

What about “open” peer review? • It merges the traditional peer reviewed journal with an “ar. Xiv. org” like preprint archive • Potentially achieving the best of both systems – – – Building a collective knowledge base (Essential) Communicating Information Satisfied by open review Validating the Quality of Research (Essential) Distributing Rewards (Essential) Building scientific communities (Irrelevant)

Nature’s open peer review trial

The Results…. • There were 1, 369 eligible papers. 71 (5%) of the authors agreed to participate. • There were a total of 92 comments. 33 of the papers (46%) did not receive any comments. • The site received an average of 5, 600 page views a week. • Comments were rated 1 (unhelpful) – 5 (influenced publication) for technical and editorial value. 1. 8 (tech) 2. 6 (ed. ) average. Non higher than 4. Nature (2006) doi: 10. 1038/nature 05535.

Results continued…. • Qualitative assessment – “Like pulling teeth to obtain comments. ” – Many authors, particularly in biology were afraid of getting “scooped” by participating. – Editors felt the comments received were not helpful.

Why does ar. Xiv. org and PLo. S One work and not the Nature Experiment? • My hypothesis is that it is something specific to “Nature” • Two other specialty journals have successfully implemented this form of open peer-review* – Preliminary results from MEO pilot • 20 of 27 (74%) agreed to participate in open peer review • Averaging between 1 and 2 comments per article but the quality of the comments tends to be reasonably good, in many cases on par with solicited reviews. • Server logs show high access rate *Electronic Transactions on Artificial Intelligence (http: //www. ida. liu. se/ext/etai/) *Atmospheric Chemistry and Physics (http: //www. copernicus. org/EGU/acp/)

My take on the results…… • Acceptance by authors varies greatly by journal • Generates relatively little discussion • Quality of the feedback does not equal traditional peer-review but can be useful • As a means of dissemination appears to be quite effective

Where do we go from here…. I do not think we should given up on this open peer-review model (culture’s change) "As people who are used to using the Web for all kinds of communication, people who are now using My. Space and Facebook, begin to infiltrate the ranks of the serious scientists, they'll be more comfortable doing this kind of thing, " she said. "What didn't work well now may work better later on. " Linda J. Miller, Nature’s U. S. executive editor as quoted in the Chronicle of Higher Ed. Thursday, January 11, 2007

"New scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it. " Max Planck, "Scientific Autobiography and Other Papers", Williams & Norgate, London (1950), pages 33 -34.