Evaluating traditional peer-review processes and their alternatives: An opinionated discussion

Aaron Weiskittel

Abstract


The advancement of science requires the timely and effective communication of important findings, which often takes the form of peer-reviewed journal articles. In the past decade, there has been significant changes in the world of scientific publishing with the rise of e-journals, open-access articles, and a greater volume of manuscript submissions. However, the overwhelming majority of journals rely on a traditional peer-review model, which is often inefficient and ineffective. In this discussion, I evaluate the core assumptions of traditional peer-review processes, assess current alternatives to traditional peer-review, and provide recommendations for authors, reviewers, Associate Editors, and Editors. Overall, the intent of the discussion is to raise the importance of this issue and provide some suggestions for change.

Keywords


Scientific publications and journals; pre- and post-publication review; high volume and independent review

Full Text:

PDF

References


Aleksic, J., Alexa, A., Attwood, T.K., Hong, N.C., Dahlӧ, M., Davey, R., Dinkel, H., Fӧrstner, K.U., Grigorov, I., Hériché, J.-K., Lahti, L., MacLean, D., Markie, M.L., Molloy, J., Schneider, M.V., Scott, C., Smith-Unna, R., Vieira, B.M., 2014. The open science peer review oath. F1000Res 3, 271.

Armstrong, J.S., 1997. Peer review for journals: Evidence on quality control, fairness, and innovation. Sci. Eng. Ethics 3, 63–84.

Bedeian, A.G., 2003. The manuscript review process: The proper roles of authors, referees, and editors. J. Manag. Inq. 12, 331–338.

Bornmann, L., Daniel, H.-D., 2009. The luck of the referee draw: the effect of exchanging reviews. Learn. Publ. 22, 117–125.

Brotons, L., 2015. Peer-review warning: system error, reviewers not found. Front. Ecol. Environ. 13, 241– 242.

Browman, H.I., 2004. The rules of the game in science publishing. Mar. Ecol. Prog. Ser. 270, 267–268.

Cieszewski, C.J., Strub, M.R., 2009. The foundation of Mathematical and Computational Forestry & Natural Resources Sciences (MCFNS). Math. Comput. For. Nat. Resour. Sci. 1, 41–46.

Emerson, G.B., Warme, W.J., Wolf, F.M., Heckman, J.D., Brand, R.A., Leopold, S.S., 2010. Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Arch. Intern. Med. 21, 1934–1939.

Ferguson, C., Marcus, A., Oransky, I., 2015. The peer-review scam. Nature 515, 480–482.

Forestry Research Advisory Council (FRAC), 2012. Report to the Secretary of Agriculture. http://www.fs.fed.us/research/docs/forestry- research-council/reports/2012-report.pdf.

Frey, B.S., 2003. Publishing as prostitution? – Choosing between ones own ideas and academic success. Public Choice 116, 205–223.

García, J.A., Rodriguez-Sánchez, R., Fdez-Valdivia, J., 2015. The principal-agent problem in peer review. J. Assoc. Inf. Sci. Technol. 66, 297–308.

Glen, A.S. 2014. A New ``Golden Rule'' for Peer Review?. Bulletin of the Ecological Society of America, 95(4), 431–434.

Hill, S., Provost, F., 2003. The myth of the double-blind review?: Author identification using only citations. ACM SIGKDD Explor. Newsl. 5, 179–184.

Hochberg, M.E., Chase, J.M., Gotelli, N.J., Hastings, A., Naeem, S., 2009. The tragedy of the reviewer commons. Ecol. Lett. 12, 2–4.

Huntoon, L.R., 2009. Tactics characteristic of sham peer review. J. Am. Physicians Surg. 14, 64–66.

Lee, C.J., Sugimoto, C.R., Zhang, G., Cronin, B., 2013. Bias in peer review. J. Am. Soc. Inf. Sci. Technol. 64, 2–17.

Kangas, A., Hujala, T. 2015. Challenges in publishing: producing, assuring, and communicating quality. Silva Fennica 49: 1304.

Macdonald, S., 2014. Emperor’s new clothes: The reinvention of peer review as myth. J. Manag. Inq. doi:10.1177/1056492614554773

McPeek, M.A., DeAngelis, D.L., Shaw, R.G., Moore, A.J., Rausher, M.D., Strong, D.R., Ellison, A.M., Barrett, L., Rieseberg, L., Breed, M.D., Sullivan, J., Osenberg, C.W., Holyoak, M., Elgar, M.A., 2009. The golden rule of reviewing. Am. Nat. 173, E155–158.

Moon, J.Y., Sproull, L.S., 2008. The role of feedback in managing the internet-based volunteer work force. Inf. Syst. Res. 19, 494–515.

Mulligan, A., Hall, L., Raphael, E., 2013. Peer review in a changing world: An international study measuring the attitudes of researchers. J. Am. Soc. Inf. Sci. Technol. 64, 132–161.

Mutz, D.C., 2015. Incentivizing the manuscript-review system using REX. Polit. Sci. Polit. 48, 73–77.

Nicholas, D., Watkinson, A., Jamali, H.R., Herman, E., Tenopir, C., Volentine, R., Allard, S., Levine, K., 2015. Peer review: still king in the digital age. Learn. Publ. 28, 15–21.

Osterloh, M., Frey, B.S., 2015. Ranking games. Eval. Rev. 39, 102–129.

Osterloh, M., Kieser, A., 2015. Double-blind peer re- view: How to slaughter a sacred cow. In: Incentives and Performance. Springer International Publishing, pp. 307–321.

Riisgård, H.U., 2003. Misuse of the peer-review system: time for countermeasures? Mar. Ecol. Prog. Ser. 258, 297–309.

Schimel, D., Strong, D.R., Ellison, A.M., Peters, D.P., Silver, S., Johnson, E.A., Belnap, J., Classen, A.T., Essington, T.E., Finely, A.O., Inouye, B.D., Stanley, E.H. 2014. Editors are editors, not oracles. Bulletin of the Ecological Society of America, 95, 342–346.

Sharik, T., Lilieholm, R., 2012. A national perspective on forestry education. West. For. 57, 1–5.

Sharik, T.L., Lilieholm, R.J., Lindquist, W., Richardson, W.W., 2015. Undergraduate enrollment in natural resource programs in the United States: Trends, drivers, and implications for the future of natural resource professions. J. For. in press. doi: http://dx.doi.org/10.5849/jof.14-146

Siler, K., Lee, K., Bero, L., 2015. Measuring the effectiveness of scientific gatekeeping. Proc. Natl. Acad. Sci. United States Am. 112, 360–365.

Sud, P., Thelwall, M., 2014. Evaluating altmetrics. Scientometrics 98, 1131–1143.

Sugimoto, C.R., Cronin, B., 2013. Citation gamesman- ship: Testing for evidence of ego bias in peer review. Scientometrics 95, 851–862.

Sugimoto, C.R., Lariviére, V., Ni, C., Cronin, B., 2013. Journal acceptance rates: A cross- disciplinary analysis of variability and relationships with journal measures. J. Informetr. 7, 897–906. doi:10.1016/j.joi.2013.08.007

Tsang, E., Frey, B.S., 2007. The as–is journal review process: let authors own their ideas. Acad. Manag. Learn. Educ. 6, 128–136.

Walker, R., Barros, B., Conejo, R., Neumann, K., Tele- font, M., 2015. Bias in peer review: a case study. F1000 Res. 4, 21.

Ware, M., 2008. Peer review in scholarly journals: Perspective of the scholarly community – Results from an international study. Inf. Serv. Use 28, 109–112.


Refbacks

  • There are currently no refbacks.


   

© 2008 Mathematical and Computational Forestry & Natural-Resource Sciences