A new reader trial approach to peer review in funding research grants : An Australian experiment
Journal article
Jayasinghe, Upali W., Marsh, Herbert W. and Bond, Nigel W.. (2006). A new reader trial approach to peer review in funding research grants : An Australian experiment. Scientometrics. 69(3), pp. 591-606. https://doi.org/10.1007/s11192-006-0171-4
Authors | Jayasinghe, Upali W., Marsh, Herbert W. and Bond, Nigel W. |
---|---|
Abstract | Peer reviews are highly valued in academic life, but are notoriously unreliable. A major problem is the substantial measurement error due to the idiosyncratic responses when large numbers of different assessors each evaluate only a single or a few submissions (e.g., journal articles, grants, etc.). To address this problem, the main funding body of academic research in Australia trialed a “reader system” in which each of a small number of senior academics read all proposals within their subdiscipline. The traditional peer review process for 1996 (2,989 proposals, 6,233 assessors) resulted in unacceptably low reliabilities comparable with those found in other research (0.475 for research project, 0.572 for researcher). For proposals from psychology and education in 1997, the new reader system resulted in substantially higher reliabilities: 0.643 and 0.881, respectively. In comparison to the traditional peer review approach, the new reader system is substantially more reliable, timely, and cost efficient - and applicable to many peer review situations. |
Year | 2006 |
Journal | Scientometrics |
Journal citation | 69 (3), pp. 591-606 |
Publisher | Akademiai Kiado |
Springer | |
ISSN | 0138-9130 |
Digital Object Identifier (DOI) | https://doi.org/10.1007/s11192-006-0171-4 |
Scopus EID | 2-s2.0-33846470568 |
Page range | 591-606 |
Funder | Australian Research Council (ARC) |
Research Group | Institute for Positive Psychology and Education |
Publisher's version | License All rights reserved File Access Level Controlled |
Output status | Published |
Publication dates | |
Online | 2006 |
https://acuresearchbank.acu.edu.au/item/8q8v1/a-new-reader-trial-approach-to-peer-review-in-funding-research-grants-an-australian-experiment
Restricted files
Publisher's version
84
total views0
total downloads3
views this month0
downloads this month