How robust are cross-country comparisons of PISA scores to the scaling model used?
Journal article
Jerrim, John, Parker, Philip, Choi, Alvaro, Chmielewski, Anna Katyn, Sälzer, Christine and Shure, Nikki. (2018). How robust are cross-country comparisons of PISA scores to the scaling model used? Educational Measurement: Issues and Practice. 37(4), pp. 28 - 39. https://doi.org/10.1111/emip.12211
Authors | Jerrim, John, Parker, Philip, Choi, Alvaro, Chmielewski, Anna Katyn, Sälzer, Christine and Shure, Nikki |
---|---|
Abstract | The Programme for International Student Assessment (PISA) is an important international study of 15‐olds' knowledge and skills. New results are released every 3 years, and have a substantial impact upon education policy. Yet, despite its influence, the methodology underpinning PISA has received significant criticism. Much of this criticism has focused upon the psychometric scaling model used to create the proficiency scores. The aim of this article is to therefore investigate the robustness of cross‐country comparisons of PISA scores to subtle changes to the underlying scaling model used. This includes the specification of the item‐response model, whether the difficulty and discrimination of items are allowed to vary across countries (item‐by‐country interactions) and how test questions not reached by pupils are treated. Our key finding is that these technical choices make little substantive difference to the overall country‐level results. |
Keywords | item-response theory; PISA; large-scale international assessments |
Year | 2018 |
Journal | Educational Measurement: Issues and Practice |
Journal citation | 37 (4), pp. 28 - 39 |
Publisher | Wiley-Blackwell |
ISSN | 0731-1745 |
Digital Object Identifier (DOI) | https://doi.org/10.1111/emip.12211 |
Scopus EID | 2-s2.0-85052521336 |
Open access | Published as green open access |
Page range | 28 - 39 |
Research Group | Institute for Positive Psychology and Education |
Author's accepted manuscript | License All rights reserved File Access Level Open |
Publisher's version | License All rights reserved File Access Level Controlled |
Place of publication | United Kingdom |
https://acuresearchbank.acu.edu.au/item/8q61v/how-robust-are-cross-country-comparisons-of-pisa-scores-to-the-scaling-model-used
Download files
Author's accepted manuscript
AM_Jerrim_2018_How_robust_are_cross_country_comparisons.pdf | |
License: All rights reserved | |
File access level: Open |
Restricted files
Publisher's version
122
total views112
total downloads3
views this month1
downloads this month