Loading...
Estimating reliability statistics and measurement error variances using instrumental variables with longitudinal data
Goldstein, Harvey Haynes, Michele Leckie, George Tran, Phuong
Goldstein, Harvey
Haynes, Michele
Leckie, George
Tran, Phuong
Abstract
The presence of randomly distributed measurement errors in scale scores such as those used in educational and behavioural assessments implies that careful adjustments are required to statistical model estimation procedures if inferences are required for ‘true’ as opposed to ‘observed’ relationships. In many cases this requires the use of external values for ‘reliability’ statistics or ‘measurement error variances’ which may be provided by a test constructor or else inferred or estimated by the data analyst. Popular measures are those described as ‘internal consistency’ estimates and sometimes other measures based on data grouping. All such measures, however, make particular assumptions that may be questionable but are often not examined. In this paper we focus on scaled scores derived from aggregating a set of indicators, and set out a general methodological framework for exploring different ways of estimating reliability statistics and measurement error variances, critiquing certain approaches and suggesting more satisfactory methods in the presence of longitudinal data. In particular, we explore the assumption of local (conditional) item response independence and show how a failure of this assumption can lead to biased estimates in statistical models using scaled scores as explanatory variables. We illustrate our methods using a large longitudinal data set of mathematics test scores from Queensland, Australia.
Keywords
instrumental variables, longitudinal data, reliability
Date
2020
Type
Journal article
Journal
Longitudinal and Life Course Studies
Book
Volume
11
Issue
3
Page Range
289-306
Article Number
ACU Department
Institute for Learning Sciences and Teacher Education (ILSTE)
Faculty of Education and Arts
Faculty of Education and Arts
Relation URI
Source URL
Event URL
Open Access Status
License
File Access
Controlled