Multilevel modeling of cognitive diagnostic assessment : The multilevel DINA example

Journal article


Wang, Wen-Chung and Qiu, Xue-Lan. (2019). Multilevel modeling of cognitive diagnostic assessment : The multilevel DINA example. Applied Psychological Measurement. 43(1), pp. 34-50. https://doi.org/10.1177/0146621618765713
AuthorsWang, Wen-Chung and Qiu, Xue-Lan
Abstract

Many multilevel linear and item response theory models have been developed to account for multilevel data structures. However, most existing cognitive diagnostic models (CDMs) are unilevel in nature and become inapplicable when data have a multilevel structure. In this study, using the log-linear CDM as the item-level model, multilevel CDMs were developed based on the latent continuous variable approach and the multivariate Bernoulli distribution approach. In a series of simulations, the newly developed multilevel deterministic input, noisy, and gate (DINA) model was used as an example to evaluate the parameter recovery and consequences of ignoring the multilevel structures. The results indicated that all parameters in the new multilevel DINA were recovered fairly well by using the freeware Just Another Gibbs Sampler (JAGS) and that ignoring multilevel structures by fitting the standard unilevel DINA model resulted in poor estimates for the student-level covariates and underestimated standard errors, as well as led to poor recovery for the latent attribute profiles for individuals. An empirical example using the 2003 Trends in International Mathematics and Science Study eighth-grade mathematical test was provided.

Keywordscognitive diagnostic assessment; multilevel models; large-scale assessment; Bayesian methods
Year2019
JournalApplied Psychological Measurement
Journal citation43 (1), pp. 34-50
PublisherSAGE Publications
ISSN0146-6216
Digital Object Identifier (DOI)https://doi.org/10.1177/0146621618765713
PubMed ID30573933
Scopus EID2-s2.0-85045033321
PubMed Central IDPMC6297912
Page range34-50
FunderResearch Grants Council, Hong Kong SAR (Special Administrative Region)
Publisher's version
License
All rights reserved
File Access Level
Controlled
Output statusPublished
Publication dates
Online03 Apr 2018
Publication process dates
Deposited15 Nov 2023
Grant ID18604515
Permalink -

https://acuresearchbank.acu.edu.au/item/8zz1v/multilevel-modeling-of-cognitive-diagnostic-assessment-the-multilevel-dina-example

Restricted files

Publisher's version

  • 48
    total views
  • 0
    total downloads
  • 1
    views this month
  • 0
    downloads this month
These values are for the period from 19th October 2020, when this repository was created.

Export as

Related outputs

Computerized adaptive testing for ipsative tests with multidimensional pairwise-comparison items : Algorithm development and applications
Qiu, Xue-Lan, de la Torre, Jimmy, Ro, Sage and Wang, Wen-Chung. (2022). Computerized adaptive testing for ipsative tests with multidimensional pairwise-comparison items : Algorithm development and applications. Applied Psychological Measurement. 46(4), pp. 255-272. https://doi.org/10.1177/01466216221084209
An empirical Q-Matrix validation method for the polytomous G-DINA model
de la Torre, Jimmy, Qiu, Xue-Lan and Santos, Kevin Carl. (2022). An empirical Q-Matrix validation method for the polytomous G-DINA model. Psychometrika. 87(2), pp. 693-724. https://doi.org/10.1007/s11336-021-09821-x
Equity in mathematics education in Hong Kong : Evidence from TIMSS 2011 to 2019
Qiu, Xue-Lan and Leung, Frederick K. S.. (2022). Equity in mathematics education in Hong Kong : Evidence from TIMSS 2011 to 2019. Large-scale Assessments in Education. 10(1), p. Article 3. https://doi.org/10.1186/s40536-022-00121-z
A new item response theory model for rater centrality using a hierarchical rater model approach
Qiu, Xue-Lan, Chiu, Ming Ming, Wang, Wen-Chung and Chen, Po-Hsi. (2022). A new item response theory model for rater centrality using a hierarchical rater model approach. Behavior Research Methods. 54(4), pp. 1854-1868. https://doi.org/10.3758/s13428-021-01699-y
Assessment of differential statement functioning in ipsative tests with multidimensional forced-choice items
Qiu, Xue-Lan and Wang, Wen-Chung. (2021). Assessment of differential statement functioning in ipsative tests with multidimensional forced-choice items. Applied Psychological Measurement. 45(2), pp. 79-94. https://doi.org/10.1177/0146621620965739
Student self-assessment : Why do they do it?
Yan, Zi, Brown, Gavin T. L., Lee, John Chi-Kin and Qiu, Xue-Lan. (2020). Student self-assessment : Why do they do it? Educational Psychology. 40(4), pp. 509-532. https://doi.org/10.1080/01443410.2019.1672038
Measuring Dynamic Goals for Marriage : Development and Validation of the Marital Goal Scale Using Rasch Modeling
Li, Tianyuan, Hiu-Ling Tsang, Vivian, Fung, H, Qiu, Xuelan and Wang, Wen-Chung. (2019). Measuring Dynamic Goals for Marriage : Development and Validation of the Marital Goal Scale Using Rasch Modeling. Psychological Assessment. 32(3), pp. 211-226. https://doi.org/10.1037/pas0000779
Item response theory modeling for examinee-selected items with rater effect
Liu, Chen-Wei, Qiu, Xue-Lan and Wang, Wen-Chung. (2019). Item response theory modeling for examinee-selected items with rater effect. Applied Psychological Measurement. 43(6), pp. 435-448. https://doi.org/10.1177/0146621618798667