Assessing the writing achievement of young struggling writers: Application of generalizability theory

This study examined the number of writing samples needed to obtain a reliable estimate of young struggling writers’ capabilities. It further assessed if performance in one genre was reflective of performance in other genres for these children. Second- and third-grade students (81 boys, 56 girls), who were identified as struggling writers in need of special assistance by their teacher and scored at the 25th percentile or lower on a norm-referenced story-writing test, wrote four compositions: a story, personal narrative, opinion essay, and informative text. Applying generalizability theory (G-theory), students’ scores on three writing measures (total number of words [TNW], vocabulary diversity, and writing quality) for the four compositions were each portioned into variance due to the following sources: students, writing tasks, and the interaction between students and writing tasks. We found that 14, 8, and 11 compositions, respectively, would be needed to obtain a reliable estimate of these students’ writing capabilities in terms of TNW, vocabulary diversity, and writing quality. Furthermore, how well these students wrote in one genre provided a weak prediction of how well they wrote in other genres.