Statistical tests conducted with school environment data: The effect of teachers being clustered in schools

Journal article

Dorman, Jeffrey Paul. (2009). Statistical tests conducted with school environment data: The effect of teachers being clustered in schools. Learning Environments Research. 12(2), pp. 85 - 99. https://doi.org/10.1007/s10984-009-9054-y

Authors	Dorman, Jeffrey Paul
Abstract	This article discusses the effect of clustering on statistical tests conducted with school environment data. Because most school environment studies involve the collection of data from teachers nested within schools, the hierarchical nature to these data cannot be ignored. In particular, this article considers the influence of intraschool correlations on tests of statistical significance conducted with the individual teacher as the unit of analysis. Theory that adjusts t test scores for nested data in two-group comparisons is presented and applied to school environment data. This article demonstrates that Type I error rates inflate greatly as the intraschool correlation increases. Because data analysis techniques that recognise the clustering of teachers in schools are essential, it is recommended that either multilevel analysis or adjustments to statistical parameters be undertaken in school environment studies involving nested data.
Year	2009
Journal	Learning Environments Research
Journal citation	12 (2), pp. 85 - 99
Publisher	Springer Netherlands
ISSN	1573-1855
Digital Object Identifier (DOI)	https://doi.org/10.1007/s10984-009-9054-y
Scopus EID	2-s2.0-67349160466
Page range	85 - 99
Research Group	School of Education
Place of publication	Netherlands

Permalink -

https://acuresearchbank.acu.edu.au/item/885vw/statistical-tests-conducted-with-school-environment-data-the-effect-of-teachers-being-clustered-in-schools

90
total views
0
total downloads
0
views this month
0
downloads this month

These values are for the period from 19th October 2020, when this repository was created.

Export as

Related outputs

The development, validation and use of the Rural and Remote Teaching, Working, Living and Learning Environment Survey (RRTWLLES)

Dorman, Jeffrey, Kennedy, Joy and Young, Janelle. (2015). The development, validation and use of the Rural and Remote Teaching, Working, Living and Learning Environment Survey (RRTWLLES). Learning Environments Research. 18(1), pp. 15 - 32. https://doi.org/10.1007/s10984-014-9171-0

The school choice decision of Catholic families

Kennedy, Joy, Mulholland, J. and Dorman, J.. (2011). The school choice decision of Catholic families. Journal of Catholic School Studies (print version). 83(1), pp. 6 - 19.

Partitioning the variance in scores on classroom environment instruments

Dorman, Jeffrey. (2009). Partitioning the variance in scores on classroom environment instruments. Australian Journal of Educational and Developmental Psychology. 9, pp. 18 - 31.

Identifying exemplary science teachers through their students' perceptions of the assessment process

Waldrip, Bruce, Fisher, Darrell and Dorman, Jeffrey Paul. (2009). Identifying exemplary science teachers through their students' perceptions of the assessment process. Research in Science and Technological Education.

Associations between psychosocial environment and outcomes in technology-rich classrooms in Australian secondary schools

Dorman, Jeffrey Paul. (2009). Associations between psychosocial environment and outcomes in technology-rich classrooms in Australian secondary schools. Research in Education.

Identifying exemplary science teachers through students' perceptions of their learning environment

Waldrip, Bruce G., Fisher, Darrell L. and Dorman, Jeffrey Paul. (2009). Identifying exemplary science teachers through students' perceptions of their learning environment. Learning Environments Research. 12(1), pp. 1 - 13. https://doi.org/10.1007/s10984-008-9049-0

Using the student perceptions of assessment questionnaire (SPAQ) to develop an assessment typology for science classes

Dorman, Jeffrey, Waldrip, Bruce and Fisher, D. L.. (2008). Using the student perceptions of assessment questionnaire (SPAQ) to develop an assessment typology for science classes. Journal of Science Education: revista de edicacion en ciencias. 9(1), pp. 13 - 17.

The effect of clustering on statistical tests: An illustration using classroom environment data

Dorman, Jeffrey. (2008). The effect of clustering on statistical tests: An illustration using classroom environment data. Educational Psychology : An international journal of experimental educational psychology. 28(5), pp. 583 - 595. https://doi.org/10.1080/01443410801954201

Conducting statistical tests with data from clustered school samples

Dorman, Jeffrey. (2008). Conducting statistical tests with data from clustered school samples. International Journal of Research and Method in Education. 31(2), pp. 113 - 124. https://doi.org/10.1080/17437270802124368

Determinants of classroom environment in Queensland secondary schools: A multilevel reanalysis

Dorman, Jeffrey. (2008). Determinants of classroom environment in Queensland secondary schools: A multilevel reanalysis. Educational Research and Evaluation: an international journal on theory and practice. 14(5), pp. 429 - 444. https://doi.org/10.1080/13803610802337640

Students' perceptions of assessment process: Questionnaire development and validation

Waldrip, Bruce, Fisher, Darrell and Dorman, Jeffrey. (2008). Students' perceptions of assessment process: Questionnaire development and validation. International Conference on Science, Mathematics and Technology Education. Australia: Curtin University of Technology. pp. 561 - 568

Use of multitrait-multimethod modelling to validate actual and preferred forms of the What Is Happening In this Class? (WIHIC) questionnaire

Dorman, Jeffrey Paul. (2008). Use of multitrait-multimethod modelling to validate actual and preferred forms of the What Is Happening In this Class? (WIHIC) questionnaire. Learning Environments Research. 11(3), pp. 179 - 193. https://doi.org/10.1007/s10984-008-9043-6

Using student perceptions to compare actual and preferred classroom environment in Queensland schools

Dorman, Jeffrey. (2008). Using student perceptions to compare actual and preferred classroom environment in Queensland schools. Educational Studies. 34(4), pp. 299 - 308. https://doi.org/10.1080/03055690802034484

Psychosocial environment and affective outcomes in technology-rich classrooms: Testing a causal model

Dorman, Jeffrey P. and Fraser, Barry J.. (2008). Psychosocial environment and affective outcomes in technology-rich classrooms: Testing a causal model. Social Psychology of Education. 12(1), pp. 77 - 99. https://doi.org/10.1007/s11218-008-9069-8

Practicum in Teacher Education :Implications for School Leaders

Kennedy, Joy and Dorman, Jeffrey Paul. (2007). Practicum in Teacher Education :Implications for School Leaders. Directions for Catholic Educational Leadership in the 21st Century: The vision, challenges and reality: Proceedings of the 2007 International Conference on Catholic Educational leadership. Sydney, Australia: Australian Catholic University. pp. 1 - 33

Using students' assessment of classroom environment to develop a typology of secondary school classrooms

Dorman, Jeffrey, Aldridge, Jill and Fraser, Barry. (2006). Using students' assessment of classroom environment to develop a typology of secondary school classrooms. International Education Journal: Comparative Perspectives. 7(7), pp. 906 - 915.

Initial use of the perceptions of assessment tasks inventory (PATI) in English secondary schools

Dorman, Jeffrey and Knightley, Wendy. (2006). Initial use of the perceptions of assessment tasks inventory (PATI) in English secondary schools. Alberta Journal of Educational Research. 52(3), pp. 196 - 199.

Development and validation of an instrument to assess secondary school students' perceptions of assessment tasks

Dorman, Jeffrey and Knightley, Wendy. (2006). Development and validation of an instrument to assess secondary school students' perceptions of assessment tasks. Educational Studies. 32(1), pp. 47 - 58. https://doi.org/10.1080/03055690500415951

Learning environments; attitudes; efficacy and perceptions of assessment: A lisrel analysis

Dorman, Jeffrey Paul and Fisher, Darrell. (2006). Learning environments; attitudes; efficacy and perceptions of assessment: A lisrel analysis. In In D. L. Fisher, L. Darrell L. and M.S. Kyine (Ed.). Contemporary Approaches to Research on Learning Environments pp. 1 - 28 World Views.

Using structural equation modelling to investigate associations between environments and outcomes in technology-rich: Outcomes-focused classrooms in Australian secondary schools

Dorman, Jeffrey Paul, Aldridge, Jill and Fraser, Barry. (2006). Using structural equation modelling to investigate associations between environments and outcomes in technology-rich: Outcomes-focused classrooms in Australian secondary schools. In In D. L. Fisher, L. Darrell L. and M.S. Kyine (Ed.). Contemporary Approaches to Research on Learning Environments pp. 425 - 447 World Views.

Assessment of science teaching efficacy of preservice teachers in an Australian university

Mulholland, Judith, Dorman, Jeffrey and Odgers, Barbara. (2004). Assessment of science teaching efficacy of preservice teachers in an Australian university. Journal of Science Teacher Education. 15(4), pp. 313 - 331.

Testing a model for teacher burnout

Dorman, Jeffrey. (2003). Testing a model for teacher burnout. Australian Journal of Educational and Developmental Psychology. 3, pp. 35 - 47.

Validation and use of the Catholic school graduate characteristics inventory

Dorman, Jeffrey. (2003). Validation and use of the Catholic school graduate characteristics inventory. Catholic Education: a journal of inquiry and practice. 7(2), pp. 165 - 180.

A cross-national investigation of students' perceptions of mathematics classroom environment and academic efficacy in secondary schools

Dorman, Jeffrey, Ada, Joan and Ferguson, Janet. (2003). A cross-national investigation of students' perceptions of mathematics classroom environment and academic efficacy in secondary schools. International Journal for Mathematics Teaching and Learning.

Leadership succession in New South Wales Catholic schools: Identifying potential principals

Dorman, Jeffrey and d'Arbon, Tony. (2003). Leadership succession in New South Wales Catholic schools: Identifying potential principals. Educational Studies. 29(March), pp. 127 - 139. https://doi.org/10.1080/03055690303281

Relationship between school and classroom environment and teacher burnout: A lisrel analysis

Dorman, Jeffrey. (2003). Relationship between school and classroom environment and teacher burnout: A lisrel analysis. Social Psychology of Education: an international journal. 6(2), pp. 107 - 127. https://doi.org/10.1023/A:1023296126723

Cross-national validation of the What is Happening in this Class? (WIHIC) questionnaire using confirmatory factor analysis

Dorman, Jeffrey. (2003). Cross-national validation of the What is Happening in this Class? (WIHIC) questionnaire using confirmatory factor analysis. Learning Environments Research. 6(231), pp. 231 - 245. https://doi.org/10.1023/A:1027355123577

Assessing impediments to leadership succession in Australian Catholic schools

d'Arbon, Tony and Dorman, Jeffrey. (2003). Assessing impediments to leadership succession in Australian Catholic schools. School Leadership and Management. 23(1), pp. 25 - 40. https://doi.org/10.1080/1363243032000080014

Psychological environment and student self-handicapping in secondary school mathematics classes: A cross-national study

Dorman, Jeffrey, Ada, Joan and Ferguson, Janet. (2002). Psychological environment and student self-handicapping in secondary school mathematics classes: A cross-national study. Educational Psychology : An international journal of experimental educational psychology. 22(5), pp. 499 - 511.

Comparing the university-level environment in the Australian Catholic university with other universities

Dorman, Jeffrey, Ada, Joan and Ferguson, Janet. (2002). Comparing the university-level environment in the Australian Catholic university with other universities. Christian Higher Education. 1(1), pp. 39 - 53.

The self-handicapping phenomenon

Dorman, Jeffrey and Ferguson, Janet. (2002). The self-handicapping phenomenon. Kappa Delta Pi Record (print version). 38(2), pp. 64 - 67.

Development and validation of the extended practicum learning environment inventory

Dorman, Jeffrey Paul and Kennedy, Joy. (2002). Development and validation of the extended practicum learning environment inventory. In J. Reid and T. Brown (Ed.). Challenging Futures: Changing Agendas in Teacher Education. Armidale, Australia: University of New England.

Associations between psychological environments in religious education classes and students' attitudes towards Christianity

Dorman, Jeffrey, McRobbie, Campbell and Foster, William. (2002). Associations between psychological environments in religious education classes and students' attitudes towards Christianity. Religious Education: a platform for the free discussion of issues in the field of religion and their bearing on education. 97(1), pp. 23 - 43.