Using multi-group confirmatory factor analysis to evaluate cross-cultural research: identifying and understanding non-invariance

Journal article


Brown, Gavin T.L., Harris, Lois, O'Quin, Chrissie and Lane, Kenneth E.. (2017). Using multi-group confirmatory factor analysis to evaluate cross-cultural research: identifying and understanding non-invariance. International Journal of Research and Method in Education. 40(1), pp. 66 - 90. https://doi.org/10.1080/1743727X.2015.1070823
AuthorsBrown, Gavin T.L., Harris, Lois, O'Quin, Chrissie and Lane, Kenneth E.
Abstract

Multi-group confirmatory factor analysis (MGCFA) allows researchers to determine whether a research inventory elicits similar response patterns across samples. If statistical equivalence in responding is found, then scale score comparisons become possible and samples can be said to be from the same population. This paper illustrates the use of MGCFA by examining survey results relating to practising teachers' conceptions of feedback in two very different jurisdictions (Louisiana, USA, n = 308; New Zealand, n = 518), highlighting challenges which can occur when conducting this kind of cross-cultural research. As the two contexts had very different policies and practices around educational assessment, it was considered possible that a common research inventory may elicit non-equivalent responding, leading to non-invariance. Independent models for each group and a joint model for all participants were tested for invariance using MGCFA and all were inadmissible for one of the two groups. Inspection of joint model differences in item loadings, scale reliabilities, and scale inter-correlations established the extent of non-invariance. This paper discusses the implications of non-invariance within this particular study and identifies difficulties in using an inventory in cross-cultural settings. It also provides suggestions about how to increase the likelihood that a common factor structure can be recovered.

Keywordsconfirmatory factor analysis; cross-cultural research; invariance testing; survey research
Year2017
JournalInternational Journal of Research and Method in Education
Journal citation40 (1), pp. 66 - 90
PublisherRoutledge, Taylor & Francis Group
ISSN1743-727X
Digital Object Identifier (DOI)https://doi.org/10.1080/1743727X.2015.1070823
Scopus EID2-s2.0-84937915425
Page range66 - 90
Research GroupInstitute for Learning Sciences and Teacher Education (ILSTE)
Publisher's version
File Access Level
Controlled
Place of publicationUnited Kingdom
Permalink -

https://acuresearchbank.acu.edu.au/item/8qw87/using-multi-group-confirmatory-factor-analysis-to-evaluate-cross-cultural-research-identifying-and-understanding-non-invariance

Restricted files

Publisher's version

  • 115
    total views
  • 0
    total downloads
  • 1
    views this month
  • 0
    downloads this month
These values are for the period from 19th October 2020, when this repository was created.

Export as

Related outputs

How are examinations inclusive for students with disabilities in higher education? A sociomaterial analysis
Tai, Joanna, Mahoney, Paige, Ajjawi, Rola, Bearman, Margaret, Dargusch, Joanne, Dracup, Mary and Harris, Lois. (2023). How are examinations inclusive for students with disabilities in higher education? A sociomaterial analysis. Assessment and Evaluation in Higher Education. 48(3), pp. 390-402. https://doi.org/10.1080/02602938.2022.2077910
Low socioeconomic status students transitioning from Vocational Education and Training (VET) to university : Examining definitions of success
Vanderburg, Robert, Harris, Lois, Dargusch, Joanne and Richardson, Susan. (2023). Low socioeconomic status students transitioning from Vocational Education and Training (VET) to university : Examining definitions of success. Higher Education Research and Development. 42(3), pp. 742-756. https://doi.org/10.1080/07294360.2022.2089098
Reviewing the evidence base : Attraction, pathways and retention : A focus on attracting candidates
Wyatt-Smith, Claire, Holloway, Jessica, Alexander, Colette, Harris, Lois and Marcy, Ariel. (2022). Reviewing the evidence base : Attraction, pathways and retention : A focus on attracting candidates Sydney, Australia: Association of Independent Schools New South Wales.
Learning progression–based assessments : A systematic review of student and teacher uses
Harris, Lois Ruth, Adie, Lenore and Wyatt-Smith, Claire. (2022). Learning progression–based assessments : A systematic review of student and teacher uses. Review of Educational Research. 92(6), pp. 996-1040. https://doi.org/10.3102/00346543221081552
‘So you can make it fast or make it up’: K–12 teachers’ perspectives on technology’s affordances and constraints when supporting distance education learning
Kate Ames, Lois Ruth Harris, Joanne Dargusch and Corey Bloomfield. (2021). ‘So you can make it fast or make it up’: K–12 teachers’ perspectives on technology’s affordances and constraints when supporting distance education learning. The Australian Educational Researcher. 48, p. 359–376. https://doi.org/10.1007/s13384-020-00395-8
Examining research into the use of data walls for teaching and learning : How are they being implemented within data use cycles?
Adie, Lenore, Harris, Lois and Wyatt-Smith, Claire. (2020). Examining research into the use of data walls for teaching and learning : How are they being implemented within data use cycles? Teaching and Teacher Education. 89, p. Article: 103012. https://doi.org/10.1016/j.tate.2019.103012
Using data walls to display assessment results : A review of their effective impacts on teachers and students
Harris, Lois, Wyatt-Smith, Claire and Adie, Lenore. (2020). Using data walls to display assessment results : A review of their effective impacts on teachers and students. Teachers and Teaching: Theory and Practice. 26(1), pp. 50-66. https://doi.org/10.1080/13540602.2020.1739018
Catering for diversity in the digital age: Reconsidering equity in assessment practices
Harris, Lois Ruth and Dargusch, Joanne. (2020). Catering for diversity in the digital age: Reconsidering equity in assessment practices. In In Margaret Bearman, Phillip Dawson, Rola Ajjawi, Joanne Tai and David Boud (Ed.). Re-imagining University Assessment in a Digital World pp. 95-110 Springer Nature. https://doi.org/10.1007/978-3-030-41956-1
Catering for ‘very different kids’: distance education teachers’ understandings of and strategies for student engagement
Harris, Lois, Dargusch, Joanne, Ames, Kate and Bloomfield, Corey. (2020). Catering for ‘very different kids’: distance education teachers’ understandings of and strategies for student engagement. International Journal of Inclusive Education. https://doi.org/10.1080/13603116.2020.1735543
Not playing the game: student assessment resistance as a form of agency
Harris, Lois, Brown, Gavin T.L. and Dargusch, Joanne. (2018). Not playing the game: student assessment resistance as a form of agency. The Australian Educational Researcher. 45(1), pp. 125 - 140. https://doi.org/10.1007/s13384-018-0264-0
Methods in feedback research
Brown, Gavin T. L. and Harris, Lois. (2018). Methods in feedback research. In In A. A. Lipnevich and J. K. Smith (Ed.). The Cambridge Handbook of Instructional Feedback pp. 97 - 119 Cambridge University Press (Anthem Press). https://doi.org/10.1017/9781316832134
Creating first-year assessment support: lecturer perspectives and student access
Dargusch, Joanne, Harris, Lois, Reid-Searl, Kerry and Taylor, Benjamin A.. (2017). Creating first-year assessment support: lecturer perspectives and student access. Distance Education. 38(1), pp. 106 - 122. https://doi.org/10.1080/01587919.2017.1299566
Understanding teacher aides’ definitions of reading: implications for classroom practice
Harris, Lois, Davidson, Christina R. and Aprile, Kerry T.. (2015). Understanding teacher aides’ definitions of reading: implications for classroom practice. The Australian Educational Researcher. 42(5), pp. 627 - 644. https://doi.org/10.1007/s13384-015-0181-4
Analysis of New Zealand primary and secondary student peer- and self-assessment comments: applying Hattie and Timperley’s feedback model
Harris, Lois, Brown, Gavin T.L. and Harnett, Jennifer A.. (2015). Analysis of New Zealand primary and secondary student peer- and self-assessment comments: applying Hattie and Timperley’s feedback model. Assessment in Education. 22(2), pp. 265 - 281. https://doi.org/10.1080/0969594X.2014.976541
'I can sort of slot into many different roles': examining teacher aide roles and their implications for practice
Harris, Lois and Aprile, Kerry T.L.. (2015). 'I can sort of slot into many different roles': examining teacher aide roles and their implications for practice. School Leadership and Management. 35(2), pp. 140 - 162. https://doi.org/10.1080/13632434.2014.992774
The future of self-assessment in classroom practice : Reframing self-assessment as a core competency
Brown, Gavin T. L. and Harris, Lois. (2014). The future of self-assessment in classroom practice : Reframing self-assessment as a core competency. Frontline Learning Research. 2(1), pp. 22-30. https://doi.org/10.14786/flr.v2i1.24
Understanding classroom feedback practices: a study of New Zealand student experiences, perceptions, and emotional responses
Harris, Lois, Brown, Gavin T.L. and Harnett, Jennifer A.. (2014). Understanding classroom feedback practices: a study of New Zealand student experiences, perceptions, and emotional responses. Educational Assessment, Evaluation and Accountability. 26(2), pp. 107 - 133. https://doi.org/10.1007/s11092-013-9187-5
Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation
Harris, Lois and Brown, Gavin T. L.. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation. Teaching and Teacher Education. 36, pp. 101 - 111. https://doi.org/10.1016/j.tate.2013.07.008