Validating and Profiling Emotional Engagement in Human–GenAI Collaborative Language Learning
pdf
html

Keywords

emotional engagement, generative artificial intelligence, human–GenAI collaboration, foreign language learning, scale validation, latent profile analysis

How to Cite

Liu, S., & Wang, Y. (2026). Validating and Profiling Emotional Engagement in Human–GenAI Collaborative Language Learning. Emerging Educational Technologies, 1(1). https://doi.org/10.65553/EET.260104

Abstract

Despite growing interest in generative artificial intelligence (GenAI) in language education, emotional engagement in human–GenAI collaborative language learning remains insufficiently conceptualized and measured. To address this gap, this study examined the psychometric properties of a 15-item measure of emotional engagement and identified learner profiles. Data came from 504 Chinese university students. Exploratory factor analysis in one random half of the sample (n = 252) yielded three factors: Positive Affective Engagement, Affective Connectedness in Collaboration, and Emotional Disaffection. Confirmatory factor analysis in the other half supported this structure (CFI = .926, TLI = .911, RMSEA = .073, SRMR = .048) and indicated better fit than one- and two-factor alternatives. The subscales showed good internal consistency (ω = .84–.87), convergent validity (AVE = .51–.58), and discriminant validity (HTMT ≤ .52). Configural, metric, scalar, and strict measurement invariance were supported across gender and year of study. Latent profile analysis identified high (29.8%), moderate (50.4%), and low (19.8%) engagement profiles that differed in level rather than shape. These findings support a multidimensional conceptualization of emotional engagement in human–GenAI collaborative language learning and provide a brief, psychometrically supported instrument for examining learners' affective experiences.

pdf
html

References

Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, Article 100745. https://doi.org/10.1016/j.asw.2023.100745

Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5(1), 184–186. https://doi.org/10.1177/2167702616657069

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.

Carlo, M. (2026). Same CEFR, different prompt: Instruction-paraphrase sensitivity in CEFR-conditioned language model writing. Applied Language Sciences, 2, 260113. https://doi.org/10.65553/ALS.260113

Celeux, G., & Soromenho, G. (1996). An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13(2), 195–212. https://doi.org/10.1007/BF01246098

Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20, Article 43. https://doi.org/10.1186/s41239-023-00411-8

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Dewaele, J.-M., & Li, C. (2021). Teacher enthusiasm and students’ social-behavioral learning engagement: The mediating role of student enjoyment and boredom in Chinese EFL classes. Language Teaching Research, 25(6), 922–945. https://doi.org/10.1177/13621688211014538

Dewaele, J.-M., & MacIntyre, P. D. (2014). The two faces of Janus? Anxiety and enjoyment in the foreign language classroom. Studies in Second Language Learning and Teaching, 4(2), 237–274. https://doi.org/10.14746/ssllt.2014.4.2.5

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218–226. https://doi.org/10.1037/0003-066X.56.3.218

Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045

Guo, N., & Li, R. (2022). Measuring Chinese English-as-a-foreign-language learners’ resilience: Development and validation of the foreign language learning resilience scale. Frontiers in Psychology, 13, 1046340. https://doi.org/10.3389/fpsyg.2022.1046340

Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But…. Communication Methods and Measures, 14(1), 1–24. https://doi.org/10.1080/19312458.2020.1718629

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8

Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2024). Engagement in language learning: A systematic review of 20 years of research methods and definitions. Language Teaching Research, 28(1), 201–230. https://doi.org/10.1177/13621688211001289

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447

Horwitz, E. K., Horwitz, M. B., & Cope, J. (1986). Foreign language classroom anxiety. The Modern Language Journal, 70(2), 125–132. https://doi.org/10.1111/j.1540-4781.1986.tb05256.x

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610

Järvelä, S., Nguyen, A., & Hadwin, A. (2023). Human and artificial intelligence collaboration for socially shared regulation in learning. British Journal of Educational Technology, 54(5), 1057–1076. https://doi.org/10.1111/bjet.13325

Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28(12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1

Jin, S.-H. (2025). Measures of learner-generative ai relationships. Computers and Education Open, 8, 100258. https://doi.org/10.1016/j.caeo.2025.100258

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868

Lee, Y.-J., Parrila, R., & Colenbrander, D. (2026). Adopt, adapt, or reject? Analysing student-AI interaction in university curriculum-aligned writing tasks. Applied Language Sciences, 2, 260102. https://doi.org/10.65553/ALS.260102

Li, C., Jiang, G., & Dewaele, J.-M. (2018). Understanding Chinese high school students’ foreign language enjoyment: Validation of the Chinese version of the Foreign Language Enjoyment scale. System, 76, 183–196. https://doi.org/10.1016/j.system.2018.06.004

Liu, G., & Ma, C. (2024). Measuring EFL learners’ use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 18(2), 125–138. https://doi.org/10.1080/17501229.2023.2240316

MacIntyre, P., & Gregersen, T. (2012). Emotions that facilitate language learning: The positive-broadening power of the imagination. Studies in Second Language Learning and Teaching, 2(2), 193–213. https://doi.org/10.14746/ssllt.2012.2.2.4

Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2

Marsh, H. W., Lüdtke, O., Trautwein, U., & Morin, A. J. S. (2009). Classical latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling, 16(2), 191–225. https://doi.org/10.1080/10705510902751010

Molenaar, I. (2022). Towards hybrid human-AI learning technologies. European Journal of Education, 57(4), 632–645. https://doi.org/10.1111/ejed.12527

Morin, A. J. S., & Marsh, H. W. (2015). Disentangling shape from level effects in person-centered analyses: An illustration based on university teachers’ multidimensional profiles of effectiveness. Structural Equation Modeling, 22(1), 39–59. https://doi.org/10.1080/10705511.2014.919825

Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103. https://doi.org/10.1111/0022-4537.00153

Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14(4), 535–569. https://doi.org/10.1080/10705510701575396

Oga-Baldwin, W. L. Q. (2019). Acting, thinking, feeling, making, collaborating: The engagement process in foreign language learning. System, 86, Article 102128. https://doi.org/10.1016/j.system.2019.102128

Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9

Philp, J., & Duchesne, S. (2016). Exploring engagement in tasks in the language classroom. Annual Review of Applied Linguistics, 36, 50–72. https://doi.org/10.1017/S0267190515000094

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

Reeve, J., & Tseng, C.-M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36(4), 257–267. https://doi.org/10.1016/j.cedpsych.2011.05.002

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68

Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13. https://doi.org/10.1080/00461520.2014.1002924

Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69(3), 493–525. https://doi.org/10.1177/0013164408323233

Skjuve, M., Følstad, A., Fostervold, K. I., & Brandtzaeg, P. B. (2021). My chatbot companion—A study of human-chatbot relationships. International Journal of Human-Computer Studies, 149, Article 102601. https://doi.org/10.1016/j.ijhcs.2021.102601

Spurk, D., Hirschi, A., Wang, M., Valero, D., & Kauffeld, S. (2020). Latent profile analysis: A review and “how to” guide of its application within vocational behavior research. Journal of Vocational Behavior, 120, Article 103445. https://doi.org/10.1016/j.jvb.2020.103445

Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, Article 100752. https://doi.org/10.1016/j.asw.2023.100752

Svalberg, A. M.-L. (2009). Engagement with language: Interrogating a construct. Language Awareness, 18(3–4), 242–258. https://doi.org/10.1080/09658410903197264

Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second language anxiety and achievement: A meta-analysis. Studies in Second Language Acquisition, 41(2), 363–387. https://doi.org/10.1017/S0272263118000311

Teng, M. F. (2026). The impact of GenAI feedback on Chinese EFL students’ emotional engagement: A mixed effect modelling approach. The Asia-Pacific Education Researcher, 1–17. https://doi.org/10.1007/s40299-025-01067-w

Teng, M. F., & Huang, J. (2025). Incorporating ChatGPT for EFL writing and its effects on writing engagement. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 15(1), 1–21. https://doi.org/10.4018/IJCALLT.367874

Teng, M. F., & Huang, J. (2026). Assessing ChatGPT feedback for EFL learners’ engagement and writing performance. International Journal of Applied Linguistics. https://doi.org/10.1111/ijal.70155

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002

Wang, X., Liu, Q., Pang, H., Tan, S. C., Lei, J., Wallace, M. P., & Li, L. (2023). What matters in AI-supported learning: A study of human-AI interactions in language learning using cluster analysis and epistemic network analysis. Computers & Education, 194, Article 104703. https://doi.org/10.1016/j.compedu.2022.104703

Wang, Y., & Xue, L. (2024). Using AI-driven chatbots to foster Chinese EFL students’ academic engagement: An intervention study. Computers in Human Behavior, 159, 108353. https://doi.org/10.1016/j.chb.2024.108353

Warschauer, M., Tseng, W., Yim, S., Webster, T., Jacob, S., Du, Q., & Tate, T. (2023). The affordances and contradictions of AI-generated text for writers of English as a second or foreign language. Journal of Second Language Writing, 62, Article 101071. https://doi.org/10.1016/j.jslw.2023.101071

Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127

Watson Todd, R. (2026). Using generative AI to revise articles: How much is too much? Applied Language Sciences, 2, 260108. https://doi.org/10.65553/ALS.260108

Wong, W. L., & Fong, S. (2025). A new era with artificial intelligence powered learner corpora for teaching and learning of English as a second language in primary school education. Applied Language Sciences, 1, 250105. https://doi.org/10.65553/ALS.250105

Xu, X. (2026). Emotions and engagement in generative AI-supported L2 writing: A systematic review. Emerging Educational Technologies, 1, 260103. https://doi.org/10.65553/EET.260103

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28(11), 13943–13967. https://doi.org/10.1007/s10639-023-11742-4

Zhang, S., & Liu, X. (2025). Learner emotions in AI-assisted English as a second/foreign language learning: A systematic review of empirical studies. Frontiers in Psychology, 16, Article 1652806. https://doi.org/10.3389/fpsyg.2025.1652806

Zou, M., & Huang, L. (2023). To use or not to use? Understanding doctoral students’ acceptance of ChatGPT in writing through technology acceptance model. Frontiers in Psychology, 14, Article 1259531. https://doi.org/10.3389/fpsyg.2023.1259531

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2026 Songming Liu, Yongliang Wang (Author)

Views: 37Downloads: 37

Downloads

Download data is not yet available.
Creative Commons License All articles published in Emerging Educational Technologies are open access and distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are properly credited. Copyright is retained by the author(s).