Emerging Educational
Technologies
Emerging Educational Technologies, 2026, 1, 260104 https://doi.org/10.65553/EET.260104
Validating and Profiling Emotional Engagement in Human–GenAI Collaborative Language Learning
Songming Liua Yongliang Wangb
aMacao
Polytechnic University, China
bNorth China University of Water Resources and Electric Power, China
Submission date: May 1, 2026; Revision date:
October 25, 2026; Online: November 20, 2026
Abstract
Despite growing interest
in generative artificial intelligence (GenAI) in language education, emotional
engagement in human–GenAI collaborative language learning remains
insufficiently conceptualized and measured. To address this gap, this study
examined the psychometric properties of a 15-item measure of emotional
engagement and identified learner profiles. Data came from 504 Chinese
university students. Exploratory factor analysis in one random half of the
sample (n = 252) yielded three factors: Positive Affective Engagement,
Affective Connectedness in Collaboration, and Emotional Disaffection.
Confirmatory factor analysis in the other half supported this structure (CFI =
.926, TLI = .911, RMSEA = .073, SRMR = .048) and indicated better fit than one-
and two-factor alternatives. The subscales showed good internal consistency (ω
= .84–.87), convergent validity (AVE = .51–.58), and discriminant validity
(HTMT ≤ .52). Configural, metric, scalar, and strict measurement invariance
were supported across gender and year of study. Latent profile analysis
identified high (29.8%), moderate (50.4%), and low (19.8%) engagement profiles
that differed in level rather than shape. These findings support a
multidimensional conceptualization of emotional engagement in human–GenAI
collaborative language learning and provide a brief, psychometrically supported
instrument for examining learners' affective experiences.
Keywords: emotional engagement,
generative artificial intelligence, human–GenAI collaboration, foreign language
learning, scale validation, latent profile analysis
Introduction
Since the public release of ChatGPT in late 2022,
generative artificial intelligence (GenAI) has rapidly become an increasingly
prominent resource in higher education (Chan & Hu, 2023; Kasneci et al., 2023) and primary education (Wong & Fong, 2025). Language learning is among the domains substantially influenced by
these developments, as large language models can engage in conversations,
provide explanations and translations, model target-language expressions, and
offer immediate feedback on learners’ writing (Barrot, 2023; Kohnke et al., 2023; Lee
et al., 2026; Warschauer et al.,
2023). Rather than merely
consulting GenAI as a supplementary tool, language learners increasingly engage
in collaborative processes involving idea generation, meaning negotiation, and
iterative revision (Su et al., 2023; Watson Todd, 2026; Yan,
2023). Such interactions reflect
an emerging form of human–AI collaboration in which learners and intelligent
systems contribute to shared learning tasks (Järvelä et al., 2023; Molenaar, 2022). This evolving relationship highlights the importance of
understanding not only how learners interact with GenAI but also how they
experience these interactions emotionally.
Human–GenAI collaboration involves emotional as well as
cognitive processes. While emotional engagement in human–GenAI collaboration
can be systematically measured, the linguistic stability of LLM-generated texts
across proficiency prompts remains high (Carlo, 2026). Previous
research has consistently demonstrated that emotions influence learners’
effort, persistence, and learning outcomes (Fredricks et al., 2004; Pekrun, 2006). In second language acquisition, research has expanded beyond its
traditional emphasis on foreign language anxiety (Horwitz et al., 1986; Teimouri et al., 2019) to encompass positive emotional experiences, particularly enjoyment (Dewaele &
MacIntyre, 2014; MacIntyre &
Gregersen, 2012). Within this
development, learner engagement has emerged as a central construct for
explaining the quality of learners’ participation in language learning
activities (Hiver et al., 2024; Philp
& Duchesne, 2016). Emotional
engagement, referring to learners’ affective reactions to and involvement in
learning activities, represents an important component of this broader
construct (Skinner et al., 2009). In human–GenAI collaborative language learning, such engagement may
involve not only enjoyment and interest in collaborative tasks but also
feelings of connection with GenAI and negative emotional responses when
collaboration proves unsatisfactory. These potentially distinct experiences
highlight the need for a multidimensional understanding of emotional engagement
in this emerging learning context.
Despite growing research interest in GenAI-assisted
language learning, several gaps remain in understanding and measuring learners’
emotional engagement. First, existing quantitative research has predominantly
examined learners’ technological acceptance, perceptions, and behavioral
intentions (Liu
& Ma, 2024; Zou & Huang, 2023), although increasing attention has been directed toward
emotional experiences and academic engagement (Wang & Xue, 2024; Zhang & Liu, 2025). Available instruments, however, have largely originated from
conventional classroom contexts and may not adequately capture the relational
characteristics of human–GenAI collaboration, particularly learners’ perceived
emotional connection with an AI partner (Nass & Moon, 2000; Skjuve et al., 2021). Moreover, positive affect and emotional disaffection have often been
examined separately, limiting an integrated understanding of their distinct
contributions to emotional engagement (Dewaele & MacIntyre, 2014). Second, limited evidence is available regarding whether
measures of emotional engagement function equivalently across different learner
groups, despite measurement invariance being essential for meaningful
comparisons (Putnick & Bornstein, 2016). Third, existing variable-centered approaches provide
valuable insights into general patterns of emotional engagement but may
overlook heterogeneity in how positive affect, affective connectedness, and
emotional disaffection coexist within individual learners (see a systematic
review, Xu, 2026). Person-centered methods such as latent profile analysis
can help identify such heterogeneity (Spurk et al., 2020). Although
person-centered approaches have increasingly been applied to engagement in
GenAI-mediated language learning (Teng & Huang, 2025, 2026),
research specifically examining the multidimensional profiles of emotional
engagement in human–GenAI collaboration remains limited (Teng, 2026).
To address these gaps, the present study examines emotional
engagement in human–GenAI collaborative language learning using data from 504
Chinese university students. Specifically, it evaluates the factor structure,
reliability, validity, and measurement invariance of a 15-item instrument
comprising Positive Affective Engagement, Affective Connectedness in
Collaboration, and Emotional Disaffection. It further employs latent profile
analysis to identify distinct patterns of emotional engagement and examines whether
profile membership varies across learner background characteristics. By
integrating psychometric evaluation with person-centered analysis, this study
seeks to advance understanding of both the multidimensional structure of
emotional engagement and the heterogeneity of learners’ affective experiences
in human–GenAI collaborative language learning.
Literature review
Emotional
Engagement in Language Learning
Engagement refers to the quality of learners’ active
involvement in learning activities. Fredricks et al. (2004) conceptualized engagement as comprising behavioral,
cognitive, and emotional dimensions, while subsequent scholarship further
recognized its agentic and social components (Philp & Duchesne, 2016; Reeve & Tseng, 2011). In language education, engagement represents an important
process through which motivation is translated into meaningful learning
participation (Hiver et al., 2024; Oga-Baldwin, 2019; Svalberg, 2009). However, emotional engagement remains less consistently
conceptualized and operationalized than other dimensions of engagement. A
systematic review of two decades of research highlighted the frequent reliance
on general indicators of enjoyment, interest, or liking to assess emotional
engagement (Hiver et al., 2024). Similarly, Sinatra et al. (2015) emphasized the importance of aligning the specificity of engagement
measures with the learning activities under investigation. These observations
suggest that emotional engagement should be conceptualized and measured in
relation to the particular characteristics of the learning context rather than
treated as a uniform emotional response across different educational settings.
Two conceptual considerations are particularly relevant to
understanding emotional engagement in language learning. The first concerns
emotional valence. Skinner et al. (2009) distinguished emotional engagement, characterized by enthusiasm,
interest, and enjoyment, from emotional disaffection, reflected in
discouragement, anxiety, and frustration. Although negatively associated, these
dimensions represent distinguishable emotional experiences, indicating that the
absence of positive engagement does not necessarily imply the presence of
negative emotions. Similar distinctions have been established in second
language acquisition. Dewaele and MacIntyre (2014) demonstrated that foreign language enjoyment and anxiety were
moderately negatively correlated rather than opposite ends of a single
continuum, a distinction subsequently supported in the Chinese language
learning context (Li et al., 2018). Control-value theory further suggests that positive and negative
achievement emotions arise from learners’ evaluations of control and task value
and may therefore have different antecedents and consequences (Pekrun, 2006).
Positive emotions may broaden attention and facilitate the development of
learning resources (Fredrickson, 2001; MacIntyre & Gregersen, 2012), whereas anxiety has been consistently associated with
poorer language achievement (Teimouri et al., 2019). Taken together, these findings underscore the importance of
distinguishing positive emotional engagement from emotional disaffection when
examining learners’ affective experiences.
The second consideration concerns the interpersonal and
relational orientation of emotional engagement. Emotional experiences in
language learning are shaped not only by learning tasks but also by
interactions with others involved in the learning process. Philp and Duchesne (2016) emphasized that engagement in language classrooms
encompasses learners’ emotional connections with interlocutors, reflecting the
inherently social nature of language learning. From the perspective of
self-determination theory, relatedness refers to the psychological need to
experience connection and support in interpersonal relationships, which
contributes to the maintenance of learning motivation (Ryan & Deci, 2000). Empirical research has similarly demonstrated
associations between teacher-created emotional climates and learners' enjoyment
and engagement (Dewaele & Li, 2021). These perspectives suggest that emotional engagement may encompass
both affective responses to learning activities and perceived connections with
interaction partners. Although such relational experiences have traditionally
been examined in human interactions, their relevance to technology-mediated
learning warrants further consideration, particularly when learners interact
with conversational systems capable of providing responsive and personalized
support.
GenAI as a
Collaborative Partner in Language Learning
Before the emergence of GenAI, language
learning chatbots generally relied on scripted interactions and demonstrated
limitations in sustaining learners’ interest over time. Fryer et al. (2017) found that learners’ interest in speaking activities
declined following initial interactions with chatbot partners, whereas
comparable declines were not observed with human partners. This potential
novelty effect was also highlighted in a subsequent systematic review (Huang et al., 2022). Compared with earlier chatbot systems, GenAI offers
greater conversational flexibility, responsiveness, and adaptability, enabling
learners to engage in open-ended exchanges, obtain personalized explanations,
and receive immediate feedback (Kohnke et al., 2023). Research on second language writing has documented the
integration of ChatGPT into different stages of composing, including idea
generation, drafting, and revision (Barrot, 2023;
Su et al., 2023; Yan, 2023). Nevertheless, analyses
of learner interaction logs suggest considerable variation in how individuals
collaborate with AI systems (Wang et al., 2023). These
developments indicate that GenAI can support increasingly interactive forms of
language learning, although learners’ experiences may differ depending on how
collaboration unfolds.
The expanding capabilities of GenAI have also
encouraged a reconsideration of the relationship between learners and
educational technologies. Molenaar (2022) conceptualized hybrid human–AI learning as a process in
which learners and intelligent systems share aspects of control over learning
activities, while Järvelä et al. (2023)
highlighted the potential contribution of AI systems to collaborative learning
regulation. Similarly, Jeon and Lee (2023)
described the relationship between teachers and ChatGPT as complementary rather
than substitutive. Within this perspective, GenAI is not merely a passive
information source but can function as an interactive partner supporting the
completion of shared learning tasks. Importantly, this collaborative
orientation introduces an affective dimension into learner–technology
interactions. Learners may evaluate collaboration not only according to the
usefulness of AI-generated responses but also in terms of whether the
interaction feels supportive, responsive, and emotionally comfortable.
Consequently, understanding human–GenAI collaboration requires attention to
learners’ subjective experiences of interacting with an intelligent
conversational partner.
The possibility of experiencing relational
responses toward non-human systems is supported by research on human–computer
interaction. Nass and Moon (2000)
demonstrated that individuals may apply social expectations and interaction
norms to computers despite recognizing their non-human nature. Research on
companion chatbots further suggests that users can develop perceived feelings
of being understood and supported through repeated interactions (Skjuve et al., 2021).
In language learning, such perceived support may be particularly relevant
because fear of negative evaluation constitutes an important component of
foreign language anxiety (Horwitz et al., 1986). Interacting
with GenAI may provide opportunities to ask questions, explore linguistic
expressions, and practice language use without the immediate interpersonal
evaluation associated with some classroom situations. However, these perceived
relational benefits should not be equated with genuine interpersonal
understanding or emotional reciprocity. Moreover, GenAI-generated responses may
be inaccurate, ambiguous, or insufficiently responsive to learners’ needs,
potentially generating disappointment, confusion, and frustration (Barrot, 2023; Warschauer et al., 2023). Human–GenAI collaboration may therefore involve
positive emotional experiences, perceived affective connectedness, and
emotional disaffection, highlighting the importance of examining these
experiences as distinguishable dimensions.
Emotional
Engagement in Human–GenAI Collaborative Language Learning
Building on these theoretical and empirical perspectives,
the present study conceptualizes emotional engagement in human–GenAI
collaborative language learning as comprising three related but distinguishable
dimensions. Positive Affective Engagement encompasses enjoyment, interest,
excitement, accomplishment, and anticipation associated with collaborative
language learning activities. This dimension reflects positive emotional
involvement in learning and corresponds to the activating positive emotions
emphasized in control-value theory (Pekrun, 2006) and the engagement
dimension proposed by Skinner et al. (2009). Affective Connectedness in Collaboration refers to learners’
perceived emotional connection with GenAI, including feelings of support,
understanding, reduced isolation, and comfort during interaction. This
dimension draws on the relational characteristics of language learning
engagement (Philp & Duchesne, 2016; Ryan & Deci, 2000) and extends consideration of perceived social responses to non-human
conversational partners (Nass & Moon, 2000; Skjuve
et al., 2021). Emotional Disaffection
encompasses discouragement, disappointment, anxiety, stress, and frustration
arising from unsatisfactory collaborative learning experiences. It is
consistent with the distinction between engagement and disaffection proposed by
Skinner et al. (2009) and with
research demonstrating the relevance of negative emotions in foreign language
learning (Dewaele & MacIntyre, 2014; Horwitz et al., 1986). Although these dimensions may be interrelated, they capture
different aspects of learners’ emotional experiences. Positive Affective
Engagement concerns enjoyment and interest in collaborative learning, Affective
Connectedness in Collaboration concerns perceived relational support during
interaction, and Emotional Disaffection concerns negative emotional responses
to difficulties encountered in collaboration.
Existing instruments provide important foundations for
examining these experiences but do not fully address their combined
characteristics in human–GenAI collaborative language learning. Measures of
foreign language enjoyment and anxiety primarily assess positive and negative
emotional experiences within conventional language learning contexts (Dewaele &
MacIntyre, 2014; Li et al., 2018). Multidimensional engagement instruments generally
position emotional engagement alongside behavioral and cognitive engagement,
often operationalizing it through a limited number of positively oriented items
(Hiver et al., 2024; Reeve & Tseng, 2011). Similarly, technology acceptance measures typically
examine enjoyment or attitudes as predictors of technology use intentions
rather than as components of multidimensional emotional engagement (Liu & Ma, 2024; Zou & Huang, 2023). Although these approaches capture relevant aspects of learners’
affective experiences, they do not explicitly integrate positive emotional
involvement, perceived affective connectedness with GenAI, and emotional
disaffection within a unified measurement framework. To address this
limitation, the present study examines items adapted from previous research on
learner–GenAI relationships and cognitive, social, and emotional experiences in
foreign language learning (Guo & Li, 2022; Jin,
2025). This approach allows the
three proposed dimensions to be evaluated within the specific context of human–GenAI
collaborative language learning.
An additional consideration concerns whether emotional
engagement can be measured consistently across different groups of learners.
Comparisons of latent constructs require evidence that measurement instruments
operate equivalently across relevant populations (Putnick & Bornstein, 2016; Vandenberg & Lance, 2000). Without establishing measurement invariance, differences
in observed scores may reflect variations in item interpretation or measurement
properties rather than substantive differences in emotional engagement. This
issue is particularly relevant to human–GenAI collaborative language learning,
where learners with different demographic and educational backgrounds may
encounter different forms of technology-mediated interaction. Nevertheless,
measurement invariance has received limited attention in existing instruments
examining GenAI-related language learning experiences. Evaluating invariance
across gender and year of study can therefore provide important evidence
concerning the comparability of emotional engagement scores and the suitability
of the instrument for subsequent group-based investigations.
Finally, understanding emotional engagement requires
consideration of both general relationships among constructs and differences in
how emotional experiences are combined within individual learners. Previous
research has predominantly adopted variable-centered approaches, which describe
associations and average tendencies across samples. In contrast,
person-centered methods, such as latent profile analysis, identify subgroups of
learners characterized by distinct configurations of emotional experiences (Spurk et al., 2020).
Importantly, such configurations may differ in level or shape. Level
differences occur when profiles display broadly similar patterns across
dimensions but differ in their overall intensity, whereas shape differences
emerge when learners exhibit qualitatively different combinations of emotional
experiences, such as high enjoyment alongside substantial emotional
disaffection (Marsh et al., 2009; Morin
& Marsh, 2015). Distinguishing between
these possibilities has implications for understanding learner heterogeneity
and informing instructional support. Profiles differing primarily in level may
indicate varying degrees of emotional involvement, whereas profiles differing
in shape may suggest more complex affective configurations requiring
differentiated forms of support. Accordingly, examining latent profiles can
complement psychometric validation by clarifying how the proposed dimensions of
emotional engagement coexist across learners in human–GenAI collaborative
language learning.
Rationale for the Present Study
The present study had two aims: to evaluate the psychometric
properties of a three-dimensional measure of emotional engagement in human–GenAI
collaborative language learning, and to describe heterogeneity in emotional
engagement among learners. Four research questions were addressed.
RQ1. What is the factor structure of the emotional
engagement measure, and does a three-factor model fit better than alternative
models?
RQ2. Do the subscales show adequate reliability, convergent
validity, and discriminant validity?
RQ3. Is the measure invariant across gender and year of
study?
RQ4. What latent profiles of emotional engagement can be
identified, and is profile membership related to gender, year of study, English
proficiency, and the GenAI tool used?
Methodology
Participants
Participants were 504 students from three universities in China who
used GenAI tools in their foreign language learning. The sample comprised 230
men (45.6%) and 274 women (54.4%). By year of study, 115 (22.8%) were
first-year undergraduates, 145 (28.8%) second-year, 108 (21.4%) third-year, and
90 (17.9%) fourth-year, and 46 (9.1%) were postgraduates. Participants named
the GenAI tool they currently used. The most frequent were ChatGPT (n = 132,
26.2%), ERNIE Bot (n = 77, 15.3%), Doubao (n = 66, 13.1%), Kimi (n = 59,
11.7%), DeepSeek (n = 48, 9.5%), and Zhipu Qingyan (n = 43, 8.5%); the
remaining 79 participants (15.7%) used Tongyi Qianwen, Claude, Tencent Yuanbao,
or iFlytek Spark. College English Test Band 4 (CET-4) scores were reported by
371 participants (M = 462.41, SD = 62.38, range = 277–639).
Procedure
Data were collected over a one-week period through Wenjuanxing, an
online survey platform widely used in China. Faculty members at the three
participating universities distributed the survey link to students who met the
eligibility criteria. The first page of the questionnaire presented an
electronic informed-consent statement explaining the purpose of the study, the
voluntary nature of participation, the confidentiality of responses, and
participants’ right to discontinue the survey. Participants reviewed this
information before proceeding to the questionnaire.
Participants first provided demographic information and then completed
the study measures. The emotional engagement items analyzed here were
administered as part of a larger questionnaire. The questionnaire presented the
English and Chinese versions of the items in parallel.
After data collection, responses were screened for data quality.
Questionnaires were classified as invalid when they showed one or more of the
following characteristics: an unreasonably short completion time, invariant
response patterns across a series of items, incomplete responses, or evident
inconsistencies among responses. On the basis of these criteria, 37
questionnaires were excluded, leaving 504 valid cases for analysis. The 15
emotional engagement items had no missing values.
Measures
Emotional engagement in human–GenAI collaborative language learning was assessed using a
15-item instrument comprising three dimensions: Positive Affective Engagement,
Affective Connectedness in Collaboration, and Emotional Disaffection, with five
items representing each dimension (see the Appendix). The items were adapted
from previous research examining learner–GenAI relationships and cognitive,
social, and emotional experiences in foreign language learning (Guo & Li, 2022; Jin, 2025). Participants responded using a 5-point Likert scale ranging from 1
(strongly agree) to 5 (strongly disagree). All responses were reverse-coded
before analysis to ensure that higher scores indicated stronger endorsement of
the construct represented by each dimension. Accordingly, higher scores on
Positive Affective Engagement and Affective Connectedness in Collaboration
reflected greater positive emotional involvement and perceived connectedness,
respectively, whereas higher scores on Emotional Disaffection indicated greater
negative emotional experiences.
Positive Affective Engagement assessed learners’ enjoyment, interest,
excitement, sense of accomplishment, and anticipation associated with human–GenAI collaborative
language learning (sample item: “I enjoy learning through Human–GenAI collaboration”).
Affective Connectedness in Collaboration captured learners’ perceived emotional
connection with GenAI, including feelings of support, understanding, reduced
isolation, and comfort during interaction (sample item: “During Human–GenAI collaborative
learning, I feel a sense of connection with GenAI”). Emotional Disaffection
measured learners’ negative emotional responses to collaborative learning
experiences, including discouragement, disappointment, anxiety, stress, and
frustration (sample item: “During Human–GenAI collaborative foreign language
learning, I feel discouraged when my progress is unsatisfactory”). Together,
these dimensions were intended to capture distinguishable aspects of learners’
affective experiences during human–GenAI collaboration.
Background variables included gender, year of study, self-reported
College English Test Band 4 (CET-4) scores, and the GenAI tool currently used
by participants.
Data
Analysis
The analysis proceeded in five steps. First, item quality was examined
through descriptive statistics, corrected item–total correlations, Cronbach’s α if an item was deleted,
and an extreme-groups comparison of the top and bottom 27% of scorers on each
subscale.
Second, the sample was randomly divided into two halves of 252 cases, a
practice recommended so that a structure is not explored and confirmed on the
same data (Fabrigar
et al., 1999; Worthington & Whittaker, 2006). Exploratory factor analysis (EFA) was conducted on the
first half using principal axis factoring with promax rotation, because the
factors were expected to correlate. The number of factors was determined by
parallel analysis (Horn, 1965) with 1,000 simulated data sets and the 95th-percentile criterion.
Third, confirmatory factor analysis (CFA) was conducted on the second
half. Four models were compared: a one-factor model (M1); a two-factor model
that combined Positive Affective Engagement and Affective Connectedness into
one positive factor (M2); the hypothesized correlated three-factor model (M3);
and a bifactor model with a general factor and three specific factors (M4). A
second-order model was not tested separately because, with three first-order
factors, it is statistically equivalent to M3 (Brown, 2015). Models were estimated by maximum likelihood. Skewness and
kurtosis were below |1| for every item, and 5-point items can be treated as
continuous under these conditions with little bias (Rhemtulla et al., 2012). Fit was judged
by the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square
error of approximation (RMSEA) with its 90% confidence interval, and the
standardized root mean square residual (SRMR). Values of CFI and TLI at or
above .90 and of RMSEA and SRMR at or below .08 were taken as acceptable, and
CFI and TLI at or above .95 with RMSEA at or below .06 as good (Hu & Bentler,
1999; Marsh et al., 2004). The Akaike and Bayesian information criteria (AIC and
BIC) were used to compare models.
Fourth, reliability and validity were evaluated in the full sample.
Internal consistency was estimated with Cronbach’s α and McDonald’s ω (Dunn et al., 2014; Hayes & Coutts, 2020), together with composite reliability (CR). Convergent
validity was assessed by the average variance extracted (AVE), with .50 as the
threshold, and discriminant validity by the Fornell–Larcker criterion (Fornell &
Larcker, 1981) and the heterotrait–monotrait ratio of
correlations (HTMT), with .85 as the threshold (Henseler et al., 2015). Measurement invariance across gender and year of study
was tested with multigroup CFA in a sequence of configural, metric, scalar, and
strict models (Vandenberg & Lance, 2000). Invariance was retained when CFI decreased by no more
than .010 and RMSEA increased by no more than .015 (Chen, 2007; Cheung & Rensvold, 2002).
Fifth, latent profile analysis (LPA) was conducted on the three
subscale mean scores, with variances constrained to be equal across profiles
and indicators assumed to be locally independent. Models with one to six
profiles were compared on AIC, BIC, the sample-size-adjusted BIC (SABIC),
entropy (Celeux &
Soromenho, 1996), the bootstrap likelihood ratio test (BLRT) with
100 bootstrap samples, and the size and interpretability of the smallest
profile (Nylund
et al., 2007; Spurk et al., 2020). Each model was estimated from multiple random starts.
Participants were assigned to their most likely profile, and profiles were
compared on background variables using chi-square tests and one-way analysis of
variance.
Because all data were self-reported at one time point, common method
variance was examined. A single factor accounted for 35.8% of the variance in
the 15 items, below the conventional 50% threshold, and the one-factor CFA
model fit the data poorly (see Section 4.3), which suggests that a single
method factor does not account for the covariation among items (Podsakoff et al.,
2003).
Results
Descriptive Statistics and Item Analysis
Table 1 presents the item statistics. Item means ranged from 3.89 to
3.95 for Positive Affective Engagement, from 3.58 to 3.62 for Affective
Connectedness in Collaboration, and from 2.30 to 2.33 for Emotional
Disaffection. Subscale means were 3.93 (SD = 0.63), 3.60 (SD = 0.68), and 2.31
(SD = 0.64), respectively. One-sample t tests showed that all three differed
from the scale midpoint of 3, t(503) = 33.25, 19.69, and −23.92, all ps <
.001. Participants thus tended to endorse positive affect and connectedness and
to disagree with statements of disaffection. Skewness ranged from −0.40 to 0.34
and kurtosis from −0.66 to 0.15, and no item showed a floor or ceiling effect.
Corrected item–total correlations ranged from .58 to .76. Every item
discriminated between high and low scorers, ts = 17.85–27.76, and deleting any
item would not have increased the α of its subscale. All 15 items were
retained.
Table 1. Descriptive
Statistics and Item Analysis for the 15 Items (N = 504)
|
Item
|
Wording
|
M
|
SD
|
Skewness
|
Kurtosis
|
CITC
|
α if
deleted
|
t
|
|
Positive
Affective Engagement (α = .835, subscale M
= 3.93, SD = 0.63)
|
|
1
|
I
enjoy learning through Human–GenAI collaboration.
|
3.93
|
0.80
|
−0.29
|
−0.51
|
.603
|
.811
|
23.20
|
|
2
|
I
feel interested when exploring new foreign language expressions, linguistic
knowledge, or related information together with AI.
|
3.89
|
0.80
|
−0.28
|
−0.50
|
.675
|
.791
|
23.39
|
|
3
|
I
feel excited when human–AI collaboration helps me make progress on
challenging foreign language learning tasks.
|
3.95
|
0.80
|
−0.25
|
−0.66
|
.637
|
.802
|
24.28
|
|
4
|
I
feel a sense of accomplishment after completing foreign language learning
tasks in collaboration with AI.
|
3.92
|
0.84
|
−0.40
|
−0.37
|
.684
|
.788
|
23.69
|
|
5
|
I
look forward to carrying out foreign language learning tasks through human–AI
collaboration.
|
3.95
|
0.80
|
−0.34
|
−0.47
|
.581
|
.817
|
19.72
|
|
Affective
Connectedness in Collaboration (α =
.871, subscale M = 3.60, SD = 0.68)
|
|
6
|
During
Human–GenAI collaborative learning, I feel a sense of connection with GenAI.
|
3.62
|
0.86
|
−0.19
|
−0.31
|
.669
|
.851
|
24.19
|
|
7
|
I
feel that the AI provides me with support.
|
3.58
|
0.83
|
−0.05
|
−0.35
|
.675
|
.849
|
22.26
|
|
8
|
When
the AI responds to my questions, I feel that my learning needs are
understood.
|
3.60
|
0.86
|
−0.10
|
−0.47
|
.758
|
.828
|
27.76
|
|
9
|
When
dealing with challenging tasks, collaborating with AI makes me feel less
isolated and helpless.
|
3.60
|
0.81
|
0.01
|
−0.43
|
.705
|
.842
|
24.57
|
|
10
|
I
feel comfortable asking the AI questions.
|
3.58
|
0.82
|
−0.22
|
−0.14
|
.677
|
.848
|
21.51
|
|
Emotional
Disaffection (α = .851, subscale M = 2.31, SD
= 0.64)
|
|
11
|
During
Human–GenAI collaborative foreign language learning, I feel discouraged when
my progress is unsatisfactory.
|
2.30
|
0.79
|
0.04
|
−0.43
|
.682
|
.816
|
21.70
|
|
12
|
I
feel disappointed when the collaboration does not bring the expected
improvement.
|
2.32
|
0.84
|
0.11
|
−0.40
|
.679
|
.816
|
23.81
|
|
13
|
I
feel anxious when I obtain unsatisfactory results.
|
2.32
|
0.80
|
0.20
|
−0.26
|
.675
|
.818
|
20.91
|
|
14
|
I
feel stressed when I am confused by AI-generated feedback.
|
2.33
|
0.83
|
0.13
|
−0.46
|
.672
|
.818
|
22.18
|
|
15
|
I
feel frustrated when the AI cannot adequately resolve my difficulties.
|
2.30
|
0.81
|
0.34
|
0.15
|
.604
|
.836
|
17.85
|
Note. CITC = corrected
item–total correlation with the remaining four items of the same subscale. The t
values compare the top and bottom 27% of scorers on the subscale total (Welch’s
test); all ps < .001.
Exploratory Factor Analysis
The data in the first half-sample were suitable for factor analysis,
KMO = .858, Bartlett’s χ2(105) = 1868.71, p < .001. The first three
eigenvalues (5.51, 2.60, and 1.55) exceeded the corresponding 95th-percentile
eigenvalues from the parallel analysis (1.53, 1.40, and 1.32), whereas the
fourth (0.83) did not (1.25; Figure 1). Three factors were extracted,
accounting for 55.7% of the variance. As Table 2 shows, every item loaded on
its intended factor, with primary loadings between .62 and .84 and no
cross-loading above .13. Communalities ranged from .41 to .70. Positive
Affective Engagement correlated .45 with Affective Connectedness in
Collaboration and −.52 with Emotional Disaffection, and the latter two factors
correlated −.26.
Figure 1 Scree Plot With
Parallel Analysis (n = 252)
Table 2.
Promax-Rotated Pattern Matrix from the Exploratory Factor Analysis (n = 252)
|
Item
|
Wording
|
Factor
1
|
Factor
2
|
Factor
3
|
h2
|
|
1
|
I
enjoy learning through Human–GenAI collaboration.
|
.648
|
.088
|
.021
|
.465
|
|
2
|
I
feel interested when exploring new foreign language expressions, linguistic
knowledge, or related information together with AI.
|
.711
|
.023
|
−.049
|
.560
|
|
3
|
I
feel excited when human–AI collaboration helps me make progress on
challenging foreign language learning tasks.
|
.768
|
−.033
|
.023
|
.552
|
|
4
|
I
feel a sense of accomplishment after completing foreign language learning
tasks in collaboration with AI.
|
.774
|
−.003
|
−.006
|
.602
|
|
5
|
I
look forward to carrying out foreign language learning tasks through human–AI
collaboration.
|
.623
|
−.029
|
−.057
|
.412
|
|
6
|
During
Human–GenAI collaborative learning, I feel a sense of connection with GenAI.
|
−.008
|
.739
|
.005
|
.539
|
|
7
|
I
feel that the AI provides me with support.
|
.063
|
.736
|
.028
|
.576
|
|
8
|
When
the AI responds to my questions, I feel that my learning needs are
understood.
|
.014
|
.840
|
.043
|
.699
|
|
9
|
When
dealing with challenging tasks, collaborating with AI makes me feel less
isolated and helpless.
|
−.026
|
.774
|
−.011
|
.587
|
|
10
|
I
feel comfortable asking the AI questions.
|
−.017
|
.783
|
−.030
|
.614
|
|
11
|
During
Human–GenAI collaborative foreign language learning, I feel discouraged when
my progress is unsatisfactory.
|
−.038
|
.054
|
.738
|
.556
|
|
12
|
I
feel disappointed when the collaboration does not bring the expected
improvement.
|
−.002
|
−.004
|
.772
|
.599
|
|
13
|
I
feel anxious when I obtain unsatisfactory results.
|
−.045
|
.062
|
.770
|
.607
|
|
14
|
I
feel stressed when I am confused by AI-generated feedback.
|
−.028
|
.013
|
.710
|
.521
|
|
15
|
I
feel frustrated when the AI cannot adequately resolve my difficulties.
|
.098
|
−.132
|
.685
|
.462
|
Note. Factor 1 = Positive
Affective Engagement; Factor 2 = Affective Connectedness in Collaboration;
Factor 3 = Emotional Disaffection. Loadings of |.40| or greater are in bold. h2
= communality.
Confirmatory Factor
Analysis
Table 3 reports the fit of the competing models. In the holdout sample,
the one-factor model (M1) and the two-factor model (M2) fit poorly. The
hypothesized three-factor model (M3) fit acceptably, χ2(87) = 204.91, CFI =
.926, TLI = .911, RMSEA = .073, 90% CI [.060, .086], SRMR = .048, and had much
lower AIC and BIC than M1 and M2. Results in the full sample were similar, χ2(87)
= 360.22, CFI = .921, TLI = .904, RMSEA = .079, 90% CI [.071, .087], SRMR =
.040. Standardized loadings in the holdout sample ranged from .61 to .83 (Table
4).
The bifactor model (M4) had somewhat better fit indices than M3 but was
not retained, for three reasons. Its BIC in the holdout sample (7899) was
higher than that of M3 (7892). Its general factor was defined mainly by Items 1–3,
whose loadings on their specific factor were near zero or negative (Table 4),
so the general factor largely duplicated Positive Affective Engagement instead
of representing variance common to all 15 items. And bifactor models tend to
fit better than correlated-factor models even when they are not the
data-generating structure (Bonifay et al., 2017; Reise,
2012). A higher-order
interpretation was not supported either. When the correlations among the three
factors were expressed as loadings on a second-order factor, the standardized
loading of Positive Affective Engagement was .988 in the holdout sample and .999
in the full sample.
The RMSEA of M3 was at the upper end of the acceptable range.
Inspection of residual correlations showed that the misfit was localized in a
few pairs of items within the same factor, most clearly Items 4 and 5, Items 9
and 10, Items 14 and 15, and Items 11 and 12. Allowing these residuals to
covary produced good fit in the holdout sample, χ2(83) = 90.28, CFI = .995, TLI
= .994, RMSEA = .019, SRMR = .037 (M5 in Table 3), without changing the pattern
of loadings. Because these modifications were made post hoc, M5 is reported for
information only, and M3 is retained as the measurement model.
Table 3. Fit Indices
for Competing Measurement Models
|
Model
|
χ2
|
df
|
χ2/df
|
CFI
|
TLI
|
RMSEA [90% CI]
|
SRMR
|
AIC
|
BIC
|
|
Holdout
sample (n = 252)
|
|
M1:
One-factor
|
840.31
|
90
|
9.34
|
.532
|
.454
|
.182 [.171, .193]
|
.145
|
8352
|
8511
|
|
M2:
Two-factor
|
511.10
|
89
|
5.74
|
.737
|
.689
|
.137 [.126, .149]
|
.114
|
8025
|
8188
|
|
M3:
Correlated three-factor
|
204.91
|
87
|
2.36
|
.926
|
.911
|
.073 [.060, .086]
|
.048
|
7723
|
7892
|
|
M4:
Bifactor
|
145.24
|
75
|
1.94
|
.956
|
.939
|
.061 [.046, .076]
|
.039
|
7687
|
7899
|
|
M5:
M3 + 4 residual covariances
|
90.28
|
83
|
1.09
|
.995
|
.994
|
.019 [.000, .041]
|
.037
|
7616
|
7800
|
|
Full
sample (N = 504)
|
|
M1:
One-factor
|
1727.72
|
90
|
19.20
|
.525
|
.445
|
.190 [.182, .198]
|
.147
|
16686
|
16876
|
|
M2:
Two-factor
|
1028.00
|
89
|
11.55
|
.727
|
.678
|
.145 [.137, .153]
|
.121
|
15988
|
16182
|
|
M3:
Correlated three-factor
|
360.22
|
87
|
4.14
|
.921
|
.904
|
.079 [.071, .087]
|
.040
|
15324
|
15527
|
|
M4:
Bifactor
|
263.51
|
75
|
3.51
|
.945
|
.923
|
.071 [.061, .080]
|
.034
|
15251
|
15505
|
|
M5:
M3 + 5 residual covariances
|
102.79
|
82
|
1.25
|
.994
|
.992
|
.022 [.000, .035]
|
.027
|
15077
|
15300
|
Note. M2 combines Positive
Affective Engagement and Affective Connectedness in Collaboration into one
factor. M5 adds residual covariances for item pairs 4–5, 9–10, 14–15, 11–12 in
the holdout sample and 14–15, 4–5, 9–10, 11–12, 2–3 in the full sample. CI = confidence
interval.
Table 4. Standardized
Factor Loadings for the Three-Factor and Bifactor Models
|
Item
|
Factor
|
M3 λ
(n = 252)
|
M3 λ
(N = 504)
|
M4 general λ
(N = 504)
|
M4 specific λ
(N = 504)
|
|
1
|
PAE
|
.661
|
.671
|
.652
|
.044
|
|
2
|
PAE
|
.796
|
.779
|
.807
|
−.091
|
|
3
|
PAE
|
.719
|
.732
|
.775
|
−.180
|
|
4
|
PAE
|
.713
|
.738
|
.735
|
.548
|
|
5
|
PAE
|
.615
|
.626
|
.583
|
.359
|
|
6
|
ACC
|
.733
|
.737
|
.314
|
.673
|
|
7
|
ACC
|
.710
|
.736
|
.370
|
.633
|
|
8
|
ACC
|
.830
|
.827
|
.365
|
.748
|
|
9
|
ACC
|
.754
|
.761
|
.356
|
.668
|
|
10
|
ACC
|
.682
|
.731
|
.343
|
.642
|
|
11
|
ED
|
.762
|
.775
|
−.357
|
.714
|
|
12
|
ED
|
.746
|
.776
|
−.395
|
.683
|
|
13
|
ED
|
.710
|
.745
|
−.400
|
.621
|
|
14
|
ED
|
.739
|
.711
|
−.399
|
.568
|
|
15
|
ED
|
.655
|
.643
|
−.362
|
.512
|
Note. PAE = Positive Affective
Engagement; ACC = Affective Connectedness in Collaboration; ED = Emotional
Disaffection; M3 = correlated three-factor model; M4 = bifactor model.
Reliability and Validity
Table 5 summarizes reliability and convergent validity. Cronbach’s α
ranged from .835 to .871 and McDonald’s ω from .836 to .872. The AVE of each
subscale exceeded .50. As Table 6 shows, the square root of each AVE (.71–.76)
was larger than the absolute correlations of that factor with the other two
(.24–.52), and the HTMT ratios (.25–.52) were well below .85. The three
subscales were therefore reliable and empirically distinct.
Table 5. Reliability and
Convergent Validity (N = 504)
|
Subscale
|
α
|
ω
|
CR
|
AVE
|
Standardized λ range
|
|
Positive
Affective Engagement
|
.835
|
.836
|
.836
|
.506
|
.626–.779
|
|
Affective
Connectedness in Collaboration
|
.871
|
.872
|
.872
|
.577
|
.731–.827
|
|
Emotional
Disaffection
|
.851
|
.851
|
.852
|
.536
|
.643–.776
|
Note. ω and composite
reliability (CR) were computed from the M3 solution. AVE = average variance
extracted; λ = standardized factor loading.
Table 6. Discriminant
Validity (N = 504)
|
Subscale
|
1
|
2
|
3
|
|
1.
Positive Affective Engagement
|
.711
|
.465
|
.524
|
|
2.
Affective Connectedness in Collaboration
|
.464
|
.759
|
.250
|
|
3.
Emotional Disaffection
|
−.524
|
−.244
|
.732
|
Note. Square roots of AVE are
on the diagonal in bold. Latent correlations from M3 are below the diagonal,
and HTMT ratios are above the diagonal in italics.
Measurement Invariance
Table 7 presents the invariance tests. For gender, constraining
loadings, intercepts, and residual variances in turn produced nonsignificant
changes in χ2, changes in CFI between .000 and +.003, and decreases in RMSEA.
For year of study, the largest decrease in CFI was .004, at the scalar step,
and RMSEA again decreased at every step. Configural, metric, scalar, and strict
invariance were therefore supported for both grouping variables. The absolute
fit of the configural models was similar to that of M3 in the full sample. The
postgraduate group was small (n = 46), and the results for year of study should
be read with this in mind.
Table 7. Tests of
Measurement Invariance Across Gender and Year of Study
|
Model
|
χ2
|
df
|
CFI
|
TLI
|
RMSEA
|
SRMR
|
Δχ2 (Δdf)
|
p
|
ΔCFI
|
ΔRMSEA
|
|
Gender
(male n = 230, female n = 274)
|
|
Configural
|
461.47
|
174
|
.917
|
.899
|
.081
|
.046
|
—
|
—
|
—
|
—
|
|
Metric
|
467.00
|
186
|
.919
|
.908
|
.077
|
.048
|
5.54 (12)
|
.938
|
+0.002
|
−0.004
|
|
Scalar
|
480.63
|
198
|
.918
|
.913
|
.075
|
.049
|
13.62 (12)
|
.325
|
0.000
|
−0.002
|
|
Strict
|
485.63
|
213
|
.921
|
.922
|
.071
|
.049
|
5.01 (15)
|
.992
|
+0.003
|
−0.004
|
|
Year
of study (five groups)
|
|
Configural
|
731.43
|
435
|
.915
|
.897
|
.082
|
.064
|
—
|
—
|
—
|
—
|
|
Metric
|
774.84
|
483
|
.916
|
.909
|
.077
|
.074
|
43.42 (48)
|
.661
|
+0.001
|
−0.005
|
|
Scalar
|
835.40
|
531
|
.912
|
.913
|
.075
|
.077
|
60.56 (48)
|
.105
|
−0.004
|
−0.002
|
|
Strict
|
887.45
|
591
|
.915
|
.924
|
.071
|
.080
|
52.04 (60)
|
.758
|
+0.002
|
−0.005
|
Note. Group sizes for year of
study were 115 (first year), 145 (second year), 108 (third year), 90 (fourth
year), and 46 (postgraduate). Each model is compared with the preceding one.
Latent Profiles of Emotional Engagement
Table 8 shows the fit of the latent profile models. BIC was lowest for
the three-profile model. SABIC was slightly lower for the four-profile model,
but the additional profile comprised only 3.5% of the sample (about 18
participants), too few to be estimated stably; profile size and
interpretability should be weighed alongside statistical indices (Spurk et al., 2020).
The BLRT was significant up to four profiles and nonsignificant thereafter. The
three-profile model was retained on the grounds of BIC, profile size, and
parsimony. Its entropy was .695 and the mean posterior probabilities of the
assigned profiles ranged from .85 to .88.
Table 8. Fit Statistics
for Latent Profile Models with One to Six Profiles (N = 504)
|
Profiles
|
LL
|
AIC
|
BIC
|
SABIC
|
Entropy
|
BLRT p
|
Profile proportions
|
|
1
|
−1492.61
|
2997.22
|
3022.55
|
3003.51
|
—
|
—
|
100.0%
|
|
2
|
−1408.29
|
2836.59
|
2878.81
|
2847.07
|
.595
|
.010
|
52.6% / 47.4%
|
|
3
|
−1387.96
|
2803.91
|
2863.03
|
2818.59
|
.695
|
.010
|
49.4% / 30.4% / 20.2%
|
|
4
|
−1380.15
|
2796.31
|
2872.31
|
2815.18
|
.784
|
.020
|
43.7% / 28.6% / 24.2% / 3.5%
|
|
5
|
−1377.31
|
2798.62
|
2891.51
|
2821.68
|
.677
|
.683
|
28.5% / 25.7% / 24.1% / 18.1% / 3.5%
|
|
6
|
−1370.73
|
2793.47
|
2903.25
|
2820.73
|
.669
|
.188
|
22.7% / 22.2% / 20.6% / 18.4% / 12.5% /
3.7%
|
Note. The retained solution is
in bold. LL = log-likelihood; SABIC = sample-size-adjusted BIC; BLRT =
bootstrap likelihood ratio test comparing k with k − 1 profiles.
The profiles are shown in Figure 2 and Table 9. The first, labeled high
engagement (n = 150, 29.8%), combined very high Positive Affective Engagement
(M = 4.67) with high Affective Connectedness (M = 3.95) and low Emotional
Disaffection (M = 1.89). The second, moderate engagement (n = 254, 50.4%), was
close to the sample means on all three subscales. The third, low engagement (n
= 100, 19.8%), scored near the scale midpoint on all three subscales (Ms =
3.05, 3.06, and 2.83). The profiles differed significantly on each subscale,
F(2, 501) = 1101.99 for Positive Affective Engagement, 64.92 for Affective
Connectedness, and 88.49 for Emotional Disaffection, all ps < .001. The
profiles were ordered in the same way on every indicator: the higher a
profile's positive affect, the higher its connectedness and the lower its
disaffection. No profile combined high positive affect with high disaffection.
Figure 2 Subscale Means
of the Three Latent Profiles
Note. Positive affect =
Positive Affective Engagement; Connectedness = Affective Connectedness in
Collaboration; Disaffection = Emotional Disaffection.
Table 9. Subscale Means
and Standard Deviations by Latent Profile
|
Profile
|
n (%)
|
Positive Affective
Engagement
|
Affective
Connectedness
|
Emotional
Disaffection
|
Mean posterior
probability
|
|
High
engagement
|
150 (29.8%)
|
4.67 (0.24)
|
3.95 (0.56)
|
1.89 (0.57)
|
.884
|
|
Moderate
engagement
|
254 (50.4%)
|
3.84 (0.28)
|
3.60 (0.63)
|
2.36 (0.53)
|
.853
|
|
Low
engagement
|
100 (19.8%)
|
3.05 (0.30)
|
3.06 (0.60)
|
2.83 (0.59)
|
.851
|
|
Total
sample
|
504
|
3.93 (0.63)
|
3.60 (0.68)
|
2.31 (0.64)
|
—
|
Note. Subscale entries are M
(SD) on the 1–5 scale. Profile membership is based on the highest
posterior probability.
Table 10 compares the profiles on background variables. Profile
membership was not significantly associated with gender, χ2(2) = 5.36, p =
.068, Cramér’s V = .10; year of study, χ2(8) = 11.06, p = .199, V = .11; or
GenAI tool, χ2(12) = 11.39, p = .496, V = .11. CET-4 scores did not differ
among profiles, F(2, 368) = 0.78, p = .458.
Table 10. Background Characteristics by Latent
Profile
|
Variable
|
High engagement
(n = 150)
|
Moderate engagement
(n = 254)
|
Low engagement
(n = 100)
|
|
Gender:
χ2(2) = 5.36, p = .068, Cramér’s V = .103
|
|
Male
|
58 (38.7%)
|
128 (50.4%)
|
44 (44.0%)
|
|
Female
|
92 (61.3%)
|
126 (49.6%)
|
56 (56.0%)
|
|
Year
of study: χ2(8) = 11.06, p = .199, Cramér’s V = .105
|
|
First
year
|
41 (27.3%)
|
57 (22.4%)
|
17 (17.0%)
|
|
Second
year
|
40 (26.7%)
|
67 (26.4%)
|
38 (38.0%)
|
|
Third
year
|
37 (24.7%)
|
52 (20.5%)
|
19 (19.0%)
|
|
Fourth
year
|
20 (13.3%)
|
53 (20.9%)
|
17 (17.0%)
|
|
Postgraduate
|
12 (8.0%)
|
25 (9.8%)
|
9 (9.0%)
|
|
Primary
AI tool: χ2(12) = 11.39, p = .496, Cramér’s V = .106
|
|
ChatGPT
|
43 (28.7%)
|
61 (24.0%)
|
28 (28.0%)
|
|
Doubao
|
19 (12.7%)
|
38 (15.0%)
|
9 (9.0%)
|
|
ERNIE
Bot
|
21 (14.0%)
|
37 (14.6%)
|
19 (19.0%)
|
|
Kimi
|
23 (15.3%)
|
29 (11.4%)
|
7 (7.0%)
|
|
Zhipu
Qingyan
|
11 (7.3%)
|
24 (9.4%)
|
8 (8.0%)
|
|
DeepSeek
|
16 (10.7%)
|
22 (8.7%)
|
10 (10.0%)
|
|
Other
|
17 (11.3%)
|
43 (16.9%)
|
19 (19.0%)
|
|
CET-4
score (n = 371): F(2, 368) = 0.78, p = .458
|
|
M (SD)
|
468.5 (69.1)
|
459.3 (59.2)
|
461.1 (59.5)
|
Note. Entries are n (%
within profile) unless otherwise indicated. Other = Tongyi Qianwen, Claude,
Tencent Yuanbao, and iFlytek Spark. CET-4 = College English Test Band 4.
Discussion
A Three-Dimensional Structure
The first research question concerned factor structure.
Exploratory and confirmatory analyses in independent half-samples converged on
three factors, and models that merged the dimensions fit far worse. Two aspects
of this result are of theoretical interest.
The first is that Affective Connectedness in Collaboration
was distinct from Positive Affective Engagement. The two correlated moderately
(latent r = .46), and a model that merged them was clearly rejected. Enjoying
work with GenAI and feeling connected to and understood by it are therefore
related but different experiences. This extends the argument that relational
feelings are part of engagement in language learning (Philp & Duchesne, 2016) to a non-human partner, and it is consistent with evidence
that people respond socially to conversational systems (Nass & Moon,
2000; Skjuve et al., 2021). Connectedness was endorsed less strongly than positive
affect (M = 3.60 vs. 3.93). A plausible reading is that most learners find
GenAI collaboration pleasant and rewarding, while a smaller share experience it
as relational. Instruments that assess only enjoyment or interest would miss
this difference.
The second is that Emotional Disaffection was a separate
factor and not the negative pole of a single continuum. Its correlation with
Positive Affective Engagement was moderate and negative (latent r = −.52),
close to the relation between enjoyment and anxiety reported for language
classrooms (Dewaele & MacIntyre, 2014) and in line with the distinction between engagement and
disaffection drawn by Skinner et al. (2009). Its correlation with Affective Connectedness was weak (latent r = −.24).
Feeling supported by GenAI thus offers only limited protection against
discouragement or frustration when collaboration fails to help. This is
understandable from the standpoint of control-value theory (Pekrun, 2006):
negative emotions in this setting are likely to arise from appraisals of low
control, for instance when feedback is confusing or a difficulty remains
unresolved, and such appraisals can occur whether or not the learner feels at
ease with the system.
Neither a general factor nor a higher-order factor was
supported. The bifactor solution did not yield an interpretable general factor,
and the second-order loading of Positive Affective Engagement approached unity.
A single total score would therefore be dominated by positive affect and would
obscure the other two dimensions. Researchers using the scale should report the
three subscale scores separately.
Fit of the three-factor model was acceptable, not excellent.
The misfit was traced to four within-factor item pairs with related content:
accomplishment and anticipation (Items 4 and 5), feeling less isolated and
feeling comfortable asking questions (Items 9 and 10), stress and frustration
(Items 14 and 15), and discouragement and disappointment (Items 11 and 12).
Such local dependence is common when items within a subscale describe
neighboring emotions, and strict application of cutoff values to multi-item
scales has been criticized as unrealistic (Marsh et al., 2004). The pattern nevertheless identifies where the scale could
be refined. Rewording one item in each pair, or replacing it with an item that
samples a different emotion, may reduce redundancy in future versions.
Reliability, Validity, and Invariance
The second and third research questions concerned the
quality and comparability of scores. All three subscales showed good internal
consistency with only five items each, exceeded the AVE threshold for
convergent validity, and met both the Fornell–Larcker and HTMT criteria for
discriminant validity. The scale is short enough to be included in
multi-construct surveys or administered repeatedly.
Full invariance across gender and across five year-of-study
groups means that men and women, and students at different stages of study,
interpret the items in the same way and use the response scale similarly.
Observed and latent mean differences between these groups can therefore be
attributed to the construct and not to the instrument (Putnick & Bornstein, 2016). Evidence of this kind is seldom reported for instruments
in this field, and it provides a foundation for group comparisons in later
research.
Profiles That Differ in Level
The fourth research question concerned heterogeneity. Three
profiles were identified. Their defining feature is that they were ordered: the
high, moderate, and low profiles kept the same rank on positive affect and
connectedness and the reverse rank on disaffection. In the terms of Morin and
Marsh (2015), the profiles differed in level and not in shape. Learners
who combined high enjoyment with high distress, or strong connectedness with
little enjoyment, did not form a distinguishable group. One explanation is that
behavioral, cognitive, and emotional engagement can diverge from one another,
whereas the facets within emotional engagement tend to move together. When
profiles differ mainly in level, a dimensional description of individual
differences is more parsimonious than a typology (Marsh et al., 2009), and for most purposes the three subscale scores will be
more informative than profile membership.
The content of the profiles is still informative. Half of
the sample was moderately engaged, and nearly a third was highly engaged, which
is consistent with the generally positive emotional responses reported in the
literature (Zhang & Liu, 2025). The low profile, one fifth of the sample, was not characterized by
strong negative emotion. Its members scored near the midpoint on all three
subscales, including disaffection. These learners appear indifferent or
undecided, not distressed. The profiles were separated chiefly by positive
affect, and much less by connectedness or disaffection. The main difference
among learners, in other words, was how much they enjoyed and valued the
collaboration.
Profile membership was unrelated to gender, year of study,
English proficiency, or the GenAI tool used. Emotional engagement with GenAI
was thus spread evenly across the demographic and proficiency groups examined
here, and no tool stood out as more or less emotionally engaging. This pattern
agrees with evidence that engagement profiles in GenAI-mediated learning are
associated with malleable features of the learning environment (Xu, 2026), and it suggests that explanations should be sought in how
learners use GenAI and in the instructional context more than in who the
learners are. The null findings should be interpreted cautiously, because the
tool categories were broad and CET-4 scores were available for only 74% of the
sample.
Conclusion
This study provides empirical evidence for a
three-dimensional conceptualization of emotional engagement in human–GenAI
collaborative language learning, comprising Positive Affective Engagement,
Affective Connectedness in Collaboration, and Emotional Disaffection. The
15-item instrument demonstrated satisfactory reliability, convergent validity,
and discriminant validity, alongside configural, metric, scalar, and strict
measurement invariance across gender and year of study. Notably, the
identification of affective connectedness as a distinct dimension highlights
the importance of perceived relational experiences in human–GenAI
collaboration, extending conventional understandings of emotional engagement
beyond enjoyment and negative emotional responses. Latent profile analysis
further identified three groups characterized by high, moderate, and low
emotional engagement, with differences primarily reflecting levels rather than
configurations of affective experiences. These findings contribute to a
multidimensional understanding of emotional engagement in technology-mediated
language learning and provide a brief instrument for examining learners’
affective responses to GenAI collaboration. From a pedagogical perspective, the
findings highlight the value of attending to learners’ enjoyment, perceived
connectedness, and emotional difficulties when designing and supporting human–GenAI
collaborative language learning activities.
Several limitations should be acknowledged. First, the study
relied on cross-sectional self-reported data from 504 students at three Chinese
universities, which limits conclusions about temporal stability and
generalizability to other educational and cultural contexts. Although the
reported analyses did not indicate that a single factor accounted for the
observed item relationships, potential common method variance cannot be
entirely excluded. Second, the psychometric evaluation primarily focused on internal
structure, reliability, and measurement invariance, without incorporating
external criteria to establish criterion-related or predictive validity. Test–retest
reliability was also not examined, and several item pairs exhibited local
dependence, indicating opportunities for further refinement. Third, the latent
profile solution showed moderate classification precision, suggesting that
individual profile assignments should be interpreted cautiously. Furthermore,
although demographic characteristics, English proficiency, and GenAI tool use
were examined in relation to profile membership, the study did not assess the
frequency or specific patterns of GenAI use. These limitations constrain the
extent to which the identified profiles can explain the contextual and
interactional processes underlying learners’ emotional engagement.
Future research should extend the present findings in
several directions. First, cross-cultural and longitudinal studies involving
learners from different educational levels and institutional contexts are
needed to evaluate the generalizability, temporal stability, and measurement
equivalence of the proposed instrument. Second, further validation could
incorporate established measures of foreign language enjoyment and anxiety,
behavioral and cognitive engagement, actual GenAI use, and language learning outcomes
to strengthen criterion-related and predictive validity evidence. Refining
items exhibiting local dependence, cross-validating the revised instrument in
independent samples, and examining its properties using estimation methods
appropriate for ordinal responses would also be valuable. Third, future studies
could investigate how the frequency, purposes, and interactional patterns of
GenAI use, together with instructional conditions, relate to different
emotional engagement profiles. Longitudinal person-centered approaches,
particularly latent transition analysis, may further clarify whether learners
move between profiles over time and which contextual factors are associated
with these transitions. Such investigations would contribute to a more comprehensive
understanding of the dynamic and context-dependent nature of emotional
engagement in human–GenAI collaborative language learning.
Acknowledgement
We gratefully acknowledge the anonymous reviewers for their diligent
efforts in reviewing this study.
Declaration
of Interest Statement
There is no potential conflict
of interest.
References
Brown, T. A.
(2015). Confirmatory factor
analysis for applied research (2nd ed.).
Guilford Press.
Celeux, G., & Soromenho, G. (1996). An
entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13(2), 195–212. https://doi.org/10.1007/BF01246098
Author Bios
Songming Liu received his M.A. degree from Inner Mongolia
University, China. He is currently working toward the Ph.D. degree in Applied
Science of Linguistics with the Faculty of Languages and Translation, Macao
Polytechnic University, Macao, China. His research interests lie at the
intersection of psycholinguistics, cognitive science, and educational
technology. Specifically, his current research focuses on artificial
intelligence-assisted language learning among trilingual learners, with a
special emphasis on metacognition and positive psychology.
E-mail: liusongming314@gmail.com
ORCiD: https://orcid.org/0009-0007-0580-797X
Yongliang
Wang is an Associate
Professor of Applied Linguistics and a Ph.D. degree holder. He works at the
School of Foreign Studies, North China University of Water Resources and
Electric Power, China, and serves as a postgraduate supervisor in Applied
Linguistics at the School of Foreign Languages and Cultures, Nanjing Normal
University, China. He also acts as a blind external examiner for Ph.D.
dissertations in language education from institutions in the United Kingdom,
Poland, Australia, the USA, and other countries. Additionally, he serves as a
peer reviewer for over 40 SSCI-indexed international journals in EFL education.
He has been recognized as one of the world’s top 2% scientists in Languages and
Linguistics by Stanford University for three consecutive years (2024–2026), and
he was named a Highly Cited Chinese Researcher by Elsevier for four consecutive
years (2022–2025). His recent academic publications have appeared in top-tier
journals such as Computers in Human Behavior, System, Journal of Multilingual and
Multicultural Development, and Applied Linguistics Review.
E-mail: wangyongliang@ncwu.edu.cn
ORCiD: http://orcid.org/0000-0002-4672-8481
Appendix
Items of the Emotional Engagement Measure
|
Item
|
Subscale
|
Wording
|
|
1
|
Positive
Affective Engagement
|
I
enjoy learning through Human–GenAI collaboration.
|
|
2
|
Positive
Affective Engagement
|
I
feel interested when exploring new foreign language expressions, linguistic
knowledge, or related information together with AI.
|
|
3
|
Positive
Affective Engagement
|
I
feel excited when human–AI collaboration helps me make progress on
challenging foreign language learning tasks.
|
|
4
|
Positive
Affective Engagement
|
I
feel a sense of accomplishment after completing foreign language learning
tasks in collaboration with AI.
|
|
5
|
Positive
Affective Engagement
|
I
look forward to carrying out foreign language learning tasks through human–AI
collaboration.
|
|
6
|
Affective
Connectedness in Collaboration
|
During
Human–GenAI collaborative learning, I feel a sense of connection with GenAI.
|
|
7
|
Affective
Connectedness in Collaboration
|
I
feel that the AI provides me with support.
|
|
8
|
Affective
Connectedness in Collaboration
|
When
the AI responds to my questions, I feel that my learning needs are
understood.
|
|
9
|
Affective
Connectedness in Collaboration
|
When
dealing with challenging tasks, collaborating with AI makes me feel less
isolated and helpless.
|
|
10
|
Affective
Connectedness in Collaboration
|
I
feel comfortable asking the AI questions.
|
|
11
|
Emotional
Disaffection
|
During
Human–GenAI collaborative foreign language learning, I feel discouraged when
my progress is unsatisfactory.
|
|
12
|
Emotional
Disaffection
|
I
feel disappointed when the collaboration does not bring the expected
improvement.
|
|
13
|
Emotional
Disaffection
|
I
feel anxious when I obtain unsatisfactory results.
|
|
14
|
Emotional
Disaffection
|
I
feel stressed when I am confused by AI-generated feedback.
|
|
15
|
Emotional
Disaffection
|
I
feel frustrated when the AI cannot adequately resolve my difficulties.
|