Rare Categories, Anxious Learners: Using Relative Efficiency Curves to Evaluate Category Collapsing in L2 Likert Scales
pdf
html

Keywords

category collapsing; item response theory; relative efficiency; foreign language anxiety

How to Cite

Teng, M. F. (2026). Rare Categories, Anxious Learners: Using Relative Efficiency Curves to Evaluate Category Collapsing in L2 Likert Scales. Brief Reports in Language Research, 1(1). https://doi.org/10.65553/BRLR.260101

Abstract

When few respondents choose some categories of a Likert scale, L2 researchers often merge those categories and report an unchanged Cronbach’s alpha as evidence that nothing was lost. This methodological note shows why that evidence is uninformative. This article thus offers a simple alternative: relative efficiency (RE) curves, which show where along the trait a merged scale loses precision. In a 10-item anxiety scale completed by 714 Chinese EFL learners, merging the two lowest categories left alpha unchanged (.92–.93) but kept only 39–45% of the information about the most anxious learners; merging the three lowest kept 10–12%. The result replicated in a held-out half of the sample, and a simulation showed the same pattern. I provide an R function and five reporting guidelines for L2 researchers.

pdf
html

References

Brysbaert, M. (2026). Are 7-point Likert scales preferable to 5-point scales in language research? Behavior Research Methods, 58(8), Article 218. https://doi.org/10.3758/s13428-026-03101-1

Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Psychology Press.

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.

Liu, J., Bao, Y., DiStefano, C., & Jiang, W. (2025). Collapsing sparse responses in Likert-type scale data: Advantages and disadvantages for model fit in CFA. Educational and Psychological Measurement. Advance online publication. https://doi.org/10.1177/00131644251401097

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum.

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144

Quan, Y., & Wang, C. (2025). Collapsing or not? A practical guide to handling sparse responses for polytomous items. Methodology, 21(1), 46–73. https://doi.org/10.5964/meth.14303

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315

Samejima, F. (1968). Estimation of latent ability using a response pattern of graded scores. ETS Research Bulletin Series, 1, Pages i-169. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0

Sudina, E. (2021). Study and scale quality in second language survey research, 2009–2019: The case of anxiety and motivation. Language Learning, 71(4), 1149–1193. https://doi.org/10.1111/lang.12468

Sudina, E., Quan, Y., & Ozdemir, H. (2026). Measurement and structural invariance testing in L2 research: A methodological synthesis. Studies in Second Language Acquisition, 48(3), 873–906. https://doi.org/10.1017/S0272263126101624

Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second language anxiety and achievement: A meta-analysis. Studies in Second Language Acquisition, 41(2), 363–387. https://doi.org/10.1017/S0272263118000311

Teng, M. F. (Under review). An integrated SEM–IRT approach to evaluating emotion regulation and language learning anxiety measures in L2 research.

Views: 53Downloads: 33

Downloads

Download data is not yet available.
Creative Commons License All articles published in Brief Reports in Language Research are open access and distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are properly credited. Copyright is retained by the author(s).