Evaluating Test Quality in English for Mathematics Course: An Item Analysis Approach

Hedia Rizki, Reri Seprina Anggraini, Indra Bagus Satrio

Abstract


This study evaluated the quality of an English for Mathematics course administered to 35 third-semester students in the Mathematics Education Study Program at UIN Sulthan Thaha Saifuddin Jambi using Classical Test Theory. The 50-item multiple-choice test covered eight mathematical topics, including numbers, rational numbers, powers and roots, statistics and probability, logic, algebra, geometry, and Measurement. Item analysis examined difficulty indices, discrimination indices, reliability (KR-20), and distractor effectiveness. Results revealed excellent reliability (r₁₁ = 0.911) and balanced difficulty distribution: 36% easy items (P > 0.70), 52% moderate items (0.30-0.70), and 12% difficult items (P < 0.30). The majority of items (86%) demonstrated acceptable discrimination capacity, with 56% achieving excellent discrimination (D ≥ 0.40). However, critical concerns emerged: difficult items clustered in geometry and measurement topics (items 39-44, 48), 14% of items showed poor discrimination, and 27% of distractors proved ineffective. Findings indicate that the instrument possesses strong overall technical quality, while requiring targeted revisions to poorly discriminating items and ineffective distractors. The study recommends enhanced instructional support for geometry and measurement topics, qualitative analysis of student reasoning, and future research employing differential item functioning analysis to distinguish mathematical ability from language proficiency effects in English for Mathematics assessment.

 

Keywords: Classical Test Theory, English for Mathematic, Evaluation

Full Text:

PDF

References


Abedi, J., & Lord, C. (2010). The Language Factor in Mathematics Tests. Applied Measurement in Education, 14(3), 219–234. https://doi.org/https://doi.org/10.1207/S15324818AME1403_2

Arikunto, S. (2013). Dasar-dasar evaluasi pendidikan (2nd ed.). Bumi Aksara.

Bailey, A. L., Blackstock-Bernstein, A., & Heritage, M. (2015). At the intersection of mathematics and language: Examining mathematical strategies and explanations by grade and English learner status. The Journal of Mathematical Behavior, 40(Part A), 60–28.

Barwell, R. (2018). From Language as a Resource to Sources of Meaning in Multilingual Mathematics Classrooms. The Journal of Mathematical Behavior, 50, 155–168. https://doi.org/https://doi.org/10.1016/j.jmathb.2018.02.007

Buana, A. N. H., Waru, M. V., & Aditia, R. (2025). Analisis Kesulitan Linguistik dalam Pemahaman Istilah Matematika Berbahasa Inggris oleh Mahasiswa Teknik Informatika. EQUALS: Jurnal Ilmiah Pendidikan Matematika, 8(2), 122–134.

Cesaria, A., & Fitri, D. Y. (2014). Efektivitas pengembangan lembar kerja mahasiswa pada perkuliahan Bahasa Inggris untuk Matematika. Jurnal Pelangi, 6(2), 85–91.

Coetzee, J., & Mammen, K. J. (2017). Language of Learning and Teaching as one of the Causes for Difficulties in Learning Fractions. Science Journal of Education, 5(2), 66–70. https://doi.org/10.11648/j.sjedu.20170502.15

DeMars, C. E. (2018). Classical test theory and item response theory. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). In Classical Test Theory and Item Response Theory (pp. 49–73). https://doi.org/https://doi.org/10.1002/9781118489772.ch2

Dudley-Evans, T., & St. John, M. J. (1998). Developments in English for specific purposes: A multi-disciplinary approach. Cambridge University Press.

Ebel, R. L., & Frisbie, D. A. (1991). Essentials of Educational Measurement (5th ed.). Prentice-Hall.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7(2), 95–105. https://doi.org/10.1016/j.iheduc.2004.02.001

Gronlund, N. E., & Waugh, C. K. (2009). Assessment of Student Achievement (9th ed.). Pearson.

Haag, N., Heppt, B., Stanat, P., Kuhl, P., & Pant, H. A. (2013). Second language learners' performance in mathematics: Disentangling the effects of academic language features. Learning and Instruction, 28, 24–34. https://doi.org/https://doi.org/10.1016/j.learninstruc.2013.04.001

Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and Validating Test Items. Routledge.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Springer Science & Business Media.

Hyland, K. (2006). English for academic purposes: An advanced resource book. Routledge.

Kurniawati, D. (2023). Teachers' linguistic challenges in teaching mathematics in English: A case study in Indonesian bilingual classrooms. Indonesian Journal of Language and Education Studies, 5(2), 55–68.

Martiniello, M. (2008). Language and the Performance of English-Language Learners in Math Word Problems. Harvard Educational Review, 78(2), 333–368. https://doi.org/https://doi.org/10.17763/haer.78.2.7893w8825576563l

Mullen, J. (2009). Enhancing Mathematical Literacy: How has open access to Fisher Digital Publications benefited you ?

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.

Prediger, S., Clarkson, P., & Bose, A. (2016). Purposefully Relating Multilingual Registers: Building Theory and Teaching Strategies for Bilingual Learners Based on an Integration of Three Traditions. International Journal of Science and Mathematics Education, 14(1), 143–164. https://doi.org/https://doi.org/10.1007/s10763-016-9720-2

Schleppegrell, M. J. (2007). The Linguistic Challenges of Mathematics Teaching and Learning: A research review. Reading & Writing Quarterly, 23(2), 139–159. https://doi.org/https://doi.org/10.1080/10573560601158461

Setati, M., & Adler, J. (2000). Between Languages and Discourses: Language Practices in Primary Multilingual Mathematics Classrooms in South Africa. Educational Studies in Mathematics, 43, 243–269. https://doi.org/https://doi.org/10.1023/A:1011996002062

Swanson, H. L., Orosco, M. J., & Lussier, C. M. (2015). Journal of Experimental Child Growth in literacy, cognition, and working memory in English language learners. Journal of Experimental Child Psychology, 132, 155–188. https://doi.org/10.1016/j.jecp.2015.01.001

Taifour, A., & Alabdulaziz, M. (2025). Challenges of English mathematical literacy among EFL learners in higher education. International Journal of Language and Education, 9(1), 22–35.




DOI: http://dx.doi.org/10.30659/kontinu.10.1.176-190

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Hedia Rizki, Reri Seprina Anggraini, Indra Bagus Satrio

License URL: https://creativecommons.org/licenses/by/4.0

Creative Commons License
This work is licensed under a Creative Commons - Attribution-NonCommercial-ShareAlike 4.0 International License.

The Kontinu Journal is published by the Mathematics Education Study Program, Faculty of Teacher Training and Education (FKIP), Universitas Islam Sultan Agung (Unissula).

Contact us: GKB, 2nd Floor, FKIP Unissula, Jl. Raya Kaligawe Km.4, Terboyo Kulon, Genuk, Semarang 50112.
Email: jurnal.kontinu@unissula.ac.id