Developing video-conferenced English speaking test (VEST) for classroom-based assessment in tertiary education
DOI:
https://doi.org/10.30659/e.8.1.242-262Kata Kunci:
speaking, videoconferenced English speaking test, VEST, technologyAbstrak
A proper speaking test may pose challenges to teachers due to suboptimal assessment literacy and the laborious administration of a speaking test. Technology can help alleviate the labor while enabling teachers to ascertain the accuracy and validity of the speaking test. In this direction, this study aims to develop a videoconferenced English-speaking test (henceforth VEST) with the aid of videoconferencing technology. Aligned with the empirics on the use of technology for speaking assessment, this study addresses the gapping void in how videoconferencing technology can be harnessed to escalate the practicality and positive impacts of a classroom-based speaking test in an Indonesian EFL setting. The development adhered to the ADDIE framework involving Analysis, Design, Development, Implementation, and Evaluation. The test prototype was evaluated using a test usefulness analysis framework. The evaluation results were coupled with reflection results of test takers’ experiences to guide test revision. Given the small sample size, this study has unveiled the potential of VEST to improve the praxis of speaking assessment and the resultant test takers’ experience. Implications and recommendations for future studies are also discussed.Referensi
Alderson, J.C., and Banerjee, J. (2002). Language testing and assessment (Part 2). Language Teaching, 35, 79–113. https://doi.org/10.1017/S0261444802001751
Bachman, L.F., and Palmer, A.S. (1996). Language testing in practice. Oxford: Oxford University Press. https://octovany.files.wordpress.com/2013/12/language-testing-in-practice-bachman-palmer.pdf
Byrne, G., and Staehr, L. (2002, June). International internet-based videoconferencing in distance education – a low cost option. Paper presented at the Informing Science IT Education Conference Proceedings, Information Science Institute, California, USA. https://doi.org/10.28945/2451 .
Chapelle, C. (1994). Are C-tests valid measures for L2 vocabulary research? Second Language Research, 10, 157–187. https://doi.org/10.1177/026765839401000203
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Council of Europe. https://www.coe.int/en/web .
Craig, D.A., and Kim, J. (2010). Anxiety and performance in videoconferenced and face-to-face oral interviews. Multimedia-Assisted Language Learning, 13, 9–32. https://doi.org/10.15702/MALL.2010.13.3.9 .
Field, J. (2011). Cognitive validity. In L. Taylor (Ed.), Examining speaking: Research and practice in assessing second language speaking (Studies in Language Testing, Vol. 30) (pp. 65–111). Cambridge, UK: Cambridge University Press. http://hdl.handle.net/10547/338249 .
Galaczi, E. D. (2010). Face-to-face and computer-based assessment of speaking: Challenges and opportunities. In L. Araújo (Ed.), Computer-based assessment of foreign language speaking skills (pp. 29–51). Luxemburg: European Union. https://www.researchgate.net/publication/281090096 .
Galaczi, E. D., Ffrench, A., Hubbard, C, and Green, A. (2011). Developing assessment scales for large-scale speaking tests: a multiple-method approach. Assessment in Education: Principles, Policy and Practice, 18(3), 217-237. http://dx.doi.org/10.1080/0969594X.2011.574605
Grayson, D., and Monk, A. (2003). Are you looking at me? Eye contact and desktop video conferencing. ACM Transactions on Computer-Human Interaction (TOCHI), 10(3), 221–243. http://dx.doi.org/10.1145/937549.937552 .
Gruba, P. (1997). The role of video media in listening assessment. System, 25, 335–345. https://doi.org/10.1016/S0346-251X(97)00026-2 .
Heins, B., Duensing, A., Stickler, U., and Batstone, C. (2007). Spoken interaction in online and face-to-face language tutorials. Computer Assisted Language Learning, 20(3), 279–295. http://dx.doi.org/10.1080/09588220701489440 .
Hirai, A., and Koizumi, R. (2009). Development of a practical speaking test with a positive impact on learning using a story retelling technique. Language Assessment Quarterly, 6(2), 151-167. http://dx.doi.org/10.1080/15434300902801925
Kehoe, M., Niederberger, N., and Bouchut, A-L. (2021). The development of a speech sound screening test for European French-speaking monolingual and bilingual children. International Journal of Speech-Language Pathology, 23(2), 135-144. https://doi.org/10.1080/17549507.2020.1750699
Kenyon, D., and Malabonga, V. (2001). Comparing examinee attitudes toward computer-assisted and other proficiency assessments. Language Learning and Technology, 5(2), 60–83. http://dx.doi.org/10125/25128
Kiddle, T., and Kormos, J. (2011). The effect of mode of response on a semidirect test of oral proficiency. Language Assessment Quarterly, 8(4), 342–360. doi:10.1080/15434303.2011.613503.
Kim, J.T. (2006). The effectiveness of test-takers’ participation in development of an innovative web-based speaking test for international teaching assistants at American colleges (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign. https://www.proquest.com/docview/622019942 .
Kim, J., & Craig, D. A. (2012). Validation of a videoconferenced speaking test. Computer Assisted Language Learning, 25(3), 257–275. https://doi:10.1080/09588221.2011.649482 .
Lee, L. (2007). Fostering second language oral communication through constructivist interaction in desktop videoconferencing. Foreign Language Annals, 40, 635–649. http://dx.doi.org/10.1111/j.1944-9720.2007.tb02885.x
Messick, S. (1989). Validity. In R.L.Linn (Ed.), Educational measurement (pp. 13–103). New York: Macmillan. https://doi.org/10.3102/0013189X018002005 .
Nakatsuhara , F., Inoue, C., and Taylor, L. (2020): Comparing Rating Modes: Analysing Live, Audio, and Video Ratings of IELTS Speaking Test Performances, Language Assessment Quarterly. https://doi.10.1080/15434303.2020.1799222
O’Sullivan, B., Weir, C. J., and Saville, N. (2002). Using observation checklists to validate speaking-test tasks. Language Testing, 19(1), 33–56. doi:10.1191/0265532202lt219oa
Schreiber, L. M., Paul, G. D., and Shibley, L. R. (2012). The development and test of the public speaking competence rubric. Communication Education, 61(3), 205-233. http://dx.doi.org/10.1080/03634523.2012.670709
Schwandt, T.A., and Halpern, E.S. (1988). Linking auditing and meta evaluation. Newbury Park, CA: Sage. https://doi.org/10.1177/109821408901000405 .
van Moere, A. (2012). A psycholinguistic approach to oral language assessment. Language Testing, 29(3), 325–344. doi:10.1177/0265532211424478
Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning and Technology, 11(1), 7–8. http://dx.doi.org/10125/44089
Wainfan, L., and Davis, P.K. (2004). Challenges in virtual collaboration: videoconferencing, audio conferencing, and computer-mediated communications. Santa Monica, CA: RAND Corporation. https://www.rand.org/content/dam/rand/pubs/monographs/2004/RAND_MG273.pdf
Wang, Y. (2004). Supporting synchronous distance language learning with desktop videoconferencing. Language Learning and Technology, 8(3), 90–121. http://dx.doi.org/10125/43997
Wang, Y. (2006). Negotiation of meaning in desktop videoconferencing-supported distance language learning. ReCALL, 18(1), 122–146. http://dx.doi.org/10.1017/S0958344006000814 .
Weir, C. J., Vidakovic, I., and Galaczi, E. (2013). Measured constructs (Studies in Language Testing, Vol. 37). Cambridge, UK: Cambridge University Press. http://hdl.handle.net/10547/338250.
Whitehead, J. (1989). Creating a Living Educational Theory from Questions of the Kind, ‘How do I Improve my Practice?’ Cambridge Journal of Education, 19(1), 41–52. https://doi.org/10.1080/0305764890190106
Xiao, M. (2007). An empirical study of using internet-based desktop videoconferencing in an EFL setting (Unpublished dissertation). Ohio University. http://rave.ohiolink.edu/etdc/view?acc_num=ohiou1194703859
Unduhan
Diterbitkan
Terbitan
Bagian
Lisensi
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
