Developing video-conferenced English speaking test (VEST) for classroom-based assessment in tertiary education

David Imamyartha, Mutiara Bilqis, Rizki Febri Andika Hudori, Areta Puspa, Eka Wahjuningsih

Abstract


A proper speaking test may pose challenges to teachers due to suboptimal assessment literacy and the laborious administration of a speaking test. Technology can help alleviate the labor while enabling teachers to ascertain the accuracy and validity of the speaking test. In this direction, this study aims to develop a videoconferenced English-speaking test (henceforth VEST) with the aid of videoconferencing technology. Aligned with the empirics on the use of technology for speaking assessment, this study addresses the gapping void in how videoconferencing technology can be harnessed to escalate the practicality and positive impacts of a classroom-based speaking test in an Indonesian EFL setting. The development adhered to the ADDIE framework involving Analysis, Design, Development, Implementation, and Evaluation. The test prototype was evaluated using a test usefulness analysis framework. The evaluation results were coupled with reflection results of test takers’ experiences to guide test revision. Given the small sample size, this study has unveiled the potential of VEST to improve the praxis of speaking assessment and the resultant test takers’ experience. Implications and recommendations for future studies are also discussed.

Keywords


speaking; videoconferenced English speaking test; VEST; technology

Full Text:

PDF

References


Alderson, J.C., and Banerjee, J. (2002). Language testing and assessment (Part 2). Language Teaching, 35, 79–113. https://doi.org/10.1017/S0261444802001751

Bachman, L.F., and Palmer, A.S. (1996). Language testing in practice. Oxford: Oxford University Press. https://octovany.files.wordpress.com/2013/12/language-testing-in-practice-bachman-palmer.pdf

Byrne, G., and Staehr, L. (2002, June). International internet-based videoconferencing in distance education – a low cost option. Paper presented at the Informing Science IT Education Conference Proceedings, Information Science Institute, California, USA. https://doi.org/10.28945/2451 .

Chapelle, C. (1994). Are C-tests valid measures for L2 vocabulary research? Second Language Research, 10, 157–187. https://doi.org/10.1177/026765839401000203

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Council of Europe. https://www.coe.int/en/web .

Craig, D.A., and Kim, J. (2010). Anxiety and performance in videoconferenced and face-to-face oral interviews. Multimedia-Assisted Language Learning, 13, 9–32. https://doi.org/10.15702/MALL.2010.13.3.9 .

Field, J. (2011). Cognitive validity. In L. Taylor (Ed.), Examining speaking: Research and practice in assessing second language speaking (Studies in Language Testing, Vol. 30) (pp. 65–111). Cambridge, UK: Cambridge University Press. http://hdl.handle.net/10547/338249 .

Galaczi, E. D. (2010). Face-to-face and computer-based assessment of speaking: Challenges and opportunities. In L. Araújo (Ed.), Computer-based assessment of foreign language speaking skills (pp. 29–51). Luxemburg: European Union. https://www.researchgate.net/publication/281090096 .

Galaczi, E. D., Ffrench, A., Hubbard, C, and Green, A. (2011). Developing assessment scales for large-scale speaking tests: a multiple-method approach. Assessment in Education: Principles, Policy and Practice, 18(3), 217-237. http://dx.doi.org/10.1080/0969594X.2011.574605

Grayson, D., and Monk, A. (2003). Are you looking at me? Eye contact and desktop video conferencing. ACM Transactions on Computer-Human Interaction (TOCHI), 10(3), 221–243. http://dx.doi.org/10.1145/937549.937552 .

Gruba, P. (1997). The role of video media in listening assessment. System, 25, 335–345. https://doi.org/10.1016/S0346-251X(97)00026-2 .

Heins, B., Duensing, A., Stickler, U., and Batstone, C. (2007). Spoken interaction in online and face-to-face language tutorials. Computer Assisted Language Learning, 20(3), 279–295. http://dx.doi.org/10.1080/09588220701489440 .

Hirai, A., and Koizumi, R. (2009). Development of a practical speaking test with a positive impact on learning using a story retelling technique. Language Assessment Quarterly, 6(2), 151-167. http://dx.doi.org/10.1080/15434300902801925

Kehoe, M., Niederberger, N., and Bouchut, A-L. (2021). The development of a speech sound screening test for European French-speaking monolingual and bilingual children. International Journal of Speech-Language Pathology, 23(2), 135-144. https://doi.org/10.1080/17549507.2020.1750699

Kenyon, D., and Malabonga, V. (2001). Comparing examinee attitudes toward computer-assisted and other proficiency assessments. Language Learning and Technology, 5(2), 60–83. http://dx.doi.org/10125/25128

Kiddle, T., and Kormos, J. (2011). The effect of mode of response on a semidirect test of oral proficiency. Language Assessment Quarterly, 8(4), 342–360. doi:10.1080/15434303.2011.613503.

Kim, J.T. (2006). The effectiveness of test-takers’ participation in development of an innovative web-based speaking test for international teaching assistants at American colleges (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign. https://www.proquest.com/docview/622019942 .

Kim, J., & Craig, D. A. (2012). Validation of a videoconferenced speaking test. Computer Assisted Language Learning, 25(3), 257–275. https://doi:10.1080/09588221.2011.649482 .

Lee, L. (2007). Fostering second language oral communication through constructivist interaction in desktop videoconferencing. Foreign Language Annals, 40, 635–649. http://dx.doi.org/10.1111/j.1944-9720.2007.tb02885.x

Messick, S. (1989). Validity. In R.L.Linn (Ed.), Educational measurement (pp. 13–103). New York: Macmillan. https://doi.org/10.3102/0013189X018002005 .

Nakatsuhara , F., Inoue, C., and Taylor, L. (2020): Comparing Rating Modes: Analysing Live, Audio, and Video Ratings of IELTS Speaking Test Performances, Language Assessment Quarterly. https://doi.10.1080/15434303.2020.1799222

O’Sullivan, B., Weir, C. J., and Saville, N. (2002). Using observation checklists to validate speaking-test tasks. Language Testing, 19(1), 33–56. doi:10.1191/0265532202lt219oa

Schreiber, L. M., Paul, G. D., and Shibley, L. R. (2012). The development and test of the public speaking competence rubric. Communication Education, 61(3), 205-233. http://dx.doi.org/10.1080/03634523.2012.670709

Schwandt, T.A., and Halpern, E.S. (1988). Linking auditing and meta evaluation. Newbury Park, CA: Sage. https://doi.org/10.1177/109821408901000405 .

van Moere, A. (2012). A psycholinguistic approach to oral language assessment. Language Testing, 29(3), 325–344. doi:10.1177/0265532211424478

Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning and Technology, 11(1), 7–8. http://dx.doi.org/10125/44089

Wainfan, L., and Davis, P.K. (2004). Challenges in virtual collaboration: videoconferencing, audio conferencing, and computer-mediated communications. Santa Monica, CA: RAND Corporation. https://www.rand.org/content/dam/rand/pubs/monographs/2004/RAND_MG273.pdf

Wang, Y. (2004). Supporting synchronous distance language learning with desktop videoconferencing. Language Learning and Technology, 8(3), 90–121. http://dx.doi.org/10125/43997

Wang, Y. (2006). Negotiation of meaning in desktop videoconferencing-supported distance language learning. ReCALL, 18(1), 122–146. http://dx.doi.org/10.1017/S0958344006000814 .

Weir, C. J., Vidakovic, I., and Galaczi, E. (2013). Measured constructs (Studies in Language Testing, Vol. 37). Cambridge, UK: Cambridge University Press. http://hdl.handle.net/10547/338250.

Whitehead, J. (1989). Creating a Living Educational Theory from Questions of the Kind, ‘How do I Improve my Practice?’ Cambridge Journal of Education, 19(1), 41–52. https://doi.org/10.1080/0305764890190106

Xiao, M. (2007). An empirical study of using internet-based desktop videoconferencing in an EFL setting (Unpublished dissertation). Ohio University. http://rave.ohiolink.edu/etdc/view?acc_num=ohiou1194703859




DOI: http://dx.doi.org/10.30659/e.8.1.242-262

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 David Imamyartha, Mutiara Bilqis, Areta Puspa, Rizki Febri Andika Hudori, Eka Wahjuningsih

License URL: https://creativecommons.org/licenses/by/4.0/