The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis

Thada Jantakoon; Thiti Jantakun; Kitsadaporn Jantakun; Weerapa Pongpanich; Rungfa Pasmala; Panita Wannapiroon; Prachyanun Nilsook

doi:10.30935/cedtech/17310

Thada Jantakoon ¹, Thiti Jantakun ² ^* , Kitsadaporn Jantakun ², Weerapa Pongpanich ³, Rungfa Pasmala ⁴, Panita Wannapiroon ⁴, Prachyanun Nilsook ⁴

More Detail

CONT ED TECHNOLOGY, Volume 17, Issue 4, Article No: ep596

https://doi.org/10.30935/cedtech/17310

Submitted: 24 September 2024, Published Online: 20 October 2025

OPEN ACCESS 350 Views 180 Downloads

Download Full Text (PDF)

Abstract

The increasing integration of artificial intelligence (AI) in education has raised significant questions about its pedagogical value, especially in language learning. This meta-analysis examines the extent to which AI contributes to the development of English-speaking and listening skills. A systematic review of the literature was conducted by the preferred reporting items for systematic reviews and meta-analyses guidelines, utilizing peer-reviewed studies indexed in Scopus, ERIC, and EBSCOhost from 2017 to 2024. Nineteen studies met the inclusion criteria, all of which utilized experimental or quasi-experimental designs with measurable learning outcomes. The analysis reveals a substantial overall effect of AI-enhanced instruction (standardized mean difference [SMD] = 0.981, 95% confidence interval [0.571, 1.391], p < .001), with particularly notable improvements in speaking proficiency (SMD = 1.033). Although listening outcomes showed a positive trend (SMD = 0.714), the effect did not attain statistical significance. Considerable heterogeneity was noted across the studies, reflecting variations in learner populations, instructional contexts, and AI applications. Quality appraisal using the risk of bias in non-randomized studies of interventions framework indicated a predominantly low to moderate risk of bias. Publication bias analysis, including funnel plot symmetry and fail-safe N, further confirmed the reliability of the results. These findings highlight the advantages of AI in enhancing speaking skills within English instruction and underscore the need for further empirical studies to investigate its impact on listening comprehension. Collectively, the results provide timely, evidence-based guidance for educators and policymakers aiming to integrate AI effectively into language education. Highlight the advantages of AI in enhancing speaking skills within English instruction and underscore the need.

Keywords: artificial intelligence, English instruction, speaking skill, listening skill, meta-analysis

References

Adipat, S. (2023). An artificial intelligence-enhanced phenomenon-based learning approach for interdisciplinary understanding and speaking skills. International Journal of Instruction, 16(3), 531–550. https://doi.org/10.29333/iji.2023.16329a
Agnes, D., & Srinivasan, R. (2024). Fostering vocabulary memorization: Exploring the impact of AI-generated mnemonic keywords on vocabulary learning through Anki flashcards. World Journal of English Language, 14(2), 434–445. https://doi.org/10.5430/wjel.v14n2p434
Ahmed Ali, S. G. (2020). Using an artificial intelligence application for developing primary school pupils’ oral language skills. Journal of Education Sohag UNV, 75(75), 67–110. https://doi.org/10.21608/edusohag.2020.97643
Alsowat, H. H. (2020). Evidence-based practices of English language teaching: A meta-analysis of meta-analyses. English Language Teaching, 13(11), 75–94. https://doi.org/10.5539/elt.v13n11p75
Amin, M. Y. M. (2023). AI and Chat GPT in language teaching: Enhancing EFL classroom support and transforming assessment techniques. International Journal of Higher Education Pedagogies, 4(4), 554–567. https://doi.org/10.33422/ijhep.v4i4.554
Bao, R., & Sile, L. (2024). Empowering English vocabulary instruction through generative artificial intelligence: The AEFC method. Journal of Educational Research Progress, 4(4). https://doi.org/10.55375/joerp.2024.4.4
Caldwell, A. R. (2022). SimplyAgree: An R package and jamovi module for simplifying agreement and reliability analyses. Journal of Open Source Software, 7(71), Article 4148. https://doi.org/10.21105/joss.04148
Chang, H. (2023). The effect of AI chatbot-based tourism English instruction on intercultural communicative competence. STEM Journal, 24(2), 15–32. https://doi.org/10.16875/stem.2023.24.2.15
Chen, C. H., Koong, C. S., & Liao, C. (2022). Influences of integrating dynamic assessment into a speech recognition learning design to support students’ English speaking skills, learning anxiety, and cognitive load. Educational Technology & Society, 25(1), 1–14.
Choe, S., Lee, K. A., & So, Y. (2020). The effects of phonemic awareness instructions on L2 listening comprehension: A meta-analysis. The Journal of AsiaTEFL, 17(4), 1294–1316. https://doi.org/10.18823/asiatefl.2020.17.4.9.1294
Chu, J., & Szlagor, M. (2023). Multi-intelligence English teaching model based on distance and open education. International Journal of Web-Based Learning and Teaching Technologies, 18(1), 1–15. https://doi.org/10.4018/ijwltt.325617
Daweli, T. W., & Mahoub, R. A. M. (2024). Exploring EFL learners’ perspectives on using AI tools and their impacts in reading instruction: An exploratory study. Arab World English Journal, 10(11). https://doi.org/10.24093/awej/call10.11
Duisenova, M. M., & Zhorabekova, A. N. (2023). The effectiveness of gamification and artificial intelligence in increasing the motivation and effectiveness of students in learning English in elementary school. Eurasia Journal of Mathematics, Science and Technology Education, 19(10), Article em2330. https://doi.org/10.29333/ejmste/13670
Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., Kerekes, M. F., Gerken, M., & Wieseler, B. (2010). Reboxetine for acute treatment of major depression: Systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 341, Article c4737. https://doi.org/10.1136/bmj.c4737
Fathi, J., Rahimi, M., & Derakhshan, A. (2024). Improving EFL learners’ speaking skills and willingness to communicate via artificial intelligence-mediated interactions. System, 121, Article 103254. https://doi.org/10.1016/j.system.2024.103254
Feng, T., Deng, J., Liu, Y., Gao, Y., Peng, L., & Fu, J. (2023). A study on application of artificial intelligence in college English teaching management and evaluation. In Proceedings of the 2023 International Conference on Emerging Information Technology for Smart Applications (pp. 128–132). ACM. https://doi.org/10.1145/3641343.3641447
Ferrer, R. L. (1998). Graphical methods for detecting bias in meta-analysis. Family Medicine, 30, 579–583.
Hao, T., Wang, Z., & Ardasheva, Y. (2021). Technology-assisted vocabulary learning for EFL learners: A meta-analysis. Journal of Research on Educational Effectiveness, 14(4), 834–862. https://doi.org/10.1080/19345747.2021.1917028
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186
Ho, P. X. P. (2024). Using ChatGPT in English language learning: A study on I.T. students’ attitudes, habits, and perceptions. International Journal of TESOL & Education, 4(1). https://doi.org/10.54855/ijte.24414
Hwang, W. Y., Luthfi, M. I., & Liu, Y. F. (2024). AI-enhanced video drama-making for improving writing and speaking skills of students learning English as a foreign language. Innovation in Language Learning and Teaching. https://doi.org/10.1080/17501229.2024.2437655
Ibrahim, K. A. A. A., Kassem, M. A. M., & Lami, D. (2024). Intelligent computer-assisted language assessment (ICALA) in philosophy-based language instruction: Unraveling the effects on critical thinking, self-evaluation, academic resilience, and speaking development. Language Testing in Asia, 14(1), Article 46. https://doi.org/10.1186/s40468-024-00320-1
Jerusha, A., & Rajakumari, R. (2024). Longitudinal analysis of student progress in AI-integrated English language courses. In Proceedings of the 2024 International Conference on Knowledge Engineering and Communication Systems (pp. 1–6). IEEE. https://doi.org/10.1109/ickecs61492.2024.10616489
John, E., & Yunus, M. M. (2021). A systematic review of social media integration to teach speaking. Sustainability, 13(16), Article 9047. https://doi.org/10.3390/su13169047
Kemelbekova, Z., Degtyareva, X., Yessenaman, S., Ismailova, D., & Seidaliyeva, G. (2024). AI in teaching English as a foreign language: Effectiveness and prospects in Kazakh higher education. XLinguae, 17(1), 69–83. https://doi.org/10.18355/XL.2024.17.01.05
Kim, N. Y. (2022). AI-integrated mobile-assisted language learning: Is it an effective way of preparing for the TOEIC test in classroom environments? English Teaching, 77(3), 79–102. https://doi.org/10.15858/engtea.77.3.202209.79
Knapp, G., & Hartung, J. (2003). Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22(17), 2693–2710. https://doi.org/10.1002/sim.1482
Konyrova, L. (2024). The evolution of language learning: Exploring AI’s impact on teaching English as a second language. Eurasian Science Review: An International Peer-Reviewed Multidisciplinary Journal, 2(2), 133–138. https://doi.org/10.63034/esr-42
Lee, J. H., Lee, H., & Lo, Y. Y. (2023). Effects of EMI-CLIL on secondary-level students’ English learning: A multilevel meta-analysis. Studies in Second Language Learning and Teaching, 13(3), 439–467. https://doi.org/10.14746/ssllt.38277
Lee, S., Kuo, L., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning, 35(1–2), 1–29. https://doi.org/10.1080/09588221.2020.1774612
Li, Y. (2024). A deep learning multimodal feature fusion approach for assessing English study. In Proceedings of the 2024 4^th Asia-Pacific Conference on Communications Technology and Computer Science (pp. 0047–0052). IEEE. https://doi.org/10.1109/acctcs61748.2024.00047
Lin, C. J., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16–35.
Lin, H., & Liu, X. (2024). InVideo AI: Creating instructional audiovisual input for English learning. RELC Journal. https://doi.org/10.1177/00336882241267773
Ling, W. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1261955
Liu, P.-L., & Chen, C.-J. (2023). Using an AI-based object detection translation application for English vocabulary learning. Educational Technology & Society, 26(3), 5–20. https://doi.org/10.30191/ETS.202307_26(3).0002
Madhavi, E., Sivapurapu, L., Koppula, V., Sreehari, V., & Rani, P. (2023). Developing learners’ English-speaking skills using ICT and AI tools. Journal of Advanced Research in Applied Sciences and Engineering Technology, 32(2), 142–153. https://doi.org/10.37934/araset.32.2.142153
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007
Murdoch, Y. D., & Lin, Y. (2023). Factors mediating the link between engagement and satisfaction among online English-mediated instruction learners during COVID-19. Electronic Journal of e-Learning, 21(3). https://doi.org/10.34190/ejel.21.3.3031
Ni, A., Cheung, A., & Shi, J. (2022). Effects of educational technology on reading achievement for Chinese K-12 English second language learners: A meta-analysis. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1025761
Peng, J., & Xie, X. (2021). English-medium instruction as a pedagogical strategy for the sustainable development of EFL learners in the Chinese context: A meta-analysis of its effectiveness. Sustainability, 13(10), Article 5637. https://doi.org/10.3390/su13105637
Qian, M., & Kong, C. (2024). Exploring the gap: The challenge of achieving human-like generalization for concept-based translation instruction using large language models. In Proceedings of the AAAI Spring Symposia. AAAI. https://doi.org/10.1609/aaaiss.v3i1.31283
Qiao, H., & Zhao, A. (2023). Artificial intelligence-based language learning: Illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1255594
Rahmati, J., Izadpanah, S., & Shahnavaz, A. (2021). A meta-analysis on educational technology in English language teaching. Language Testing in Asia, 11(1), Article 37. https://doi.org/10.1186/s40468-021-00121-w
Rosenthal, R. (1979). The ‘file drawer problem’ and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
Sharifi, M., Seyedagha Sharifi, M. B., Rostami AbuSaeedi, A., Jafarigohar, M., & Zandi, B. (2018). Retrospect and prospect of computer assisted English language learning: A meta-analysis of the empirical literature. Computer Assisted Language Learning, 31(4), 413–436. https://doi.org/10.1080/09588221.2017.1412325
Song, F., Khan, K. S., Dinnes, J., & Sutton, A. J. (2002). Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. International journal of epidemiology, 31(1), 88–95. https://doi.org/10.1093/ije/31.1.88
Sonsaat-Hegelheimer, S., & Kurt, Ş. (2024). The impact of generative AI-powered chatbots on L2 comprehensibility. Journal of Second Language Pronunciation, 10(3), 339–374. https://doi.org/10.1075/jslp.24053.son
Tang, J. L., & Liu, J. L. (2000). Misleading funnel plot for bias detection in meta-analysis. Journal of Clinical Epidemiology, 53(5), 477–484. http://doi.org/10.1016/s0895-4356(99)00204-8
Um, H., Kim, H., Choi, D., & Oh, H. (2023). An AI-based English education platform during the COVID-19 pandemic. Universal Access in the Information Society, 23, 1233–1248. https://doi.org/10.1007/s10209-023-01046-2
Wang, C., Zou, B., Du, Y., & Wang, Z. (2024a). The impact of different conversational generative AI chatbots on EFL learners: An analysis of willingness to communicate, foreign language speaking anxiety, and self-perceived communicative competence. System, 127, Article 103533. https://doi.org/10.1016/j.system.2024.103533
Wang, X., Zhang, Q., Chen, H., Neitzel, A. J., & Davis, M. H. (2024b). The effects of language learning strategy instruction on college students’ English achievement and learner autonomy in mainland China: A meta-analysis. International Journal of Educational Research, 124, Article 102442. https://doi.org/10.1016/j.ijer.2024.102442
Wang, Y., & Zheng, L. (2022). A novel deep framework for English communication based on educational psychology perspective. Frontiers in Public Health, 10. https://doi.org/10.3389/fpubh.2022.916101
Wu, Y. (2024). Study on the impact of utilizing ChatGPT and other AI tools for feedback in EAP writing classrooms on the discursive writing performance of English major students. Transactions on Social Science, Education and Humanities Research, 4, 143–150. https://doi.org/10.62051/4se95x52
Xatamova, N. (2024). The future of legal English learning: Integrating AI into ESP education. SPAST Reports, 1(7). https://doi.org/10.52773/tsuull.conf.teach.foreign.lang.2024.8.5/pnxs6025
Yang, T. (2024). Impact of artificial intelligence software on English learning motivation and achievement. SHS Web of Conferences, 193, Article 02011. https://doi.org/10.1051/shsconf/202419302011
Yaseen, M. G. (2023). Exploring the evolution of AI integration in English as a foreign language education: A Scopus-based bibliometric analysis (1997-2023). Mesopotamian Journal of Computer Science, 2(2), 19–31. https://doi.org/10.58496/mjcsc/2023/019
Zhang, C., Meng, Y., & Ma, X. (2024). Artificial intelligence in EFL speaking: Impact on enjoyment, anxiety, and willingness to communicate. System, 121, Article 103259. https://doi.org/10.1016/j.system.2024.103259
Zhao, Y. (2013). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(1), 7–27. https://doi.org/10.1558/cj.v21i1.7-27
Zheng, C., Chen, X., Zhang, H., & Chai, C.S. (2024). Automated versus peer assessment: Effects on learners’ English public speaking. Language Learning & Technology, 28(2), 210–228.
Zou, B., Guan, X., Shao, Y., & Chen, P. (2023). Supporting speaking practice by social network-based interaction in artificial intelligence (AI)-assisted language learning. Sustainability, 15(4), Article 2872. https://doi.org/10.3390/su15042872

Citation

Jantakoon, T., Jantakun, T., Jantakun, K., Pongpanich, W., Pasmala, R., Wannapiroon, P., & Nilsook, P. (2025). The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. Contemporary Educational Technology, 17(4), ep596. https://doi.org/10.30935/cedtech/17310

Jantakoon, T., Jantakun, T., Jantakun, K., Pongpanich, W., Pasmala, R., Wannapiroon, P., and Nilsook, P. (2025). The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. Contemporary Educational Technology, 17(4), ep596. https://doi.org/10.30935/cedtech/17310

Jantakoon T, Jantakun T, Jantakun K, et al. The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. CONT ED TECHNOLOGY. 2025;17(4), ep596. https://doi.org/10.30935/cedtech/17310

Jantakoon, Thada, Thiti Jantakun, Kitsadaporn Jantakun, Weerapa Pongpanich, Rungfa Pasmala, Panita Wannapiroon, and Prachyanun Nilsook. "The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis". Contemporary Educational Technology 2025 17 no. 4 (2025): ep596. https://doi.org/10.30935/cedtech/17310

Jantakoon, Thada et al. "The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis". Contemporary Educational Technology, vol. 17, no. 4, 2025, ep596. https://doi.org/10.30935/cedtech/17310

Jantakoon T, Jantakun T, Jantakun K, Pongpanich W, Pasmala R, Wannapiroon P, et al. The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. CONT ED TECHNOLOGY. 2025;17(4):ep596. https://doi.org/10.30935/cedtech/17310