CONTEMPORARY EDUCATIONAL TECHNOLOGY
e-ISSN: 1309-517X
Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes

Sumie Tsz Sum Chan 1, Noble Po Kan Lo 2 3 * , Alan Man Him Wong 1

CONT ED TECHNOLOGY, Volume 16, Issue 4, Article No: ep541

https://doi.org/10.30935/cedtech/15607

Submitted: 16 July 2024, Published Online: 13 November 2024

OPEN ACCESS   2593 Views   225 Downloads

Download Full Text (PDF)

Abstract

This paper investigates the effects of large language model (LLM) based feedback on the essay writing proficiency of university students in Hong Kong. It focuses on exploring the potential improvements that generative artificial intelligence (AI) can bring to student essay revisions, its effect on student engagement with writing tasks, and the emotions students experience while undergoing the process of revising written work. Utilizing a randomized controlled trial, it draws comparisons between the experiences and performance of 918 language students at a Hong Kong university, some of whom received generated feedback (GPT-3.5-turbo LLM) and some of whom did not. The impact of AI-generated feedback is assessed not only through quantifiable metrics, entailing statistical analysis of the impact of AI feedback on essay grading, but also through subjective indices, student surveys that captured motivational levels and emotional states, as well as thematic analysis of interviews with participating students. The incorporation of AI-generated feedback into the revision process demonstrated significant improvements in the caliber of students’ essays. The quantitative data suggests notable effect sizes of statistical significance, while qualitative feedback from students highlights increases in engagement and motivation as well as a mixed emotional experience during revision among those who received AI feedback.

References

Citation

The articles published in this journal are licensed under the CC-BY Creative Commons Attribution International License.
This website uses cookies to provide necessary website functionality. By using our website, you are agree to our Privacy Policy.