The Use of AI-Generated Feedback in an Indonesian EFL Writing Class: Student Achievement and Perceptions

Muhammad Ahkam Arifin

Research has shown that teacher and peer feedback can facilitate the development of EFL (English as a foreign language) students’ writing. However, due to workload, teachers often have limited time to provide individualized, detailed, and immediate feedback, while students often express concerns about the accuracy of peer feedback. Currently, an increasing number of studies have shown that GenAI (generative artificial intelligence) chatbots (e.g., ChatGPT), which can provide automated, detailed feedback for EFL writing, have the potential to help students improve their writing skills. However, research exploring whether AI-generated feedback can be a comparable alternative to teacher or peer feedback remains limited. Accordingly, this research examined how ChatGPT-generated feedback, compared to teacher feedback and peer feedback, could enhance EFL students’ writing skills. The analysis of co-variance (ANCOVA) indicated that the participants across the three groups significantly improved their writing performance on the posttest based on the overall scores, and that no significant differences were observed among the groups. However, follow-up t-tests revealed that the AI feedback group showed no significant improvements in lexical resources, yet their lexical scores were not significantly different from those of the teacher and peer feedback groups. This indicates that more research is needed to see whether and how AI feedback can support vocabulary acquisition as well as humans do. Further, the Likert-scale questionnaire revealed that initially the participants were less optimistic about the ability of ChatGPT in providing quality feedback on writing aspects that require higher-order thinking (e.g., content, organization). Nevertheless, having participated in this nine-week research and learnt how to use ChatGPT in helping them revise their essays, the participants significantly became more confident in the tool’s ability to provide accurate and reliable feedback not only on surface-level (e.g., grammar, lexis) but also on deep-level aspects of writing. This suggests that students’ familiarity with GenAI chatbots can change their perceptions and use of the technology. The semi-structured interviews and open-ended questionnaires found that students can seek feedback from ChatGPT at any time without feeling anxious or uncomfortable. The participants mentioned that ChatGPT can help them provide feedback both on sentence-level features (e.g., grammar and mechanics) and on discourse-level elements (e.g., organization and coherence) of writing. Lastly, they stated that ChatGPT has the potential to be used as a reliable automated essay scoring (AES) tool for assessing essays. Overall, these findings indicate that AI-generated feedback can be as effective as that of teachers and peers and, thus, has the potential to help reduce EFL teachers’ workload in providing detailed feedback to student essays. With AI chatbots, EFL students now have access to an on-demand personalized learning tutor, which they can always consult at any time and from anywhere for immediate, detailed feedback. However, because GenAI chatbots can hallucinate and generate inconsistent and inaccurate feedback, EFL teachers need to provide professional development to students on how they can effectively and ethically leverage AI affordances to support their writing development without engaging in academic dishonesty (e.g., plagiarism). For example, teachers can provide formally evaluated or empirically validated prompts that can generate feedback appropriate to students’ needs and can help students accurately revise their essays. They can also show students how they can critically evaluate AI-suggested revisions.

The Use of AI-Generated Feedback in an Indonesian EFL Writing Class: Student Achievement and Perceptions

Files and links (1)

Abstract

Metrics

Details