EFFECTIVENESS OF AI VERSUS HUMAN FEEDBACK ON SSC STUDENTS' L2 ENGLISH LANGUAGE WRITING PROFICIENCY: A COMPARATIVE ANALYSIS
DOI:
https://doi.org/10.5281/fs8vvq22Keywords:
AI Feedback, ESL Assessment, Human Teacher Feedback, L2 Writing, SSC Students PakistanAbstract
The integration of artificial intelligence (AI) into English language teaching provides new possibilities for timely and efficient feedback on students’ writing. This study compares human teacher feedback and AI-generated feedback in assessing the L2 English language writing proficiency of Secondary School Certificate (SSC) students in Pakistan. The data was collected from purposively sampled student of SSC who secured more than 95% marks in their English paper in SSC examination conducted by BISE Mardan. A quantitative comparative research design was employed using 26 student compositions evaluated through the ESL Composition Profile developed by Jacobs et al. (1981), which measures content, organization, vocabulary, language use, and mechanics. The students were asked to write an English essay regarding their exam experience. The essay was evaluated from subject specialist in English with over 10 years of teaching experience. The same essay was evaluated from ChatGPT (AI) via predefined scoring profile. The human teacher showed common Pakistani SSC assessment practices, and focused on content coverage and communicative effectiveness, while AI applied standardized criteria with greater focus on grammatical accuracy and coherence. Descriptive statistics, paired-samples t-tests, Pearson correlation, and intraclass correlation coefficients (ICC) were used for analysis. Results showed that human teachers assigned significantly higher scores (M = 68.42, SD = 8.31) than AI feedback (M = 61.15, SD = 9.02), t(25) = 5.84, p < .001. Despite these differences, strong agreement was observed (r = .82; ICC = .76). The findings suggest that AI can effectively supplement, but not replace, human feedback in Pakistani ESL classrooms.
Downloads
References
Aljaafreh, A. L., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal, 78(4), 465–483. https://doi.org/10.1111/j.1540-4781.1994.tb02064
Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence: Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62.
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17(2), 102–118. https://doi.org/10.1016/j.jslw.2007.11.004
Bitchener, J., & Ferris, D. R. (2012). Written corrective feedback in second language acquisition and writing. Routledge.
Bitchener, J., & Knoch, U. (2010). The contribution of written corrective feedback to language development: A ten-month investigation. Applied Linguistics, 31(2), 193–214. https://doi.org/10.1093/applin/amp016
Bitchener, J., & Storch, N. (2016). Written corrective feedback for L2 development. Multilingual Matters.
Crosthwaite, P., & Sun, C. (2025). Generative AI and L2 written feedback: A comprehensive review. Journal of Second Language Writing. Advance online publication.
Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1(1), 3–18. https://doi.org/10.5070/L2.V1I1.9054
Ferris, D. R. (2003). Response to student writing: Implications for second language students. Lawrence Erlbaum Associates.
Ferris, D. R. (2014). Treatment of error in second language student writing (2nd ed.). University of Michigan Press.
Ferris, D. R., & Roberts, B. (2001). Error feedback in L2 writing classes: How explicit does it need to be? Journal of Second Language Writing, 10(3), 161–184.
Fu, Q., Zou, D., Xie, H., & Cheng, G. (2024). A review of automated writing evaluation feedback: Types, learning outcomes, and implications. Computer Assisted Language Learning, 37(1), 179–221.
Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing assistant’s impact on English language learners. Computers and Education: Artificial Intelligence, 3, Article 100055.
Godwin-Jones, R. (2022). Partnering with AI: Intelligent writing assistance and instructed language learning. Language Learning & Technology, 26(2), 5–24.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
Hyland, K. (2003). Second language writing. Cambridge University Press.
Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39(2), 83–101. https://doi.org/10.1017/S0261444806003399
Jacobs, H. L., Zinkgraf, S. A., Wormuth, D. R., Hartfiel, V. F., & Hughey, J. B. (1981). Testing ESL composition: A practical approach. Newbury House.
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E.,
Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeiffer, F., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274.
Lee, I. (2017). Classroom writing assessment and feedback in L2 school contexts. Springer.
Li, X., et al. (2024). Preliminary investigation into the role of ChatGPT in English as a foreign language writing assessments. Assessing Writing. Advance online publication.
Mahapatra, S. (2024). The pedagogical effectiveness of ChatGPT in English language writing among ESL learners. Computer Assisted Language Learning. Advance online publication.
Polakova, P., & Ivenz, R. (2024). The impact of ChatGPT feedback on EFL students’ writing skills. Language Learning & Technology. Advance online publication.
Stevenson, M., & Phakiti, A. (2019). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65.
Teng, F. (2024). EFL learners’ perceptions and experiences regarding the use of ChatGPT for writing feedback. System. Advance online publication.
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel? ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15.
Yan, X., & Zhang, Z. (2024). Learners’ engagement with automated written corrective feedback generated by ChatGPT: A mixed-method multiple-case study. The Modern Language Journal. Advance online publication.
Yoon, H., Miszoglad, E., & Pierce, R. (2023). Investigating the quality of ChatGPT-generated feedback on English language learners’ essays. TESOL Quarterly. Advance online publication.
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education: Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Shehzad Khan, Sawairoon Hayat, Aisha Hussain, Aizaz khan, Maryam Salam (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
