Usability of ChatGPT in the english essay writing proficiency of the BEED students
DOI:
https://doi.org/10.56003/jse.v4i2.338Keywords:
ChatGPT, english essay, BEED students, AI technology, writing proficiencyAbstract
The advancement of AI technology, especially in language processing systems like ChatGPT, creates opportunities for students to elevate their writing skills. This study aimed to explore the relationship between the usability of ChatGPT and the English essay writing proficiency of the BEED students. The study utilized a descriptive correlational design to investigate the relationship. A sample of 58 participants was selected to write an essay with the assistance of ChatGPT. Researchers developed and employed a rubric to assess the quality and ethical use of ChatGPT. Then, the participants answered the USE questionnaire to measure their perceived usability of ChatGPT. The result revealed that BEED students perceived ChatGPT as beneficial in enhancing their essays and found the system easy to use, and learn, and provided satisfying results. The participants also expressed a strong sense of contentment regarding using ChatGPT. The essay writing proficiency assessment revealed that BEED students were proficient writers. However, a Pearson correlation coefficient of 0.042 showed a statistically significant but very weak relationship observed between ChatGPT usability and English essay writing proficiency levels. It can be concluded that ChatGPT may improve students' essay writing proficiency to a minimal extent. Moreover, this study recommended further research to explore ChatGPT's use in other academic outputs, refine the rubrics to include vocabulary, grammar, and sentence structure criteria, and provide training for effective AI tool management among students. Also, policymakers should establish clear policies and guidelines to ensure the responsible and ethical use of ChatGPT, maintaining academic integrity and enhancing writing proficiency.
Downloads
References
Amaro, I. R., Greca, A., Francese, R., Tortora, G., & Tucci, C. (2023). AI Unreliable Answers: a case study on ChatGPT. In Lecture Notes in Computer Science (pp. 23–40). https://doi.org/10.1007/978-3-031-35894-4_2
Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
Bašić, Ž., Banovac, A., Kružić, I., & Jerković, I. (2023). Better by You, better than Me? ChatGPT-3 as writing assistance in students’ essays.
Bhandari, P. (2021). An introduction to correlational research. https://www.scribbr.com/methodology/correlational-research
Bin-Nashwan, S. A., Sadallah, M., & Bouteraa, M. (2023). Use of ChatGPT in academia: Academic integrity hangs in the balance. Technology in Society, 75, 102370.
Biswas, S., Logan, N. S., Davies, L. N., Sheppard, A. L., & Wolffsohn, J. S. (2023). Assessing the utility of ChatGPT as an artificial intelligence‐based large language model for information to answer questions on myopia. Ophthalmic and Physiological Optics, 43(6), 1562–1570. https://doi.org/10.1111/opo.13207
Bozkurt, A., Xiao, J., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., Farrow, R., Bond, M., Nerantzi, C., Honeychurch, S., Bali, M., Dron, J., Mir, K., Stewart, B., Costello, E., Mason, J., Stracke, C. M., Romero-Hall, E., Koutropoulos, A., Toquero, C. M., Singh, L., Tlili, A., Lee, K., Nichols, M., Ossiannilsson, E., Brown, M., Irvine, V., Raffaghelli, J. E., Santos-Hermosa, G., Farrell, O., Adam, T., Thong, Y. L., Sani-Bozkurt, S., Sharma, R. C., Hrastinski, S., & Jandrić, P. (2023). Speculative Futures on ChatGPT and Generative Artificial Intelligence (AI): A Collective Reflection from the Educational Landscape. Asian Journal of Distance Education, 18(1). Retrieved from https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/709
Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1). Retrieved from https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/718
Campbell, M. (2019). Teaching academic writing in higher education. Educ. Quar. Rev. 2, 608–614. https://10.31014/aior.1993.02.03.92
Caratiquit, K. D., & Caratiquit, L. J. C. (2023). ChatGPT as an academic support tool on the academic performance among students: The mediating role of learning motivation. Journal of Studies in Higher Education, 4(1), November. https://doi.org/10.35912/jshe.v4i1.1558
Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1-12.
Dergaa, I., Chamari, K., Zmijewski, P., & Saad, H. B. (2023). From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40(2), 615-622.
Dharani, M., Jyostna, J. V. S. L., Sucharitha, E., Likitha, R., & Manne, S. (2020). Interactive Transport Enquiry with AI Chatbot. 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS), 1271–1276.
Duval, A., Lamson, T., de Kérouara, G. D. L., & Gallé, M. (2020). Breaking writer’s block: Low-cost fine-tuning of natural language generation models. arXiv preprint arXiv:2101.03216.
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., . . . Wright, R. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Fitria, T. N. (2023). Artificial intelligence (AI) technology in OpenAI ChatGPT application: A review of ChatGPT in writing English essay. ELT Forum: Journal of English Language Teaching, 12(1), 44-58.
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089.
Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic integrity in online assessment: A research review. In Frontiers in Education (Vol. 6, p. 639814). Frontiers Media SA. https://doi.org/10.1109/ICICCS48265.2020.9120905
Huy, N. T. (2015). Problems affecting learning writing skill of grade 11 at Thong Linh high school. Asian Journal of Educational Research, 3(2).
Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature. Contemporary Educational Technology, 15(4), ep464. https://doi.org/10.30935/cedtech/13605
Karaköse, T. (2023). The Utility of ChatGPT in Educational Research—Potential Opportunities and Pitfalls. Educational Process: International Journal, 12(2). https://doi.org/10.22521/edupij.2023.122.1
Lee, S., & Park, G. (2023). Exploring the impact of ChatGPT literacy on user satisfaction: The mediating role of user motivations. Cyberpsychology, Behavior, and Social Networking, 26(12), 913–918. https://doi.org/10.1089/cyber.2023.0312
Limna, P., Kraiwanit, T., Jangjarat, K., Klayklung, P., and Chocksathaporn, P. (2023). The use of ChatGPT in the digital era: perspectives on chatbot implementation. J. Appl. Learn. Teach. 6, 64–74. https://10.37074/jalt.2023.6.1.32
Liu, B. (2023). Chinese University Students’ Attitudes and Perceptions in Learning English Using ChatGPT. International Journal of Education and Humanities, 3(2), 132-140.
Lund, A.M. (2001). Measuring Usability with the USE Questionnaire. STC Usability SIG Newsletter, 8:2
Luo, X., Tong, S., Fang, Z., & Qu, Z. (2019). Frontiers: Machines vs. humans: The impact of artificial intelligence chatbot disclosure on customer purchases. Marketing Science, 38(6), 937-947.
Malik, A. R., Pratiwi, Y., Andajani, K., Numertayasa, I. W., Suharti, S., & Darwis, A. (2023). Exploring Artificial Intelligence in Academic Essay: Higher Education Student's Perspective. International Journal of Educational Research Open, 5, 100296.
Mahapatra, S. (2024). Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study. Smart Learn. Environ. 11, 9. https://doi.org/10.1186/s40561-024-00295-9
Marzuki, S., Widiati, U., Rusdin, D., Darwin, R., & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2), 2236469. https://doi.org/10.1080/2331186X.2023.2236469
Mulia, A. P., Piri, P. R., & Tho, C. (2023). Usability Analysis of Text Generation by ChatGPT OpenAI Using System Usability Scale Method. Procedia Computer Science, 227, 381-38.
Napitupulu, D., Rahim, R., Abdullah, D., Setiawan, M. I., Abdillah, L. A., Ahmar, A. S., Simarmata, J., Park, H., Nurdiyanto, H., & Pranolo, A. (2018). Analysis of student satisfaction toward quality of service facility. Journal of Physics: Conference Series, 954, 012019. https://doi.org/10.1088/1742-6596/954/1/012019
Ngo, T. T. A. (2023). The perception by university students of the use of CHATGPT in education. International Journal of Emerging Technologies in Learning (Ijet), 18(17), 4–19. https://doi.org/10.3991/ijet.v18i17.39019
Posecion, O. T., Go, M. B., & Albano, H. P. (2011). Language research: Principles and application.
Rodríguez, J. M. R., Ramírez-Montoya, M. S., Fernández, M. B., & Lara, F. L. (2023). Use of ChaTGPT at university as a tool for complex thinking: students’ perceived usefulness. Journal of New Approaches in Educational Research, 12(2), 323. https://doi.org/10.7821/naer.2023.7.1458
Roe, J., & Perkins, M. (2022). What are Automated Paraphrasing Tools and how do we address them? A review of a growing threat to academic integrity. International Journal for Educational Integrity, 18(1), 15.
Shaikh, S., Yayilgan, S. Y., Klimova, B., & Pikhart, M. (2023). Assessing the usability of ChatGPT for formal English language learning. European Journal of Investigation in Health, Psychology and Education, 13(9), 1937-1960.
Sharma, S., & Yadav, R. (2022). Chat GPT – a technological remedy or challenge for education system. https://www.gjeis.com/index.php/GJEIS/article/view/698
Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays—Should academics worry. Nature. https://doi.org/10.1038/d41586-022-04397-7
Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation. Educ. Inf. Technol. 28, 13943–13967. https://doi.org/10.1007/s10639-023-11742-4
Yoon, C. (2011). Concordancing in L2 writing class: an overview of research and issues. J. Engl. Acad. Purp. 10, 130–139. https://doi.org/10.1016/j.jeap.2011.03.003
Zhou, C., Qiu, C., & Acuna, D. E. (2022). Paraphrase Identification with Deep Learning: A Review of Datasets and Methods (arXiv:2212.06933). arXiv. https://doi.org/10.48550/arXiv.2212.06933
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Karl Alvin Aglibot, Edrian Mark Tomines, Janel Talaman, Janabeth Soguilon, Rowena Bongolto, Rhona Marielle Abayon
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
