Whilst still a relatively new type of AI to be introduced into the field of language learning, there is a body of research which has examined how students view its usefulness. Syahid et al. (2023) focused on the perceived usefulness of ChatGPT in EFL writing classes from the perspective of Indonesian university students. He found that while this technology has the potential to aid student learning in certain aspects, there was some reluctance to rely on it. A further study also done in Indonesia (Zebua & Ketemba, 2024) found that 82 university students had overall positive perceptions learning writing using ChatGPT and felt it can help with improving the motivation to learn and help to correct grammatical errors. These results were mirrored in a different study (Rahim, 2023) which collected data from 181 students across various university campuses in Malaysia concluding most respondents felt ChatGPT can help to improve writing skills and that it adds an extra level of interest to their learning experience. In addition, they also felt ChatGPT to be helpful with writing specific skills of refining vocabulary usage, correcting grammatical errors, and getting personalized feedback.

In Vietnam, Phan (2023), conducted a study on 100 university students’ perceptions of general AI technology use in writing classes. This research included discussions on several different AI technologies, including Google Translate, Grammarly, and Paraphrasing Tool and there was no mention of ChatGPT. The results showed overall feelings towards AI were positive. However, a lack of learning opportunities combined with limited instructional variety and inhibitions related to the use of technology meant perceptions were not as high as they could have been. Also in Vietnam, and more specific to ChatGPT, Vo & Nguyen (2024) surveyed 369 students across several different universities and found that due to its ease of use, ChatGPT was a useful tool for language learning. This study did not focus solely on writing, instead choosing to examine the effectiveness of ChatGPT in relation to the four main language skills. Specifically, they felt this technology is limited in improving the oral skills of speaking and listening. Interestingly, they had more positive perceptions in terms of developing reading and writing skills, although these skills were not discussed in any detail.

In Korea, Van Horn (2024) explored the perceptions of university students towards the use of ChatGPT in EFL classes. Similar to the studies discussed above, perceptions were high and led to increased collaboration amongst students along with higher levels of autonomous learning. There were still some persistent technical challenges among some of the participants as well as lingering skepticism on the benefits of using ChatGPT to learn English. However, this study focused on conversation-based classes instead of writing-based ones. Bok and Cho, (2023), explored the perceptions of university students of using ChatGPT but solely as a tool for revising work rather than including as part of the instructional process. These results once again reveal positive perceptions based on ease of use, unrestricted use across time and space, and accuracy in terms of vocabulary, grammar, and written cohesion.

One way to understand the effectiveness and potential benefits and drawbacks of ChatGPT, is to compare it to learning in a traditional classroom. Conventional EFL instruction, which normally relies on textbooks, tools such as PowerPoint files, videos, classroom discussions, and direct interactions between teachers and students. However, integrating ChatGPT changes this traditional method by adding a dynamic and interactive aspect to language learning. One study (Shahid et al., 2023) looked at a comparative analysis of a traditional EFL class and one incorporating ChatGPT. The study suggested both ethical considerations and the responsible use of ChatGPT are needed to offer a balanced view of the implications of integrating this technology into EFL instruction. The study also discovered EFL learners using ChatGPT showed a statistically significant improvement in writing skills compared to those using traditional teaching methods. However, vocabulary acquisition showed a statistically significant difference favoring the traditional method. Likewise, a study (Boudouaia, 2024) with 76 undergraduate students from a private school in Algeria divided participants into an experimental group who were allowed access to ChatGPT and a control group who were not. The results showed that the former outperformed the latter and demonstrated significant improvements in post-test scores, perceived usefulness, ease of use, attitudes, and behavioral intentions. ChatGPT was found to enhance students’ EFL writing skills, leading to its acceptance and interest due to its practical applications.

However, in a more specific study (Niloy et al., 2023) regarding learners’ creative ability in writing proved that using ChatGPT was detrimental to students’ creative learning skills. The study involved 600 students from the top 10 ranked universities in Bangladesh divided into a control group taught without the aid of ChatGPT and an experimental group who could use it in their creative writing process. It evaluated originality, content presentation, accuracy, and elaboration through the Wilcoxon Signed-Rank Test. The results indicated a negative link between ChatGPT use and university students’ creative writing abilities. Both machine-generated and human-generated assessments confirmed qualitative observations on ChatGPT’s adverse impact on creative writing. The research showed the need for a cautious approach to AI integration in creative writing, considering both benefits and drawbacks. These insights provide guidance for future research and education to effectively integrate AI while nurturing students’ writing skills.

Moving away from the classroom and learning and not specific to writing, a study from the Netherlands (De Winter, 2013) compared ChatGPT and high school students that completed the English national high school exam focusing on English reading comprehension. The same exam questions were entered through the ChatGPT and according to official standards, ChatGPT achieved an average score of 7.3 on the Dutch grading scale of 1 to 10, which is similar to the average score of all students who took the exam in the Netherlands, which was 6.99. However, ChatGPT sometimes needed additional prompts to provide a clearer definitive answer. Without these prompts, the overall grade was 6.5. Although this study is a reference to testing it could be applied in a comparable manner to that of learning, whereby ChatGPT will not solely take over the traditional classroom room setting but would rather help to assist language learners and use it as a supporting tool by incorporating it into the traditional classroom way of learning.

The final part of this project attempts to understand how ChatGPT will affect the need for and design of EFL classes at Korean universities. A limited amount of work has been done regarding this issue in Korea. As a result, existing research done on other countries was examined. In Vietnam (Nguyen, 2023) explored the insights of university EFL language teachers on how ChatGPT could be implemented concluding professional training for both teachers and students for ChatGPT to play a more effective role in the classroom. However, exactly what training is needed and how that should be done was unclear. Madhi (2023) in Saudi Arabia found that college freshman enrolled in EFL writing classes felt ChatGPT to be helpful in terms of grammar usage, vocabulary choice, and syntax clarity. However, they felt it should be a tool that both students and teachers can look towards only as a supplemental resource in the future and should not be relied on for any more than that. Other key factors in the writing classroom such as engaging learners, managing classrooms, and choosing appropriate learning methods cannot be handled well enough by ChatGPT for it to assume a greater role. These results are mirrored in a different study (Meniado, 2023) which expanded on the potential limitations of greater reliance on ChatGPT to include issues such as inaccurate responses, plagiarism, skill deterioration, and generic responses. One study done in Korea (Cho, Richards, & Jones, 2019) gathered university students’ perceptions on how 4IR technology could impact EFL classes of the future. This study found that 4IR technology would undoubtedly be a driving factor behind changes in this field. Despite this, it also discovered a desire for mandatory classes to continue in the future and elements such as having a human teacher, classes being conducted offline, and student ownership of the learning process should be valued. However, it should be noted that this study was done several years before the release of ChatGPT, and it could be interesting to gauge if any of these perceptions have changed since then.

While all the above research provides valuable insight into how university students based both inside and outside of Korea feel about using ChatGPT in writing classes, there is some scope to clarify how Korean university students feel about using this technology as part of their day-to-day writing lessons. Consequently, this study will attempt to answer the following research questions:

Research for this study was conducted in the spring/ summer of 2024 and focused on students enrolled in mandatory EFL writing classes at one university in the greater Seoul area. 247 people participated in a survey, 44.4% of whom were female and 55.6% male. 9.6% of respondents indicated they were aged 19, 51.5% were aged 20, 25.9% were aged 21, 6.7% were 22, and the remaining 6.3% were aged 23 or older. 97.9% of respondents were in their freshman year of university and the other 2.1% were either sophomore or senior students. The data were collected from students of varying English ability with 55.2% of respondents saying they were in basic level classes, 35.1% were in intermediate classes, and 9.6% were enrolled in advanced level classes. The vast majority of respondents indicated their major was either in the field of engineering or economics (36.6% and 26.4% respectively). The remaining respondents were studying a variety of other areas, including but not limited to humanities, law, social sciences, and computer science.

Data was collected between June 4th and July 10th, 2024, through a survey questionnaire (Appendix A) designed and written by the research team using Google Forms. This method was chosen for several reasons as outlined by Evans and Mather (2005). It provided flexibility for the project and enabled data collection without being limited by space and time. Additionally, it allowed the inclusion of various question types to obtain a wide range of responses and facilitated the collection of a large dataset. Furthermore, given that survey questionnaires are highly effective in gauging perceptions on specific topics or situations (Young, 2016), the research team determined it was an appropriate choice for this project.

The target respondents were enrolled in a mandatory EFL writing class, named College English 1 (CE1) at this university. These classes were all conducted in-person, and the focus of the course was to improve students’ paragraph / essay writing skills. The survey was distributed via a hyperlink and QR code posted to the announcements section of the learning management system (LMS) for each CE1 class taught by the research team. Students were then requested to complete the survey on their own outside of class time. They were clearly informed that it would have no effect on any class grades or scores and they were under no obligation to complete it. In addition, a total of 6 colleagues of the research team who were also teaching CE1 classes were contacted via email and requested to share the survey with their students by following the same procedure outlined above.

The initial survey item was designed as a control question to ensure all respondents were enrolled in a CE1 class at the time they took the survey. 99.2% responded affirmatively while data from the 2 respondents who answered negatively was discarded. Furthermore, the final survey question requested permission from respondents to use their responses as part of this research. Data from those who rejected this request (6) was also discarded, leaving data from 239 respondents to be used in the analysis phase. The remaining survey items were made up of a diverse range of questions, including short and long answer questions, multiple choice questions, and 5-point Likert scale questions. They were designed to focus on students’ experience of using ChatGPT, if and how they incorporated it into their EFL writing classes, their perceptions of how ChatGPT could influence EFL writing classes of the future, and potential pedagogical implications. A Cronbach Alpha test was applied to the data from the multiple-choice Likert scale questions and a score of 0.96 indicates an excellent level of internal consistency. The final survey question asked respondents if they would be willing to discuss their responses in more detail. Those who responded positively were asked to leave their email addresses and were contacted in early July for a more in-depth discussion of responses that the research team felt relevant.

During the data analysis phase of the study, the researchers systematically organized and scrutinized survey responses, which included both Likert scale and multiple-choice questions, using descriptive statistical methods. For short and long-answer questions that provided deeper insights into participant opinions, the researchers employed a general analysis of qualitative research. Initially, they individually reviewed the data, identifying recurring themes. Following this, they convened several times to compare findings, exchange notes, and identify overarching trends and patterns. These meetings included discussions to refine the overall results and resolve any discrepancies that arose.

Question 7 asked how often the participants used ChatGPT to learn English. A substantial proportion of the participants (63.2%) said they have never used ChatGPT to learn English. 42 respondents (17.6%) used it only once a month, whereas 23 (9.6%) mentioned they used it several times a month. 29 (9.6%) respondents used it approximately once a week with 14 (5.9%) using the tool several times per week, only 1 (0.4%) participant used it once a day with 3 (1.3%) participants using ChatGPT several times a day.

More specifically, survey question 8 asked if the participants ever used ChatGPT to assist them in their CE1 class. Again, a sizable portion of the total respondents, 164 out of 239 (68.6%) said they never used it while the remaining 75 respondents (31.4%) said they did use it to assist them.

Those 75 respondents were then guided to a section of the survey designed to analyze specifically how ChatGPT assisted them. For question 9, respondents were asked to select which aspects of writing ChatGPT helped them with. The options of vocabulary and grammar were the most frequently selected (49.3% and 48%) respectively which could indicate a lack of confidence when tackling these aspects of writing. This notion was confirmed during follow-up email discussions from which the quotes in Table 1 have been taken.

The remaining responses were mixed with and can be seen listed in Table 2. Interestingly, only 10 respondents indicated they found ChatGPT assisted them with spelling although this could be explained by the ‘vocabulary’ option which also provided the correct spelling if needed.

Following on from this, question 10 asked the same 75 respondents to what extent they felt ChatGPT helped with learning EFL writing compared to what they covered in their CE1 classes and why or why not. A relatively small percentage indicated they either found it very helpful or extremely helpful, 14.4% and 6.7%, respectively. 21 (28%) selected somewhat helpful, 25 (33.3%) selected slightly helpful, and 13 (17.3%) said they found it extremely unhelpful. It could be logical to suggest these responses were based on everyone’s English proficiency level. However, an in-depth analysis of these data points shows no clear related pattern. Students from advanced classes commonly selected each of the options multiple times, likewise students from beginner classes did the same. This demonstrates ChatGPT is not particularly helpful or unhelpful for any specific ability group.

As a follow-up, question 11 asked respondents to clarify why or why not, in addition to helping with grammar and vocabulary as discussed earlier, and there were several other emergent positive themes including help choosing topics / brainstorming, teacher-like interactions, speed and ease of use, getting feedback, and help with written organization. Table 3 summarizes these themes and includes a selection some of the corresponding comments:

One interesting point from the above data points regards how students deem ChatGPT to be quick and easy to use. However, the comments provided suggest some of the respondents used it to produce writing which they claimed as their own which surely raises questions about plagiarism and authenticity. This may indicate students require further clarification on the acceptable uses of ChatGPT and how it can be used as a learning resource.

On the other hand, there were also negative themes. The first of which suggests that students do not trust ChatGPT. One participant wrote: “ChatGPT is often wrong or conveys different content.” A different student wrote: “Sometimes there’s false information, and if you don’t actively check it, you can write the wrong thing and maybe lose points.” In a follow up email with one of these respondents elaborated:

When I was writing one of my assignments for my College English 1 class, I translated some sentences from Korean into English using ChatGPT, but they did not feel correct. After checking several times, it appeared ChatGPT had translated it differently each time. Also, ChatGPT suggested using grammar and punctuation that was very different from what I learned from my CE1 professor.

Another negatively perceived aspect of ChatGPT was students seeing no purpose behind using it. There were several comments suggesting that over-reliance on it would result in a decreased ability to learn and understand English independently of using AI. “I think that if you use ChatGPT, you can depend on ChatGPT. So, your English skill cannot be developed.;’ “College English 1 classes are far more useful to develop English skills than ChatGPT.”; “My professor is much better at helping me to improve my writing than ChatGPT will ever be”. This suggests that ownership of the learning process is important to these students, and it is not something ChatGPT can help with.”

The final negative aspect is surprising in that several respondents suggested other AI technology is preferable to ChatGPT when it comes to learning English. One respondent wrote: “I think the translator like Papago or Google translate is better, and there seems to be no reason to use GPT instead of a translator.” Another stated: “For learning writing there are other AIs which are much more suitable. ChatGPT is better for other subjects like math or coding.” Although this type of comment was not as prevalent as others, it does reveal an underlying feeling that this type of 4IR technology may not be the most suited to help with EFL writing.

For the second research question, the researchers hoped to generate some information on how to determine the participants’ perceptions of ChatGPT as a learning option compared to traditional methods of instruction in a mandatory EFL writing class. Survey Question 12 asked if they felt ChatGPT had any advantages over traditionally taught English writing classes such as CE1 with question 13 asking for more details in the form of short responses. 239 participants answered this question with only 105 (43.9%) answering that it did. Several participants stated reasons due to class size and not fully understanding what the professor taught, and the speed and ease of using ChatGPT were beneficial in a writing class. Responses from participants: “It is fast, convenient, and saves time on writing;” “It has the advantage of being able to get an immediate answer” and one saying “ChatGPT doesn’t make mistakes or errors compared to humans.” In a longer correspondence regarding this, one respondent said:

There are multiple students, but there is only one professor who teaches them, so it can be difficult to receive and answer detailed questions, but using ChatGPT can answer questions about things your professor did not teach you and help you understand things you did not understand.

However, the majority, 134 (56.1%) answered negatively, stating there were no benefits, and a traditionally taught class is still preferred as ChatGPT may provide incorrect information and answers. Examples from the respondents included, “I think it would be more helpful to learn from a native professor than ChatGPT’. This is because ChatGPT still provides a lot of inaccurate information, and I think it is more beneficial to discuss and teach classes directly with the social community” and one respondent stating “This is because ChatGPT still provides a lot of inaccurate information, and I think it is more beneficial to discuss and teach classes directly with the social community”.

Question 14 was the opposite of the previous one and asked the participants if they felt traditionally taught English writing classes such as CE1 have any advantages over using ChatGPT to learn English. Remarkably, out of the 239 who responded to this question, 206 (86.2%) answered yes to this question and a deeper investigation into why (question 15) revealed an obvious reason. Despite the numerous perceived benefits to ChatGPT discussed earlier, many respondents indicated the human element of traditionally taught writing classes is something they strongly desire. This includes being taught by a human teacher, communicating with them, and discussing and comparing ideas with their peers. A selection of such related comments can be seen in Table 4.

Data from the 33 respondents who answered negatively to question 15 showed no clear identifiable patterns and consisted of noticeably short, often obscure answers; thus, it was hard to establish why they felt this way. Further studies might be needed to explore this area in more detail.

The 5-point Likert scale in question 16 asked the participants if they felt mandatory English writing classes such as CE1 are still necessary in the current era of ChatGPT. Figure 1 shows 186 (77.9%) respondents selected agree or strongly agree in response to this question. Despite having access to ChatGPT, over three quarters of the pool of respondents still believe CE1 classes and those similar should still be mandatory at university.

The final research question aimed to assess opinions on how ChatGPT might impact the future of EFL writing classes. Survey question 17 asked respondents to select their preferred type of class based on their experience with CE1 and using ChatGPT: The data shows that almost half of the participants (116 respondents, 48.5%). prefer a face-to-face class while also having access to ChatGPT for support. The second most preferred option (84 respondents, 35.1%) was an in-person class only without any AI influence. Fewer respondents favored a combination of ChatGPT and an online class (28 respondents, 11.7%), while only 11 respondents (4.6%) preferred solely to just use ChatGPT. Further studies are needed to explore the reasons behind these preferences.

The participants were asked (Question 18) if they felt eventually ChatGPT could replace the need for English writing classes such as CE1. The overall response to this question was clear with 166 (69.4%) indicating they either disagreed or strongly disagreed (Figure 2). Interestingly, almost one-fifth of participants gave a neutral response indicating some feeling of hesitation relating to this question. One possible consideration that should be given to the results of both survey question 16 and 18 is that the participants may have been reluctant to express their negative feeling both their CE1 course and their instructor leading to them indicating more favorably in terms of maintaining the mandatory status of CE1 classes and less favorably in terms of greater AI adoption.

Survey question 19 continued by asking participants to explain why or why not. The main trend of these responses mirror those discussed in question 15 in that there was strong desire for human interaction which cannot be replicated by ChatGPT: “You can’t learn the format of writing in English from ChatGPT, which is mainly used by native speakers, you need CE1 to learn this unique English skill”. Another participant stated, “Communication and learning writing skills are most helpful when they are cultivated through direct contact with people.” A more detailed comment from a different participant:

I felt ChatGPT was not as useful as a face-to-face class, even when I was in a completely online class due to the coronavirus, but people, including myself, as well as other students, felt the advantages of face-to-face classes which included concentration, interaction with the teacher and social communication. This cannot be replaced by ChatGPT.

However, the few participants who felt ChatGPT could replace English writing classes all commented about the development of AI technology and ChatGPT could one day substitute traditionally taught writing classes:

This is because the learning ability and information transmission ability of ChatGPT is satisfactory, and it is likely that the development of ChatGPT will surpass the excellent teaching ability in the future” and “With the development of AI, it will be possible to replace it one day.

Participants were then asked if they thought CE1 teachers should integrate ChatGPT into their weekly lessons (Question 20). Continuing with a reluctance to fully embrace ChatGPT as seen in previous questions, slightly over half (53.3%) indicated negative feelings towards this potential pedagogical modification (Figure 3). It is significant to note that almost as many participants indicated a neutral response which could indicate a level of indecision on this issue. Question 21 gave the participants the opportunity to express themselves in more detail. Once again, as discussed previously, it was clear participants seemed to embrace not only the human element of having face to face classes but also the possibility over overreliance on ChatGPT: “If GPT is integrated, it will be difficult to develop students’ independent power, so GPT isn’t necessary” and “In order to improve your skills, it is better not to use ChatGPT.” One other respondent elaborated in greater detail:

This is because they believe that what they learn in college English 1 is far better from what they can learn in ChatGPT. Of course, you can refer to ChatGPT for grammatical content, but I do not think you should use ChatGPT in college English class because English writing requires a variety of things other than grammatical elements, and the professors’ information is the most accurate and practical. Personally, I believe that if students use and rely on ChatGPT as a necessity, their own language skills will not improve.

Interestingly, those who believe teachers should integrate ChatGPT in the lessons all noted the same concern along with the feeling the it will inevitably have some effect of the education of the future: “Now that artificial intelligence has penetrated deeply into our lives, I don’t think we can unconditionally exclude it”; “With the development of artificial intelligence, I think it is necessary to use something like ChatGPT in class so that students can use it as well” and finally,

College English 1 has the advantage of face-to-face classes where you actually meet and communicate with people, so I think it would be a shame to get rid of it. However, I think that ChatGPT should be introduced in a class because it is a trend in the world.

If ChatGPT were to be introduced, survey question 22 (Figure 4) asked if students should receive training on how to it to improve their English writing skills. Almost half (45.2%) selected strongly agree or agree when answering this question, meaning 55% of the participant pool were either undecided or responded negatively. This shows that training students on how this technology could be integrated into traditionally taught classes is not as urgent as some would suggest.

Survey question 23 (Figure 5) asked if teachers should receive training on how to incorporate AI tools like ChatGPT into writing classes. The data somewhat mirrors that of the previous question with 93 (38.9%) responding with strongly agree or agree, 67 (28%) selecting strongly disagree or disagree. Once again, the neutral option was selected the most often meaning 112 (46.8%) were indifferent on this matter.

Some of the most interesting data from this study was extracted from question 24 which asked if they believed English writing classes such as College English 1 should continue to be a required subject in future semesters. The results were conclusive with 200 (83.7%) either answering strongly agree or agree and only 13 (5.5%) answering strongly disagree or disagree. Despite developments of AI technology and ChatGPT and it being more accessible, the participants still perceive courses such as CE1 should play an integral role in Korean university programs of the future. However, one caveat to this is the potential for the data from this question to be slightly skewed based on the participants’ intention not to offend or upset their instructor by responding to this question in a negative way.

The final question asked the respondents to elaborate on their answer to question 24 by asking why or why not. Analysis of these responses show three clearly identifiable themes. Firstly, students value the knowledge and experience of learning English in a traditional setting as opposed to using ChatGPT and seem to believe English is going to continue to be important for both their life at university and beyond. A considerable number of comments noted how English is important not only for other courses / university life but also for life after graduation. In a brief discussion relating to this via email, one participant commented:

English writing is indirectly incorporated into all disciplines, so it is difficult to do other classes without learning writing through College English 1. Even though I may not have to write English essays in other classes. Other skills that I have learned such as grammar, organization, referencing and writing patterns help me read textbooks and understand ideas in my major lessons.

The second theme relates to the perceived limitations of using ChatGPT to learn EFL writing and indicates that respondents do not believe they can be solved anytime soon. The final theme re-visits a previously mentioned idea that students desire human interaction from both teacher and classmates when taking EFL writing classes. A selection of further comments linked to each of them can be seen in Table 4.