4.1. Prevalence of ChatGPT Usage
Research question 1 attempts to develop an understanding on how prevalent ChatGPT usage is for homework assignments in courses taught in English. Survey question 7 asked the participants if they had ever done so. 171 (67.1%) answered ‘yes’ and their responses were compared with other demographic data points such as age, major, and how many English courses they had taken, but there were no clear identifiable trends. However, there was one interesting data point found when examining students’ English level and response to question 7. As shown in
Table 1, students from intermediate level courses appear to be using ChatGPT for home assignments more often than students from the other two levels. Although the reasons behind this were not clear, it might be possible to suggest lower-level students find it more difficult to apply ChatGPT to their homework assignments whereas higher level learners are skilled enough to not need to use it.
<Table 1>
Percentage of students who use ChatGPT for homework based on English level
English Level |
n |
No |
Yes |
Beginner |
42 |
19 (45.2%) |
23 (54.7%) |
|
Intermediate |
176 |
50 (27.8%) |
127 (72.1%) |
|
Advanced |
37 |
16 (43.2%) |
21 (56.7%) |
Those who answered ‘yes’ to question 7 were directed to a section of the survey designed to further understand how they used ChatGPT and any perceived benefits / drawbacks. The first part of this section (survey question 8) asked the extent to which the participants found ChatGPT helpful for homework assignments. Slightly over 40% of the 171 students indicated they found it either very helpful or extremely helpful. A similar number (45%) said they found it somewhat helpful with only 11.7% saying it was slightly helpful. Only 1 student selected the “not very helpful” option. Based on the findings indicated in
Table 1, data from question 8 was also compared to students’ English level. Once again, there are clear indications (
Table 2) that students who took intermediate classes are not only using ChatGPT more often but also find it to be the most helpful.
<Table 2>
Percentage of students who find ChatGPT most helpful based on English level
English Level |
n |
Not very helpful |
Slightly helpful |
Somewhat helpful |
Very helpful |
Extremely helpful |
Beginner |
23 |
1 (4.3%) |
5 (21.7%) |
12 (52.1%) |
3 (13.0%) |
2 (8.7%) |
|
Intermediate |
127 |
0 (0%) |
12 (9.4%) |
55 (43.3%) |
38 (29.9%) |
22 (17.3%) |
|
Advanced |
21 |
0 (0%) |
3 (14.2%) |
10 (47.6%) |
6 (28.5%) |
2 (9.5%) |
Survey question 9 asked the students to explain why or why not. Analysis of responses reveal there were a number of recurring themes and patterns including ChatGPT helping with grammar and sentence structure, assistance with brainstorming and developing ideas, ChatGPT being better than other available technology, and ChatGPT reducing time and increasing efficiency when completing assignments. A summary of these, along with a selection of student responses can be seen in
Table 3.
<Table 3>
Selection of comments from 171 participants on why ChatGPT was helpful
Grammar & Sentence Structure Improvement |
“ChatGPT corrects grammar mistakes and improves sentence flow.” |
“It simplifies complex sentences while maintaining meaning.” “It corrects grammar errors better than I can.” |
“When I ask about unfamiliar grammar or words, it provides example sentences and exercises.” |
|
Idea Generation, Brainstorming & Creative Thinking |
“Sometimes I struggle to start writing, but ChatGPT helps me get started.” |
“It gives me ideas I wouldn’t have thought of myself.” |
“It helps me think of things I wouldn’t have thought of.” |
“There’s a limit to how much I can think on my own.” |
|
ChatGPT superiority over other available technology |
“Helps in structuring sentences more naturally compared to other translation tools.” |
“Unlike other translation apps, it minimizes grammatical errors and provides various ways to express the same meaning.” |
“Papago sometimes fails to translate smoothly, but ChatGPT almost never does.” |
“Provides more natural English translations compared to Papago or dictionaries.” |
|
Time Efficiency & Convenience |
“Looking things up in a dictionary would take me days, but ChatGPT reduces that time to just a day”. |
“Searching online often leads to vague explanations. ChatGPT helps simplify complex topics for better understanding”. |
“It saves time in searching for information”. |
“ChatGPT can generate results in seconds or minutes that would take an ordinary person hours or days to think about, and these results are often better than expected”. |
In an email follow-up, one participant offered a brief summary of all these benefits while also commenting on how they feel ChatGPT cannot replace the human input needed to complete English homework assignments. However, they also felt it has helped them to improve their overall ability to complete English homework assignments:
Since ChatGPT is generative AI, it doesn’t help me create original sentences, but it’s useful for checking words and flow based on large language models (LLM). It doesn’t replace human creativity but helps with structure and flow. It quickly turns ideas into actual results with just a command. It helps me realize errors in my thinking, and sometimes what I thought was a good idea isn’t actually good. My overall improvement has been faster since using GPT. (Participant A, male, age 20, intermediate level).
Despite all the above listed benefits, there was one area in which the students were unhappy about ChatGPT performance when completing assignments. They felt it is not always accurate and provides incorrect information / responses to prompts. There were a number of comments relating to how students used ChatGPT to help with an assignment, but they felt the information was either incorrect or not suitable, leading to students using their own knowledge of English to complete it. A summary of such comments can be seen in
Table 4. This not only demonstrates awareness of the limitations of this technology but the fact the students knew ChatGPT was wrong and were able to apply their own knowledge to rectify the mistakes.
<Table 4>
Selection of comments from 171 participants on why ChatGPT was unhelpful
“ChatGPT helps me find information and translate English, but it sometimes gives incorrect information that I don’t feel comfortable using in my assignment.” |
“Sometimes, it gives false information, so I double-check with online sources”. |
“Sometimes, ChatGPT overcomplicates my sentences. It’s only helpful occasionally, and I often end up rewriting sentences myself”. |
“Sometimes, ChatGPT provides the answers I want, but sometimes it doesn’t. I often end up using ChatGPT to begin my assignment but give up and use my own knowledge to complete it”. |
“It’s helpful, but GPT isn’t perfect, so I need to rewrite or revise things myself”. |
The survey continued by asking the participants what types of assignments they use ChatGPT for and why. In survey question 10, respondents were presented with a list of commonly assigned types of English homework. They were asked to choose what type of assignment they used ChatGPT for and multiple selections were possible. There was also the option of selecting ‘other’ and entering any other types of assignment not covered in the initial list. The data from this question can be seen in
Table 5.
<Table 5>
Types of assignment that students most commonly used ChatGPT for
Aspect |
Selected (n) |
% |
Essay Writing |
86 |
50.3 |
|
Presentation Preparation |
74 |
43.3 |
|
Paragraph Writing |
53 |
31 |
|
Grammar Assignment / Quiz |
40 |
23.4 |
|
Vocabulary Assignment / Quiz |
24 |
14 |
|
Speaking Assignment / Quiz |
15 |
8.8 |
|
Other (Translation |
2 |
1.2 |
|
Other (Brainstorming) |
2 |
1.2 |
Given that previous questions have revealed some interesting finds when comparing question data to English level, the same principle was applied to this data. It showed that students who took intermediate classes were more likely to use ChatGPT for a wider variety of assignments. Students from the intermediate level generally selected a greater number of assignment types compared to students from other levels suggesting they are more likely to leverage ChatGPT for different types of tasks. Advanced students tend to use ChatGPT for more complex combinations of assignments, such as essay writing combined with grammar or speaking assignments. Basic level students primarily use ChatGPT for basic grammar or vocabulary assignments.
Survey question 11 asked the participants the reasons behind why they opted to use ChatGPT for those particular types of assignments. General observations from the data indicate ChatGPT is particularly helpful for tasks that require extensive writing and organization of ideas, such as essays, paragraphs and preparing presentations. There were indications that it was somewhat helpful for vocabulary and grammar assignments, yet its use for speaking assignments was variable. More specifically, there were clear trends that emerged when examining each type of specific assignment as shown in
Table 6.
<Table 6>
Selection of reasons from 171 participants for using ChatGPT based on assignment type
Type of Assignment |
Common perceived themes |
Selection of Student Responses |
Essay / Paragraph Writing |
Improving Cohesion / Clarity |
“I didn’t want to repeat the same words, so I used ChatGPT to find alternative words.” |
“I often use it to find synonyms. It is easy to find various synonyms and learn their nuances, when they are appropriate to use, and whether they sound natural” |
“My teacher told me to avoid too much repetition of the same words, but it’s hard for me to notice that. I use ChatGPT to help me” |
|
Improving Quality of Sentences |
“As someone who is not proficient in English, I used it for grammar and to make sentences smoother.” |
“ChatGPT expresses sentences in natural-sounding words.” |
“I wanted to use English expressions that native speakers use.” |
|
Time efficiency |
“There’s no reason not to use it. Everyone is using it to save time and work efficiently. I use GPT as an assistant, but I still make the final decisions and review everything.” |
“Because it quickly provides the answers I need.” |
“It allows multi-angle analysis through various questions and is highly efficient in a short time.” |
|
Preparing Presentations |
Improving Organization |
“When I had determined my presentation content, I asked ChatGPT to generate a PPT outline. This way, I could use it as a reference for structuring my slides.” |
“ChatGPT organizes information well and fills in the parts I missed.” |
“To check if my presentation flows naturally.” |
|
Improved Understanding of complex concepts in English |
“My class PPT was about circuit connections in English. Since I had an exam coming up, I thought simply translating it might cause me to miss important information.” |
“I tried searching the internet for presentation script content, but I couldn’t find the answers I wanted. ChatGPT helped me to do this |
|
Grammar Assignment / Quiz |
Improving Quality of Grammar |
“Since I am not good at English, I used it to construct correct grammar.” |
“I lack English skills, so I use it for grammar corrections” |
“ChatGPT corrects awkward grammatical expressions in English.” |
|
Improved knowledge of grammar rules |
“To study unfamiliar grammar rules and be able to use them by myself” |
“It helps me understand grammar forms that we do not have in Korean” |
|
Vocabulary Assignment / Quiz |
Improved Understanding |
“It corrects my vocabulary and explains why I was wrong.” |
“It finds words with meanings that better fit my writing, making it sound more natural and native-like.” |
|
Time efficiency |
“I didn’t know the word, so I searched for it—it’s more convenient than a dictionary.” |
“Because it allows me to use simple yet effective vocabulary. I don’t need to spend time checking my words other kinds of app or translators” |
Aside from the themes and comments listed above, there was one other notable theme found in the responses to question 11 which relates back to an idea discussed earlier. ChatGPT now appears to be the go-to form of technology used for home assignments with students seemingly abandoning previously used resources such as Google Translate and Naver’s Papago. One participant stated: “I felt that using Papago for translation had its limitations, so I wanted to use more advanced English offered by ChatGPT”. Another commented: “It is difficult to get the correct grammar using Google or Naver, but ChatGPT is better and more natural”. A different participant elaborated: “Because it is the most widely used program among students because it generates excellent English results directly without needing a separate translation process”. This indicates a shift in approach to how students are using technology to assist with homework assignments.
This section concluded with question 12, which asked the students if they felt using ChatGPT helped them get a higher score on the homework assignments. Almost 60% of respondents selected agree or strongly agree. Interestingly, only 12 (7.1%) out of the 171 answered negatively and felt it did not help them achieve higher scores. The data for this question can be seen in
Figure 1.
[Figure 1]
Does ChatGPT help student get higher assignment scores
A more detailed analysis of the question once again reveals students who took intermediate classes were more likely to agree as shown in
Table 7. This could be due to ChatGPT’s ability to provide better vocabulary, correct grammar, and enhance sentence structure, which are crucial for students who have a general level of understanding but need assistance with more complex tasks. Basic level students might not feel as strongly that ChatGPT helps them achieve higher scores because they are still developing foundational language skills and might struggle to understand the suggestions provided by ChatGPT. Advanced learners might agree to a lesser extent, as they are more confident in their language abilities and may feel ChatGPT is less helpful than students in other levels 84 participants (32.9%) stated they had never used ChatGPT to help with English homework assignments. Thus, were not required to respond to any of the survey items discussed above. Instead, they were directed to question 13 asking them to explain their reasons for not doing so. Analysis of these responses has uncovered two clear reasons which could be considered as drawbacks to this technology.
<Table 7>
Percentage of students by level who feel ChatGPT helped with homework scores
English Level |
n |
Strongly Disagree |
Disagree |
Neutral |
Agree |
Strongly Agree |
Beginner |
23 |
0 (0.0%) |
4 (19.0%) |
8 (38.1%) |
5 (23.8%) |
4 (19.0%) |
|
Intermediate |
127 |
1 (0.7%) |
4 (3.1%) |
45 (35.4%) |
57 (44.8%) |
20 (15.7%) |
|
Advanced |
21 |
0 (0.0%) |
4 (19.0%) |
8 (38.1%) |
5 (23.8%) |
4 (19.0%) |
The most frequently mentioned reason was a strong desire to be able complete assignments independently of ChatGPT and therefore maintaining ownership of the process of learning English. Many of these students appear to value being able to complete and submit assignments without having to rely on AI based technology. Furthermore, they feel that doing so allows them to develop as students and learn the language at a pace which is comfortable for them. One survey response stated: “Submitting ChatGPT’s answers that I didn’t create is no benefit to me.” Another elaborated: “I have to think for myself. If I use GPT for assignments, my English skills won’t improve”. Another commented: “It ruins my English learning a bit”. Several students were contacted to discuss this idea in more detail. In an email thread, one stated:
I believe using ChatGPT means missing out on opportunities to grow and study on my own. While it might get you points by doing your assignments, as a student who wants to learn, I think it’s more important to study and work on assignments myself at my own speed. I don’t plan on using it in the future, and I think we should stop students who use ChatGPT instead of doing their own work. (Participant B, female, age 20, intermediate level).
This idea was also somewhat reflected in conversation with a different participant:
My English class was within my ability, and I thought doing it myself would benefit my academic achievement more. It’s easier for me to do my English assignments in my own words and easier to memorize that way. I could do it on my own, so there was no particular reason to use ChatGPT. I just didn’t feel the need to use it. The tasks weren’t difficult, and I heard most classes prohibit GPT”. (Participant C, male, age 21, advanced level).
This comment from a Zoom call mirrored these sentiments while also implying that using ChatGPT is a waste tuition fee:
Using ChatGPT in a class meant to help me learn language skills is not beneficial at all. We pay tuition, and if ChatGPT does all the assignments, the efficiency of learning goes down, and it defeats the purpose. I don’t use it because I don’t want to waste my tuition money. (Participant D, male, age 20, intermediate level).
The second reason for not using ChatGPT for homework assignments relates to authenticity and accuracy of content produced. Students appear to place high value on submitting work that is original and created by themselves. Furthermore, there are high levels of distrust with the answers produced by Chat GPT. A selection of student comments elaborating on this can be seen in
Table 8.
<Table 8>
Selection of comments from 84 participants relating dissatisfaction with GPT’s content
Area of distrust |
Selection of comments |
Authenticity |
“I can’t be sure if GPT’s output is valid, so I can’t trust its reliability. Also, since it doesn’t use my own expressions, it might be inconvenient”. |
“I heard that using ChatGPT in e-learning systems can cause originality issues. I don’t use ChatGPT to avoid that problem”. “It was better to do the work myself in terms of quality or originality”. |
|
Accuracy |
“I decided not to use it because there might be inaccurate information mixed in.” |
“The accuracy of GPT is low”. |
“There can be inaccurate answers. When I was a freshman, ChatGPT didn’t exist, and in the classes, I’ve taken this year, there weren’t any assignments that required ChatGPT. Also, using it reduces my own thinking time”. |
While not as apparent in the data as the reasons discussed above, there was also one other theme evident in the responses to question 13. It relates to concerns over plagiarism and a fear of punishment by their professors for using ChatGPT for homework assignments. Several students commented they were worried about getting into trouble for submitting content they could not verify and could be viewed as plagiarized. One student stated: “I heard that using ChatGPT in e-learning systems can cause originality issues. I don’t use ChatGPT to avoid that problem”. Another said: “I was worried about plagiarism issues so decided to do all my work by myself and it felt good”. When discussing possible reprisals for using ChatGPT, one comment read: “I heard most classes prohibit GPT”. Another said: “My friends told me not to use ChatGPT because they heard about one student who got punished for using it”. Both of these comments imply that they had not been directly told not to use ChatGPT, rather heard it second-hand which affected their decision.
4.2. The Future Role of ChatGPT
Research question 2 asked students how they feel about the role of ChatGPT for completing English homework assignments and its pedagogical implications. Survey question 14 asked the participants if they should be allowed to use ChatGPT for their homeworks, assignments and quizzes. A combined total of 109 (42.7%) of the participants answered favorably by strongly agreeing and agreeing to this question, whereas only 57 (22. 4%) participants answered strongly disagree and disagree. 89 (34.9%) of the participants answered neutrally. Inspecting the data more closely and contrary to previous questions, it indicates a very weak positive correlation between the English level of students and their opinion on allowing ChatGPT usage in English-taught classes. The data for this question can be seen in
Figure 2.
[Figure 2]
Should ChatGPT be used for homework / assignments / quizzes
Following this, question 15 asked for more in-depth responses. Participants who were in favor of allowing ChatGPT to aid them with out of class assignments noted comments such as: “By using ChatGPT’s assistance, one could achieve better grades”.; “Using ChatGPT can improve one’s insufficient English skills, so if used correctly, it can be helpful”. and “ChatGPT provides significant assistance and makes things easier to understand, so it is very helpful”. To receive a more detailed response, several participants were contacted via email. One participant stated:
In this rapidly changing digital age, it seems inevitable to use these tools, especially when needed. Because it helps quickly with tasks and if you don’t use it, the gap in skill levels becomes too large. (Participant E, female, age 22, intermediate level).
Another participant responded:
It simplifies understanding. Since language is based on speaking and writing, it helps with grammar and things that may not show up on Google. It aids in accumulating knowledge and therefore supports learning. (Participant F, female, age 20, beginner level).
Conversely, participants who disagree with the use of ChatGPT for homeworks and out of class assignments made comments such as; “It may help with their vocabulary and grammar, but if they just copy and paste, it could be detrimental”.; “It can make one’s brain dull, so I’m not sure about it”.; “It could cause losing the ability to think for oneself, making the mind stagnant”. and “I think using ChatGPT has no positive effect on learning”. Participants who answered neutrally, understood the help ChatGPT gives but they shouldn’t be too reliant on its use noted the following comments; “I think there’s a middle ground with pros and cons to the choice”.; “It can be ambiguous because depending on the student, they could use GPT well or abuse it”. and “I think it’s good to get some help, but I don’t think I should be too dependent on it”.
Survey question 16 asked the participants if their professors in any of the classes taught in English had told them not to use ChatGPT for homework assignments. 199 (78%) answered “no” and 56 (22%) answered “yes” with responses to the follow up question (Question 17) being somewhat mixed
A number of students who answered negatively seemed to favor this approach with several appearing to appreciate the freedom to explore the use of new technology for English homework assignments. Comments included: “I thought allowing students to use new tools to do homework is a good thing”; “I think ChatGPT is a useful learning tool, so it should be allowed”.; “Since it’s the trend of the times, banning it seems strange”.; “The ability to use ChatGPT well is a skill itself. If used properly, it’s an excellent tool”.) Others appeared to be grateful to the professor for not restricting its use “I’m glad we can use ChatGPT”.; “I was relieved that I could use it”.; “I liked that the professor let us make our own judgments”. However, there were also those who felt indifferent noting that allowing its use did not alter their approach to homework assignments. “I had no strong opinions about AI policies”.; “I didn’t think much about it”.; “I had no particular thoughts”; “Not really bothered by it. I can do it on my own”.
Data from those who responded with ‘yes’ to question 16 also revealed very mixed feelings. There were those who agreed with the restriction and understood the rationale behind it, emphasizing the importance of using English independently of ChatGPT for homework assignments to help them learn by themselves. “Since the purpose is to improve one’s English skills, I agree with the restriction on ChatGPT use”.; “To improve one’s English learning, limiting ChatGPT use is necessary”.; “It fosters critical thinking skills, so I viewed it positively”.; “I think not using it would be more helpful for learning, as it provides answers too easily”. Others expressed feelings of indifference indicating that it did not significantly impact how they chose to complete homework assignments. Interestingly, there were also students who appeared to criticize the restriction of using ChatGPT highlighting how useful it can be for those who struggle with English or are not confident in their ability. “For those who are not good at English, it can be difficult and burdensome without GPT”.; “I thought it might be necessary when using a foreign language”.
In addition, regardless of whether students were told they were allowed to use ChatGPT or not, one common response to question 17 implied that they were willing to follow the guidelines set by their professors. If ChatGPT usage was permitted they would make use of it, likewise, would not use it if told not to. (“If the professor says not to use it, I won’t”; “I agree with the professor”.; “The professor didn’t mention it, so I thought it was okay to use”; “I follow the professor’s rules”.
Overall, the analysis reveals that the majority of students who were not restricted from using ChatGPT felt positively or were indifferent about it. This suggests a general acceptance and appreciation for the freedom to use AI tools in their academic work. However, the lack of clarity or detail to some responses highlights the need for more detailed feedback to understand students’ perspectives better.
Since this technology is relatively new and both students and teachers need time to adapt to its influence, one area that could be important to consider is in what ways teachers allow students to use it for homework assignments. In previous years, due to perceptions that it could encourage plagiarism and cheating, the default approach would have been telling students not to use it. Survey question 18 asked the participants if they felt teachers of courses taught in English need to be more flexible in their position on using ChatGPT for homework assignments. The data showed a strong preference for more leniency with almost 60% saying they agreed or strongly agreed. One very interesting point can be seen in
Figure 3 with only 9.8% selecting either disagree or strongly disagree.
[Figure 3]
Should professor be more flexible in their approach to using ChatGPT be used for homework / assignments / quizzes
The reasons behind these responses expanded on in question 19, which asked the participants to explain why they felt this way. Several key themes emerged from this data. Firstly, the idea that we are now living in a new era in which there is no option but to adopt new technologies such as ChatGPT was one of the most frequent comments. This implies the students feel teachers need to adapt their approach to homework assignments accordingly. Secondly, the benefits in terms of giving students access to more information and content which they can apply to their assignment and consequently learn more was also mentioned. Finally, there were strong indications that students would like to see ChatGPT appropriately integrated into lessons and homework so it can be used as a positive aid to learning rather than being viewed as a threat. A summary of these themes and a selection of responses to question 19 can be seen in
Table 9.
<Table 9>
Selection of comments from 173 participants on why teachers should be more flexible in their approach to ChatGPT
Changing Landscape of Education
|
“A university cannot be considered a university if it does not utilize the latest AI technologies”. |
“Because we are in the Fourth Industrial Revolution era”. |
“The world has changed. The education system should adapt as well”. |
“It’s the AI era; we should use it”. |
|
Beneficial Resource
|
“It can help students understand assignments and allow many of them to concentrate better”. |
“Because GPT’s explanations are very helpful”. |
“It allows students to see and learn from a wider variety of perspectives”. |
“Using AI is not bad; fully relying on AI for assignments is. |
If used correctly, AI can lead to better results”. |
|
Aid rather than threat
|
“If it does not directly impact grades, it would be helpful for professors to teach students how to use it effectively”. |
“It is better to encourage appropriate use so that students can integrate their own thoughts and opinions”. |
“ChatGPT is a great supplementary tool for learning and homework”. |
“More effective to guide GPT use rather than restrict it”. |
Finally, the last question (20) asked the participants if they feel the university administration should provide clearer guidelines for students on if and how they can use ChatGPT. Almost two thirds (65.1%) of the participants were in favor of the university administration department doing so, with 101 (39.6%) and 65 (25.5%) participants selecting agree and strongly agree respectively. Only 6 (2.4%) participants selected strongly disagree, 16 (6.3%) chose to disagree, while 67 participants (26.3%) remained neutral. This illustrates the participants’ belief that the university administration should establish more explicit and comprehensive guidelines regarding the permissible use of ChatGPT for homework assignments. These guidelines should clarify whether students are allowed to utilize the tool and, if so, outline the specific contexts, limitations, and ethical considerations associated with its use in academic work.