핵심역량기반 개편 후 대학교양영어 성과진단 연구 -I 대학교 사례를 중심으로
A Study on Performance Diagnosis of College English Courses in General Education after Core Competency-Based Reform -The Case of I University
Article information
Abstract
I 대학은 2021년 대학기본역량진단을 앞두고 핵심역량을 중심으로 교양교육과정을 개편한 것을 토대로 학습성과진단을 실시하였고, 본 연구는 그 중 기초교양교육에 해당하는 대학영어 교과목에 대한 진단결과를 바탕으로 진행하였다. 기초교양교육의 외국어 영역은 I대학의 5가지 핵심 역량 중에 ‘커뮤니케이션’ 역량에 중점을 두었다. 본 연구는 이러한 대학영어 교과목들이 대학의 핵심 역량에 적합한지 알아보기 위해 대학영어 수업을 수강하는 학생들의 반응을 살펴보았다. 각 대학영어 수업이 ‘커뮤니케이션’에 중점을 둔 핵심역량에 적합한지, 특히 영어 커뮤니케이션 능력을 위한 기본지식의 학습과 그 지식을 바탕으로 하는 경험이 적합한지, 학생들의 영어 커뮤니케이션 능력 향상을 위해 교과 과정에 어떤 변화가 필요한지 살펴보았다.
설문조사를 통해 대학영어I(독해), 대학영어II(듣기), 대학영어II(말하기) 수업에 참여한 학생들의 교육적 향상을 비교 분석하였고, 이러한 학습성과진단을 토대로 대학영어 교과목의 교과과정 개선을 위한 기초 자료로 활용하고자 하였다. 학습성과진단을 위해 한국교양기초교육원(KNIGE)에서 2016년에 제시한 성과진단 도구를 선택 활용하였다. 5개 영역의 93문항 중 ‘기본학습능력’과 ‘지식과 경험’ 영역에서 각각의 교과목에 해당하는 문항을 선택 조사하였고, 각 문항은 리커트 5점 척도를 사용하여 측정되었다. 이번 성과진단 조사에는 세 개의 대학영어 교과목을 수강한 전체 학생 2063명 중 1,366명(66.2%)이 참여하였다.
학습성과진단 분석결과, 일반적으로 학생들은 각 교과목에서 영어 의사소통 능력이 향상되었다고 반응하였는데, 이는 I대학의 기초교양 외국어영역에서 추구하는 핵심 역량과 해당 교과목에서 추구하는 교육의 방향이 적합하다는 것으로 간주될 수 있는 결과였다. 다만, 기초학습영역과는 사뭇 다르게, 지식과 경험 영역 중 수업활동과 관련된 문항들은 다른 문항들보다 낮게 나타났다. 학기 동안에 코로나 상황에 따라 대면과 비대면 수업을 병행하였는데, 이러한 상황이 학생들로 하여금 교수 및 동료들과 상호작용 하는데 있어서 불편함이 있었다는 것으로 사료된다. 대면 수업에서뿐 만 아니라, ‘위드 코로나’ 시대를 대비하기 위해서라도 효과적인 비대면 혹은 온라인 수업에서의 상호작용을 위한 활동 개발과 관련된 논의가 시급해 보였다.
Trans Abstract
This study conducted a basic learning performance diagnosis for each College English course falling under the purview of basic general education. In line with the five core competencies of I University, the area of foreign language in basic general education focused on communication competency. This study explored the reactions of students taking College English courses to examine whether each course matched the core competency focused on communication, especially the basic learning competence for English communication skills and experiences acquired through the knowledge provided and what changes were needed in the curriculum to improve students’ English communication competence.
Through a survey, educational improvements recognized by students who participated in College English I (reading), College English II (listening), and College English III (speaking) classes were compared and analyzed. The performance diagnosis for the study aimed to measure learning performance obtained through each course and utilize it as a basic mean for improving curricula for College English courses. For performance diagnosis, the Learning Performance Diagnosis Tool developed by the Korea National Institute for General Education’s General Education Standards in 2016 was selected. Diagnosis questions from the areas of “basic learning competency” and “knowledge and experience” were selected, and each question was measured using a 5-point Likert scale. The performance diagnosis survey involved 1,366 of 2,063 students (66.2%) in all three courses.
Based on the analysis result, students reported that their English communication competency had improved in each course. This result could be interpreted as indicating that the core competency pursed in the area of foreign language at I University and the direction of education pursued in College English courses were appropriate. However, slightly different from basic learning competence area, the performance indicators relating to class activities questions in knowledge and experience area were found to be lower than that of other questions. During the semester, online and offline classes had been conducted depending on the COVID-19 situation, and it can be assumed that this situation resulted in many inconveniences in terms of utilizing effective class activities where students could interact with the professor and classmates. It also seemed urgent to develop effective non-face-to-face or online class activities to prepare for the COVID-19 pandemic era.
1. Introduction
According to the Korea National Institute for General Education’s (KNIGE) General Education Standards, basic general education includes the development of basic knowledge and autonomous academic competence required throughout university education. It is defined as universal education required by all students beyond the expertise of academic distribution. In the new era of a global information society, it is defined as education that cultivates the qualities of autonomously leading the cultural life of the community through critical and creative thinking and open communication. In this context, diverse literacy, information acceptance, knowledge generation, communication and empathy, cooperation, rational thinking, and emotion are the abilities and qualities needed to cultivate general education in college.
Following the introduction of the University Basic Competency Diagnosis in 2021, a nationwide evaluation of universities, many universities in Korea launched various efforts to reorganize their general education curriculum based on core competencies,1) and improve the quality of the general education curriculum operation (Jung et al., 2020; Kim, 2020; Lee et al., 2020; Son et al., 2021). In 2018, I University, a local private university, also reorganized its curriculum completion system of general education based on basic studies to strengthen competency-based general education through consultations with KNIGE.
As the first step, the Liberal Arts College of I University (i-LAC) dedicated to general education was established. i-LAC had already made various attempts to improve the quality of its curriculum in line with the 5Cs (Creativity, Convergence, Communication, Collaboration, and Challenge), which were the core competencies defined by I University. Under the curriculum of basic general education in i-LAC, the foreign language subjects defined communication as their key competency.
A variety of foreign language courses are offered in i-LAC’s basic general education, including Chinese I & II, Chinese Character I & II, College English I, II, III, & IV, Japanese I & II, Basic German, Basic Italian, and Basic Spanish. Completion of each foreign language course provides two credits. To complete the requisite curriculum, first-year students can select four credits of courses, and second to fourth-year students two credits of courses to complete a total of six credits in the area of foreign language communication.
As aforementioned, College English courses in I University belong to the foreign language communication area in basic general education, and their main purpose is to strengthen students’ English communication competency. College English I, II, and III are beginning-level courses of grammar and reading comprehension, listening comprehension, and speaking, respectively, while College English IV2) is focused on intermediate-level English conversation.
This study assumed that these beginning-level courses of College English I, II, and III needed to be evaluated to determine whether they could improve basic English skills and knowledge and experience related to the subjects based on the responses of students taking these classes.3) The results might help to determine to what extent students have reached the goals set forth in the curriculum. In addition, when educational knowledge and experience are measured on a class-by-class or on a course-by-course basis, it was assumed that specific suggestions could be given to professors in order to improve the quality of the curriculum. Accordingly, a performance diagnosis was conducted for each course via online and offline surveys. The educational improvements perceived by the students themselves in the subjects were compared and analyzed. The purpose was to measure the learning outcomes obtained through this diagnosis process and use the results as basic data for improving the curriculum of College English courses in basic general education at I University.
To conduct the learning performance diagnosis mentioned in this study, a general education performance diagnosis tool developed by Hong et al. (2016) of KNIGE was utilized.4) This tool consists of five areas: 1) basic learning competence areas that evaluate basic preparation for taking liberal arts classes; 2) knowledge and experience areas that evaluate knowledge, understanding the purpose of subjects, and participation in class activities acquired after taking liberal arts classes; 3) thinking areas that evaluate the ability to think as a result of the liberal arts classes; 4) value areas that evaluate the formation of values after taking liberal arts classes; and 5) integrated areas that evaluate the degree of improvement in the ability to solve complex problems by fusing knowledge, skills, attitudes, and so on, and apply them to new environments.
Among the five areas, this study focused on the responses of students in the areas of 1) basic learning competence and 2) knowledge and experience, which are appropriate for communication, the core competency of i-LAC’s foreign language area. The research questions for the study are as follows:
What were the students’ reactions to basic learning competence in each College English course?
What were the students’ reactions to knowledge and experience in each College English course?
Were there any other distinctive implications in the students’ responses?
2. Literature Review
Learning performance refers to the achievements gained through learning in the curriculum, and is a process that checks how far learners have progressed to achieve the intended goal of the curriculum. It can be easily measured by curriculum. The educational experience must also be measured on a class basis to provide specific implications to professors in order to improve the quality of the curriculum. Bae et al. (2011) categorized the achievements of participation in university general education into 11 areas (information technology literacy, world citizenship, creative problem solving skills, mathematical thinking skills, understanding and values of society, personality development, critical and analytical thinking skills, understanding and values of oneself, basic knowledge of major subjects, communication skills, and foreign language proficiency) and measured them on the Likert 5-point scale. Song et al. (2015) also examined the core competencies of learning participation and college students as learning process and performance factors to analyze the quality and performance of basic general education. Learning engagement involved 20 questions on the Likert 5-point scale of cognitive, emotional, and behavioral engagement. Core competencies included eight questions (cognitive, intrapersonal, and interpersonal competence), which were also measured on the Likert 5-point scale.
In addition, Hong et al. (2016), limited the scope of the study to the area of liberal arts education to measure the learning performance recognized by students who took liberal arts courses. They organized the achievements of liberal arts education into five areas such as basis learning competency, knowledge and experience, thinking, value, and integrated areas. To summarize the aforementioned performance study of students in general educations conducted at KNIGE, the results of general education for students are largely measured by two methods. The first is a variety of competency-oriented components, and the second is the achievement of general education classes themselves outside of student competence, such as satisfaction with class participation. According to Hong et al. (2016), the tools for measuring general education and higher education learning performance can be broadly classified into two types. First, student achievement or competence is evaluated directly by the professor, such as through GpA. Second, students’ perceptions or satisfaction are measured through self-report surveys. In this case, both quantitative and qualitative assessments are being utilized. The quantitative evaluation uses the Likert 5-point scale, while for qualitative evaluations, students’ responses are scored through questions or tasks presented. In qualitative evaluation, it is time consuming and expensive to secure and grade the exam for evaluation. Therefore, quantitative assessments, which take relatively less time and effort by respondents, are mainly being utilized. This diagnostic tool was compiled using the Delphi method. The tool was derived by examining domestic and international college student-related educational and diagnostic tools (CAAP, MAAP, CEQ of Australia, AHELO of OECK, K-CESA of the Korean Ministry of Education and the Korea Research Institute for Vocational Education and Training (KRIVET)), and rubrics created by the Association of American Colleges and Universities (AAC&U) to evaluate learning outcomes through general education. In addition, the tool is made available to professors by selecting some questions that match the purpose and characteristics of classes by referring to the area-specific utilization method (Hong et al., 2016).
Considering the educational and diagnostic tools in the United States in detail, the Collegiate Assessment of Academic Proficiency (CAAP) is organized by ACT, and is an evaluation tool that has been used in various university sites. It was developed for the purpose of improving the quality of the curriculum through the evaluation of the learning performance of the college’s general education curriculum. The evaluation consist of six areas: reading, writing, essay, mathematics, science, and critical thinking. The Measure of Academic Proficiency and Progress (MAAP) was developed by the ETS to measure the performance of general education at universities in four areas: reading, critical thinking, writing, and mathematics. Additionally, the AAC&U was established by the Association of American Colleges to evaluate educational achievements through general education. The evaluation consists of three core (intellectual and practical skills, responsibility, integrated and applied learning) and 16 detailed areas (exploration and analysis, critical thinking, creative thinking, writing skills, presentation skills, reading skills, mathematical reasoning skills, teamwork, problem solving, multiculturalism, ethical reasoning, global reading, and integrated learning). Since its distribution, the rubrics have been used as a guideline for measuring intrinsic learning performance, with access reaching 70,000 from 5,895 institutions by 2015 (AAC & U, 2020).
The Course Experience Questionnaire (CEQ) is a diagnostic assessment of the quality of university classes conducted by Australian college graduates to collect data on their education and learning experiences in universities. The evaluation consists of three essential (professor learning quality, general core competence, and overall satisfaction) and eight optional areas (learning goals, learning volume, evaluation, intellectual motivation, student support, graduates’ level, learning resources, and learning community). CEQ is a self-reported survey conducted for the purpose of the Graduate Destination Survey. It is currently being conducted for college graduates from all universities in Australia, and it asks about the education and learning experience related to the quality of teaching. Based on the results of the CEQ, many Australian universities have come up with support measures to provide students with wider experience and quality curriculum. Despite CEQ’s advantages of collecting large volumes of information using minimal cost and time, there is a limit to performance assessment across the curriculum relying on individual subjective perceptions (Hong et al., 2016; Mclnnis et al., 2001).
An evaluation tool organized by the Organization for Economic Cooperation and Development (OECD) is the Assessment of Higher Education Learning Outcomes (AHELO), which evaluates education learning outcomes acquired through university education. The areas of evaluation consist of general core (critical thinking, analytical reason, and problem-solving skills), major, and contextual skills (OECD, 2011, 2015).
Lastly, the Korea Collegiate Essential Skill Assessment (K-CESA) is an evaluation system organized by the Korean Ministry of Education and the Korea Research Institute for Vocational Education and Training (KRIVET) to evaluate the core competencies of college students. The evaluation consists of six areas: communication, comprehensive thinking, resource, information and technology utilization, global, self-management, and interpersonal skills (KRIVET, 2020; Son et al., 2019).
Comprehensively, Hong et al. (2016), closely related to this study, mentioned that effective general education performance diagnosis tools should evaluate whether learning outcomes suitable for the purpose of general education curriculum are carried out in the curriculum of each university. In addition, they emphasized the need to analyze whether teaching and learning conducted in individual classes help learners to strengthen their competencies. As shown in [Figure 1], the diagnostic tools developed by them consist of five areas: 1) Basic learning competence areas that evaluate basic preparation for taking liberal arts classes; 2) Knowledge and experience areas that evaluate knowledge, understanding the purpose of subjects, and participation in class activities acquired after taking liberal arts classes; 3) Thinking areas that evaluate the ability to think as a result of the liberal arts classes; 4) Value areas that evaluate the formation of values after taking liberal arts classes; 5) Integrated areas that evaluate the degree of improvement in the ability to solve complex problems by fusing knowledge, skills, attitudes, etc., and to apply them to new environments, with 39 performance indicators, 40 goal statements, and 93 measurement questions.
In this respect, the general education performance diagnostic tools developed by Hong et al. (2016) were considered appropriate for the evaluation of the academic performance of the i-LAC curriculum. Accordingly, the academic performance evaluation for all courses in basic general education of i-LAC was carried out, and only the data for College English courses in the foreign language area were extracted and utilized in this study.
3. Methods
In this study, a general education performance diagnosis tool developed by Hong et al. (2016) was used to measure performance diagnosis of beginning-level College English courses in basic general education at I University. Subsequently, diagnostic questions relevant to each College English course were selected and utilized in the survey.
The survey was conducted online using Naver Office Form or offline in class. The online survey for performance diagnosis was conducted for 25 days at the end of the semester (from December 2-December 27, 2020) for all students taking courses in College English I, II, and III.
3.1 performance diagnostic participants
The total number of participants enrolled in College English I, II, and III was 2,063, while the number of participants in the diagnosis was 1,366 (66.2%). The detailed classification of the number of participants in the survey by subject is by subject is, as indicated in <Table 1>.
3.2 performance diagnostic questionnaires
Among the 93 diagnostic questions in the five areas developed by Hong et al. (2016), this study focused on the responses of students in the areas of 1) basic learning competence and 2) knowledge and experience, which are appropriate for communication, the core competency of i-LAC’s foreign language area. The reason for selecting these performance diagnosis questions was to examine if beginning-level College English courses could improve basic learning skills needed for English communication and improve knowledge and experience as a result, based on the responses of students taking these classes. It was deemed important to reduce the number of survey questions to ensure that students answered the survey faithfully without difficulty and ambiguity. Thus, only 17, 14, and 23 questions from the areas of “basic learning competency” and “knowledge and experience” for College English I, College English II, and College English III, respectively, were selected.
3.2.1 College English I
College English I aims to teach students the basic grammar needed to construct English sentences and foster students’ reading comprehension skills. For performance diagnosis questions for College English I, 17 questions were selected and measured from the two areas and five sub-areas, as shown in <Table 2>.5)
3.2.2 College English II
College English II aims to improve English listening comprehension by having students listen to spoken English and understand the main ideas and specific information. For College English II, a total of 14 questions were selected and measured from the two areas and five sub-areas as the performance diagnosis questions, as shown in <Table 3>.6)
3.2.3 College English III
College English III aims to improve basic communications skills such as listening and speaking through conversation practice. The course strengthens knowledge through repeated key phrases and expressions that are useful to review and practice previously acquired English language skills. In the case of College English III, a total of 23 questions were selected and measured from the two areas and seven sub-areas, as shown in <Table 4>.7)
3.3 Data analysis
An analysis of questions was conducted based on the results of students’ responses to the questions measuring performance. Subsequently, the questionnaires were collected for each subject, and the mean, standard deviation, F-value, and p-value were investigated using SPSS 26, ANOVA to determine the variance of means and the statistic differences. The results were measured and analyzed on a 5-point Likert scale: strongly disagree (1 point), disagree (2 points), neither agree nor disagree (3 points), agree (4 points), and strongly agree (5 points).
4. Results and Discussion
4.1 Results of the performance diagnostic analysis from College English I
Generally, a t-test is used to determine whether there is a significant difference between the means of two groups, and the t-value measures the magnitude of the difference relative to the variation in the sample data. However, most of the participants in the study were first and second-year students (90.2%), and the ratio of men and women was almost 50/50. Thus, it was meaningless to find out the significance probability through t-tests evaluating these groups. Instead, ANOVA was conducted to determine F and p-values, that is, the variance of means and the statistically significant differences in each professor from each College English course.
4.1.1 General status of respondents
As College English I was one of basic general education courses, 706 students (92.5%) who took the course were first and second-year students. As shown in <Table 5>, the number of respondents was 556, with a high response rate of 72.9% for students taking College English I.
4.1.2 Analysis results of performance areas from College English I
① performance of basic learning competency for College English I
As indicated in <Table 6>, the performance of basic learning competency area shows a mean score of 4.12 with a standard deviation of 0.71 for College English I.
The highest score among the sub-areas was data utilization (4.24). By the measurement of the questions, #9 (I can find the necessary data or information using libraries or the Internet) in data utilization and #18 (I am prepared to do my best with the assignments and learning required by this class) in learning preparedness had the highest points, with a mean of 4.24. Meanwhile, the mean score of 3.91 for #17 (I registered for the class because I was interested in the subject) in learning preparedness was relatively low.
Analysis of basic learning competency area by professor showed statistically significant differences in acceptance, data utilization, and learning readiness areas, as shown in <Table 7>. In particular, Professor E’s students’ performance was relatively low.
In the case of Professor A, #9 (I can find the necessary data or information using libraries or the Internet) in data utilization was the highest, with a mean of 4.28 points, and #17 (I registered for the class because I was interested in the subject) in learning preparedness was relatively low, with a mean of 3.79 points. For Professors B and C, a noticeable point was that both the highest and the lowest mean scores were in learning preparedness: #18 (I am prepared to do my best with the assignments and learning required by this class) in learning preparedness was the highest, with scores of 4.37 and 4.24, respectively, and #17 (I registered for the class because I was interested in the subject) was the lowest, with scores of 3.85 and 3.91, respectively. Professor D had the highest scores in all areas, with the mean scores ranging from 4.20 to 4.47 points. Professor E’s case was also noticeable because of the lowest scores in all areas were quite contrary to those of Professor D. In particular, both #3 and #4 (ability to read and understand the core content and the writer’s intentions) in acceptance averaged 3.59 which were only slightly above the normal level, and class attitude (#18) in learning preparedness was also low at 3.66 points.
According to the students’ responses, #17 (interests in the subject) was the lowest (p<.001), and #18 (class attitude) was the highest (p<.001) in most classes of College English I. It could be assumed that the subject of grammar and reading comprehension was not very interesting to students but that they recognized that it was important to acquire linguistic ability and thus to do their best in the class.
② Knowledge and experience for College English I
The overall mean for knowledge and experience performance for College English I was 3.76, with a standard deviation of 0.80, as indicated in <Table 8>.
By the measurement of survey questions, #23 (I can tell the contents I learned from this class), with a mean of 4.16 points in knowledge acquisition, was the highest, followed by #19 (While reading newspapers, books, and Internet materials, I can recognize if they were covered in the class), with 4.13 points. Conversely, in the sub-area of class activities, both #32 (I asked the professor questions in various ways including online to better understand the contents of the class.) and #33 (I exchanged help with other students for learning and understanding the content, presentation, exam preparation, discussion, assignment, etc.) were slightly above the middle, with mean scores of 3.27 and 3.11, respectively. Due to the COVID-19 circumstances, College English I courses were conducted both online and offline. Given the low mean scores of interactions for class activities, it could be assumed that these circumstances made it difficult for students to interact with professors and their classmates.
Analysis of knowledge and experience among professors in <Table 9> showed statistically significant differences in the areas of knowledge acquisition and class activity. Especially for students from the classes of Professor E, “knowledge and experience” improvements were significantly lower.
In knowledge and experience for College English I, there were very noticeable responses from the students. In every professor’s class, the two questions related to class activities, which were #32 (interactions with the class professor) and #33 (interactions with classmates), had significantly and relatively negative assessments. As mentioned earlier in relation to <Table 8>, College English I courses had to implement both online and offline classes due to COVID-19. Given the much lower performance indicators for interactions (#32 and #33) in class activities, utilizing the online method of lecture in class seemed to have a negative impact, especially on interactions for class activities. Professor E’s case was also noticeable here in the area of knowledge and experience. In particular, four questions related to participation and utilization experience of class activities were found to be significantly below normal, ranging from 1.69 to 1.94 points. Even considering the COVID-19 situation, it was clear that there were significant differences between professors using online and offline class activities for interactions.
In the case of College English I, basic learning skills seemed to be well delivered in line with communication, the core competence required in the foreign language area. However, it seemed urgent to discuss the pros and cons of the methods of lecture and activity that each professor used for both online and offline classes to improve the curriculum of College English I.
4.2 Results of the performance diagnostic analysis from College English II
4.2.1 General status of respondents for College English II
In College English II, 484 students (84.6%) who took the course were first and second-year students. The number of respondents was 447, and the response rate of students taking College English II was quite high at 78.1%, as shown in <Table 10>.
4.2.2 Analysis results of performance areas from College English II
① performance of basic learning competency for College English II
As indicated in <Table 11> for College English II, the performance of basic learning competency area shows a mean score of 4.00, with a standard deviation of 0.74.
From the performance indicators of survey questions, #18 (I am prepared to do my best with the assignments and learning required by this class) in learning preparedness was the highest, with 4.20 points. Meanwhile, #17 (I registered for the class because I was interested in the subject) in the same sub-area (learning preparedness) was relatively low, with 3.80 points.
A comparison of basic learning competency area by professor in <Table 12> also showed statistically significant differences in data utilization and learning preparedness. All performance indicators were above average overall. However, it could be seen that Professor A’s students’ indicators, in particular, were relatively low in the College English II classes.
In the case of “basic learning competence” for College English II, #9 (I can find necessary data or information using books or the Internet) and #18 ((I am prepared to work hard on the tasks and learning required by this class) mean scores were high overall, but #1 (I can summarize key contents by listening to others in public and private situations) and #2 (I can understand the hidden meaning by listening to others), which were directly related to listening comprehension, were relatively very low, along with #17 (I registered for the class because I was interested in the subject of this class) in all classes of College English II. According to the indicators based on students’ responses, students’ interest in basic learning competence for acquiring listening skills through College English II (listening comprehension) seemed low. These figures could indicate that there need to be discussions and changes in the College English II (listening comprehension) curriculum at I University.
② Knowledge and experience for College English II
For “knowledge and experience” performance of College English II, the overall mean score was 3.78, with a standard deviation of 0.77, as shown in <Table 13>.
By the measurement of survey questions, the mean score for #19 (While reading newspapers, books, and Internet materials, I can see if they were covered in the class) was the highest, with 4.10 points followed by #23 (I can tell the contents I learned from this class), with a mean score of 4.07. However, #32 (the experience of asking the professor about various methods such as online) and #38 (I applied ideas, experiences, knowledge, and information gained from this class in my daily life) were relatively low, with 3.34 and 3.57 points, respectively.
The <Table 14> indicates the comparison of “knowledge and experience” among professors, which also indicated statistically significant differences.
For knowledge and experience in College English II, compared to other professors’ data, in the case of Professors A, performance in class activities was significantly below the average score, with means ranging from 1.69 to 2.03. There seemed to be no experience exchanging help with others for learning (#34) or using and applying ideas, experiences, knowledge, and information gained from this class (#36, #37 and #38). Based on the responses from students who participated in College English II, it was also clear that there were significant differences between professors using online and offline class activities. It seemed necessary to exchange and understand the pros and cons of class activities that each professor used through discussion and to improve the curriculum of College English II through changes.
4.3 Results of the performance diagnostic analysis from College English III
4.3.1 General status of respondents for College English III
Similar to the other courses, 728 students (92.0%) who took the course were first and second-year students. However, the number of respondents for College English III was only 363, with a relatively low response rate of 49.9% compared to the other classes of College English I and II, as indicated in <Table 15>. A possible reason the survey participation rate was significantly lower than in other College English subjects is that the explanation of and encouragement to participate in the survey were not accurately delivered because the native professors spoke only in English in College English III classes.
4.3.2 Analysis results of performance areas from College English III
① performance of basic learning competency for College English III
In the performance of basic learning competency for College English III, the overall mean score was 4.18, with a standard deviation of 0.67, as shown in <Table 16>.
By indicators of survey questions, #15 (I understand and respect each team member has a different mindset) in team-work was the highest, with 4.45 points. Meanwhile, #7 (I can write sentences according to spelling and grammar) in expression was relatively low, with only 3.91 points. Judging from these figures, it could be assumed that students were exposed to little writing, although there were beginner-level writing activities in the textbook itself. It could be because College English III is a course that focuses on practicing English conversation as a group. Other than that, there were many positive comments above 4.0 points for all other performance indicators; that is, other basic learning competences other than writing skills seemed to have been appropriate for students in College English III.
As in <Table 17>, the analysis of basic learning competency areas by professor showed statistically significant differences in acceptance, expression, data utilization, and teamwork. There were 33 students who did not indicate their class section and the data from these students were excluded from the analysis.
According to performance indicators, most of them looked similar, and they were above the normal level in basic learning competence area for College English III. However, Professor H’s students were relatively low in all language skills (from #1 to #8), using materials (#10), and interest in the course (#17), except for teamwork (#13, #15) and learning preparedness (#18). Other than that, in most professors’ classes, all language skills remained similarly higher than average without significant changes. Above all, Professor F’s basic learning competence areas were high, ranging from 4.19 to 4.67 points. In particular, teamwork and learning preparedness required by the class were quite high. Based on the performance indicators for College English III, it could also be assumed that even if the same textbook was used, the activities used by each professor were different. It also means that it is necessary to exchange the strengths and weaknesses of the activities and teaching methods through discussion.
② Knowledge and experience for College English III
The mean for knowledge and experience performance for College English III was 4.16, with a standard deviation of 0.73, as indicated in <Table 18>.
Overall, there were many positive results for all performance indicators for knowledge and experience areas in College English III. By measurement of survey questions, #23 (I can tell the contents I learned from the class) was the highest, with 4.37 points. However, #32 (I asked the professor questions in various ways including online to better understand the contents of the class) was relatively lower than other questions, with 3.93 points. This semester, both offline and online classes were combined, depending on the COVID-19 circumstances. If students had questions, they would have been able to interact enough in the classroom or via Microsoft Teams chat and emails. Nevertheless, improving the question- and-answer environment between students and professors always seemed to be a challenge.
Examining the data and p-values in <Table 19>, a comparison of knowledge and experience areas by professor indicated statistically significant differences in knowledge acquisition and class activities.
Most professors in College English III classes had knowledge and experience performance indicators far exceeding 4.00 points (higher than average level) overall compared to the other courses of College English I and II. Specifically, the indicators for knowledge acquisition (#21, #22, and #23) and class activities with classmates (#33) were quite high. However, in the case of Professor H, the performance indicators from knowledge and experience areas were just above the average level, and the indicators in class activities were relatively low. In particular, the interactions with the professor (#32) and with classmates (#33) were the lowest. Specifically, in the case of College English III, the number of students was less than 20 because the course purpose was to improve and practice beginner-level English conversation skills. Thus, depending on the COVID-19 situation, offline (face-to-face) classes or real-time online classes were recommended to professors. Nevertheless, it was curious that the performance indicators from Professor H’s classes showed that there was little interaction between the professor and the students, until it was discovered that professor H continued to conduct online video lessons due to personal reasons. In other words, compared to face-to-face classes, it could be assumed that non-face- to-face classes undermined the interaction between professors and students. To prepare for the post-COVID-19 era, it also seems necessary to discuss effective online activities that can guide interaction with others in non-face- to-face classes for English language learning.
Overall, basic learning competence was found to have higher performance evaluations than knowledge and experience area in all College English courses. Students responded that their English communication competences had improved in each subject. This could indicate that communication which was the core competency pursued in the foreign language area of i-LAC as well as the direction of education pursued in College English courses, was appropriate. However, unlike the results from basic learning competence, some performance evaluations in knowledge and experience area were significantly low. Specifically, questions #32 (interactions with the professor) and #33 (interactions with classmates) related to class activities were found to be significantly lower than other questions. The COVID-19 situation, which resulted in a mix of online and offline classes, might have caused many inconveniences in terms of utilizing class activities where students could interact with professors and classmates. Furthermore, it was noticeable that there were statistically significant differences between professors even in the same subject. Thus, it seems necessary to discuss the strengths and weaknesses of class activities conducted in each class and to develop effective class activities both for offline and online classes to prepare for the post- COVID-19 era.
5. Conclusion
COVID-19 has triggered rapid changes in the environment surrounding higher education. Amid these changes, efforts are being made to verify students’ performance in various ways to foster the quality of higher education, not only in each university’s core competencies but also in governments and research institutes. This is because there is a growing awareness that the performance of college education should be statistically analyzed and that the quality of education should be improved based on the result of analysis.
This study was intended to measure the performance indicators of university students in line with communication, the core competency for foreign language in general basic education at I University, a local private university. Based on the aforementioned findings, the conclusions and significance of the study are as follows. Performance diagnosis was conducted on the College English I, II, and III courses in the area of foreign language communication in Liberal Arts College at I University. The performance diagnosis was analyzed by having students respond to the survey questions selected by the professors in charge of each subject and comparing the results. The response rates were 72.9%, 78.1%, and 49.9% for College English I, II, and III, respectively. The mean for College English I basic learning competency was 4.12 and that for knowledge and experience was 3.76, which is far above the average level. College English II was also well above the average level, with a mean score of 4.0 in basic learning competency, and 3.78 in knowledge and experience. In the case of College English III, the mean scores for basic learning competency and knowledge and experience were generally high at 4.18 and 4.16 points, respectively. In general, there seemed to be an improvement in students’ competencies in areas where each course emphasized the purpose (target) in line with communication, which was the core competency of foreign language in I University. This could indicate that communication competency pursued in the foreign language area of i-LAC as well as the direction of education pursued in College English courses, was appropriate.
This study has several limitations. First, when choosing the performance diagnostic questions, it seemed necessary to coordinate opinions with other professors who taught the same subject. For the current study, the professors in charge of the course chose the performance diagnosis evaluation questions. However, as there were many performance diagnostic questions that could be necessary regarding the target of courses, the professors in charge of each course needed to discuss and select the ones that corresponded to course targets. Second, it was presupposed that students would answer in the survey questions faithfully, although it seemed necessary to identify effective ways to increase students’ participation in the survey. In the case of online surveys using the Naver Office form, students were encouraged to voluntarily participate in the survey, but the survey participation for students in College English III classes, where native professors only spoke English in class, was generally low. The participation rate would have been higher if there had been a process to induce participation in Korean. Last, in this study, only two areas of basic learning competence and knowledge and experience were diagnosed and analyzed in line with communication, which was the core competency for foreign language in i-LAC. In future studies, it may be useful to go one step further and analyze thinking, value, and integration competences to examine the status and effectiveness of general English education holistically as cultural education as well as general English education in the area of foreign language.
This study suggests on the future studies using tools developed by KNIGE. First, there will have to be a process for professors of each subject to review the feasibility of the questions when choosing diagnostic tool questions. This is because performance indicators, which are considered important by professors, may vary, so it seems desirable to select appropriate diagnosis questions for each subject after sufficient discussion. Second, in order to effectively use tools developed by KNIGE, it seems necessary for professors to predict student achievement when selecting diagnostic questions and comparing them to actual results of responses. In future studies, this process should be supplemented, so that professors can use it as materials for improving their classes.
References
Appendices
Notes
The paradigm of domestic university education emphasizes the need for general education to enhance students’ core competencies. Its core competencies include critical, creative and comprehensive thinking, communication, self-directed learning, cooperation, problem-solving, physical and mental health care, reflective and integrated learning.
College English IV was excluded from the study because it was an intermediate-level course of English conversation and because the focus of the study was to examine the reactions of students in beginning-level courses of College English.
The data for this study were partially extracted from the performance diagnosis evaluation conducted by the Liberal Arts College of I University in preparation for the University Basic Competency Diagnosis implementation in 2021.
See Appendix 1 for detailed information on the area of tools for diagnosing performance in general education.
See Appendices 2 and 5 to check out detailed information on survey questions and weekly class syllabus for College English I.
See Appendices 3 and 6 to check out detailed information on survey questions and weekly class syllabus for College English II.
See Appendices 4 and 7 to check out detailed information on survey questions and weekly class syllabus for College English III.