Next Article in Journal
Social and Emotional Learning during Pandemic-Related Remote and Hybrid Instruction: Teacher Strategies in Response to Trauma
Next Article in Special Issue
Modeling Students’ Perceptions of Chatbots in Learning: Integrating Technology Acceptance with the Value-Based Adoption Model
Previous Article in Journal
A Human Capability Perspective on the Progression of Low-SES Students to Higher Education in Ireland and the UK
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 4
10.3390/educsci13040410
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature

by
Chung Kwan Lo
Department of Mathematics and Information Technology, The Education University of Hong Kong, Hong Kong SAR 999077, China
Educ. Sci. 2023, 13(4), 410; https://doi.org/10.3390/educsci13040410
Submission received: 13 March 2023 / Revised: 6 April 2023 / Accepted: 13 April 2023 / Published: 18 April 2023
(This article belongs to the Special Issue Teaching and Learning in the Era of AI Conversation Chatbots: Opportunities and Challenges)
Browse Figures
Versions Notes

Abstract

:
An artificial intelligence-based chatbot, ChatGPT, was launched in November 2022 and is capable of generating cohesive and informative human-like responses to user input. This rapid review of the literature aims to enrich our understanding of ChatGPT’s capabilities across subject domains, how it can be used in education, and potential issues raised by researchers during the first three months of its release (i.e., December 2022 to February 2023). A search of the relevant databases and Google Scholar yielded 50 articles for content analysis (i.e., open coding, axial coding, and selective coding). The findings of this review suggest that ChatGPT’s performance varied across subject domains, ranging from outstanding (e.g., economics) and satisfactory (e.g., programming) to unsatisfactory (e.g., mathematics). Although ChatGPT has the potential to serve as an assistant for instructors (e.g., to generate course materials and provide suggestions) and a virtual tutor for students (e.g., to answer questions and facilitate collaboration), there were challenges associated with its use (e.g., generating incorrect or fake information and bypassing plagiarism detectors). Immediate action should be taken to update the assessment methods and institutional policies in schools and universities. Instructor training and student education are also essential to respond to the impact of ChatGPT on the educational environment.

1. Introduction

Artificial intelligence (AI) has developed rapidly in recent years, leading to various applications in different disciplines, such as healthcare [1] and education [2]. AI systems can be trained to simulate the human brain and carry out routine work using large amounts of data [3]. In healthcare, for example, AI can aid professionals in their work by synthesising patient records, interpreting diagnostic images, and highlighting health concerns [4]. AI applications have also been utilised in education to enhance administrative services and academic support [2]. One representative example is intelligent tutoring systems (ITS), which can be used to simulate one-to-one personal tutoring. The results of a meta-analysis indicated that ITS generally had a moderately positive effect on the academic achievement of college students [5]. However, the development of ITS can be challenging, as it involves not only content creation and design but also the refinement of feedback phrasing and dialogue strategies [6].
ChatGPT, a recently developed conversational chatbot created by OpenAI [7], may make it easier for instructors to apply AI in teaching and learning. ChatGPT uses natural language processing to generate human-like responses to user input. It has gained attention worldwide for its impressive performance in generating coherent, systematic, and informative responses [8]. In a surprising achievement, ChatGPT passed four separate examinations at the University of Minnesota Law School [9]. Although its scores were not (yet) very good, the results demonstrate that this AI application is capable of earning a university degree [10]. Since its release on 30 November 2022, ChatGPT has become the fastest-growing user application in history, reaching 100 million active users as of January 2023, just two months after its launch [11].
Despite its success, ChatGPT has introduced new challenges and threats to education. With its ability to provide specific answers to user questions, it can be used to complete written assignments and examinations on behalf of students, leading to concerns about AI-assisted cheating. In response, some schools have banned access to ChatGPT on campus [12]. The implications of GhatGPT in the field of education were explored in a review by Mhlanga [13]. He analysed eight articles on ChatGPT and revealed that educators had concerns regarding the use of ChatGPT in education. They expressed worries that students may outsource their work to ChatGPT because of its ability to rapidly generate acceptable texts. Therefore, Mhlanga [13] emphasised the importance of the responsible and ethical use of ChatGPT. Sallam [14] reviewed 60 articles on ChatGPT in the fields of healthcare, medical education, and academia. He found various concerns across the studies, ranging from plagiarism to incorrect responses and inaccurate citations. Therefore, the implications of ChatGPT-assisted learning require immediate attention to ensure that its benefits are optimised while its drawbacks are minimised.
Although Mhlanga [13] and Sallam [14] shed some light on the concerns regarding the use of ChatGPT, a more comprehensive review is needed to gain a deeper understanding of its potential benefits and threats to education. Furthermore, current knowledge about ChatGPT’s capabilities comes mainly from media reports (e.g., [9,15]). It is necessary to examine the results of research on ChatGPT to evaluate its true performance across different subject domains. Hence, the following research questions (RQ1 to RQ3) are posed to guide the review.
  • RQ1: How does ChatGPT perform in different subject domains?
  • RQ2: How can ChatGPT be used to enhance teaching and learning?
  • RQ3: What are the potential issues associated with ChatGPT, and how can they be addressed?

2. Methods

2.1. The Rapid Review Approach

As ChatGPT continues to receive great attention and is increasingly used by students, there is a pressing need to understand its impact on education and take immediate action in response to its possible threats. However, a comprehensive systematic review can take several months or even years to conduct [16,17], which is not ideal for catching up with the rapidly evolving ChatGPT landscape. Therefore, a rapid review approach was used. According to Tricco et al. [17], “a rapid review is a type of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a short period of time” (p. 2). This approach enabled a timely synthesis and overview of recently published articles and their key findings. Accordingly, this review could provide valuable insights enabling researchers, practitioners, and policymakers to respond promptly to the influence of ChatGPT on the field of education.

2.2. Search Strategies

This rapid review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement when selecting relevant articles [18]. The final search was conducted on 28 February 2023 (i.e., three months after the release of ChatGPT). Therefore, the ChatGPT involved in these articles was its original release (dated 30 November 2022) based on Generative Pre-trained Transformer 3.5 (GPT-3.5). Seven electronic databases were used: (1) Academic Search Ultimate, (2) ACM Digital Library, (3) Education Research Complete, (4) ERIC, (5) IEEE Xplore, (6) Scopus, and (7) Web of Science. The search string “ChatGPT” was used in each database to search for relevant articles that included the term “ChatGPT” in their title, abstract, or keywords. The publication period was specified as 2022 to the present. Despite the use of multiple databases, only a limited number of relevant articles were found. Therefore, a title search for the term “ChatGPT” was conducted using Google Scholar within the same publication period. This approach enabled the retrieval of additional relevant articles that were not captured in the initial database search.

2.3. Inclusion and Exclusion Criteria

Academic articles published between 1 January 2022 and 28 February 2023 (the date of the final search) were reviewed, including advanced online publications and preprints. Non-academic articles (e.g., articles from mass and social media) were excluded. At the time of writing, this period covered all of the articles that had been published about ChatGPT, as it was released on 30 November 2022. To be included in this rapid review, articles had to discuss ChatGPT in the field of education, with no constraints on any specific educational contexts. Literature reviews, if retrieved, were used as background references. However, they were excluded from the synthesis to avoid duplicate findings. In addition, only English-language articles were included in this review. Table 1 summarises the inclusion and exclusion criteria for article selection.

2.4. Content Analysis

Guided by the research questions set forth above, a content analysis of the included articles was conducted. Creswell’s [19] coding techniques were employed in the process of analysis and interpretation. His approach comprised of the following three main stages.
  • Open coding: Open coding is the initial stage of coding in which the researcher examines the data and generates codes to describe and categorise the data. It allows a deep exploration and understanding of the data without imposing preconceived ideas.
  • Axial coding: After the initial process of open coding, axial coding is used to analyse the relationships and connections between the codes generated during open coding. It involves organising the codes into broader categories or themes and establishing the relationships between them.
  • Selective coding: Selective coding is the final stage of coding. It involves further refining and integrating the categories or themes identified during open and axial coding to develop a comprehensive picture. It identifies the core category or central phenomenon that emerged from the data.
Following Creswell’s [19] approach, codes were assigned to pieces of data without using predominant frameworks in the stage of open coding. After the completion of coding the first 15 articles, all the codes assigned were reviewed and grouped by similarity, thereby reducing redundant codes in the stage of axial coding. The preliminary list of codes was used to analyse the remaining articles. Hence, any new codes that emerged were added to the list. Specific quotes were identified to further support the codes. Finally, similar codes were organised into categories in the stage of selective coding. To enhance the consistency of classification, several exemplary quotes that clearly illustrated each category were identified. Multiple reviews of the data were performed to maintain consistency in coding.

3. Findings

3.1. General Description of the Literature

Figure 1 shows that 50 and 367 outcomes were retrieved through the database search and Google Scholar, respectively. Due to duplication across databases and Google Scholar, some of the articles were removed, yielding 363 unique records for screening. Notably, many records retrieved were outside the scope of this review, including articles from mass and social media found by Google Scholar. After reviewing the unique records’ titles, abstracts, and publication sources, 55 full-text articles were assessed for eligibility. Five articles were then excluded: (1) two articles that were not related to education, (2) one preprint article that was incomplete, and (3) two articles that were literature reviews. However, some of the excluded articles were used as background references. Ultimately, 50 articles [8,10,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67] were selected for synthesis. The list of the included articles is provided in the Supplementary Materials. Figure 1 provides an overview of the article selection process.
As shown in Figure 2, the majority of the included articles were written by researchers from the United States (N = 19), followed by the United Kingdom (N = 4) and Austria (N = 3). Figure 3 shows that 16 of the 50 included articles (32%) were published in journals, and two were published as an eBook (2%) and report (2%), respectively. The remaining 32 articles (64%) were preprints uploaded on SSRN [68] (N = 12), followed by arXiv [69] (N = 7) and ResearchGate [70] (N = 5), among others.

3.2. RQ1: How Does ChatGPT Perform in Different Subject Domains?

As shown in Table 2, ChatGPT-3.5′s performance was evaluated in 21 studies using tests, exams, or interviews. All of these studies, except for de Winter’s [20] study that used high school exams, were conducted in the context of higher education. ChatGPT’s performance varied across subject domains. It demonstrated outstanding results in critical and higher-order thinking [21] and economics [22]. However, its performance was not entirely satisfactory in other subject domains, such as law [10,23], medical education [24,25,26,27,28,29,30,31,32], and mathematics [33]. These results are consistent with Newton’s [34] study, which used multiple-choice questions-based exams to assess ChatGPT’s performance in economics, medical education, law, and physics. In economics, ChatGPT outperformed average students, but in other subjects, it underperformed average students by 8 to 40 points (out of 100).
Notably, medical education was a subject domain for which there was relatively more evidence of ChatGPT’s performance. These results came from multiple countries. Kung et al. [24] and Gilson et al. [25] evaluated ChatGPT’s performance using the United States Medical Licensing Examination. Their results suggested that ChatGPT could yield moderate accuracy and, thus, a passing score. However, Fijačko [26] found that ChatGPT failed the exams of the American Heart Association. Han et al. [27] noted that ChatGPT provided incorrect and insufficient information about cardiovascular diseases. In the context of pharmacology education in Malaysia, Nisar and Aslam [28] found that although ChatGPT could provide relevant and accurate answers, these answers lacked references and sources. In China, Wang et al. [29] testified ChatGPT using the Chinese National Medical Licensing Examination. They reported that ChatGPT’s performance was lower than medical students’ average score. Similar results were reported by researchers in Korea [30], India [31], and Singapore [32]. The findings of these studies suggested that, in general, ChatGPT’s performance was not entirely satisfactory in the domain of medical education.

3.3. RQ2: How Can ChatGPT Be Used to Enhance Teaching and Learning?

The findings of this review suggested that ChatGPT can serve as an assistant for both instructors and students. Concerning instructors, Table 3 categorises ChatGPT’s five major functions into two main aspects, namely teaching preparation (i.e., generating course materials, providing suggestions, and performing language translation) and assessment (i.e., generating assessment tasks and evaluating student performance).
Concerning the teaching preparation aspect, ChatGPT can make suggestions that assist instructors. In the words of one instructor, “ChatGPT can be a useful tool for teachers and educators to remind them of what knowledge and skills should be included in their curriculum, by providing an outline” [40] (pp. 7–8). Megahed et al. [37] asked ChatGPT to generate a course syllabus for an undergraduate statistics course. They noted that its teaching suggestions could be adopted without the need for major changes. Zhai [41] found that ChatGPT was able to provide recommendations related to special education. He commented, “These recommendations are beneficial for students with special learning needs” (p. 4).
Concerning the assessment aspect, ChatGPT can help instructors generate exercises, quizzes, and scenarios for student assessment [29,45]. However, Al-Worafi et al. [49] cautioned that the assessment tasks suggested by ChatGPT might not cover all targeted learning objectives. Therefore, they recommended using ChatGPT to guide instructors in their preparation of assessments rather than to completely replace their efforts. For example, Han et al. [27] instructed ChatGPT to create a multiple-choice question with a vignette and lab values for a medical topic. The question’s outcome was “a reasonable basic question to assess students’ knowledge” (p. 7). The instructors could refine the language of and information contained in the question to increase its relevancy to their course requirements.
For students, ChatGPT can serve as a virtual tutor to support their learning. Table 4 categorises its six major functions into two main aspects, namely learning (i.e., answering questions, summarising information, and facilitating collaboration) and assessment (i.e., concept checking and exam preparation, drafting assistance, and providing feedback).
Consider ChatGPT’s ability to facilitate collaboration in the learning aspect. Rudolph et al. [52] suggested that ChatGPT can generate different scenarios for students to work collaboratively in group activities. It can then provide a discussion structure, real-time feedback, and personalised guidance to facilitate group discussions and debates [44]. As noted by Gilson et al. [25], the enhanced small-group discourse in problem-solving benefits student learning.
Concerning the assessment aspect, students can benefit from using ChatGPT as a scaffolding tool for their initial draft and then refining the draft by correcting errors and adding references to the final versions of their written assignments [10,23]. Gilson et al. [25] noted that ChatGPT’s initial answer could prompt further questioning and encourage students to apply their knowledge and reasoning skills. However, Rudolph et al. [52] cautioned that ChatGPT should not replace critical thinking and original work but should instead serve as an aid to improve writing and research skills.

3.4. RQ3: What Are the Potential Issues Associated with ChatGPT and How Can They Be Addressed?

This review identified five major issues associated with ChatGPT in education. Table 5 categorises these issues into two main aspects, namely accuracy and reliability (i.e., relying on biased data, having limited up-to-date knowledge, and generating incorrect or fake information) and plagiarism prevention (i.e., student plagiarism and bypassing plagiarism detectors).
Consider the issue of generating incorrect or fake information in the accuracy and reliability aspect. Mogali [32] raised concerns about ChatGPT’s well-written but inaccurate information. It has commonly been observed that the bibliographic citations generated by ChatGPT can be fake [32,39,46,56,58]. Concerning subject knowledge, Megahed et al. [37] found that it generated incorrect programming codes and was unable to detect and resolve its errors. This concern was echoed by Jalil et al. [38], who commented that “ChatGPT is a poor judge of its own correctness” (p. 8). ChatGPT’s accuracy and reliability have also been found to be questionable in other subject domains, such as mathematics [33], sports science and psychology [39], and health professions [26,27,32].
Concerning the plagiarism prevention aspect, ChatGPT-generated texts can bypass conventional plagiarism detectors. For example, Ventayen [62] asked ChatGPT to write an essay based on existing publications and checked its output for originality using Turnitin (a plagiarism detection application [71]). However, the application found a low similarity index between the document and existing work, and no plagiarism could be detected. Khalil and Er [61] asked ChatGPT to generate 50 essays based on different open-ended questions. Half of the essays were checked using Turnitin [71], which gave an average similarity score of 13.72%. The other half were checked using iThenticate [72], another plagiarism detection application, which gave an average similarity score of 8.76%. With these scores, the ChatGPT documents were considered highly original.
The included articles proposed several strategies to address the potential issues associated with ChatGPT in education. Table 6 categorises these strategies into three main aspects, namely task design (i.e., incorporating multimedia resources, adopting novel question types, and using digital-free assessment formats), the identification of AI writing (i.e., using AI-based writing detection tools and checking references), and institutional policy (i.e., establishing anti-plagiarism guidelines and providing student education).
Consider the issue of adopting novel question types in the task design aspect. Zhai [8] proposed exploring innovative formats that encourage students to be creative and engage in critical thinking. Choi et al. [10] emphasised the importance of requiring students to analyse cases rather than simply recalling knowledge. Similarly, Geerling et al. [22] suggested requiring students to apply the concepts that they learn in their courses and even to create new materials that AI cannot replicate. Stutz et al. [36] concluded that future assessments should focus on the higher levels of Bloom’s taxonomy, such as application, analysis, and creation [73].
Other strategies concerned the identification of AI writing and institutional policy aspects. In the former aspect, the findings of this review suggested that using AI-based writing detection tools and checking references were the two major strategies. Szabo [39] reported that although conventional plagiarism detectors failed to identify ChatGPT-generated texts, AI detectors were (still) able to detect them. Moreover, ChatGPT’s possible failure to generate a correct reference list (see Table 5) can be a telltale sign for instructors seeking to identify whether a student has used ChatGPT [50,51,56]. In addition to detecting student plagiarism, researchers have emphasised the importance of establishing anti-plagiarism guidelines and educating students about academic integrity [52,56,61].

4. Discussion

In this review, 50 articles published on or before 28 February 2023 were analysed. Therefore, the ChatGPT involved in these articles was its original version based on GPT-3.5, instead of GPT-4 [74]. The findings suggested that ChatGPT has the potential to enhance teaching and learning (see Table 3 and Table 4). However, its knowledge and performance were not entirely satisfactory across subject domains (see Table 2). The use of ChatGPT also presents various potential issues, such as generating incorrect or fake information and student plagiarism (see Table 5). Therefore, immediate action is needed to address these potential issues (see Table 6) and optimise the use of ChatGPT in education.

4.1. Leveraging ChatGPT in Teaching and Learning

ChatGPT can be a valuable tool for instructors, providing a starting point for creating course syllabi, teaching materials, and assessment tasks. However, concerns regarding the accuracy of its generated content must be addressed. One possible solution would be to use ChatGPT to generate raw materials to train course-specific chatbots. For example, Topsakal and Topsakal [42] used ChatGPT to create dialogues to aid students’ English language learning. After verifying the accuracy of the materials, instructors can ask GhatGPT to convert them into a format suitable for use with AI-based chatbots, such as Google Dialogflow [75], providing students with an interactive and personalised learning environment.
ChatGPT can also enhance active learning approaches. For example, Rudolph et al. [52] suggested using flipped learning, in which students are required to prepare for lessons by studying pre-class materials. This instructional approach can free up class time for interactive learning activities, such as group discussions. In conventional flipped classes, however, students may encounter difficulties in pre-class learning [76]. In-class participation also needs to be improved [77]. This issue became apparent during the COVID-19 pandemic, during which fully online flipped learning led to poor in-class participation and disengagement from peer sharing [78,79]. As a virtual tutor, ChatGPT can assist students in online independent study by answering their questions [28] and enhance group dynamics by suggesting a discussion structure and providing real-time feedback [44].

4.2. Challenges and Threats Posed by ChatGPT in Education

According to Sallam [14], the use of ChatGPT in education poses challenges related to its accuracy and reliability. Because ChatGPT is trained on a large corpus of data, it may be biased or contain inaccuracies. Mbakwe et al. [57] noted that bias may stem from the use of research primarily conducted in high-income countries or textbooks that are not universally applicable. As evidenced by Pavlik [54], for example, ChatGPT is not familiar with hedge fund ownership of news media. In addition, ChatGPT’s knowledge is limited and has not (yet) been updated with data after 2021 [25,45,46]. Therefore, its responses may not always be accurate or reliable, particularly for specialised subject matters and recent events. Furthermore, ChatGPT may generate incorrect or even fake information [37,46,51]. This issue can be problematic for students who rely on ChatGPT to inform their learning.
Student plagiarism has become a significant concern in education. Plagiarism detection applications (e.g., Turnitin [71] and iThenticate [72]) are commonly used to identify copied content in student assignments. However, studies have found that ChatGPT can bypass these detectors by generating seemingly original content [61,62]. Bašić et al. [59] provided evidence that students who used ChatGPT were more likely to commit plagiarism than those who did not use it. ChatGPT’s ability to facilitate plagiarism not only impairs academic integrity but also defeats the purpose of assessment, which is to evaluate student learning fairly. According to Cotton et al. [50], students who use ChatGPT to generate high-quality work gain an unfair advantage over their peers who do not have access to it. More importantly, instructors cannot accurately evaluate student performance when ChatGPT is involved, making it difficult to follow up on students’ learning problems.

4.3. Immediate Action in Response to the Impact of ChatGPT

Immediate action must be taken to mitigate the impact of ChatGPT on education. Assessment methods and institutional policies need to be updated to address the challenges posed by the emergence of AI-generated content in student assignments. Before the launch of GPT-4 (dated 14 March 2023), instructors could refine the design of their assessment tasks by incorporating multimedia resources to reduce the risk of plagiarism. The original release of ChatGPT could not process images and videos, which resulted in the missing context that increased barriers for students seeking to use it to cheat [21,34,52]. However, GPT-4, a large multimodal model created by OpenAI [74], is able to process images. Therefore, instructors have to consider other strategies, as shown in Table 6, such as incorporating digital-free components (e.g., oral presentations [40,52,58]) into their assessment tasks. These components require students to demonstrate their abilities in real-time and in person. At the institutional level, AI-based writing detection tools should be made available to instructors. Furthermore, anti-plagiarism guidelines should be established to clarify the boundaries of ChatGPT’s involvement in student learning.
Instructor training and student education are also critical in responding to the impact of ChatGPT [64]. It is essential to train instructors on how to identify the use of ChatGPT in student assignments, which can be achieved by using AI detection tools. Instructors should also be trained on how to fully use ChatGPT in their teaching preparation and course assessment, as shown in Table 3. For students, it is crucial to introduce them to the limitations of ChatGPT, such as its reliance on biased data, limited up-to-date knowledge, and potential for generating incorrect or fake information. Therefore, instructors should teach students to use other authoritative sources (e.g., reference books) to verify, evaluate, and corroborate the factual correctness of information provided by ChatGPT [39,44]. It is also important to increase students’ awareness of academic integrity policies and their understanding of the consequences of academic misconduct [52,60]. To achieve this goal, instructors should openly discuss ChatGPT in their courses and emphasise the importance of academic honesty.

4.4. Limitations and Recommendations for Future Research

4.4.1. Limitations of this Rapid Review

This rapid review has limitations that must be considered when interpreting the findings. First, similar to the reviews by Mhlanga [13] and Sallam [14], the majority of the included articles were preprints, meaning that they have not undergone rigorous peer review. The quality of their evidence is, therefore, questionable. Follow-up systematic reviews are needed once more peer-reviewed articles on ChatGPT are published.
Second, most of the included articles were written in the Western context, particularly in medical and higher education. Thus, the findings of this review may be biased towards these specific contexts. Further studies in other subject domains (e.g., mathematics and language education) and education contexts (e.g., primary and secondary education) are recommended.
Third, this review only focused on the original release of ChatGPT. As a result, the findings might not be applicable to other applications of GPT and GPT-4 [74] that have been launched beyond the time period of this review. Future research can test the performance of GPT-4, explore its possibilities in supporting teaching and learning, and discuss its potential threats and solutions. Furthermore, future reviews can expand their scope to include other applications of GPT in education. By conducting more in-depth research and comprehensive reviews, educators can better understand the capabilities and limitations of GPT technology, as well as develop appropriate guidelines and policies to ensure its responsible and ethical use.

4.4.2. Limitations of the Included Articles

In addition to the limitations of this review, there are three major limitations associated with the included articles. First, very few studies empirically examined the influence of ChatGPT on student performance and behaviour. From the study by Bašić et al. [59], for example, we learnt that using ChatGPT to support student writing might not improve performance but could instead lead to more plagiarism issues. However, these researchers acknowledged that the generalisability of their study was limited due to the small number of research participants (N = 18). Further research is needed to evaluate the benefits and potential problems of ChatGPT-assisted learning for students.
Second, some of the suggestions made in the included articles were based on the researchers’ intuitive beliefs rather than empirical evidence. For example, some researchers [8,52,62] suggested designing assessment tasks that focus on creativity and critical thinking. However, specific strategies to achieve this goal were not always thoroughly discussed. Therefore, more rigorous studies are needed to provide evidence-based recommendations for using ChatGPT in education.
Finally, some researchers used ChatGPT to generate some of their content or suggestions, leading to repetitive ideas across articles. For example, the researchers of different articles asked ChatGPT how it can benefit teaching and learning, resulting in very similar content, such as personalised learning and language translation [8,43,45,46,48,51]. This observation provides another reason to discourage the direct adoption of ChatGPT- or AI-generated texts as personal ideas in research articles, in addition to Thorp’s [80] concern about “plagiarism of existing works” (p. 313). When ChatGPT is used as a co-author, it may generate similar points in different articles that do not acknowledge each other, which is the equivalent of self-plagiarism. If researchers plan to involve ChatGPT in writing, additional analysis of its ideas should be conducted.

5. Conclusions

This rapid review of 50 articles highlighted ChatGPT’s varied performance across different subject domains and its potential benefits when serving as an assistant for instructors and as a virtual tutor for students. However, its use raises various concerns, such as its generation of incorrect or fake information and the threat it poses to academic integrity. The findings of this review call for immediate action by schools and universities to update their guidelines and policies for academic integrity and plagiarism prevention. Furthermore, instructors should be trained on how to use ChatGPT effectively and detect student plagiarism. Students should also be educated on the use and limitations of ChatGPT and its potential impact on academic integrity.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/educsci13040410/s1, Table S1: A summary of the included articles (N = 50).

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data samples and detailed coding procedures can be accessed by contacting the corresponding author.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Xu, L.; Sanders, L.; Li, K.; Chow, J.C.L. Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review. JMIR Cancer 2021, 7, e27850. [Google Scholar] [CrossRef] [PubMed]
  2. Zawacki-Richter, O.; Marín, V.I.; Bond, M.; Gouverneur, F. Systematic Review of Research on Artificial Intelligence Applications in Higher Education—Where are the Educators? Int. J. Educ. Technol. High. Educ. 2019, 16, 39. [Google Scholar] [CrossRef]
  3. Bengio, Y.; Lecun, Y.; Hinton, G. Deep Learning for AI. Commun. ACM 2021, 64, 58–65. [Google Scholar] [CrossRef]
  4. Aung, Y.Y.M.; Wong, D.C.S.; Ting, D.S.W. The Promise of Artificial Intelligence: A Review of the Opportunities and Challenges of Artificial Intelligence in Healthcare. Br. Med. Bull. 2021, 139, 4–15. [Google Scholar] [CrossRef]
  5. Steenbergen-Hu, S.; Cooper, H. A Meta-analysis of the Effectiveness of Intelligent Tutoring Systems on College Students’ Academic Learning. J. Educ. Psychol. 2014, 106, 331–347. [Google Scholar] [CrossRef]
  6. Afzal, S.; Dhamecha, T.; Mukhi, N.; Sindhgatta Rajan, R.; Marvaniya, S.; Ventura, M.; Yarbro, J. Development and deployment of a large-scale dialog-based intelligent tutoring system. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2–7 June 2019; Association for Computational Linguistics: Stroudsburg, PA, USA, 2019; Volume 2, pp. 114–121. [Google Scholar]
  7. OpenAI. Available online: https://openai.com (accessed on 10 March 2023).
  8. Zhai, X. ChatGPT User Experience: Implications for Education. SSRN 2022, 4312418. [Google Scholar] [CrossRef]
  9. ChatGPT Passes Exams from Law and Business Schools. Available online: https://edition.cnn.com/2023/01/26/tech/chatgpt-passes-exams (accessed on 10 March 2023).
  10. Choi, J.H.; Hickman, K.E.; Monahan, A.; Schwarcz, D. Chatgpt Goes to Law School. SSRN 2023, 4335905. Available online: https://ssrn.com/abstract=4335905 (accessed on 10 March 2023).
  11. ChatGPT Sets Record for Fastest-Growing User Base—Analyst Note. Available online: https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01 (accessed on 10 March 2023).
  12. Schools Ban ChatGPT Amid Fears of Artificial Intelligence-Assisted Cheating. Available online: https://www.voanews.com/a/schools-ban-chatgpt-amid-fears-of-artificial-intelligence-assisted-cheating/6949800.html (accessed on 10 March 2023).
  13. Mhlanga, D. Open AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning. SSRN 2023, 4354422. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4354422 (accessed on 10 March 2023).
  14. Sallam, M. The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. Available online: https://www.medrxiv.org/content/10.1101/2023.02.19.23286155v1 (accessed on 1 March 2023).
  15. ChatGPT Could Be a Stanford Medical Student, a Lawyer, or a Financial Analyst. Here’s a List of Advanced Exams the AI Bot Has Passed So Far. Available online: https://www.businessinsider.com/list-here-are-the-exams-chatgpt-has-passed-so-far-2023-1 (accessed on 10 March 2023).
  16. Khangura, S.; Konnyu, K.; Cushman, R.; Grimshaw, J.; Moher, D. Evidence Summaries: The Evolution of a Rapid Review Approach. Syst. Rev. 2012, 1, 10. [Google Scholar] [CrossRef] [PubMed]
  17. Tricco, A.C.; Antony, J.; Zarin, W.; Strifler, L.; Ghassemi, M.; Ivory, J.; Perrier, L.; Hutton, B.; Moher, D.; Straus, S.E. A scoping review of rapid review methods. BMC Med. 2015, 13, 224. [Google Scholar] [CrossRef]
  18. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; The PRISMA Group. Reprint-preferred Reporting Items for Systematic Reviews and Meta-analyses: The PRISMA statement. Phys. Ther. 2009, 89, 873–880. [Google Scholar] [CrossRef]
  19. Creswell, J.W. Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 4th ed.; Pearson: Boston, MA, USA, 2012. [Google Scholar]
  20. de Winter, J.C.F. Can ChatGPT Pass High School Exams on English Language Comprehension? Available online: https://www.researchgate.net/publication/366659237_Can_ChatGPT_pass_high_school_exams_on_English_Language_Comprehension (accessed on 1 March 2023).
  21. Susnjak, T. ChatGPT: The End of Online Exam Integrity? arXiv 2022, arXiv:2212.09292v1. [Google Scholar]
  22. Geerling, W.; Mateer, G.D.; Wooten, J.; Damodaran, N. Is ChatGPT Smarter than a Student in Principles of Economics? SSRN 2023, 4356034. [Google Scholar] [CrossRef]
  23. Hargreaves, S. ‘Words are Flowing out Like Endless Rain into a Paper Cup’: ChatGPT & Law School Assessments. SSRN 2023, 4359407. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4359407 (accessed on 10 March 2023).
  24. Kung, T.H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepaño, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; et al. Performance of ChatGPT on USMLE: Potential for AI-assisted Medical Education Using Large Language Models. PLoS Digital Health 2023, 2, e0000198. [Google Scholar] [CrossRef] [PubMed]
  25. Gilson, A.; Safranek, C.W.; Huang, T.; Socrates, V.; Chi, L.; Taylor, R.A.; Chartash, D. How does CHATGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med. Educ. 2023, 9, e45312. [Google Scholar] [CrossRef]
  26. Fijačko, N.; Gosak, L.; Štiglic, G.; Picard, C.T.; Douma, M.J. Can ChatGPT Pass the Life Support Exams without Entering the American Heart Association Course? Resuscitation 2023, 185, 109732. [Google Scholar] [CrossRef]
  27. Han, Z.; Battaglia, F.; Udaiyar, A.; Fooks, A.; Terlecky, S.R. An Explorative Assessment of ChatGPT as an Aid in Medical Education: Use it with Caution. medRxiv 2023. [Google Scholar] [CrossRef]
  28. Nisar, S.; Aslam, M. Is ChatGPT a Good Tool for T&CM Students in Studying Pharmacology? SSRN 2023, 4324310. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4324310 (accessed on 10 March 2023).
  29. Wang, X.; Gong, Z.; Wang, G.; Jia, J.; Xu, Y.; Zhao, J.; Fan, Q.; Wu, S.; Hu, W.; Li, X. ChatGPT Performs on the Chinese National Medical Licensing Examination. J. Med. Syst. 2023. [Google Scholar] [CrossRef]
  30. Huh, S. Are ChatGPT’s Knowledge and Interpretation Ability Comparable to Those of Medical Students in Korea for Taking a Parasitology Examination? A Descriptive Study. J. Educ. Eval. Health Prof. 2023, 20, 1. [Google Scholar] [CrossRef] [PubMed]
  31. Hisan, U.; Amri, M. ChatGPT and Medical Education: A Double-Edged Sword. J. Pedagog. Educ. Sci. 2023, 2, 71–89. [Google Scholar] [CrossRef]
  32. Mogali, S.R. Initial Impressions of ChatGPT for Anatomy Education. Anat. Sci. Educ. 2023, 1–4. [Google Scholar] [CrossRef] [PubMed]
  33. Frieder, S.; Pinchetti, L.; Griffiths, R.R.; Salvatori, T.; Lukasiewicz, T.; Petersen, P.C.; Chevalier, A.; Berner, J. Mathematical Capabilities of ChatGPT. arXiv 2023, arXiv:2301.13867. [Google Scholar]
  34. Newton, P.M. ChatGPT Performance on MCQ-based Exams. EdArXiv 2023. [Google Scholar] [CrossRef]
  35. Buchberger, B. Is ChatGPT Smarter than Master’s Applicants? Research Institute for Symbolic Computation: Linz, Austria, 2023. [Google Scholar] [CrossRef]
  36. Stutz, P.; Elixhauser, M.; Grubinger-Preiner, J.; Linner, V.; Reibersdorfer-Adelsberger, E.; Traun, C.; Wallentin, G.; Whös, K.; Zuberbühler, T. Ch(e)atGPT? An Anecdotal Approach on the Impact of ChatGPT on Teaching and Learning GIScience. EdArXiv 2023. [Google Scholar] [CrossRef]
  37. Megahed, F.M.; Chen, Y.J.; Ferris, J.A.; Knoth, S.; Jones-Farmer, L.A. How Generative AI Models such as ChatGPT Can be (Mis)Used in SPC Practice, Education, and Research? An Exploratory Study. arXiv 2023, arXiv:2302.10916. [Google Scholar]
  38. Jalil, S.; Rafi, S.; LaToza, T.D.; Moran, K.; Lam, W. ChatGPT and Software Testing Education: Promises & Perils. arXiv 2023, arXiv:2302.03287. [Google Scholar]
  39. Szabo, A. ChatGPT a Breakthrough in Science and Education: Can it Fail a Test? OSF Prepr. 2023. [Google Scholar] [CrossRef]
  40. Tlili, A.; Shehata, B.; Adarkwah, M.A.; Bozkurt, A.; Hickey, D.T.; Huang, R.; Agyemang, B. What if the Devil is My Guardian Angel: ChatGPT as a Case Study of Using Chatbots in Education. Smart Learn. Env. 2023, 10, 15. [Google Scholar] [CrossRef]
  41. Zhai, X. ChatGPT for Next Generation Science Learning. SSRN 2023, 4331313. Available online: https://ssrn.com/abstract=4331313 (accessed on 10 March 2023).
  42. Topsakal, O.; Topsakal, E. Framework for a Foreign Language Teaching Software for Children Utilizing AR, Voicebots and ChatGPT (Large Language Models). J. Cogn. Syst. 2022, 7, 33–38. [Google Scholar] [CrossRef]
  43. Ali, F. Let the Devil Speak for Itself: Should ChatGPT Be Allowed or Banned in Hospitality and Tourism Schools? J. Glob. Hosp. Tour. 2023, 2, 1–6. Available online: https://digitalcommons.usf.edu/jght/vol2/iss1/1/ (accessed on 10 March 2023).
  44. Kasneci, E.; Seßler, K.; Küchemann, S.; Bannert, M.; Dementieva, D.; Fischer, F.; Gasser, U.; Groh, G.; Günnemann, S.; Hüllermeier, E.; et al. ChatGPT for Good? On Oppor-tunities and Challenges of Large Language Models for Education. Learn. Individ. Differ. 2023, 103, 102274. [Google Scholar] [CrossRef]
  45. Khan, R.A.; Jawaid, M.; Khan, A.R.; Sajjad, M. ChatGPT—Reshaping Medical Education and Clinical Management. Pak. J. Med. Sci. 2023, 39, 605–607. [Google Scholar] [CrossRef]
  46. Baidoo-Anu, D.; Owusu Ansah, L. Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. SSRN 2023, 4337484. Available online: https://ssrn.com/abstract=4337484 (accessed on 10 March 2023).
  47. Bishop, L. A Computer Wrote This Paper: What ChatGPT Means for Education, Research, and Writing. SSRN 2023, 4338981. Available online: https://ssrn.com/abstract=4338981 (accessed on 10 March 2023).
  48. Atlas, S. ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI; University of Rhode Island: Kingston, RI, USA, 2023; Available online: https://digitalcommons.uri.edu/cba_facpubs/548 (accessed on 1 March 2023).
  49. Al-Worafi, Y.M.; Hermansyah, A.; Goh, K.W.; Ming, L.C. Artificial Intelligence Use in University: Should We Ban ChatGPT? Preprints.org 2023. [Google Scholar] [CrossRef]
  50. Cotton, D.R.; Cotton, P.A.; Shipway, J.R. Chatting and Cheating: Ensuring Academic Integrity in the Era of ChatGPT. EdArXiv 2023. [Google Scholar] [CrossRef]
  51. Qadir, J. Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education. TechRxiv 2023. [Google Scholar] [CrossRef]
  52. Rudolph, J.; Tan, S.; Tan, S. ChatGPT: Bullshit Spewer or the End of Traditional Assessments in Higher Education? J. Appl. Learn. Teach. 2023, 6, 1–22. [Google Scholar] [CrossRef]
  53. Pardos, Z.A.; Bhandari, S. Learning Gain Differences between ChatGPT and Human Tutor Generated Algebra Hints. arXiv 2023, arXiv:2302.06871. [Google Scholar]
  54. Pavlik, J.V. Collaborating with ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. J. Mass Commun. Educ. 2023, 78, 84–93. [Google Scholar] [CrossRef]
  55. Zhang, B. Preparing Educators and Students for ChatGPT and AI Technology in Higher Education: Benefits, Limitations, Strategies, and Implications of ChatGPT & AI Technologies. Available online: https://www.researchgate.net/profile/Bo-Zhang-376/publication/367380845_Preparing_Educators_and_Students_for_ChatGPT_and_AI_Technology_in_Higher_EducationBenefits_Limitations_Strategies_and_Implications_of_ChatGPT_AI_Technologies/links/63d06849e922c50e99bd4d09/Preparing-Educators-and-Students-for-ChatGPT-and-AI-Technology-in-Higher-EducationBenefits-Limitations-Strategies-and-Implications-of-ChatGPT-AI-Technologies.pdf (accessed on 1 March 2023). [CrossRef]
  56. Perkins, M. Academic Integrity Considerations of AI Large Language Models in the Post-pandemic Era: ChatGPT and beyond. J. Univ. Teach. Learn. Pract. 2023, 20, 7. [Google Scholar] [CrossRef]
  57. Mbakwe, A.B.; Lourentzou, I.; Celi, L.A.; Mechanic, O.J.; Dagan, A. ChatGPT Passing USMLE Shines a Spotlight on the Flaws of Medical Education. PLoS Digital Health 2023, 2, e0000205. [Google Scholar] [CrossRef]
  58. King, M.R. A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cell. Mol. Bioeng. 2023, 16, 1–2. [Google Scholar] [CrossRef]
  59. Bašić, Ž.; Banovac, A.; Kružić, I.; Jerković, I. Better by You, Better than Me? ChatGPT-3 as Writing Assistance in Students’ Essays. arXiv 2023, arXiv:2302.04536. [Google Scholar]
  60. Ventayen, R.J.M. ChatGPT by OpenAI: Students’ Viewpoint on Cheating Using Artificial Intelligence-Based Application. SSRN 2023, 4361548. Available online: https://ssrn.com/abstract=4361548 (accessed on 10 March 2023).
  61. Khalil, M.; Er, E. Will ChatGPT Get You Caught? Rethinking of Plagiarism Detection. arXiv 2023, arXiv:2302.04335. [Google Scholar]
  62. Ventayen, R.J.M. OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence-Based Contents. SSRN 2023, 4332664. Available online: https://ssrn.com/abstract=4332664 (accessed on 10 March 2023).
  63. Marusenko, R. New Challenges in Assessing Students’ Knowledge: Chatbot ChatGPT and Real-Time Deepfakes. Available online: https://www.researchgate.net/profile/Roman-Marusenko/publication/368450665_New_challenges_in_assessing_students'_knowledge_chatbot_ChatGPT_and_real-time_deepfakes/links/63e815e8e2e1515b6b8bad02/New-challenges-in-assessing-students-knowledge-chatbot-ChatGPT-and-real-time-deepfakes.pdf (accessed on 1 March 2023).
  64. García-Peñalvo, F.J. The Perception of Artificial Intelligence in Educational Contexts after the Launch of ChatGPT: Disruption or Panic? Educ. Knowl. Soc. 2023, 24, 1–9. [Google Scholar] [CrossRef]
  65. Willems, J. ChatGPT at Universities—The Least of Our Concerns. SSRN 2023, 4334162. Available online: https://ssrn.com/abstract=4334162 (accessed on 10 March 2023).
  66. Rospigliosi, P.A. Artificial Intelligence in Teaching and Learning: What Questions Should We Ask of ChatGPT? Interact. Learn. Environ. 2023, 31, 1–3. [Google Scholar] [CrossRef]
  67. Shiri, A. ChatGPT and Academic Integrity. SSRN 2023, 4360052. Available online: https://ssrn.com/abstract=4360052 (accessed on 10 March 2023).
  68. SSRN. Available online: https://www.ssrn.com (accessed on 10 March 2023).
  69. arXiv. Available online: https://arxiv.org (accessed on 10 March 2023).
  70. ResearchGate. Available online: https://www.researchgate.net (accessed on 10 March 2023).
  71. Turnitin. Available online: https://www.turnitin.com (accessed on 10 March 2023).
  72. iThenticate. Available online: https://www.ithenticate.com (accessed on 10 March 2023).
  73. Bloom, B.S. Taxonomy of Educational Objectives: The Classification of Educational Goals; David McKay: New York, NY, USA, 1956. [Google Scholar]
  74. GPT-4. Available online: https://openai.com/research/gpt-4 (accessed on 5 April 2023).
  75. Google Cloud: Dialogflow. Available online: https://cloud.google.com/dialogflow (accessed on 10 March 2023).
  76. Lo, C.K.; Hew, K.F. A Critical Review of Flipped Classroom Challenges in K-12 Education: Possible Solutions and Recommendations for Future Research. Res. Pract. Technol. Enhanc. Learn. 2017, 12, 4. [Google Scholar] [CrossRef]
  77. Hew, K.F.; Bai, S.; Dawson, P.; Lo, C.K. Meta-analyses of Flipped Classroom Studies: A Review of Methodology. Educ. Res. Rev. 2021, 33, 100393. [Google Scholar] [CrossRef]
  78. Lo, C.K.; Hew, K.F. Design Principles for Fully Online Flipped Learning in Health Professions Education: A Systematic Review of Research During the COVID-19 Pandemic. BMC Med. Educ. 2022, 22, 720. [Google Scholar] [CrossRef]
  79. Lo, C.K. Strategies for Enhancing Online Flipped Learning: A Systematic Review of Empirical Studies During the COVID-19 pandemic. Interact. Learn. Environ. 2023, 1–29. [Google Scholar] [CrossRef]
  80. Thorp, H.H. ChatGPT is Fun, But not an Author. Science 2023, 379, 313. [Google Scholar] [CrossRef]
Education 13 00410 g001 550
Figure 1. PRISMA flow diagram of article selection.
Figure 1. PRISMA flow diagram of article selection.
Education 13 00410 g001
Education 13 00410 g002 550
Figure 2. Countries/locations of the included articles (N = 50).
Figure 2. Countries/locations of the included articles (N = 50).
Education 13 00410 g002
Education 13 00410 g003 550
Figure 3. Publication sources of the included articles (N = 50).
Figure 3. Publication sources of the included articles (N = 50).
Education 13 00410 g003
Table
Table 1. Inclusion and exclusion criteria for article selection.
Table 1. Inclusion and exclusion criteria for article selection.
CriterionInclusionExclusion
Article topicDiscuss ChatGPT in the field of educationDo not discuss GhatGPT in the field of education
Article typeAcademic articlesNon-academic articles, such as articles from mass and social media
Time period1 January 2022 to 28 February 2023Articles outside the time period
LanguageEnglishNon-English
Table
Table 2. Performance evaluation of ChatGPT-3.5 across subject domains (from high to low).
Table 2. Performance evaluation of ChatGPT-3.5 across subject domains (from high to low).
Subject DomainStudiesOverall
Performance		Representative Researcher Comments
Critical and higher-order thinking[21]Outstanding“The generated responses can be assessed as being clear in exposition, precise with respect to examples used, relevant to the requests…” [21] (p. 13).
Economics[22]OutstandingIn TUCE, “ChatGPT ranks at the 99th percentile in macroeconomics and the 91st percentile in microeconomics, when compared to [other] students” [22] (p. 7).
Programming[35,36,37]Outstanding to satisfactory“Nearly all answers of ChatGPT were completely correct and nicely explained… the explanations by ChatGPT were amazingly clear and goal-oriented” [35] (p. 1).
“The lecturer graded the assignment submission with a total score of 71 out of 100 points, resulting in a grade of ‘Satisfactory’” [36] (p. 4).
English language comprehension[20]Satisfactory“the mean grade achieved by ChatGPT according to official norms was 7.18, which is similar to the mean grade of all students in the Netherlands” [20] (p. 1).
Law[10,23]Barely satisfactory to unsatisfactory“ChatGPT performed on average at the level of a C+ student, achieving a low but passing grade” [10] (p. 1).
“It performed worst on exams that featured problem-style or ‘issue spotting’ questions” [23] (p. 1)
Medical education[24,25,26,27,28,29,30,31,32]Barely satisfactory to unsatisfactory“ChatGPT yields moderate accuracy approaching passing performance on USMLE” [24] (p. 4).
In American Heart Association BLS and ACLS exams, “ChatGPT did not reach the passing threshold for any of the exams” [26] (p. 109732).
Mathematics[33]Unsatisfactory“ChatGPT’s mathematical abilities are significantly below those of an average mathematics graduate student” [33] (p. 1).
Software testing[38]Unsatisfactory“ChatGPT is not able, by itself, to pass a software testing course. In total, ChatGPT was able to correctly answer 37.5% of the questions we posed” [38] (p. 8).
Sports science and psychology[39]Unsatisfactory“ChatGPT gave several correct answers to 20 questions but failed the test with [a score of] 45%” [39] (p. 2).
MCQ-based exams across subjects[34]Unsatisfactory“ChatGPT fails to meet the passing grade on almost every MCQ exam that it is tested against, and performs significantly worse than the average human student” [34] (p. 1).
ACLS = Advanced Cardiovascular Life Support; BLS = Basic Life Support; MCQ = Multiple-choice questions; TUCE = Test of Understanding in College Economics; USMLE = United States Medical Licensing Examination.
Table
Table 3. ChatGPT functions to support instructors in their teaching routines.
Table 3. ChatGPT functions to support instructors in their teaching routines.
AspectFunctionRepresentative QuotesOther
Representative Support
Teaching preparationGenerating course materials“we asked ChatGPT to create one of the dialogues above in a format that can be imported into DialogFlow. ChatGPT was able to generate it successfully” [42] (p. 37).[43,44,45]
Providing suggestions“I told ChatGPT that the learner was with dyslexia, and eventually ChatGPT recommended specific learning materials for the learner” [41] (p. 1).[8,27,37]
Performing language translation“ChatGPT can be used to translate educational materials into different languages” [46] (p. 8).[45,47,48]
AssessmentGenerating assessment tasks“Another use of ChatGPT is its ability to generate exercises, quizzes, and scenarios which can be used in the classroom to aid in practice and assessment” [29] (p. 5).[27,45,49]
Evaluating student performance“ChatGPT can be trained to grade student essays, providing teachers with more time to focus on other aspects of teaching” [29] (p. 8).[46,50,51]
Table
Table 4. ChatGPT functions to support student learning.
Table 4. ChatGPT functions to support student learning.
AspectFunctionRepresentative QuotesOther
Representative Support
LearningAnswering questions“The answer[s] given by ChatGPT were relevant and accurate… The tool can used as quick reference and self-studying instrument” [28] (p. 1).[25,46,53]
Summarising information“it is very good at processing information, distilling it, and presenting it verbally” [54] (p. 92).[23,29,44]
Facilitating collaboration“The ChatGPT application has the potential to serve as a means of generating different scenarios for students to work together to solve problems and achieve goals” [52] (p. 13).[25,44,50]
AssessmentConcept checking and exam preparation“ChatGPT has shown promising results in becoming a powerful reference and self-learning tool for preparing for the life support exams” [26] (p. 1).[10,28,32]
Drafting assistance“students could be encouraged to use AI to give a ‘first draft’ answer, and then asked to improve upon that answer manually” [23] (p. 21).[10,25,55]
Providing feedback“ChatGPT could be used to grade assignments and provide feedback to students in real-time” [50] (p. 2).[46,48,56]
Table
Table 5. Major potential issues associated with ChatGPT.
Table 5. Major potential issues associated with ChatGPT.
AspectIssuesRepresentative quotesOther
Representative Support
Accuracy and reliabilityRelying on biased data“These biases stem from research performed in high-income countries and textbooks [i.e., the training data of ChatGPT]” [57] (p. 2).[40,46,54]
Having limited up-to-date knowledge“ChatGPT has no idea of the world after 2021 and hence it could not add any references or information after 2021” [46] (p. 14).[25,45,56]
Generating incorrect/fake information“ChatGPT included a make-up article which does not exist and even provided full bibliographic details of the article with a non-functional URL” [46] (p. 14).[32,38,58]
Plagiarism preventionStudent plagiarism“Our experimental group [with ChatGPT support] had slightly more problems with plagiarism than the control group [without ChatGPT support]” [59] (p. 7).[40,51,60]
Bypassing plagiarism detectors“Of the 50 essays inspected, the plagiarism-detection software considered 40 of them with a high level of originality, as evidenced by a similarity score of 20% or less” [61] (p. 10).[36,52,62]
Table
Table 6. Strategies in response to plagiarism issues created by ChatGPT.
Table 6. Strategies in response to plagiarism issues created by ChatGPT.
AspectStrategiesRepresentative QuotesOther
Representative Support
Task designIncorporating multimedia resources“Embedding images to exam questions can make it more difficult for students to cheat and for ChatGPT to generate accurate responses” [21] (p. 16).[22,34,52]
Adopting novel question types“[Instructors] might also reconsider the types of questions they pose to students, focusing on those that require analysis rather than those that simply require [a] recall of legal rules” [10] (p. 12).[8,22,63]
Employing digital-free assessment formats“blanket solution would be to make all assessments of the ‘in-class’ variety… it will be essentially impossible for ChatGPT to be leveraged in an unscrupulous fashion” [23] (p. 19).[21,31,36]
Identification of AI writingUsing AI-based writing detection tools“Plagiarism detectors could not identify the AI-originated text, but AI detectors did” [39] (p. 2).[21,55,56]
Checking references“Although in text citations and references were provided…, these were all fabricated… This does open a potential avenue of detection by academic staff” [56] (p. 5).[39,50,52]
Institutional policyEstablishing anti-plagiarism guidelines“Administrations should consider how to reshape honor codes to regulate the use of language models in general” [10] (p. 12).[52,56,61]
Providing student education“Our recommendations for students are to… be aware of academic integrity policies and understand the consequences of academic misconduct… provide training on academic integrity for students” [52] (pp. 14–15).[54,64,65]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lo, C.K. What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Educ. Sci. 2023, 13, 410. https://doi.org/10.3390/educsci13040410

AMA Style

Lo CK. What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Education Sciences. 2023; 13(4):410. https://doi.org/10.3390/educsci13040410

Chicago/Turabian Style

Lo, Chung Kwan. 2023. "What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature" Education Sciences 13, no. 4: 410. https://doi.org/10.3390/educsci13040410

APA Style

Lo, C. K. (2023). What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop