The inquiry topic: How generative AI affect education?
The topic is very new. (Most papers were published in 2023, which is expected for ChatGPT rolled out in November 2022 only, there is not enough time for in depth research yet.)
Therefore, I widened the topic from working on students' learning in Science to education in general.
In today's class, I have done all the literature review, 5 academic journal and 3 blogs. The latex code of the references are at the end of this blog.
The direction currently is to review the effectiveness of assistance from AI in teaching, for example helping to make teaching material and act as a tutor. There are some concerns about integrity too, however, in the current state, generative AI is not yet able to produce highly sophisticated answers to homework or tests.
The reflection on the quotes will be done done the line as the project proceed.
I also have interviewed my School Advisor and some of the students on generative AI already. Again, I just need to write out and paraphrase and give my own reflection on them.
In the future, I will also need to make presentation slides and plan an in class activity. My current idea for the in class activity is to have everyone play with ChatGPT in class. We will try gathering some Math/Phy question in class then prompt it to chatGPT to answer those questions. We will evaluate the effectiveness in class as an experiment.
@article{Perkins2023,
abstract = {This paper explores the academic integrity considerations of students’ use of Artificial Intelligence (AI) tools using Large Language Models (LLMs) such as ChatGPT in formal assessments. We examine the evolution of these tools, and highlight the potential ways that LLMs can support in the education of students in digital writing and beyond, including the teaching of writing and composition, the possibilities of co-creation between humans and AI, supporting EFL learners, and improving Automated Writing Evaluations (AWE). We describe and demonstrate the potential that these tools have in creating original, coherent text that can avoid detection by existing technological methods of detection and trained academic staff alike, demonstrating a major academic integrity concern related to the use of these tools by students. Analysing the various issues related to academic integrity that LLMs raise for both Higher Education Institutions (HEIs) and students, we conclude that it is not the student use of any AI tools that defines whether plagiarism or a breach of academic integrity has occurred, but whether any use is made clear by the student. Deciding whether any particular use of LLMs by students can be defined as academic misconduct is determined by the academic integrity policies of any given HEI, which must be updated to consider how these tools will be used in future educational environments. Practitioner Notes 1. Students now have easy access to advanced Artificial Intelligence based tools such as ChatGPT. These tools use Large Language Models (LLMs) and can be used to create original written content that students may use in their assessments. 2. These tools can be accessed using commercial services built on this software, often targeted to students as a means of ‘assisting’ students with assessments. 3. The output created by these LLMs is coherent enough for it not to be detected by academic staff members, or traditional text-matching software used to detect plagiarism, but falsified references may hint at their use if unchanged by students. 4. The use of these tools may not necessarily be considered as plagiarism if students are transparent in how they have been used in any submission, however it may be a breach of academic integrity policies of any given Higher Education Institution (HEI). 5. There are legitimate uses of these tools in supporting the education of students, meaning HEIs must carefully consider how policies dealing with student use of this software are created.},
author = {Mike Perkins},
doi = {10.53761/1.20.02.07},
issn = {14499789},
issue = {2},
journal = {Journal of University Teaching and Learning Practice},
title = {Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond},
volume = {20},
year = {2023},
}
@article{Khan2023,
abstract = {Artificial Intelligence is no more the talk of the fiction read in novels or seen in movies. It has been making inroads slowly and gradually in medical education and clinical management of patients apart from all other walks of life. Recently, chatbots particularly ChatGPT, were developed and trained, using a huge amount of textual data from the internet. This has made a significant impact on our approach in medical science. Though there are benefits of this new technology, a lot of caution is required for its use.},
author = {Rehan Ahmed Khan and Masood Jawaid and Aymen Rehan Khan and Madiha Sajjad},
doi = {10.12669/pjms.39.2.7653},
issn = {1682024X},
issue = {2},
journal = {Pakistan Journal of Medical Sciences},
title = {ChatGPT-Reshaping medical education and clinical management},
volume = {39},
year = {2023},
}
@article{Frieder2023,
abstract = {We investigate the mathematical capabilities of two iterations of ChatGPT (released 9-January-2023 and 30-January-2023) and of GPT-4 by testing them on publicly available datasets, as well as hand-crafted ones, using a novel methodology. In contrast to formal mathematics, where large databases of formal proofs are available (e.g., the Lean Mathematical Library), current datasets of natural-language mathematics, used to benchmark language models, either cover only elementary mathematics or are very small. We address this by publicly releasing two new datasets: GHOSTS and miniGHOSTS. These are the first natural-language datasets curated by working researchers in mathematics that (1) aim to cover graduate-level mathematics, (2) provide a holistic overview of the mathematical capabilities of language models, and (3) distinguish multiple dimensions of mathematical reasoning. These datasets also test whether ChatGPT and GPT-4 can be helpful assistants to professional mathematicians by emulating use cases that arise in the daily professional activities of mathematicians. We benchmark the models on a range of fine-grained performance metrics. For advanced mathematics, this is the most detailed evaluation effort to date. We find that ChatGPT can be used most successfully as a mathematical assistant for querying facts, acting as a mathematical search engine and knowledge base interface. GPT-4 can additionally be used for undergraduate-level mathematics but fails on graduate-level difficulty. Contrary to many positive reports in the media about GPT-4 and ChatGPT's exam-solving abilities (a potential case of selection bias), their overall mathematical performance is well below the level of a graduate student. Hence, if your goal is to use ChatGPT to pass a graduate-level math exam, you would be better off copying from your average peer!},
author = {Simon Frieder and Luca Pinchetti and Alexis Chevalier and Ryan-Rhys Griffiths and Tommaso Salvatori and Thomas Lukasiewicz and Philipp Christian Petersen and Julius Berner},
month = {1},
title = {Mathematical Capabilities of ChatGPT},
url = {http://arxiv.org/abs/2301.13867},
year = {2023},
}
@article{Megahed2023,
abstract = {Generative Artificial Intelligence (AI) models such as OpenAI’s ChatGPT have the potential to revolutionize Statistical Process Control (SPC) practice, learning, and research. However, these tools are in the early stages of development and can be easily misused or misunderstood. In this paper, we give an overview of the development of Generative AI. Specifically, we explore ChatGPT’s ability to provide code, explain basic concepts, and create knowledge related to SPC practice, learning, and research. By investigating responses to structured prompts, we highlight the benefits and limitations of the results. Our study indicates that the current version of ChatGPT performs well for structured tasks, such as translating code from one language to another and explaining well-known concepts but struggles with more nuanced tasks, such as explaining less widely known terms and creating code from scratch. We find that using new AI tools may help practitioners, educators, and researchers to be more efficient and productive. However, in their current stages of development, some results are misleading and wrong. Overall, the use of generative AI models in SPC must be properly validated and used in conjunction with other methods to ensure accurate results.},
author = {Fadel M. Megahed and Ying Ju Chen and Joshua A. Ferris and Sven Knoth and L. Allison Jones-Farmer},
doi = {10.1080/08982112.2023.2206479},
issn = {15324222},
journal = {Quality Engineering},
title = {How generative AI models such as ChatGPT can be (mis)used in SPC practice, education, and research? An exploratory study},
year = {2023},
}
@misc{Lo2023,
abstract = {An artificial intelligence-based chatbot, ChatGPT, was launched in November 2022 and is capable of generating cohesive and informative human-like responses to user input. This rapid review of the literature aims to enrich our understanding of ChatGPT’s capabilities across subject domains, how it can be used in education, and potential issues raised by researchers during the first three months of its release (i.e., December 2022 to February 2023). A search of the relevant databases and Google Scholar yielded 50 articles for content analysis (i.e., open coding, axial coding, and selective coding). The findings of this review suggest that ChatGPT’s performance varied across subject domains, ranging from outstanding (e.g., economics) and satisfactory (e.g., programming) to unsatisfactory (e.g., mathematics). Although ChatGPT has the potential to serve as an assistant for instructors (e.g., to generate course materials and provide suggestions) and a virtual tutor for students (e.g., to answer questions and facilitate collaboration), there were challenges associated with its use (e.g., generating incorrect or fake information and bypassing plagiarism detectors). Immediate action should be taken to update the assessment methods and institutional policies in schools and universities. Instructor training and student education are also essential to respond to the impact of ChatGPT on the educational environment.},
author = {Chung Kwan Lo},
doi = {10.3390/educsci13040410},
issn = {22277102},
issue = {4},
journal = {Education Sciences},
title = {What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature},
volume = {13},
year = {2023},
}