Journal of Data Science

The rapid development of artificial intelligence (AI) tools, particularly generative models, has introduced significant challenges in academic assessment. Students increasingly rely on AI to complete assignments, complicating the evaluation of their true understanding and effort. This paper examines the limitations of AI detection tools, the inadequacies of traditional teaching methods in this context, and the potential for responsibly integrating AI into educational practices. Drawing on insights from educators and recent developments in AI, the paper proposes strategies for adapting assessment methods to ensure academic integrity while embracing technological advancements. The findings underscore the need for a balanced approach that leverages AI’s benefits while mitigating its risks.

Many believe that use of generative AI as a private tutor has the potential to shrink access and achievement gaps between students and schools with abundant resources versus those with fewer resources. Shrinking the gap is possible only if paid and free versions of the platforms perform with the same accuracy. In this experiment, we investigate the performance of GPT versions 3.5, 4.0, and 4o-mini on the same 16-question statistics exam given to a class of first-year graduate students. While we do not advocate using any generative AI platform to complete an exam, the use of exam questions allows us to explore aspects of ChatGPT’s responses to typical questions that students might encounter in a statistics course. Results on accuracy indicate that GPT 3.5 would fail the exam, GPT4 would perform well, and GPT4o-mini would perform somewhere in between. While we acknowledge the existence of other Generative AI/LLMs, our discussion concerns only ChatGPT because it is the most widely used platform on college campuses at this time. We further investigate differences among the AI platforms in the answers for each problem using methods developed for text analytics, such as reading level evaluation and topic modeling. Results indicate that GPT3.5 and 4o-mini have characteristics that are more similar than either of them have with GPT4.