/prod01/prodbucket01/media/durham-university/departments-/physics/pexels-photo.jpeg)
The Death of the Short-Form Physics Essay in the Coming AI Revolution
How smart are the latest AI chatbots? In late 2022, OpenAI released ChatGPT to the world. This freely available AI-powered chatbot could generate detailed answers on more or less any subject within seconds. It didn’t take long for people to notice its potential use as an essay-writing tool. However, how good really is this text generation technology? Could it pass say, a university level module?
Our research
In a recent paper, researchers in the Physics Education and Scholarship (PhES) group sought to answer this question. Using OpenAI’s davinci-003 text model, a series of submissions to an essay assignment used in the Physics in Society module were generated. These submissions were then marked by PhES researchers finding that, on average, they would be awarded 71 ± 2 % – enough for a first-class degree categorisation! Further, the plagiarism detection software TurnItIn and Grammerly failed to raise issue with the submissions with them both finding < 10 % of the work to potentially be plagiarized.
Check out the full paper here:
https://arxiv.org/abs/2212.11661
The Death of the Short-Form Physics Essay in the Coming AI Revolution - arxiv.org
Results
The results show that the latest AI text completion models can write high quality essays on complex topics capable of passing university assignments with ease. Further, they are potentially very hard to identify as AI written. This poses a clear challenge for authentic assessment. These technologies could potentially be used to write entire essays using a prompt as simple as ‘Write an essay on…’ which are then submitted as student-written work.
Where next?
Are short form essays still a viable form of assessment? Potentially, but they must adapt as this AI revolution shows no sign of stopping. With the Google owned Deepmind chatbot Sparrow muted to be entering private beta and OpenAI announcing GPT-4 educators must adjust their practice soon to account for the availability of these tools.