Logo image
Sieving Coding Assignments Over Submissions Generated by AI and Novice Programmers
Conference proceeding

Sieving Coding Assignments Over Submissions Generated by AI and Novice Programmers

Cyrille Jegourel, Jung Yi Ong, Oka Kurniawan, Lim Meng Shin, Kushat Chitluru, ASSOC COMPUTING MACHINERY and Cyrille Pierre Joseph Jegourel
Proceedings of the 24th Koli Calling International Conference on Computing Education Research, pp.1-11
ACM Other Conferences
Koli Calling '24: 24th Koli Calling International Conference on Computing Education Research
12/11/2024

Abstract

Applied computing -- Computer forensics Applied computing -- Education Computing methodologies -- Artificial intelligence Computing methodologies -- Machine learning Information systems -- Clustering Software and its engineering -- Software verification and validation
In the era of AI tools like ChatGPT and GitHub Copilot, and with the numerous online resources, computer science education faces the challenge of students potentially submitting plagiarised coding assignments or assignments generated by these technologies. Distinguishing between AI-generated and human-written text is notoriously difficult. In this study, we applied two text distance algorithms, commonly used for machine translation and document comparisons, to detect similarities between various computer Python code submissions and employed hierarchical clustering to analyze them from both AI tools and human programmers. Our results indicate that the distances to the cluster representatives can effectively predict whether a code submission is generated by AI or by novice programmers, achieving an accuracy of over 90%. These findings demonstrate the significant potential of text distance algorithms in identifying the origin of coding submissions, whether generated by AI or by novice programmers.

Metrics

1 Record Views

Details

Logo image