Agency abandons checking exams using anti-plagiarism software

The Education and Youth Board started using unreliable AI-based plagiarism detection software to check exams this year. Following the public revelation of this issue, the board is discontinuing the practice.
This year, over 10,000 young people took national exams. The Education and Youth Board (Harno), responsible for grading, suspected the use of external assistance in ten cases, two of which involved the use of artificial intelligence.
"When different parts of the exam are so contradictory that one part is written at a B1 level and another at a C1 level, there are justified suspicions," said Auli Udde, head of Harno's test and research administration office.
This was precisely the case with a student from Saaremaa High School, who was suspected of using artificial intelligence. To evaluate their work, AI-based plagiarism detection software was also used, resulting in a score of zero.
According to Rasmus Kits, project manager at Tallinn University of Technology's AI center of excellence, the AI-based software currently in use is not reliable enough.
"We have not thoroughly studied or tested these tools in laboratory conditions. We also do not have agreements in the education sector to use such tools. I would be very cautious about using such tools because we cannot yet confirm or guarantee their reliability," Kits said.
"AI often relies on large datasets and probabilities. And probability is never 100 percent. Therefore, we can never fully assert that a result analyzed by an AI-based system is exactly true," Kits added.
According to the Ministry of Education, Harno started using plagiarism detection software on its own initiative.
"Harno did not receive any specific directive. According to scientists, there is no plagiarism detection software that can 100 percent determine whether external or AI assistance was used. The ministry's appeals committee concluded that the use of the software was not justified and that evaluating the student's work with a score of zero was not justified," said Liina Põld, deputy secretary general of the Ministry of Education and Research.
But if it is known that these programs cannot reliably determine whether a person used AI or not, why were they used at all?
"To achieve the most fair and accurate result possible," responded Auli Udde, head of Harno's test and research administration office.
The Education and Youth Board stated that AI has not been used for grading exam results in the past and will not continue this practice in the future.
--
Follow ERR News on Facebook and Twitter and never miss an update!
Editor: Marko Tooming, Marcus Turovski