Artificial Intelligence (AI) Detection in Turnitin

This helpsheet is designed for academic staff to understand how to access and interpret the new ‘AI Writing Indicator’ that has been released by Turnitin. This feature is available at the University of Lincoln from April 2023.

In response to the recent emergence of AI tools (such as ChatGPT) capable of generating convincing human-written text, Turnitin have now released a new AI detection tool within the Feedback Studio. At this point, this new tool should only be used as an indicator and contributing evidence as part of a wider investigation and not seen as proof of use alone.

The Digital Education team have produced some online assessment guidance relating to the use of Artificial Intelligence, you can find out more about our ‘Promote Prevent Detect’ process that can be employed to support in mitigating the misuse of AI. This guidance is available on the Digital Education blog: Artificial intelligence in assessment – Digital Education Support (lincoln.ac.uk)

Accessing the ‘AI Writing Indicator’

AI detection is automatically enabled on all assignments created with the Turnitin LTI which was introduced to the University of Lincoln in 21/22 Academic Year.

In order to access the AI Writing Indicator for a student’s assignment simply open the paper in the Feedback Studio in the same way that you would if you were marking and providing feedback.

The report icon is displayed on the right-hand toolbar, and the icon can be selected to open up a more detailed analysis of the submission.

In addition to a score, which indicates the amount of qualifying text within the submission that Turnitin’s AI writing detection model determines was generated by AI, you will also see text highlighted informing you which parts of the document this percentage score is referring to. This is similar to the Similarity Report which highlights text that is taken from other sources.

Interpreting the ‘AI Writing Indicator’

The following definition provided by Turnitin will allow you to understand what the AI Writing Indicator is telling you about your student’s submission:

“The percentage shown in the AI writing detection indicator and in the AI writing report is the amount of qualifying text within the submission that Turnitin’s AI writing detection model determines was generated by AI (with 98% confidence based on data that was collected and verified in a controlled lab environment).“

After accessing the Feedback Studio, you can click on the AI Writing Indicator icon (regardless of what percentage is displayed) to open a new tab in your Web Browser, which will allow you to access the detailed report, in which the suspected AI-written text will be highlighted to draw your attention to it. The image below shows an example of this report.

If you find that a student on your module has flagged as potentially having used AI to write all or part of their assignment, we strongly recommend that you use this new service as only one part of your evidence gathering, and use the new AI indicator as additional supporting information only. You should continue to use your own experience and knowledge, such as accurate referencing or familiarity with a student’s ability, writing style or a lack of critical analysis, as indicators to inform whether to undertake further investigation.

Frequently Asked Questions (FAQs)

We have provided a short list of frequently asked questions below, however if you would like further detailed information, you can access Turnitin’s own list of questions for academic staff, or you can reach out to the Digital Education Team.

Turnitin’s guidance can be accessed on the following link: AI Writing Detection | Turnitin

Will the AI Report be visible to my students?

The Turnitin AI Report is only visible to users who are enrolled on the Blackboard module in a staff capacity such as instructors or administrators. Unlike the Similarity Report which can be made visible to students, it will not be possible to make this report available to students.

Will this AI Report be available on previous assignments and submissions?

The Turnitin AI Report will only generate for submissions that were made after the launch of the service which occurred in the late-afternoon of Tuesday the 4th April 2023. If there are papers that have already been submitted by students that you would like to report on, you will need to re-upload it to the assignment inbox, ensuring that you record the original submission date and time in the title of this submission, as doing so would mark it as late and may cause the student to be wrongly penalised.

Has this AI Report replaced the Similarity Report?

The AI Report has not replaced the Similarity Report. These reports are completely independent of each other and can be reviewed separately.

Will the AI Report tell me which AI tools or AI websites have been used?

At this time, the AI Report will only inform you of the percentage of the assignment that it believes has been generated by AI, as well as which parts of the assignment have been flagged in the AI detection process.

How should I interpret a low percentage score on my AI Report?

Try to be mindful of false positives. Turnitin says that the AI Report has a less than 1% false positive rating (incorrectly marking human written work as AI written). However, false positives are a possibility. Even with a very low false positive percentage, when you have thousands of students the chances of this happening increases. Turnitin have identified that low scores are more likely to be false positives and have now added a pop-up when you hover over the percentage to indicate this:

Also, there will also be some slight variance either way from the score given. Testing from the team at Digital Education alongside several other institutions has suggested the report is reliable at detecting obvious examples of AI use, accurately identifying AI generated reports.

We encourage you however, to use the indicator as only one piece of evidence in a wider investigation, if necessary, we recommend that you use the full range of tools available to you and your wider knowledge of your students to draw your conclusions.