How AI Detection Actually Works

AI detectors are sold with a lot of confidence. It helps to know what's going on inside them, because it explains both why they work reasonably well on raw model output and why they sometimes get it badly wrong.

The core idea: predictability

A language model writes by repeatedly choosing a likely next word. So text a model wrote tends to be text a model finds unsurprising. Detectors exploit this. Perplexity is the name for how surprised a model is by a passage: low perplexity means very predictable. Burstiness describes how much that predictability varies from sentence to sentence. People tend to swing between plain and surprising sentences; models are steadier.

Modern detectors go further

Early tools leaned almost entirely on those two numbers. Current products such as GPTZero, Turnitin and Originality.ai train classifiers on large collections of human and AI writing and combine many signals. GPTZero, for example, has said its main detector has been a deep-learning model since late 2023, with perplexity and burstiness as inputs rather than the whole story.

Why they fail

Every detector outputs a probability, and every one has false positives. Short texts are hard. Formulaic human writing (lab reports, legal summaries, answers to common questions) looks predictable. Writing by non-native English speakers is flagged far more than it should be; a 2023 Stanford study found several detectors misclassified more than half of a set of TOEFL essays as AI-written.

Even OpenAI gave up on its own classifier. It launched one in early 2023, reported that it caught only 26% of AI-written text while wrongly flagging 9% of human text, and withdrew it that July for low accuracy.

What this means for you

Treat a detector score as a hint, not a verdict, whether it's about your writing or someone else's. And if you're editing AI drafts, the changes that make text better for readers (varied rhythm, specific detail, plain words) are the same ones that make it read as human. That's the approach WriteHumane is built on.