Why AI Detectors Flag Human Writing as Machine-Made

AI detectors flag human writing because they do not detect AI at all. They estimate how predictable your words are and treat predictable text as machine-made. Plain vocabulary, tidy grammar and common phrasing all look "AI-like" to that math. That is why careful writers, students and non-native English speakers get flagged for work they wrote themselves.

I read the research on this so you do not have to, and one finding keeps nagging at me. The same quality a writing teacher praises, clear and simple sentences, is the quality an AI detector punishes. That is not a glitch in one product. It is baked into how many of these tools work.

This feature is for anyone who has had a paper, a cover letter or a blog post come back with a scary percentage. It explains what the score measures, who gets caught by it, and how regulators have responded.

What Does an AI Detector Actually Measure?

An AI detector measures how statistically likely your text is, not who wrote it. Many tools lean on a score called perplexity. A language model reads your sentence and asks how surprised it is by each next word. Low surprise means low perplexity, and low perplexity gets labeled machine-written.

That logic has a surface appeal. Chatbots pick likely words by design, so their output tends to be smooth and predictable. The trouble is that plenty of humans write smooth, predictable text too. Instruction manuals, lab reports, legal letters and second-language essays all reward plain word choice.

Notice what is missing from the measurement. The tool never sees your drafts, your notes or your keyboard. It sees only the finished words and makes a guess about their origin. I would not call that detection. It is closer to a style assessment wearing a lab coat.

The Stanford Study That Changed the Conversation

The clearest evidence came from a team of Stanford University computer scientists led by Weixin Liang and James Zou. Their paper, "GPT detectors are biased against non-native English writers," appeared in the journal Patterns on July 10, 2023. It is archived by the National Library of Medicine’s PubMed Central.

The setup was simple. The researchers ran seven widely used detectors on 91 essays written for the TOEFL, a standard English test for non-native speakers. They compared them with 88 essays by US eighth graders. Every essay was written by a person.

The results were lopsided.

The flagged TOEFL essays had noticeably lower perplexity. In other words, the writers used common words in common patterns, which is exactly what you would expect from someone writing carefully in a second language.

Fancy Words Fool the Detector Both Ways

The Stanford team then ran the experiment I find most telling. They used ChatGPT to "enhance the word choices" of the TOEFL essays so they sounded more like a native speaker. The average false-positive rate dropped from 61.3% to 11.6%. Same ideas, same human author, just fancier vocabulary.

They also went the other way. Simplifying the word choices in the eighth-grade essays made the detectors flag them far more often. The tool was rewarding vocabulary, not honesty.

The final test is the one cheaters will like. The researchers had ChatGPT write college admission essays, which detectors caught at first. Then they asked it to "elevate the provided text by employing literary language." Detection rates fell to near zero. A single extra prompt was enough.

Put those results together and the problem is plain. The writers most likely to be falsely accused are the ones with the fewest words to spare. The people most able to dodge the tool are the ones willing to ask a chatbot for a polish.

Broader Testing Found the Same Weakness

One study can be a fluke, so I looked for others. A group of European academic-integrity researchers led by Debora Weber-Wulff tested 12 publicly available tools plus two commercial systems, Turnitin and PlagiarismCheck. Their paper, "Testing of Detection Tools for AI-Generated Text," was published in the International Journal for Educational Integrity in 2023.

Their verdict was blunt. The tools were "neither accurate nor reliable." They also leaned toward calling text human-written, so they missed a lot of real AI output. Light disguise, such as machine translation or paraphrasing, made performance significantly worse.

So the errors run in both directions. Some human writing gets flagged. Plenty of AI writing slips through. A score in either direction tells you less than its confident percentage suggests.

When a 98% Claim Met the FTC

Marketing pages tell a different story from the research. In our experience reading them, accuracy figures almost never come with a public test set or a note on what kind of text was tested.

One case shows why that matters. The Federal Trade Commission (FTC) said a company called Workado advertised its AI Content Detector as "98 percent" accurate. According to the FTC’s April 28, 2025 announcement, independent testing put its accuracy on general-purpose content at 53 percent. "The product did no better than a coin toss," said Chris Mufarrige, director of the FTC’s Bureau of Consumer Protection.

The FTC also alleged a mismatch in the training data. Workado said its model learned from a wide range of material, including blog posts and Wikipedia. The agency said it was trained or fine-tuned to classify only academic content. The commission approved a final order on August 28, 2025, by a 3-0 vote. Workado now needs reliable evidence before making accuracy claims.

Is There a Better Way to Spot AI Text?

There are better approaches, but none of them is a magic scanner. The most promising ideas label content where it is made, rather than guessing after the fact. Watermarks hidden in AI output and provenance records that track where a file came from are two examples. Both depend on the AI companies taking part.

The National Institute of Standards and Technology (NIST) surveyed these options in a November 20, 2024 report, "Reducing Risks Posed by Synthetic Content" (NIST AI 100-4). It covers provenance tracking, watermarking and detection side by side, along with how to test detection software. Detection is treated as one tool in a kit, not a verdict machine.

The idea mirrors how security works elsewhere online. Our explainer on why a passkey won’t work on a fake website shows the stronger pattern. A rule checked at the source beats a guess made by looking at the surface.

Why AI Detector Scores Still Get Trusted

Given all this, it is fair to ask why schools and employers still use these tools. The honest answer is that a number feels objective. A teacher with a tall stack of essays to grade wants a quick filter, and a percentage on a screen looks like data.

The Stanford authors flagged this directly. They wrote that claims of "99% accuracy" are often taken at face value, even though most detectors are closed products. Outsiders usually cannot see the test data, the model or the training material. That makes it hard for anyone to check the claim before relying on it.

There is also a quiet asymmetry in who pays for a mistake. When the tool misses AI text, nobody notices. When it flags a human, a real person has to defend work they did honestly. I think that imbalance should make institutions far more careful than they often are.

None of this means AI-assisted cheating is imaginary. It plainly happens, and teachers are right to care. The point is narrower. A statistical guess about word choice is a weak basis for a serious accusation, and the people most exposed to that weakness are the ones least able to argue back.

An Experiment You Can Try With Your Own Writing

You can test the tools yourself in about ten minutes. Pick something you wrote before late 2022, when ChatGPT launched, so you know no chatbot touched it. Then run it through two or three free detectors.

If your scores swing around, you have seen the perplexity effect firsthand. It is a strange feeling to have your own old email called machine-made. It is also a good cure for trusting the number.

What This Means If You Are Flagged

A flag from an AI detector is a hint, not evidence, and that is still true as of October 2026. If you write plainly, in a second language or in a technical field, you are more likely to trip it through no fault of your own. Keep drafts and version history, ask which tool was used, and point to the published research.

If you are the one reading the score, treat it as a reason to talk with the writer, not to accuse them. For more on the odd ways these systems behave, browse our AI Oddities section.