Cyberpediaby Brain

AI text detectors: why their results are not enough

An AI text detector gives a score, not proof: what research says about its errors, from false positives to paraphrase attacks.

On this page
  1. 1In 30 seconds
  2. 2What it is
  3. 3Straight answers
  4. 4How it works
  5. 5What you do
  6. 6Pitfalls
  7. 7Sources
  8. 8Cite this article

In 30 seconds

Where it happens
Several of the studies cited deal with student texts and academic manuscripts, where these detectors are used[4][5].
What the score is worth
A mathematical analysis concludes that a detector that relies only on the text and has useful detection power necessarily produces false accusations, at a rate tied to the overlap between student writing and AI output[5].
The reflex
Do not make a score the sole proof, because, according to the mathematical analysis, a useful detector that judges only the text necessarily produces false accusations[5].

What it is

An AI text detector gives a score, and that score can be wrong in both directions. Researchers found that these tools often flag as AI-generated a text written by a human and then lightly polished with AI tools[1]. In an attack tested by researchers, repeated paraphrasing can sharply lower detection rates[2]. When texts change domain or generator, classifiers that perform very well on their original domain degrade sharply[3].

False positive rates on human-written texts vary widely from one detector to another[4]. The author of a mathematical analysis argues that a detection score should not serve as the sole evidence in a disciplinary procedure[5]. This article explains these limits to someone who receives a score or has to read one.

Straight answers

Can a detector prove that a text comes from an AI?

According to a researcher's mathematical analysis[5], a useful detector that judges only the text, in a single pass, necessarily produces false accusations when evaluating students. This limit comes from the diversity of the people who write. It does not depend on the quality of the AI model and cannot be fixed by better detector engineering or by technology[5]. The author therefore argues that a score should not be the sole evidence in a disciplinary procedure[5].

Can a human-written text be flagged by mistake?

Yes. Researchers evaluated twelve detectors on 14.7 thousand samples polished to varying degrees of AI involvement: the detectors frequently flag even very lightly polished texts as AI-generated[1]. The authors note that this kind of classification can lead to false accusations of plagiarism[1].

Is a text lightly polished by an AI "AI-generated"?

Detectors struggle to tell the degrees of AI involvement apart, according to the same study[1]. It adds that they show biases against older and smaller models[1].

Can a detector be fooled?

Researchers tested a recursive paraphrasing attack on passages of about 300 tokens, against watermarking, neural-network, zero-shot classification and retrieval-based detectors[2]. This attack can significantly reduce detection rates, while degrading text quality only slightly in many cases[2]. They also studied spoofing attacks on watermarked models, aimed at getting a human text classified as written by an AI: an attacker can infer the hidden signatures of an AI text without full access to the detector[2].

Does a detector stay reliable outside the test setting?

Not necessarily. A team trained a model on 30 linguistic features: on two reference corpora, it reaches an F1 score of 0.9734[3]. But an evaluation on other domains and other generators reveals a major failure to generalize[3]. The features that matter most change a great deal from one corpus to another, which indicates that detectors often rely on style cues specific to a dataset rather than on stable signals of machine writing[3].

Do results change from one detector to another?

A great deal. Across 135,389 document pairs (manuscripts by non-native writers and versions corrected by native speakers) run through 13 detectors, false positive rates on human texts range from 0.0% to 100.0%[4]. The same corrections raise the score of some detectors and lower it for others[4]. Score changes are correlated with the extent of the correction[4].

How it works

A detector assigns a score to a text. Detectors differ in method: watermarking, neural network, zero-shot classifier or retrieval-based search[2]. Researchers also distinguish figures obtained on a benchmark from reliability in real conditions: many systems show high accuracy on a benchmark, but their reliability in real conditions remains uncertain[3].

When grading student work, the examiner generally does not know each student's own way of writing[5]. According to the mathematical analysis, this situation is one of the reasons that make false accusations unavoidable for a useful detector[5].

What you do

  • You receive a score: it should not be the sole evidence in a disciplinary procedure[5].
  • You read two different scores: the same corrections raise the score of some detectors and lower it for others, and false positive rates on human texts range from 0.0% to 100.0% depending on the detector[4].
  • You want to know whether a proofread or corrected text is human: detectors struggle to tell the degrees of AI involvement apart[1], and the style of a professional correction influences their scores[4].

Pitfalls

  • Trusting a benchmark figure: a detector that performs well on one corpus degrades sharply when the domain or the generator changes[3].
  • Ignoring style: the study on non-native writers identifies the style of a professional correction as a confounding variable in detector results[4]. Earlier studies reported high false positive rates for non-native English[4].
  • Believing a better tool will solve the problem: the mathematical analysis judges that this constraint cannot be overcome by better engineering[5].
  • Confusing a text polished by an AI with a text generated by an AI: detectors poorly distinguish the degrees of AI involvement[1].

Sources

  1. arXiv, “Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing”, 21 February 2025. https://arxiv.org/abs/2502.15666 (accessed 14 November 2026)
  2. arXiv, “Can AI-Generated Text be Reliably Detected?”, 17 March 2023. https://arxiv.org/abs/2303.11156 (accessed 14 November 2026)
  3. arXiv, “Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy”, 24 March 2026. https://arxiv.org/abs/2603.23146 (accessed 14 November 2026)
  4. arXiv, “Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing”, 27 August 2026. https://arxiv.org/abs/2608.26710 (accessed 14 November 2026)
  5. arXiv, “AI Detectors Fail Diverse Student Populations: A Mathematical Framing of Structural Detection Limits”, 11 March 2026. https://arxiv.org/abs/2603.20254 (accessed 14 November 2026)