AI-text detectors have a bad reputation, mostly earned. Plenty of them are black boxes that flag a nervous student's essay as a robot and offer no way to check the claim. This page is the opposite: the corpus, the calibration, the operating point, and the parts it gets wrong, with the false-positive number that matters put first.
The detector reads prose and scores how much it looks like current-model AI writing, from a set of measurable features: the density of the words and stock phrases these models overuse, sentence-length uniformity, connective-opener overuse, chat-UI artifacts left in a paste, and a few dozen more. Those features feed a calibrated model that returns a probability. It is not a vibe and it is not a language model judging another language model; it is a scorer whose every input you can inspect.
Every number on this page is self-reported: we ran the eval, we picked the sets, nobody audited it. Saying that plainly is the point. For a tool that flags contributions, the number that matters is how often it is wrong about a human, and the hardest version of that is careful non-native English, which sits in the same formal register the models write in. On a held-out set of 4,300 non-native-English essays, real writing from real ESL students, the false-positive rate is 7.8% at the shipped operating point. Not zero. That is the honest ceiling on trusting any single flag, and it is why the extension never calls a flag proof.
Measured on data the calibration never saw, split by how current the generator is. These are the detector's real reported numbers, not a small in-house sample:
| Held-out set | What it is | AUROC | Catch rate at 5% FPR |
|---|---|---|---|
| DetectRL-X | 2025-era generators (DeepSeek-V3 and similar), plus paraphrase and zero-width-space attacks | 0.983 | 92% |
| RAID-test | 2022-23 generators (GPT-2/3, Cohere, MPT, Mistral) | 0.89 | 67% |
| MAGE (paraphrase) | GPT text run through a paraphraser, out of domain | 0.81 | 39% |
| humanizer-attack | Ghostbuster "undetectable" essays | - | 99% caught |
For comparison, GPTZero's public RAID submission is 0.9724 catch rate at 5% false positives: ahead of this detector on RAID's older models, behind it on the ESL false-positive axis, where that class of detector runs around 60%. The RAID gap on old base-model text is real and named, not hidden.
A separate 40-document labeled set (19 human, 21 AI) rides along in the repo, stacked with the writing most likely to trip a detector: formal 19th-century essayists (Twain, Thoreau, James, Cather), journalism, kernel commit messages, a translated legal code. It is not a benchmark and is not quoted as one; it is the calibration guard that catches a change that would start flagging Mark Twain as a robot before it ships.
The detector is tuned to a 5% false-positive operating point on a broad mix of English text: it would rather miss a bot than accuse a person. That is the target the threshold is set from, not a promise that every group lands on it. Non-native English comes in above it, at 7.8%, which is the whole point of putting that number first. Below roughly twenty words, or on text it reads as anything other than English, it does not guess at all. It says it abstained and shows you why. A one-line "lgtm" gets no verdict, because no honest detector can give one.
Short comments needed their own layer. On real GitHub review comments the statistical detector above is no better than chance (AUC 0.4925 on a corpus of real public review comments and assistant-drafted comments from one drafting pipeline), so the extension carries a second, separate signal that scores the genre itself: comment length, comma density, contractions, whether the text points at an issue, a person, or a link, and a few pattern tells like opening by confirming what the other person just said. It catches comment-style tells of assistant-drafted text. It is never merged into the probability above; it renders as its own chip, and a text can carry both signals, either, or neither.
Its held-out numbers, from repo-grouped 5-fold cross-validation, flagging above 7 points: it catches 25.0% of assistant-drafted comments (n=128) at a 0.92% false-positive rate on 5,316 real GitHub comments, with 0 of 84 flags on a hand-checked real-world human set. Run as a union with the main detector, the catch rate is 25.8% at 5.76% on GitHub comments; all eight human false positives in that union come from the main detector, none from the style layer. A label-shuffle null test lands at the base flag rate, so the harness is not grading its own answer key.
Its limits, plainly. The catch rate is measured against drafts from one assistant drafting pipeline, so 25% is not a general AI-detection rate and is not presented as one. The layer is also easy to evade: drafts filtered against this layer before posting evade it. On a fresh holdout of exactly such drafts, gathered after the corpus freeze, the catch rate fell to 1 in 21. Anyone who tunes drafts against the score walks past it, which is why the chip says the text reads assistant-drafted rather than claiming who wrote it. The fresh human sample is only 10 comments, too few to stand up a false-positive claim on its own, and its single flag reads assistant-drafted on inspection anyway. A style flag is a reason to look, nothing more.
Two failure modes, named rather than buried. First: text a person drafted and then had a model polish for tone or grammar. The words on the page really were shaped by the model at that point, and a style detector built to catch AI phrasing has no principled way to tell "polished" from "written," because in the features it measures they can look the same. Second, and the one to take most seriously: non-native and heavily formal English. The classifier abstains outright on anything it doesn't read as English, but a non-native speaker writing carefully in English, or anyone writing in a dense, formal register, can still land close to the same stylistic territory as current AI prose. The 7.8% false-positive rate on 4,300 real ESL essays above is exactly this failure mode measured at scale rather than papered over, and it is not zero. It is a claim that some non-native and formal writers will be flagged, which is why a flag is a signal to look, never proof. Here is a concrete example I will not hide: text written densely about the vocabulary these models overuse trips the style signals, so some of my own writing about this detector scores as AI-ish. That is the failure mode, shown rather than papered over. It is exactly why every result in the extension is labeled a signal at a stated false-positive rate, never proof, and why the advice is always the same: judge the contribution, not the author. The detector points you at text worth a second look. You do the looking.
The held-out generator sets above (DetectRL-X, RAID-test, MAGE, the humanizer-attack set, and the 4,300-essay ESL gate) are licensed for evaluation use and are not redistributed, so this page cannot hand you a one-line command to reproduce them the way an open dataset would. What is committed to: they are the same frozen sets on every release, the fitting never touches them, and the numbers are the direct output of the eval against the shipped model. The 40-document calibration set that guards against regressions does ship in the repo and runs with one command:
python3 eval/run_eval.py
That prints the per-sample table for the small guard set. It is the linter's tripwire, not the detector benchmark, and it is labeled that way here so the two are never confused.