Detection systems are affected by the text they receive. Model family, prompt style, language, document length, human editing, and time all influence signals. A fair evaluation therefore compares performance across a defined test set instead of presenting one impressive score as a universal answer.
Product reference: the public ChatGPT Detector page captured on July 19, 2026.
Last reviewed: July 19, 2026 Use case: teams evaluating an AI detector before using it in an educational, editorial, or compliance workflow
Build the test set before selecting a tool
Create a small corpus that reflects the work you actually review. Keep the provenance of every item: who wrote it, what model produced it when applicable, the prompt or assignment, language, length, and any human editing. Do not label examples by guesswork.
At minimum, include four groups:
| Group | Example | Why it matters |
|---|---|---|
| Human baseline | Verified human work from the same domain | Measures false-positive exposure |
| Model output | Text from several current model families | Tests model variation |
| Edited output | Model text revised by a human | Reflects common real-world drafts |
| Mixed authorship | Human work with permitted AI assistance | Tests whether the tool supports nuance |
Measure decisions, not marketing labels
For each group, record the tool's output, the threshold you would use for a review, and the human outcome. Then calculate practical questions: How often would an innocent writer be escalated? How often would a high-risk sample receive no review? Which document lengths produce unstable results?
Demonstration sample: Run the same 300-word factual brief through two model families using the same prompt, then compare it with a human rewrite of the brief. Treat differences as information about the test set, not proof that one output is “obviously AI.”
Avoid moving the threshold after seeing individual outcomes. Set the review rule before the pilot, document it, and revise it only after examining the complete set.
Re-test after meaningful change
Models change, writing habits change, and detectors change. Re-run the evaluation when a provider substantially changes its model, when the audience shifts to a new language or document type, or when the detector changes its scoring method. Keep old results so that a performance trend can be explained rather than guessed.
NIST's generative-AI profile highlights the importance of measuring and managing risks in context. For detection workflows, context means the actual population, language, stakes, and escalation consequences—not a benchmark chosen because it looks favorable.
Publish the limits
An honest evaluation report states the corpus size, date, languages, document lengths, threshold, and known blind spots. It also says that a probability signal is not authorship proof. This gives decision-makers enough information to decide whether the system belongs in a low-stakes triage process, a pilot, or nowhere at all.