Model Governance · Updated October 6, 2026
AI text provenance: watermarking, disclosure and the limits of detection
A machine-readable mark can help identify AI-generated text, but it cannot prove authorship, accuracy or responsibility. A defensible provenance system combines provider signals with records, disclosure and human review.
Treat a text watermark as evidence about a generation system, not a verdict about a person or document. The mark is useful only when its scope, error rate and chain of custody are understood.
Regulatory boundary
Provider marking and deployer disclosure are different duties
Article 50 of the EU AI Act requires providers of generative systems to make covered outputs machine-readable and detectable as AI-generated or manipulated. The European Commission says the obligation applies to synthetic audio, image, video and text, subject to defined exclusions such as source code, machine-to-machine output and standard editing.
Deployers have a separate disclosure duty for AI-generated or manipulated text published to inform the public on matters of public interest when it has not received substantive human review or editorial control. A hidden provider mark does not replace a clear label where deployer disclosure is required.
Do not collapse provider provenance, user-facing disclosure and editorial accountability into one checkbox. They answer different questions and may be enforced by different controls.
Mechanism
Watermarks alter statistical choices, not visible characters
A text watermark can bias the model toward selected token or word choices according to a secret or published scheme. A detector later tests whether the passage carries that statistical pattern. The output remains readable and copyable, unlike a visible label or metadata field.
This makes watermarking more durable than metadata alone, but less durable than a cryptographic signature over immutable bytes. Translation, paraphrasing, synonym replacement, truncation and constrained domains can weaken the signal because they change—or sharply limit—the choices on which detection depends.
Reference architecture
Layer provenance instead of trusting one detector
Generation request
↓
Model output + provider watermark
↓
Signed generation record and model/version metadata
↓
Human review or policy decision
↓
Visible disclosure when required
↓
Published artifact + retained audit evidence| Evidence layer | What it can establish | What it cannot establish |
|---|---|---|
| Invisible watermark | A compatible system likely generated or processed the passage | Who used it, whether the text is accurate, or how much a human changed |
| Signed generation record | Which service, model and policy handled a recorded output | Whether the published text is unchanged unless hashes are verified |
| Visible disclosure | What the publisher tells the audience | Technical provenance by itself |
| Human editorial record | Who reviewed, changed or approved the substance | Whether every upstream claim is true without fact-checking |
Detection limits
A detector result is probabilistic evidence
Every detector has false positives and false negatives. Thresholds trade one error type against the other, and performance varies with length, language, domain, editing and model version. A positive result should therefore trigger a defined review process—not automatic punishment or a public accusation.
Absence of a mark does not prove human authorship. The text may be too short, heavily edited, translated, generated by an unsupported system or created before watermarking was enabled. Conversely, a detected mark says nothing about factual accuracy, ownership, lawful use or the amount of human judgment involved.
Operational takeaway
Never use watermark detection as the sole basis for a high-impact decision.Record the detector version, threshold, passage length and confidence; require corroborating evidence and an appeal path.Production operations
Make provenance observable and reversible
- Inventory which models, regions and output types carry a watermark and which remain unmarked.
- Version the watermark and detector configuration alongside the model release.
- Store only the minimum generation metadata needed for audit, with clear retention and access controls.
- Hash the reviewed artifact so later edits can be distinguished from the approved version.
- Separate automatic marking from the policy that determines visible disclosure.
- Measure detector performance on your languages, domains and realistic editing patterns.
- Define escalation, human review and appeal before using detector results in moderation or compliance workflows.
Dated development
OpenAI starts a phased text-provenance rollout
On October 5, 2026, OpenAI made text watermarking opt-in for API customers on select models and said eligible ChatGPT and Codex output in the EU would receive an invisible watermark over the following weeks. Detector access initially goes to approved researchers and expert organizations.
OpenAI calls its system textGrain. The company says it adds a statistical signal to word choice and plans to make the technology available in open source. Its published evaluation also shows why layered controls matter: at a 1% target false-positive rate, detection was about 80% for 200-token psychology passages and 95% for 400-token passages, while performance was lower for constrained mathematical text. Replacing 10% of words in a 400-token passage reduced detection from about 92% to 66%; replacing 25% reduced it to 17%.
Those are company-reported results under stated test conditions, not a universal detector guarantee. Teams should validate the behavior on their own content and avoid inferring human misconduct from a score alone.
Implementation checklist
Before relying on AI text provenance
- Have you separated provider marking from deployer disclosure?
- Can you identify the exact model, detector and policy version used?
- Have you tested short, translated, technical and edited text?
- Is a positive result corroborated before any high-impact action?
- Can a person challenge a classification and receive human review?
- Do stored prompts, outputs and hashes follow data-minimization rules?
- Does your interface explain what the provenance signal does—and does not—mean?
Provenance is strongest when it supports accountable publishing, not automated certainty.