Cui A new Microsoft Research paper dropped quietly last week and it deserves more attention than it’s getting. The title is dry — “LLMs Corrupt Your Documents When You Delegate” — but the finding it contains is one of the more practical warnings you’ll encounter if you’re building agentic systems.
Here’s the short version: when you let an LLM handle multi-step modifications to important artifacts without verification checkpoints, frontier models degrade fidelity by 19–34% across 20 delegated iterations. Python-based workflows are the exception, showing less than 1% degradation on average.
That’s a wide gap. And it has direct implications for how you architect anything that chains AI actions on documents, spreadsheets, structured files, or code.
What “Delegation” Actually Means Here
The paper is careful to define its scope. It’s not testing whether AI can write code or summarize documents. It’s testing a specific pattern: delegated execution — where a user hands off a multi-step task to an AI with limited supervision between steps.
Think of it as the difference between:
- Supervised: You ask the AI to do something, review the output, then proceed
- Delegated: You describe the full workflow, walk away, and the AI completes it across many steps
The benchmark they built — DELEGATE-52 — intentionally stress-tests this pattern using transformation-and-inversion tasks. The AI transforms an artifact (document, spreadsheet, code), then is asked to invert that transformation to return to the original state. Errors in the underlying semantic content are what get measured — not formatting or stylistic drift.
It’s a clean methodology because it provides a ground truth for correctness.
The Core Numbers
Two findings stand out:
1. 19–34% semantic degradation across 20 iterations (frontier models)
This is the headline stat. Across strong state-of-the-art models, running 20 cycles of delegated transformations produced measurable corruption in artifact content — not hallucination in the traditional sense, but quiet semantic drift. The kind of thing that’s hard to notice on a quick scan but matters deeply when the document is a contract, a configuration file, or a financial model.
2. Python workflows: < 1% degradation
This is the signal buried in the noise. When the AI has access to structured execution environments — Python + file operations — the error rate collapses. The reason is intuitive: code is verifiable. Each transformation produces a deterministic output that can be checked, compared, and validated without human review. Language-native artifacts don’t have that property.
Why This Matters More Than It Sounds
The failure mode here isn’t dramatic. You won’t notice it on any individual step. The AI will produce clean-looking output at each stage. The degradation is sparse, accumulative, and only becomes consequential over longer chains.
That’s exactly the failure mode that kills trust in agentic systems in production.
The scenarios where this bites hardest:
- Document drafting pipelines where an AI refines, expands, and edits across multiple passes
- Structured data transformation — converting formats, merging datasets, applying rules iteratively
- Code generation chains that compound on previous AI-generated code without test gates
- Report generation workflows that pull, summarize, and cross-reference across many sources
If your pipeline looks like any of these, this paper is telling you something specific about where your reliability ceiling is.
What the Authors Are Actually Arguing
The Microsoft Research team took pains to clarify what the paper does and does not claim. They’re not arguing against using AI in professional workflows. They’re arguing that reliable long-horizon delegation is an unsolved engineering problem, not a solved deployment question.
From their follow-up clarification post:
“The results suggest that strong short-horizon benchmark performance alone may not guarantee dependable delegated execution over extended workflows. At the same time, the findings should not be interpreted as evidence that AI systems lack practical value in real-world work today.”
This is a careful, honest framing. The gap they’re identifying is between benchmark performance and the specific demands of trusted, long-chain delegation.
The Engineering Response
If you’re building agentic systems that touch important artifacts, here’s what this research implies for architecture:
1. Add Verification Loops
Don’t trust accumulated state. At defined intervals in any long-horizon workflow, checkpoint the artifact against a semantic ground truth — a schema, a hash of required fields, a parsing step that confirms structure. The paper notes that production systems mitigate degradation through exactly this mechanism.
2. Prefer Structured Environments
The < 1% Python result isn’t a fluke — it reflects that code execution provides natural verification. Where possible, express your transformation logic in a format that can be run and checked, not just described and trusted.
3. Shorten Chains with Human Gates
The 20-iteration scenario in the benchmark is a deliberate stress test. Most production workflows don’t need to run 20 uninspected cycles. Design for human checkpoints at natural task boundaries. This isn’t about distrust of AI — it’s basic reliability engineering.
4. Don’t Conflate Short-Task Benchmark Performance with Long-Horizon Reliability
A model that aces coding benchmarks or document tasks in isolation may still accumulate errors in a chained context. Evaluate your specific pipeline’s failure modes, not just the underlying model’s benchmark scores.
5. Orchestration Matters
The paper explicitly notes that production-grade orchestration layers, memory systems, and verification procedures reduce observed failure rates. Treating the raw model as your reliability layer is the architectural error. The model is a component; the system needs to be robust.
The Bigger Picture
This paper is part of a growing body of work that’s filling in the gap between “LLMs are impressive” and “LLMs are deployable for serious workloads.”
We’re at an inflection point where enterprises are pushing AI further into consequential workflows — legal documents, financial models, medical records, infrastructure configuration. The DELEGATE-52 findings matter because they put numbers on a failure mode that was previously anecdotal.
The 34% degradation stat will probably get used out of context to argue against agentic AI. That’s a mistake. The more useful read: the failure mode is known, it’s measurable, and it has engineering solutions. That’s good news for practitioners who are building carefully.
What it rules out is the assumption that you can delegate indefinitely to frontier models without verification infrastructure and expect reliable outcomes. The trust that enables serious enterprise adoption of AI agents has to be earned through system design, not assumed from model capability.
That’s not a warning about AI’s limits. It’s a roadmap for building something that actually works.
Key Takeaways
- Microsoft Research’s DELEGATE-52 benchmark reveals 19–34% semantic fidelity degradation across frontier models under 20-iteration delegated workflows
- Python-native workflows show < 1% degradation — structured execution enables natural verification
- The failure mode is sparse, accumulative, and low-visibility — exactly the kind that erodes production trust without triggering obvious alarms
- Mitigation through verification loops, orchestration layers, and human checkpoints is well-established in production deployments
- The finding doesn’t argue against agentic AI — it argues for reliability-first architecture in long-horizon delegation
The paper: LLMs Corrupt Your Documents When You Delegate
Follow-up clarification: Further Notes on Our Recent Research
Markdown Is Dying… Karpathy Agrees: HTML Is the New Standard for AI-Era Work
Click to load Disqus comments