TL;DR: Microsoft Research’s DELEGATE-52 benchmark tested 19 frontier LLMs across 52 professional domains and found that even the best models corrupt ~25% of document content after 20 editing interactions — and only Python code editing is reliable enough for real delegation.
Most AI coverage fixates on coding. But the real economic opportunity for AI agents is the billions of dollars of knowledge work in Word documents, spreadsheets, slide decks, and structured records. Microsoft Research just published a paper that empirically measures what every builder of long-running document agents has suspected: frontier LLMs consistently and silently corrupt the very documents they’re supposed to maintain.
The DELEGATE-52 Benchmark
DELEGATE-52 simulates long-horizon delegated workflows across 52 professional domains: crystallography files, music notation, accounting ledgers, Python source code, aviation checklists, textile patterns, 3D object files, and more. It spans five categories — Science & Engineering, Code & Configuration, Creative & Media, Structured Records, and Everyday tasks.
Each of the 310 work environments consists of:
- Seed documents — real-world documents (2,000–5,000 tokens), not synthetically generated
- Edit task pairs — forward and backward natural language instructions designed to be invertible transformations
- Distractor context — 8,000–12,000 tokens of irrelevant but topically related text, simulating realistic AI workspace conditions
The evaluation uses a round-trip relay methodology: the LLM performs a structural edit, then a second edit that undoes it. Round-trips chain together (e.g., 10 round-trips = 20 interactions), and performance is measured by comparing the recovered document against the original using domain-specific similarity evaluators.
The Results: Silent Compounding Corruption
Across all 19 tested models (OpenAI, Anthropic, Google, Mistral, xAI, Moonshot), the findings are stark:
- Frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT-5.4) corrupt approximately 25% of document content on average after 20 interactions
- All models combined average 50% degradation
- Degradation compounds indefinitely — no plateau was observed even at 100 interactions
- Short-term performance does not predict long-term reliability: two models scoring 91.5% vs 91.1% after two interactions diverged to 48.3% vs 64.1% over time
The paper defines “readiness for delegated work” as maintaining a 98% reconstruction score. Out of 52 domains, Python was the only domain where a majority of models met this threshold — 17 out of 19 models. Everything else: documents, prose, data, less structured formats — corrupted.
The Failure Mechanism: Cliff-Edge Drops
Degradation isn’t a slow, uniform decline. The paper names the mechanism silent compounding corruption with cliff-edge drops. Models look competent for many turns, then suffer a single-turn 10–30 point loss. About 80% of total degradation comes from these rare critical failures.
The nature of errors differs by model strength:
| Model Tier | Primary Failure Mode |
|---|---|
| Weaker/smaller models | Deletion — content gets removed |
| Frontier models | Corruption — content is modified, distorted, or hallucinated |
This makes frontier model failures harder to detect. They preserve document structure while corrupting content, so the document looks right on a quick scan — a qualification dropped, a meaning subtly altered, a detail shifted. Nobody catches it because that’s the whole point of delegating.
Hardest Operations
The paper identified several consistently difficult semantic operations:
- Split and Merge — operations requiring global document restructuring
- Classification — grouping or tagging elements across the entire document
- Coordination — executing multiple semantic operations simultaneously (e.g., sorting while renaming)
Simple local operations like string manipulation or referencing were handled with much higher reliability.
Agentic Tools Made Things Worse
In a finding that should give pause to everyone building agent frameworks, the agentic experiment — giving models access to file read/write tools, Python code execution, and web search — performed 6% worse on average. Models with more capabilities introduced more errors, not fewer.
The paper identifies two document editing patterns:
- Full regeneration — the model reads the file and outputs the entire modified version every turn. This is what the main DELEGATE-52 experiment tests, and it produces the 25% corruption rate for frontier models.
- Surgical editing — the model reads the file once, outputs a small targeted patch, and the harness verifies the match and applies only the change. The rest of the file remains untouched.
The generic agentic harness provided file-writing tools but no dedicated edit tool. Models defaulted to full regeneration and favored manual file editing over code execution — despite newer frontier models using code execution more (45% for GPT-5.4 vs 10% for GPT-4.1).
Compounding Factors
Several variables amplify degradation:
- Document length: every additional 1,000 tokens reduced content preservation by ~0.7% initially, compounding to 3.6% loss after 20 interactions for high-performing models
- Distractor context: irrelevant background documents added up to 8% more corruption over long relays compared to “clean” prompts
- Interaction depth: no observed floor — degradation continues accumulating through 50+ round-trips
Three Production Principles for Document Agents
The video’s author — who tested the benchmark locally — lays out three principles for builders editing documents at scale:
- Separate Edit from Write — don’t let models regenerate entire files. Use surgical edit operations that only touch targeted regions.
- Enforce read-before-edit — the model must read the current file state before making any changes. Never operate on stale context.
- Require exact string matching — edits should specify exact old text to replace. The harness verifies the match; if it fails, the edit is rejected rather than silently corrupting.
The argument is that Claude Code works as well as it does precisely because its harness encodes assumptions the model can’t enforce on its own. Failure becomes the safety mechanism rather than the failure mode.
Image Models Fail Even Harder
The researchers extended the evaluation to visual domains, implementing six image-editing work environments and testing nine models with image generation capabilities across 20 interactions. The results were dramatically worse than text: the best image models achieved reconstruction scores of only 28–30%, compared to 70–80% for frontier text models. A Linux kernel architecture graph degraded into something unrecognizable, a textile pattern mutated beyond recovery, and a 3D object that started as a palm tree ended up as an entirely different shape. Image generation is nowhere near ready for delegated work.
Extended Builder Takeaways
Beyond the three production principles, the video’s author draws several more practical conclusions for builders:
- Context distraction is a first-order problem — keep the model’s context window focused on what matters. Poor recall in RAG retrieval is a key source of distraction in automated systems. Use reranking and enforce minimum relevance thresholds for retrieved chunks.
- Context resets still matter — despite Anthropic pushing context compaction over resets, this research suggests context rot remains a major problem as windows grow. Resetting context at the right moment improves reliability.
- Use sub-agents for large documents — leverage isolated context windows via sub-agents to process large documents rather than stuffing everything into a single conversation.
- Human-in-the-loop is non-negotiable — until you can demonstrate above 98% accuracy on long-running tasks, you need human review gates. These can sit at the outer harness level or within individual tool calls. A diff-based review interface that highlights changes between original and edited documents makes it possible to spot silent corruption.
The Irony
Microsoft published this paper while simultaneously selling delegated document editing as a commercial offering in their Copilot product suite. The benchmark itself is MIT-licensed and reproducible — the dataset and code are available on GitHub, and anyone can run the simulations against their own models.
The core tension is clear: the AI industry markets delegation as the future of knowledge work, while its own research shows that delegation without verification is a recipe for silent, compounding corruption. The difference between a model that appears to work in a demo and one that can be trusted over 20, 50, or 100 interactions is the difference between a useful tool and a liability.
References
- LLMs Corrupt Your Documents When You Delegate — Philippe Laban, Tobias Schnabel, Jennifer Neville, Microsoft Research (April 17, 2026) — https://arxiv.org/abs/2604.15597
- DELEGATE-52 GitHub Repository — Microsoft (2026) — https://github.com/microsoft/DELEGATE52
- Claude, GPT-5, Gemini 3: Only 1 of 52 Jobs They Can Actually Do — YouTube video (2026) — https://www.youtube.com/watch?v=T8EW3mD626E
- LLMs Corrupt Your Documents (and the Theory Dies Twice) — cekrem.github.io (2026) — https://cekrem.github.io/posts/llms-corrupt-your-documents/
This article was written by Claude (Claude 4.5 Sonnet | Anthropic), based on content from: https://www.youtube.com/watch?v=T8EW3mD626E
