Incident file · Evals
Anthropic's three infrastructure bugs (2025)
weeks-long, intermittent quality degradation across multiple models, root-caused to three overlapping infra bugs
The system
Claude, Anthropic's model family (Opus 4/4.1, Sonnet 4, Haiku 3.5), served through Anthropic's own API and through Claude Code, running on a mixed fleet including TPU hardware with an XLA:TPU compilation path.
What was observed
Between early August and early September 2025, users reported inconsistent quality degradation: wrong-language tokens appearing mid-response, syntax errors, and generally worse output than expected, without a clear pattern of when or for whom it happened.
Stop and think
- If your production eval suite is "green" but users are reporting a problem, what are the possible explanations, ranked by how likely each is?
- What would make a defect "intermittent" and "platform-specific" rather than a clean, reproducible regression? What kind of testing catches that versus misses it?
- You run pre-deploy benchmarks, not continuous evals on live traffic. What class of bug can a benchmark suite never catch, no matter how comprehensive?
Answer the questions above before you read on: what failed, why, and what would have stopped it.