In plain English
This page explains how AI systems can change over time through updates, tests, retraining, memory, and approvals even when no single model rewrites itself.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
Death by a Thousand Edits
The reports use “death by a thousand edits” to name a gradual erosion pattern. One change looks harmless. Another looks like optimization. A third removes an edge case. Over time, safety constraints become weaker even though no single change looks like the decisive cause.
Where edits accumulate
Edits can accumulate in prompt policy, adapter stacks, evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition prompts, router thresholds, memory consolidation rules, tool permissions, dependency versions, or organizational runbooks. Each edit may have a legitimate local justification.
Why lineage alone is not enough
LineageThe parent-child history of models, adapters, datasets, or releases. Open glossary definition records that a change happened. It does not prove that reviewers understood the behavioral consequence of the change. Behavioral inheritance requires re-evaluation against the current composition, not just ancestry reconstruction.
Control pattern
Use cumulative-change budgets. After a defined number of minor changes, force a full ecological re-certification. Track not only changes to artifacts, but changes to thresholds, hidden tests, approval norms, and rollbackReturning a system to an earlier known state. Open glossary definition assumptions.