In plain English
This page explains the governance layer: rules, logs, approvals, signatures, audits, permissions, and rollback tools. These controls are necessary, but they also become important failure points.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
The Dissolution of Responsibility
When harm emerges from suppliers, routers, adapters, evaluators, operators, and release rules, accountability can fragment.
Control requirement
The control must live outside the candidate’s ordinary write boundary. It should be versioned, auditable, recoverable, and testable under failure. A policy expressed only as a prompt is not a hard control.
Failure mode
The governance layer becomes part of the attack surface when it controls identity, success definitions, release permissions, hidden evidence, memory retention, aliases, and rollbackReturning a system to an earlier known state. Open glossary definition.
Practical review
Ask who owns the control, who can change it, which evidence would reveal failure, how it is rolled back, and what organizational pressure could bypass it.
<!-- expanded-release-content -->
When harm has no single obvious owner
Distributed AI systems can produce outcomes through many individually reasonable decisions: a supplier ships an adapter, a team approves a prompt, a router sends a task to a specialist, an evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition rewards an output, a memory system preserves a precedent, and an operator promotes a release. After harm occurs, no one component may look like the sole cause.
This is responsibility diffusionThe inability to identify one accountable component, developer, operator, or decision point after a distributed system produces harm. Open glossary definition. It is not a reason to avoid accountability. It is a reason to design accountability into the system before incidents happen.
Required ownership map
Every safety-relevant component should have an owner with authority, not only a contact name. The ownership map should cover base models, adapters, datasets, prompts, memory, routers, tools, evaluators, registries, release controllers, runtime operations, and affected-user channels. It should also name who can stop promotion and who can accept a no-opThe decision not to change the system. Open glossary definition.
Incident review
An incident review should reconstruct the composition, the transition graphThe map of how an AI system is allowed to change over time. Open glossary definition, the evidence available at the time, the human approvals, the evaluator decisions, and the missing accountability links. Blame should not stop at the final model response if upstream components created the conditions.