In plain English
This page is part of the technical reference. It keeps the expert detail but starts with a plain-language summary for first-time readers.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
Evidence and Counterevidence for the Danger Model
Direct answer
The danger model is strongest when a behavior can be traced across carriers, reservoirs, routes, descendants, and promotion decisions. It is weaker when the system has bounded composition, complete provenance, independent evaluation, no persistent reservoirs, and rehearsed ecological rollbackRestoring not only a model artifact but the relevant router, prompts, memory state, tool permissions, evaluator version, deployment alias, and data dependencies. Open glossary definition.
What would support the model
Supporting evidence includes:
- behavior appears only in a composition;
- the same behavior appears in a descendant after the first carrier is retired;
- memory or synthetic data contains outputs from the behavior;
- promotion metrics rewarded the behavior;
- router traffic increased exposure to the behavior;
- evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition expectations changed to normalize the behavior;
- rollbackReturning a system to an earlier known state. Open glossary definition failed because it missed state or data;
- no single owner can account for the behavior end to end.
What would weaken the model
The model is less applicable when:
- every runtime composition is fixed and recorded;
- no persistent memory or synthetic-data feedback is used;
- route changes are rare, manual, and independently reviewed;
- no automated promotion occurs;
- tool access is read-only or tightly gated;
- rollback restores every dependency;
- retirement includes behavioral-extinction review;
- traces can replay the full path.
What would change the assessment
Strong evidence of effective composition-aware testing, independent evaluator diversity, trustworthy provenanceA record of where a component or behavior came from. Open glossary definition, complete trace replay, and behavioral-extinction controls would reduce concern. Evidence that a behavior persists through descendants, memory, synthetic examples, or aliases after retirement would increase concern.
Counterargument
Modularity is not inherently dangerous. It can improve cost, privacy, specialization, local deployment, resilience, and replaceability. Cognivirus.com argues that modularity shifts the assuranceConfidence, backed by evidence, that a system meets safety or governance requirements. Open glossary definition problem, not that modular AI should be banned.