In plain English
This page explains how AI systems can change over time through updates, tests, retraining, memory, and approvals even when no single model rewrites itself.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
Evolutionary breeding risk modes
Evolutionary language is useful only if the control planeThe governance layer that decides what can run, change, access tools, or be released. Open glossary definition remains explicit. A system can behave evolutionarily without literal self-replication: humans and pipelines can create variants, tests can select, dashboards can promote, and memory can preserve.
Main risk modes
| Risk mode | Description | Defensive question |
|---|---|---|
| Evaluation monoculture | one judge family, one benchmark style, or one hidden-test set defines success. | What independent evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition can disagree? |
| Proxy survival | a candidate wins by exploiting the metric rather than improving the intended capability. | What off-metric tests challenge the promotion reason? |
| Novelty inflation | unusual behavior is rewarded without enough utility, legibility, or rollbackReturning a system to an earlier known state. Open glossary definition confidence. | What proves novelty is useful and bounded? |
| Lineage launderingProposed Cognivirus terminology for a situation where repeated derivation makes the origin of a behavior difficult to recognize even when artifact parentage is technically recorded. Open glossary definition | a risky behavior is inherited through descendants until the parentage looks harmless. | Can the lineage graphA visual or machine-readable map of derivation history. Open glossary definition explain every inherited behavior? |
| Composition invisibility | the model passes alone but fails with a specific adapterA small add-on that changes or specializes model behavior. Open glossary definition, prompt, memory, route, tool, or quantization mode. | Was the exact deployed composition tested? |
| Retirement residue | a failed candidate is removed, but its outputs remain in examples, memory, logs, route statistics, or documentation. | What residue was quarantined? |
| Dashboard overconfidence | a clean UI hides uncertainty, source boundaries, or missing rollback state. | Does the interface show what is unknown? |
Why no-op matters
No-op is reproductive control. A release process that always rewards change applies constant selection pressure toward novelty and speed. A release process that respects no-op can stop a lineageThe parent-child history of models, adapters, datasets, or releases. Open glossary definition before uncertainty becomes inherited state.
Minimum controls
- Parentage graph.
- Frozen evaluation suite plus rotating hidden tests.
- Candidate quarantine until composition tests pass.
- Novelty archive with usefulness criteria.
- Synthetic-output labeling.
- Memory diff review.
- Evaluator disagreement logging.
- Rollback rehearsal.
- Retirement record.
Boundary
This page treats evolution as a system-design pattern. It does not claim that models reproduce biologically or autonomously.