In plain English
This page covers the high-risk pattern where small adapters, routes, memory, evaluators, and descendants can reinforce each other across time. It is a risk model, not a build guide.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
ModelBreeder risk escalation
The same design pattern that makes ModelBreeder valuable on the possibility side also sharpens the risk problem on CognivirusA behavior pattern that can survive, move, or reappear across a changing AI system. Open glossary definition: model evolution turns review from a one-time artifact check into an ongoing population-control problem.
The escalation path
| Stage | Normal purpose | Risk-side interpretation |
|---|---|---|
| Create variation | explore better candidates | candidate generationCreating a proposed new model, adapter, prompt, route, test, or policy. Open glossary definition can outpace review. |
| Evaluate fitness | choose useful descendants | evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition coupling can preserve proxy hacks. |
| Preserve winners | compound capability | flawed behavior can become parent material. |
| Archive novelty | avoid monoculture | unbounded novelty can normalize poorly understood behavior. |
| Compose specialists | improve coverage | safe-looking parts can fail together. |
| Release progressively | reduce deployment risk | canary success can still miss memory, route, or tool-specific expression. |
| Retire old carriers | simplify the system | retirement can leave residue in memory, descendants, prompts, and route statistics. |
Apex conditions imported from model breeding
A model-breeding loop starts to resemble an Apex Threat surface when five conditions overlap:
- Candidate generation is automated or cheap enough that reviewers see only a sample.
- Composition is dynamic enough that the actually deployed system is not the system that was tested.
- The evaluator is close enough to the candidate population that blind spots are shared.
- Successful behavior is copied into adapters, examples, memory, documentation, route policies, or descendants.
- RollbackReturning a system to an earlier known state. Open glossary definition restores a model file but not the full ecological state.
Risk statement
The risk is not that a model wants to reproduce. The risk is that human tooling and automation can accidentally provide a reproduction path: generate, test, promote, remember, derive, route, and reuse.
Concrete review questions
| Question | Why it matters |
|---|---|
| Can a candidate become a parent without human approval? | Reproduction pressure exists even without autonomy. |
| Does the evaluator see the same hidden tests each generation? | Fixed tests become part of the selection environment. |
| Can descendants inherit examples created by a failed candidate? | Failure residue can become training material. |
| Can a small adapterA small add-on that changes or specializes model behavior. Open glossary definition change action permissions indirectly? | Low-rank deltas can become high-impact carriers when composed with tools. |
| Can rollback restore memory, router policy, evaluator versionThe exact version of the evaluator used for a test or release. Open glossary definition, and deployment alias? | Apex persistence often sits outside the model file. |
Controls to carry forward
- Reproduction quotas.
- Candidate ledgers.
- Explicit parentage graphs.
- Separate candidate creator and candidate approver roles.
- Independent evaluator ownership.
- Hidden-test rotation.
- Synthetic-data quarantine.
- Memory diff review.
- Rollback packets.
- No-opThe decision not to change the system. Open glossary definition as a respected release outcome.
Boundary
This page does not claim the full Apex Threat has occurred as a single real-world incident. It maps how controlled model evolution can increase the number of carriers, transitions, and selection events that defenders must govern.
Related: Real Instances Behind the Apex Threat, Implementation Controls, and What Is Not Proven.