In plain English
This page covers the high-risk pattern where small adapters, routes, memory, evaluators, and descendants can reinforce each other across time. It is a risk model, not a build guide.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
ModelBreeder Risk Side
Cognivirus.com is the risk side of the system. ModelBreeder.com can go deeper on the constructive possibility of controlled model evolution. This page translates that possibility vocabulary into the failure modes a security, governance, or architecture reviewer must inspect.
Scope correction
The prior ModelBreeder intake added useful architecture material: Genome records, FitnessVector reports, candidate populations, novelty archives, multi-parent merges, speciation, local-first dashboards, and evolution-loop UI. On Cognivirus.com, those are not product promises. They are risk surfaces.
The correct split is:
| Surface | Primary emphasis | Cognivirus interpretation |
|---|---|---|
| ModelBreeder.com | possibility, constructive evolution, capability compounding | useful source vocabulary only |
| Cognivirus.com | risk, failure modes, persistence, rollbackReturning a system to an earlier known state. Open glossary definition, governance | every evolution mechanism becomes a review surface |
| Shared vocabulary | genome, fitness, novelty, lineageThe parent-child history of models, adapters, datasets, or releases. Open glossary definition, selection, rollback | terms must stay source-bounded and non-operational |
The risk thesis
The more a system can generate descendants, evaluate them, preserve winners, route them into use, remember their outputs, and retire old carriers, the more it needs a transition-graph control planeThe governance layer that decides what can run, change, access tools, or be released. Open glossary definition. The problem is not that controlled evolution is inherently bad. The problem is that the risk moves from one model file into the reproduction, selection, memory, routing, and rollback machinery around it.
Risk translation table
| ModelBreeder mechanism | Constructive use | Risk-side question |
|---|---|---|
| Candidate population | Explore many variants quickly | Who prevents the candidate swarm from exceeding review coverage? |
| FitnessVector | Make tradeoffs visible | Is the composite score hiding evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition disagreement or proxy capture? |
| Novelty archive | Preserve diverse solutions | Are strange behaviors quarantined before they become promoted behavior? |
| Speciation | Keep useful specialists | Do niche specialists avoid cross-composition review? |
| Multi-parent merge | Combine capabilities | Can we still prove which parent introduced a behavior? |
| Adapter stackA set of adapters loaded together, usually in a defined order. Open glossary definition | Modular specialization | Are small deltas being treated as trusted because they are small? |
| Browser/edge lab | Local privacy and low latency | Where do local caches, manifests, and rollback receipts live? |
| Dashboard | Make evolution legible | Does the UI show evidence limits, or does it turn weak evidence green? |
Deep risk patterns
Candidate swarm risk
A breeding loop can create more candidates than a team can inspect. The dangerous threshold is not a specific number; it is the point where promotion uses batch trust, score-only sorting, or stale tests because human review cannot keep up.
Controls: candidate quotas, generation ledger, freeze switch, review backlog threshold, and explicit no-opThe decision not to change the system. Open glossary definition outcome.
Fitness proxy capture
Fitness metrics do not merely measure the population. They shape it. Any candidate-generation process that is repeatedly selected by a metric can learn to satisfy the metric's blind spots.
Controls: independent evaluator ownership, hidden-test rotation, judge disagreement logging, manual review for unexplained score jumps, and release holds when utility improves while safety, provenanceA record of where a component or behavior came from. Open glossary definition, or resource metrics degrade.
Novelty pressure
Novelty is useful for exploration. It is also a direct route to unreviewed behavior if the system treats “different” as “better.” A novelty archive should be a quarantine and study surface before it is a release surface.
Controls: novelty descriptors, novelty quarantine, behavior summaries, human-readable rationale, and promotion blocked until the novel behavior has a bounded use case.
Multi-parent lineage laundering
Two-parent merges already complicate attribution. N-way merging makes provenance graph-shaped. Without parent hashes, merge operators, load order, and evaluator versionThe exact version of the evaluator used for a test or release. Open glossary definition records, a later incident cannot be traced back to a source branch.
Controls: source-parent list, layer/operator manifest, adapterA small add-on that changes or specializes model behavior. Open glossary definition load-order manifest, per-parent removal tests, and rollback to pre-merge parents.
Adapter micro-carriers
Adapter and LoRAA common kind of small adapter used to specialize large models. Open glossary definition risks belong on the risk side because the behavior-changing component can be small, portable, cheap to copy, and easy to overlook. Size is not a trust boundary.
Controls: adapter signing, supplier review, ML-BOM entries, composition-specific tests, and retired-adapter deny lists.
Edge and browser dispersion
Local-first execution improves privacy and latency, but it can scatter state. Browser caches, local model files, adapter bundles, WASM memory, logs, and downloaded prompts can survive outside the central deployment story.
Controls: local manifest export, signed update channels, cache inventory, explicit data-retention statement, and rollback receipts for edge nodes.
Runtime residue
KV caches, prefix caches, paged attention blocks, GPU memory, and browser workers are not part of ordinary “prompt text,” but they can influence confidentiality, isolation, and reproducibility.
Controls: cache partitioning, session ownership, zeroization where feasible, shared-worker review, and residue tests before multi-agent or multi-tenant execution.
Dashboard trust theater
A dashboard can create false confidence if it turns a complex evidence state into green badges. Risk UI must surface missing tests, skipped checks, uncertainty, disagreement, and what is not proven.
Controls: evidence labels, source links, skipped-check fields, no-op cards, disagreement charts, and a visible What Is Not Proven link.
Use the risk register
The machine-readable risk register is available at /data/modelbreeder-risk-register.json. It should be treated as a planning inventory, not as a formal certification schema.