In plain English
This page explains how AI systems can change over time through updates, tests, retraining, memory, and approvals even when no single model rewrites itself.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
Distillation as Behavioral Transmission
A student model trained on a teacher’s outputs may inherit useful or problematic patterns without sharing the teacher’s weights.
Mechanism
Variation, evaluation, selection, inheritance, and succession can exist as properties of the broader development process. The model does not need to rewrite itself at runtime. The ecology changes because operators, pipelines, routers, and release controllers alter the population.
Assurance implication
A descendant needs fresh evidence for safety-relevant behavior. A content hash can identify an artifact, but it cannot prove that a related descendant preserved all relevant guardrails.
Review question
What behavior is being tracked, where could it be encoded, which descendants or reservoirs may carry it, and what evidence would count as absence across active compositions?