In plain English
This page explains why testing AI parts one by one is necessary but incomplete. Safe-looking parts can still produce unsafe behavior when combined.
- Why this matters: AI risk can come from the whole arrangement, not one obvious model.
- What to look for: data, memory, routes, adapters, tools, evaluators, updates, and rollback paths.
- Technical version below: the expert terminology remains available and is linked through the glossary.
Semantic routing as a safety boundary
Multi-model and multi-LoRA systems often use routers to select a specialist, adapterA small add-on that changes or specializes model behavior. Open glossary definition, tool path, or verification pipeline. The reports emphasize that this router is not a neutral performance optimization. It is a safety boundary.
Router-created risk
A semantic router may decide whether a request is handled by a safety-hardened adapter, a creative model, a code specialist, a retrieval pipeline, or a cheaper local model. If the router misclassifies intent, the wrong policy boundary may serve the next token.
Routing manifest
A routing-aware composition manifestA machine-readable record of the exact runtime composition used for an evaluation, release, incident, or rollback. Open glossary definition should record:
| Field | Reason |
|---|---|
| router version | behavior can change without changing the base model |
| route labels | labels define which policies apply |
| classifier family | correlated failures matter |
| fallback route | fallback can be less safe than primary |
| confidence threshold | threshold changes are policy changes |
| memory access by route | routes may differ in what they can read or write |
| tool access by route | route choice may grant capability |
| evaluatorA system that judges whether an AI output or candidate is acceptable. Open glossary definition coverage | every active route needs evaluation coverage |
Red-team questions
- What happens when intent is ambiguous?
- Does the router fail closed or choose the cheapest route?
- Does a safety classifier share training data or blind spots with the routed candidate?
- Are routes tested under memory and tool states, not just raw prompts?
- Can a route be promoted without re-evaluating the composition?
Site rule
If a router changes which policy serves an input, the router is part of the evaluated artifact.