Model labels and the two-stage Auto router

Maintainer, 2026-10-01: "auto should choose which model is appropriate. let's also calibrate this process that we have labels like coding where we can filter available models" · "auto must triage first the kind of task, then which model to take."

The three pieces of data

The kind vocabulary is data, not an enum: Ai:Router:Kinds (comma-separated), default coding, triage, review, reasoning, chat, utility. An unknown label never drops a model or an agent; it simply matches nothing (ModelLabels, the same rule as ModelDefinition.Tier).

The flow (one bounded routing pass, AgentChatClient.ApplyAutoRouterSelectionAsync)

It runs when the round's model is Auto (auto, Provider/Auto/auto, or auto:<labels>), or when the serving agent declares modelLabels and no model was picked.

1. Floor. The synchronous dispatch has already parked the round on a usable model: the agent's tier, or the deployment default. Every failure below keeps it. A residency-bound floor is never routed away from.
2. Stage 1: kind and difficulty, on a CHEAP fixed model. The classifier is the utility tier (else chat), never the floor, which may be the most expensive model. It answers {"kind", "difficulty", "reason"}. Difficulty is one of simple, standard, hard, hardest. With pinned labels the kind is fixed and only the difficulty is assessed.
3. Filter. The candidates are the loaded, non-router models that pass the caller's admission gate (usable credentials AND the instance's and agent's data residency, so EU-only holds by construction) and carry ALL the required labels (the pinned ones, else the classified kind). They are capped at 12. If no candidate is left, the floor is kept and that decision is recorded. With exactly one, it is taken without a call.
4. Stage 2: the model. The floor model is shown each candidate's labels, its price in and out per million tokens, its reasoning efforts, and a calibration line for the kind. It is told to pick the CHEAPEST candidate that does this difficulty well. An id outside the candidates keeps the floor.
5. Recorded. AutoModelRouter.RoutingDecision is logged at Information and stamped on the response cell as ThreadMessage.ModelRouting: kind=coding (one-line fix); difficulty=simple; model=kimi-code (cheapest that can do it); floor=….

The decision logic is pure and unit-tested (AutoModelRouter.AfterStage1 / AfterStage2, ModelLabelsAutoRoutingTest). The client only makes the two bounded calls (10 s each) and applies the result.

Calibration

IModelCalibration.For(kind, model) returns ModelCalibrationStats, which the prompt renders as observed for coding: 12 tasks, 7 succeeded, $3.10 per success, median 4 rounds.

The platform registers NodeModelCalibration (TryAdd, so an in-process producer's own registration wins in either order). It is empty, exactly as EmptyModelCalibration, until Ai:Router:CalibrationNode names a node. From then on, every replica's NodeModelCalibrationFeed serves that node's calibration: [{kind, model, tasks, succeeded, totalCost, medianRounds}].

A producer builds ModelOutcome records (kind, model, input/output tokens, prices at the time, succeeded, rounds) from its own items and aggregates them with ModelCalibration.Aggregate. A node-native producer cannot register a service, and its pass runs on one replica while rounds route on all of them, so it writes the rows onto that node. The bug-fix pool is the first producer, writing to Hosting/BugFix:

• triage: succeeded when a difficulty was recorded;
• coding: succeeded when the fix merged, failed when the rounds or the ladder ran out;
• a bug blocked only for want of PR tooling is not counted (Essentials/BugTriage → The outcome ledger).

Entry points that use this router

Not yet

• The bug-fix pool is the calibration producer: it writes its ledger onto Hosting/BugFix → calibration, and an instance with Ai:Router:CalibrationNode: Hosting/BugFix serves those rows to the router on EVERY replica (NodeModelCalibration + NodeModelCalibrationFeed, a live query of that one node). Unset, stage 2 decides on labels, price and difficulty alone.
• Model nodes generated from a deployment record's ai.openRouterEU.models list carry no labels until the record format grows them. Label the nodes, or add the field there.