The live-only config audit — nothing lives only on the cluster

The rule this machinery enforces: nothing lives only on the cluster. Every value is a typed field on the Hosting/Deployment record, rendered by the chart (HelmValues); secrets are Key Vault NAMES on the record, never values anywhere. Anything else — a kubectl set env, a live-edited ConfigMap key, a hand-applied CronJob — is configuration only the cluster knows about, and it does not fail a deploy: helm upgrade preserves a live-edited ConfigMap key and never touches an object it does not own. It fails later, as an outage. On 2026-08-30 the memex portal crashed at boot because its Email section existed TWICE — the chart's ConfigMap defaults and a cluster-only secret patched onto the Deployment as env — and .NET picks the winner at random per pod start. The audit run by hand that morning found 25 env patches, 10 live-edited ConfigMap keys, sidecar containers and a volume on one namespace alone, plus a plain-text registry token in the pod spec. That measured inventory is Systemorph/Memex#148; this page is the machinery that keeps it measured.

Run one

Create a Hosting/InstanceAction with requestedAction: Audit. It is read-only: no backup question, no confirmation, nothing disturbed — safe to run against any instance at any time.

{ "id": "audit-memex", "namespace": "Ops/Actions", "name": "Audit memex",
  "nodeType": "Hosting/InstanceAction",
  "content": { "$type": "InstanceActionContent",
    "deployment": "memex", "requestedAction": "Audit" } }

The run launches the operator's hosting-audit (one step), which takes the truth from helm get manifest + helm get hooks on the instance's release and diffs the live namespace against it. The run node shows the verdict (auditVerdict) and links the report (auditPath).

What is detected

All of it by name, never by value — some of the things being hunted ARE secrets, so a report that quoted a value would be the finding one level up.

How to read the report

The report is a Hosting/ConfigAudit node at {space}/Audit/{deployment} (the operational space — Ops unless Hosting:OperationalSpace says otherwise, resolved identically by writer and reader) — one per instance, replaced on every audit. Its page leads with the verdict and lists every finding as a (category, object, items) table; the Fleet area on any audit node shows every instance's verdict side by side; and the instance's own Deployment page carries the latest verdict as a strip under its status line.

• clean — the namespace runs what the audited helm revision renders, and nothing else. The revision and the audit instant are part of the claim: a clean verdict from last month says nothing about today.
• drift — each finding is configuration that lives only on the cluster. A finding closes when its value has a home in a repository AND the live patch is removed — per Memex#148: A/B config keys become typed fields on DeploymentContent rendered by HelmValues; secrets become Key Vault names (the SecretProviderClass maps them); sidecars, volumes, replicas and probe budgets are record fields already; unmanaged objects become chart templates driven by record fields. Removing the patch without landing the home just breaks the instance; landing the home without removing the patch leaves the coin toss in place. Both, same change. A volume below record finding is the one category the record already owns — it closes by running Reconcile, whose first steps grow every declared claim to the record's size. An inline env: entry (envLiveOnly / plainSecretEntries) whose key already has its declared home closes the same way once the record's inlineEnv entry carries retiredBy: Reconcile's RetireInlineEnv remedy removes it from every container after re-measuring, in-cluster, that the pod falls back to an EQUAL value — and refuses, by name, a sole source or a credential whose equality is not recorded (Plugins#1593).
• never audited — deliberately rendered as unknown, not as clean. The strip and the fleet board never infer a verdict from absence.

Three trust rules, enforced in code: the verdict is derived from the lists by the mesh (LiveOnlyConfigReport.Normalize), never believed from the script; a run whose output carries no report fails rather than recording anything (silence is not clean); and an audit that cannot read a kind (RBAC) or find the release refuses rather than reporting what it never measured.

Scheduling — the standing run

The operator has no scheduler of its own; runs are Hosting/InstanceAction nodes, and anything that can create one can schedule the audit. The intended standing hook lives in the fleet's config repo (Systemorph/Memex), beside its deploy-drift.yml sentinel — the two are complementary: deploy-drift answers "is every merged overlay value LIVE?", this audit answers "is everything live RENDERED?". The workflow to add there (do not hand-roll a second detector — the detection stays in hosting-audit):

# .github/workflows/config-audit.yml (Systemorph/Memex) — sketch
on:
  schedule:
    - cron: "45 6 * * *"     # daily, after chart-drift's 06:15 pass
  workflow_dispatch:
inputs/env:  AUDIT_MCP_URL   = https://<control-instance>/mcp
             AUDIT_API_TOKEN = a mesh API token for a fleet operator account (Key Vault, not a repo secret value)
steps:  for each environment in deployments/aks/envs.json, call the control instance's MCP
        endpoint (tools/call → create) to file an Ops/Actions/audit-{ns}-{run} node with
        requestedAction: Audit — then poll the same node (tools/call → get) until state is
        Done/Failed and fail the job on Failed or on verdict drift, so the Actions tab carries
        the standing red the same way deploy-drift does.

Until that lands, the audit is on-demand: run one after every deploy, every break-glass kubectl, and before declaring any Memex#148 row closed — its definition of done ends with "the audit reports zero live-only items for that namespace".

The moving parts