Cross-process change classification — node_type on the notify payload
Issue Systemorph/MeshWeaver#2194. Measured on memex-cloud on 2026-09-02 after the roll to
ci.7616: portal pods light (0.1–1.3 cores) while Azure Postgres memexaks-pg ran at 94–98 % CPU
with 225–292 active connections; [CrossSchema] SLOW averaged 4.0 s, 2 917 lines in five minutes
across 8 pods — every one a UNION ALL over 199 partition schemas. Core's census of that window
(Doc/Architecture/CrossSchemaFanOutElimination → The 2026-09-02 census) attributed the shapes
and named the lever that lived in this repo: the multiplier.
The multiplier
A path-less live query — historically the bell (nodeType:Notification, anchored since
MeshWeaver#3156), the thread list (nodeType:Thread),
the permission fold's three globals, the user directory, the UI-contribution catalog — is served by
PostgreSqlPartitionedMeshQuery.FanOutQuery. It re-runs its cross-schema UNION whenever a change
notification is relevant, and relevance is decided from the notification itself: a MeshNode
entity is matched against the query (QueryEvaluator.Matches, node type included), a delete of a
path not in the result set is dismissed — and anything it cannot classify re-queries, because
under-notifying is a security hole and over-notifying only costs a query.
Every notification from ANOTHER process was unclassifiable by construction. The pg_notify payload
notify_mesh_node_changes() emitted was {path, op}; PostgreSqlChangeListener therefore
published DataChangeNotification(path, kind, Entity: null, …), and Entity is not MeshNode → return true. On an 8-pod portal 7/8 of all writes arrive that way, so every write anywhere
re-ran every unanchored live query on every pod — which is how a process-wide, subscribed-once
membership fold showed 172 SLOW lines in five minutes, and how the bell, per circuit, showed 444.
What changed
- The trigger names the row's type.
notify_mesh_node_changes()— one function per partition schema, fired bymesh_node_notifyonmesh_nodesand by{table}_notifyon every satellite table — now buildsjson_build_object('path', …, 'op', TG_OP, 'node_type', NEW.node_type)(OLD.node_typeon DELETE).PostgreSqlSchemaInitializercarries the body for new schemas; migration V55 re-applies it in every existing schema (SET LOCAL search_pathper schema, the V24 shape), andDbVersion.Latestis 55. The payload stays far inside NOTIFY's 8000-byte limit: identifiers only. - The listener publishes a descriptor, never a node.
PostgreSqlChangeListener.ParsePayloadturns the payload intoEntity = new ChangedNodeDescriptor(path, nodeType). It is deliberately NOT aMeshNodewith blank content:MeshDataSource's hydration path pattern-matchesEntity is MeshNodeand applies what it gets, and a NOTIFY payload bypasses row-level security (the bypass #1250 removed) — so a node built from it would be both wrong and leaked. A distinct type keeps every existing consumer on its re-read branch exactly asnulldoes. A payload withoutnode_type(a schema V55 has not reached yet) still yieldsEntity = null— unclassifiable, re-query — so the rollout can only fail to skip a change, never miss one. - The gate reads the type.
FanOutQuery.IsRelevantgained one branch between the full matcher and the fail-open default: for aChangedNodeDescriptorwith a type, the change is relevant iff some fan-out query of the requestCouldAdmitNodeType— i.e. names no node type, or names exactly this one.ParsedQuery.ExtractNodeTypeanswers only for a simple equality (one value, not negated, not under an OR); every other shape answers null → re-query. The fail-safe is preserved shape for shape.
What it does and does not remove
It removes the multiplier from every fan-out that survives: a write of type X no longer wakes a live query for type Y on any pod. It does not anchor a query — a bell subscription still UNIONs every schema once at Initial and once per Notification write, which is rare. The remaining fan-outs are anchoring work, and the census below says which ones this repo still cannot anchor.
The census (ShellQueryShapesTest, MeshWeaver.Blazor.Portal.Test)
Two censuses, both classified with the router's own decision,
PostgreSqlPartitionedMeshQuery.ResolvePinnedPartition — not a mirror of it. The TYPED census reads
the declared builders (NotificationQueries.Bell, ThreadQueries.*). The SOURCE census scans every
.razor/.cs under src/MeshWeaver.Blazor.Portal for string literals that are queries (they start
with a qualifier), substitutes X for each interpolation hole and classifies the result — so a query
inlined in a view cannot escape by not being registered. The test pins the fan-out set EXACTLY: a
new unanchored shell query fails it; anchoring one of the declared ones fails it too, so the list is
updated rather than going stale. Declared today, with the reason each cannot be anchored on the
current data model:
| Shape | Why it fans out | What would anchor it |
|---|---|---|
NotificationQueries.Bell — nodeType:Notification sort:CreatedAt-desc |
it was written as a satellite of the entity it is about, {entity}/_Notification/{id}, in THAT entity's partition, so the viewer's partition named nothing |
Done (MeshWeaver#3156/#3216/#3238): a notification is DELIVERED to its addressee, so the bell is two PINNED reads — namespace:{viewer}/_Notification and, for a viewer hub.IsGlobalAdmin() confirms, namespace:Admin/_Notification. 🚨 Two queries, never a namespace:A\|B alternation: an alternation leaves ParsedQuery.Path null, takes the fan-out route, and is narrowed by INTERSECTION with searchable_schemas — which excludes Admin, so it would silently drop the platform bell. See core's Doc/Architecture/AddressedNotifications |
ThreadQueries.MyThreads / MyOpenThreads — nodeType:Thread content.createdBy:{user} … |
a thread is created at {contextPath}/_Thread/{id} for whatever node it was started from; "my threads" spans every partition by design (the ThreadQueries remarks say why scoping it to the page answers empty everywhere but one node) |
a per-user thread index (a {user}/_Thread mirror or a recipient-side record), or a product decision that the picker is page-scoped — a scope call, not a refactor |
| ChatHistorySelector.razor — nodeType:Thread createdBy:{user} (inlined) | the same "my threads" question, spelled with the createdBy: COLUMN the ThreadQueries remarks warn is NULL on every satellite read | route it through ThreadQueries.MyThreads — one shape, one place |
| MeshSearch.cs — source:accessed scope:descendants is:main … limit:{n} (inlined) | the empty search box lists what the viewer accessed recently; those nodes live in every partition the viewer ever visited, so the accessed-join has no single home (NeedsFanOut routes every source: join here) | narrow the UNION to the partitions the viewer's UserActivity rows name |
| CreateNode.razor — namespace: nodeType:NodeType (inlined) | NodeType registrations are declared per partition, so the create page's list has no single home | NodeType instance locations (the fourth narrowing, #1127) |
ThreadQueries.ThreadsUnder(path) is the positive control: namespace:{path} scope:descendants
pins to the path's partition. NotificationTriageService is outside the census by construction —
one live subscription per PROCESS, not per circuit — and now DECLARES its span
(MeshWideQuery.OfType(Notification)) rather than being grandfathered; narrowing it to only the
users who authored routing rules is #1294. The React client's NotificationCenter.tsx is anchored
to the viewer (it omits the platform leg, having no admin verdict on the wire — #1295).
Verifying it on a deployment
- The migration Job logs
Repair v55: notify_mesh_node_changes() now emits node_type in N partition schema(s)andDatabase migration completed. Version: 55;DbVersionGaterefuses to serve a portal ahead of that. [CrossSchema] SLOWlines for the shell shapes should collapse from "one per write anywhere × circuits" to "one per write of that type × circuits". A shape whose count does not move is either not going throughFanOutQuery(check[FanOut] Query decisionat Debug) or is a payload withoutnode_type—SELECT prosrc FROM pg_proc WHERE proname = 'notify_mesh_node_changes'per schema shows which body is installed.
Related
- Core:
Doc/Architecture/CrossSchemaFanOutElimination(the census and the plan),Doc/Architecture/UnanchoredSecurityReads(why the fold's globals stay global). - This repo: NodeType instance locations (the fourth narrowing, and why none of the seven measured shapes can be narrowed by declaration today).