Static-Repo Import

A static repo is authored content that ships with the build — embedded documentation (MeshWeaver.Documentation), the built-in agents (MeshWeaver.AI), the model catalog, sample graphs (samples/Graph/Data). It is the source, never the live serving copy. This doc defines the one way to get a static repo into a partition so the partition is served from the database like any other.

The rule

A static repo is materialized into its partition through the single canonical upsert verb (CreateOrUpdateNodeRequest) — content + prerender + a Space root — idempotently per content-version, tracked as a content-addressed Activity, then reconciled (prune absent). No bespoke SQL, no hand-rolled CreateNode/stream-Overwrite, no per-instance races, no content-NULL shells.

How to import a static repo (the whole recipe)

To materialize partition P from a static repo:

1. Implement IStaticRepoSource:
   • Partition → the target partition name (e.g. "Doc"). This is the target; it defaults to the repo's own partition — there is no separate "target" argument, you set it here.
   • Versioned → false for authored content (fingerprint on content hash so an edited file re-imports); true if the nodes carry meaningful versions.
   • SyncMode (optional) → the partition's PartitionSyncMode — what the import PRUNES. Defaults to FullReplace (mirror the partition to the repo). Override to Additive if users add their own nodes to this partition (the built-in AI catalogs do), or UpsertOnly to never prune.
   • EnumerateSourceNodes() → the partition's children, with full Content (e.g. MarkdownContent). Children + satellites only — never the namespace="" root.
   • PartitionRoot (optional) → a curated Space root (NodeType = "Space", MarkdownContent welcome). Return null to get a generic synthesized root.
2. Register it: services.AddSingleton<IStaticRepoSource>(new MyRepoSource()), gated behind Features:StaticRepoSync:Partitions via AddStaticRepoSync(serveFromPartition). For a synced partition the in-memory read-only static provider is skipped so Postgres serves + accepts the import.
3. That's it. On boot, StaticRepoImporter.ImportAll(hub) runs every registered source. To import one source directly: StaticRepoImporter.Import(hub, source).

Reference implementations: DocumentationStaticRepoSource, AgentStaticRepoSource, ModelStaticRepoSource. The importer: StaticRepoImporter.

Where the import runs — a dedicated hub, never the mesh router

The import runs on its own reachable hub (import/{meshHubId}), NEVER the root mesh hub. The bulk upsert traffic — every CreateOrUpdateNodeRequest, plus the inner CreateNodeRequest each one self-dispatches — must not be processed on the root mesh hub's single action block. That hub is the irreplaceable router; the moment it is busy creating import nodes it stops routing, every node op times out, and the whole portal wedges (the 2026-06-11 production outage: 11× CreateOrUpdateNodeRequest + 3× CreateNodeRequest@mesh/<self> stale >60s while real user SubscribeRequests starved).

StaticRepoImporter.ImportAll therefore creates one dedicated import hub and runs the whole import on it — the same reachable-hosted-hub pattern as the MeshNodeStreamCache cache hub:

meshHub.GetHostedHub(
    new Address(ImportAddressType /* "import" */, meshHub.Address.Id),   // process-unique
    config => config
        .AddData()                       // IWorkspace, so the upsert-of-existing path can dispatch
        .WithNodeOperationExecution()    // Create/CreateOrUpdate handled on THIS action block —
                                         // and IMeshService targets it instead of the router
        .WithInitialization(h => h.RegisterForDisposal(routingService.RegisterStream(h))),
    HostedHubCreation.Always);

• The import address-type is declared stream-routed — registered modularly by the owning module via MeshBuilder.AddStreamRoutedAddressType(StaticRepoImporter.ImportAddressType) in AddGraph (NOT hard-coded into the core MeshConfiguration.DefaultStreamRoutedAddressTypes, which keeps only the framework-core portal/client/cache). The silo's RoutingGrain dispatches to it over the cluster memory stream, and RegisterStream makes responses (query results, ImportContent acks) route back.
• Because the hub carries WithNodeOperationExecution, the bulk upserts are handled locally on the import hub (the inner self-posted CreateNodeRequest stays on it too). The mesh router only sees the occasional read query — never the create storm.
• 🚨 That opt-in is what makes the isolation real. Registering the handlers alone was not enough: IMeshService used to post every create/move/delete to hub.GetMeshHub() unconditionally, so anything the import routed through IMeshService walked straight back onto the router and this hub bought nothing. MeshExtensions.NodeOperationTarget now walks up to the nearest hub that declared WithNodeOperationExecution. The marker is deliberately separate from WithNodeOperationHandlers: every per-node hub registers the handlers (it must receive ops addressed to its own node), but a per-node hub also carries AddAccessControlPipeline, whose permission check is anchored at the receiving hub's path — so targeting one would evaluate Create against whatever node the caller happened to be rendering.
• Even a total import failure is isolated: the boot subscription (StaticRepoImportHostedService) is fire-and-forget with an onError terminal, so a wedged or failing import can never take down the router. The portal serves regardless.

What the importer does, per source, per boot

1. Resolve the Space root — source.PartitionRoot or a synthesized generic Space — and fold it into the source node set.
2. Provision the partition schema (IPartitionStorageProvider.EnsurePartitionProvisioned, lowercased, idempotent/promise-cached) before anything is written — the marker node in step 4 lives at {P}/_Activity/…, inside the partition schema, so a fresh partition would otherwise fault (42P01 — there is no lazy schema create; see [GhostSchemaInvariantTests]).
3. Fingerprint + short-circuit — PartitionSourceFingerprint.Compute(nodes + root). A converged Succeeded activity at {P}/_Activity/import-{fingerprint} may skip only when the current authoritative import manifest matches the source, including its root (the common case on every boot).
4. Stamp the marker, then open a fresh attempt — upsert {P}/_Activity/import-{fingerprint} to Running. That deterministic node is the durable "version vN imported at T" record — nothing else. It is not a lock: a marker left Running is deliberately reclaimed rather than obeyed, so two replicas booting together can both import. That is safe because every write on the path is an upsert. The run's own log goes to a fresh {P}/_Activity/import-{fingerprint}-{timestamp}-{rand} node, one per attempt. See Content-addressed Activity — the marker + the short-circuit below for why the two roles are split.
5. Ensure the Space root (standard step) via the canonical upsert — creating a Space triggers eager schema provisioning + the Admin/Partition/{P} routing prime + the admin grant; an existing root is updated. This makes the partition routable, listed in public.top_level_index, and gives it a landing page. Exception — a claimed root is left untouched: if the existing root carries SyncBehavior != Include (i.e. an admin set ExcludeThisAndChildren = "sync: none"), EnsureRoot does not re-materialise it. Re-materialising would reset the root's SyncBehavior back to Include and silently re-enable sync — see Decoupling a partition (sync: none) below.
6. Upsert every source node through CreateOrUpdateNodeRequest — the single canonical verb (the same one NodeCopyHelper uses). It creates absent nodes and updates existing ones (the owner re-stamps Version), running the full pipeline: prerender (MarkdownContent.Parse), embedding, satellites, access. Claimed subtrees are skipped — both a child claimed in the snapshot (SyncBehavior != Include) and an entire partition whose root is claimed (ExcludeThisAndChildren). The partition-root claim is read authoritatively (GetMeshNodeStream), NOT from the eventually-consistent query snapshot, so a just-set decouple is honoured before the read-model catches up (the snapshot lags writes — reading the claim from it re-synced the partition and clobbered the admin's edits: a production Provider/Anthropic key reset, 2026-06-25). Each upsert is independently guarded (per-file try/catch): a single node faulting (bad content, a validator reject, a transient owner timeout) logs a ⚠ Failed to import {path} line into the import activity and the import continues — the first failure never aborts the rest of the partition. Failures are tallied. 🚨 The writes are ORDERED, not a flat fan-out: a NodeType node lands before every instance that names it, and a type's Source/Test nodes land before the type — see Import Write Ordering, which also settles what happens to a type that arrives from another partition or repo, and the cycle policy. Without it a repo shipping an instance of a type it introduces was refused NodeType 'X' is not registered and the retry re-ran the identical ordering forever (issue #2556: 6,902 refusals in 90 minutes on memex-cloud).
7. Prune (per the partition's sync mode) — delete target nodes absent from the source (except governance _Policy/_Access/_Activity, claimed subtrees, and — in Additive mode — user-added nodes the source never owned). 🚨 A NodeType definition that still has instances is HELD, never pruned (NodeTypeInstanceProbe, since 2026-09-08): it stays with its own Source//Test/ subtree, is stamped pendingRetirement, is named on a ⏸ line, and counts as preserved so the run is not converged — the next sync asks again and prunes it once the instances are retyped or deleted (Dangling NodeTypes → hold while instances exist). Then write the terminal status atomically onto the ATTEMPT node via NodeTypeCompilationActivity.Complete: Succeeded when every node imported, Warning ("N FAILED (see ⚠ above)") when any per-file upsert failed — and Warning too when a claim refused to let a declared node be CREATED (🚫 … CANNOT BE CREATED, naming the paths; see A claim protects a node from being OVERWRITTEN below) — so the activity log never shows a green Succeeded while hiding failures, and the ⚠ lines pinpoint exactly which files to investigate. A hard fault in provisioning/root/read still MarkFaileds the attempt. The same verdict is then stamped on the marker (upsert), because that is what the next boot's short-circuit reads.

All writes run under AccessService.ImpersonateAsSystem (re-established at each write's own subscribe, since the System identity must reach the cross-hub write — see AccessContextPropagation.md). That includes the queued leg: IMeshNodeStreamCache serialises writes per path through a Concat, so every write after the first is subscribed on its predecessor's settle thread — the enqueuer's identity is captured at enqueue and carried onto the request, or the import's own progress write would go out unattributed and be refused by the very partition it is repairing (issue #919).

Per-partition sync mode (what gets pruned)

Upserting the source's nodes is the same in every partition; what differs is the prune — which live nodes an import removes after the upsert. That policy is the partition's PartitionSyncMode (MeshWeaver.Mesh.Contract), set on the source via IStaticRepoSource.SyncMode:

How Additive knows what to keep. Every import writes a per-partition manifest ({P}/_Activity/import-manifest) listing exactly the paths the source owned that run. On the next import, Additive prunes only (previous-manifest paths) ∖ (current-source paths) — so a node the repo dropped is still cleaned up, while a node the user created (never in a manifest) is never a prune candidate. On the very first import the manifest is empty, so Additive prunes nothing.

Defaults. FullReplace is the default for any source that doesn't opt in. The built-in AI catalogs — Skill, Agent, Provider, Harness — default to Additive (their IStaticRepoSource.SyncMode returns it), so a user's own skills/agents live safely next to the shipped ones. An operator can override any partition's mode by name via config: Features:StaticRepoSync:Modes:{Partition} = FullReplace | Additive | UpsertOnly (env form Features__StaticRepoSync__Modes__Skill=UpsertOnly).

Sync mode is per-partition; SyncBehavior is per-node — they compose. The mode decides which extras get pruned; the per-node SyncBehavior (below) still claims/protects individual nodes in every mode. A node marked ExcludeThisAndChildren is never overwritten or pruned regardless of the partition's mode; Additive/UpsertOnly additionally spare nodes the source never owned.

Add your own skill/agent that survives sync

Because Skill/Agent/Provider/Harness are Additive, you can simply create a node in that partition (e.g. a new nodeType:Skill node under Skill, from the GUI or MCP create) and it survives every re-import — it was never in a shipped manifest, so the importer never prunes it. Editing a shipped node instead? Claim it with SyncBehavior = ExcludeThisAndChildren (see below) so the next content-version doesn't overwrite your edit. (In a FullReplace partition like Doc, a hand-added node WOULD be pruned — claim its subtree or switch the partition's mode if you need it to persist.)

Decoupling a partition (sync: none)

A partition is DB-owned once seeded: an admin edits a synced node (a provider's API key, a doc page) and the change must survive the next import. The control is the node's SyncBehavior (MeshWeaver.Mesh.Contract) — which applies within every sync mode:

Set ExcludeThisAndChildren on the partition ROOT to decouple the WHOLE partition. On the namespace="" root (@Provider, @Doc, …) it makes the importer skip the root and every descendant — the partition becomes fully DB-owned and the static source inert. This is how you turn off sync for an entire catalog (e.g. Provider, so admin-managed AI keys are never reset). Flip it from the GUI (StopSyncLayoutArea toggles Include ⇄ ExcludeThisAndChildren) or via workspace.GetMeshNodeStream(root).Update(n => n with { SyncBehavior = SyncBehavior.ExcludeThisAndChildren }).

Two importer rules make a root claim durable (StaticRepoImporter):

• EnsureRoot leaves a claimed root untouched — it never re-materialises a root whose SyncBehavior != Include (re-materialising would reset the claim to Include and re-enable sync).
• The root claim is read authoritatively. ReadClaimedRoots reads each partition root via GetMeshNodeStream (the authoritative single-node read), not the eventually-consistent snapshot query — a claim set moments before an import would otherwise read back as Include and the partition would be re-synced. This is the CQRS rule: never decide on a single node's content from Query. (The lagged snapshot reading the claim is exactly what re-synced Provider and clobbered the admin's Anthropic key, 2026-06-25.)

🚨 A claim protects a node from being OVERWRITTEN — never from being CREATED

Claim the narrowest thing that carries the edit. ExcludeThisOnly claims the node an admin edits and leaves its children syncing; ExcludeThisAndChildren claims the whole branch. Both are right in their place, and picking the wide one "to be safe" is not safe — it also refuses the creation of every node the source declares underneath, forever. There is nothing to protect on a path the mesh has no node at, so a claim that blocks a create is wider than the thing it protects.

That is exactly how a deployment's configured AI models went missing (issue #2211): the catalog seeded Provider/{name} as ExcludeThisAndChildren to protect an admin's key, which also froze its LanguageModel children — so a model configured after the provider node existed could never materialize. It now seeds ExcludeThisOnly; the children carry no credential and stay synced, which is what makes the configured model list enforceable at all.

The import reports what a claim refused. Every claimed skip is counted, and a claimed skip of a path the mesh has no node at is collected as a blocked create (StaticRepoImportResult.BlockedCreatePaths) and NAMED in the activity summary and the log. When there is one, the run's terminal status is Warning, never Succeeded, and the outcome is ImportedWithBlockedCreates — because Succeeded at a fingerprint IS the durable short-circuit, so recording it there froze the divergence permanently and invisibly. A partition the operator decoupled wholesale ("sync: none" on its root) is the one carve-out: declining everything there is the instruction, so it is stated rather than escalated.

🚨 Decouple by claiming the root — never by unregistering the source. Dropping a partition from AddStaticRepoSync (config or code) removes it from the current source set → ReconcileSourceOwnedPartitions treats it as an orphan and DeleteNodes the whole partition (every node + key). Keep the source registered and claim the root instead.

Why CreateOrUpdateNodeRequest, not CreateNode + Overwrite

The importer must be idempotent over existing rows (re-imports, eventually-consistent snapshots, and especially the migration backfill's content-NULL shadow rows). Plain CreateNode faults on an existing node; a hand-rolled stream-Overwrite re-asserts the same Version, which the owner drops as not-newer — so content silently never lands. The canonical CreateOrUpdateNodeRequest does the right thing for both cases and increments the Version on update, so the write is accepted and persists. This is non-negotiable: do not re-implement create/update in the importer.

Creates travel in BULK; updates do not — and why that is not a compromise

An import used to cost one write request and one single-row upsert per node. Storage has had the bulk contract all along — IStorageAdapter.WriteMany, which the PostgreSQL adapter implements as one NpgsqlBatch per (schema, table) window: N upserts, one round-trip, one implicit transaction, the same upsert SQL the singular Write uses. Nothing on the import path reached it, because a node write has to route through the mesh's canonical verbs, and WriteMany is an IStorageAdapter call one level below them. Calling the adapter from the importer would have bought the batching by dropping the typing, access check and per-node-hub observability the write depends on.

The answer is a bulk verb, not a bypass. CreateNodesRequest is the bulk sibling of CreateNodeRequest and runs the identical pipeline — partition bootstrap once per partition, every validator including RLS for every node, the type-existence probe per distinct type, then one WriteMany, the Created change-feed publishes in caller order post-commit, and the post-creation handlers. So the importer splits each write stage (see Import Write Ordering — everything inside a stage may be written concurrently, a stage begins only after the previous one completed):

Two properties are load-bearing and neither is free:

1. Per-file isolation survives. CreateNodesRequest is validate-all-then-write: one offender refuses the whole request and can name at most one path, while the import's contract is the opposite — every other node still has to land, and the one that did not has to be named, because Failed > 0 is what holds the git baseline. So a refused or faulted batch is re-run one node at a time, which attributes the failure to the file that caused it and lands everything else. The extra pass is paid only on the failing chunk. Likewise a path the batch reports in CreateNodesResponse.Existing — the snapshot said "absent", the authoritative read disagreed — is an update, and falls through to the per-node verb rather than being dropped.
2. The chunks are MERGED, not concatenated. BatchSize bounds concurrent heavy operations, and one bulk request is internally sequential. Running the chunks back to back would cut the import's concurrency from ~5 to 1 and could make a large first import slower than the per-node path it replaces. Merged at BatchSize, the bound is unchanged and each of those in-flight operations now carries 25 nodes instead of one.

StaticRepoImportResult.WriteRequests reports the round-trip count — ⌈bulk creates ÷ 25⌉ plus one per node written individually — and the import's summary log line carries it, so "did this import batch?" is answerable from an operator's log rather than by reading code. A first import of 40 fresh nodes costs 2 requests; it used to cost 40.

Scope: mesh nodes AND content-collection files

The import materializes both mesh nodes (a node's Content + prerendered HTML, via the node upsert above) and, for sources that need it, the node's content-collection files — the assets a node references through the content collection, e.g. an @@content/logo.svg image embed on a Space page.

Those files live in a per-node content collection, not the node row — and where that collection is mapped is a host decision, not something every node hub gets automatically. The memex portal (MemexConfiguration.ConfigureDefaultNodeHub) maps a writable content collection only on Space/partition roots (nodePath with no /), rooted at {Storage:BasePath}/content/{nodePath}; a read-only embedded source like Doc instead maps each node's own embedded Content/<subpath> subfolder as its read-only content collection (AddDocumentation.ConfigureDefaultNodeHub), so a doc page's @@content/<file> embed is served straight from the shipped assembly — no copy needed. Files are read via IFileContentProvider.GetFileContent("content", "<file>") on that node's hub.

How content files are synced (collection → collection)

This copy path is for a source whose target nodes have a writable per-node content collection (e.g. a GitSync Space whose assets ship in a FileSystem source). The built-in Doc partition does NOT use it — a child doc node like Doc/DataMesh/UnifiedPath has no writable content collection (the portal maps writable content only on Space roots), so the copy had nowhere to land. Instead, AddDocumentation maps each Doc node's own embedded Content/<subpath> subfolder as its read-only content collection, and @@content/<file> resolves directly from the shipped assembly. Verified by DocContentEmbedRenderTest.

1. A source that needs the copy declares its imports by overriding IStaticRepoSource.EnumerateContentImports() → StaticContentImport(NodePath, SourceCollection, SourcePath, TargetCollection="content", TargetPath=""), shipping the assets in a source content collection readable on the owning node's hub.

2. After the node upsert the importer's SyncContentImports posts the canonical ImportContentRequest per entry, under ImpersonateAsSystem, via the fluent API in MeshWeaver.ContentCollections:

   hub.ImportContent(nodePath)
      .From("<sourceCollection>", "<sourceFolder>")  // source collection + folder
      .To("content")                                 // WRITABLE target collection on the node
      .Post()                                        // → ImportContentRequest to the OWNING node's hub
   

   The handler (registered by AddContentCollectionsInfrastructure, so every content-enabled node hub has it) resolves both collections via IContentService and copies each direct-child file of the source folder stream-to-stream so binary assets (svg/png) survive — GetFiles/GetContentAsync on the source, SaveFileAsync on the target — with the whole copy sealed in one IIoPool operation (the hub action block only subscribes + returns; async never runs on the hub). The file is then served through /api/content/{address}/content/<file> — the access-controlled content route, gated on Read of the owning node (issue #587). No hand-rolled cross-hub IFileContentProvider write, no async on a hub path, and no second ImportContentRequest (the type is wire-registered — a duplicate collides). The source collection must be exposed on the node hub (via ConfigureDefaultNodeHub) so the node-hub handler can read it.

This is how a source with a writable target collection gets its @@content/<file> assets to land on a fresh deployment (e.g. a FileSystem source at /mnt/content): the importer copies them into the runtime content collection on boot, alongside the nodes. Tests: ContentImportSyncTest (monolith, filesystem + embedded sources) and OrleansContentImportSyncTest (the distributed cross-grain path — the shape that must not deadlock).

The same landing, reached from a REGISTRY install (issue #848)

The two paths above are for sources compiled into the portal. A course or plugin installed from the plugin registry is not one — so for a long time nothing copied its assets, and its content/** binaries had to be uploaded to each portal out of band. PackageInstaller.SyncPackageContent closes that: after the nodes land it classifies the package's {package}/content/** files with the same ContentAssetMapper and posts the same SyncContentFilesRequest (the byte-carrying sibling of ImportContentRequest) to the partition ROOT — the one hub where the per-Space content collection resolves.

Two things make that work end to end, and both are easy to break again:

• The bytes must survive the transport. RepoFileCodec classifies a non-UTF-8 blob as binary and puts its bytes on RepoFile.Binary, leaving Content deliberately empty. Any projection that copies only Content therefore publishes an empty file — which is exactly what POST /api/plugins/files did (content = 0 chars). PackageFile mirrors RepoFile (Content + Binary + Bytes) and every package source must carry both.
• The install is ADDITIVE, never a mirror. GitSync mirrors because it owns the whole Space; an install does not, and portals carry assets uploaded by hand that the repo has never tracked. Note that SyncContentFilesRequest.Mirror is a declared-true bool and so carries [JsonIgnore(Condition = Never)] — without it the hub serializer drops an explicit Mirror = false (it is the CLR default) and the receiving hub rebuilds it as true, silently turning an additive sync into a pruning one.

Test: PackageContentAssetInstallTest — the bytes land at the exact layout /api/content/{root}/{file} resolves to, a text asset still round-trips, and an untracked upload survives a re-install.

🚨 One sync is MANY deliveries — the bytes are never sent whole (#2885)

ContentAssetMapper.ToContentSyncs still emits one StaticContentSync per Space — the mirror has to be one authoritative pass over the whole collection — but SyncContentFilesBuilder.Post no longer turns that into one message. It partitions the files against DeliveryPayloadBounds.MemoryStreamBlockBytes (measured on the base64 form, which is what the message weighs) and posts them with Concat, so the portal ever holds one batch. A whole-Space message was 28 MB of course video for AgenticBusiness — ~114 MB of transient allocation per hop, which took a production pod down twice — and 141 MB packaged for AgenticEngineering, over the frame limit entirely, so that Space's assets silently stopped syncing once the producer-side refusals landed.

The prune does not chunk: it rides the first delivery and carries SyncContentFilesRequest.MirrorKeepPaths, the full keep set as paths. First rather than last is the safety property — an over-prune would be repaired by the writes queued behind it. Full argument and arithmetic: Oversized Delivery Refusal → "The producer that was building it whole". Tests: ContentSyncIsNeverBuiltWholeTest, ContentSyncMirrorSurvivesTheSplitTest.

The two primitives

1. Source fingerprint — the content-version

A deterministic, order-independent hash over the source node set (children + the Space root):

for each source node:  line = path + "\0" + (Versioned ? version : sha256(content))
sort lines by path                     // order MUST NOT affect the hash
fingerprint = sha256( join(lines, "\n") )[..16]

Changes iff a node is added, removed, or modified — including an edited welcome (the root is in the set). Helper: PartitionSourceFingerprint.Compute.

2. Content-addressed Activity — the marker + the short-circuit

The import is governed by TWO Activity nodes with deliberately different lifetimes.

The marker — {Partition}/_Activity/import-{fingerprint}, id = the fingerprint:

• A Succeeded activity records that this fingerprint was imported previously. The skip also checks its convergence verdict and the current authoritative import manifest; a historical marker alone does not establish the partition's current source. This is the marker's whole job, and it is why the id must be derived from the content.
• Changed source ⇒ new id ⇒ a fresh import runs. Old import-{prevHash} markers remain as a visible import history.
• It is written only through the idempotent upsert (CreateOrUpdateNodeRequest), twice per run: Running at the start, then the terminal verdict. That verb floors the version on the durable row it just read, so a forked/ghost row is repaired in place rather than silently discarded (#902/#909).
• 🚨 It is not a lock, and nothing else serialises replicas. A marker left Running — by a crashed import, or by a rollout briefly running two pods — is reclaimed, not obeyed: the guard re-imports on anything that is not Succeeded. Obeying it was the old behaviour and it wedged a partition into "AlreadyRunning, 0 nodes" forever (the prod Agent/Harness/Command wedge). So two replicas on the same fingerprint can import concurrently. That is safe only because every write on the path is an upsert of byte-identical content — do not add a step here that is not idempotent, and do not treat "the lock protects me" as an available argument.

The attempt — {Partition}/_Activity/import-{fingerprint}-{timestamp}-{rand}, a fresh id per run, carrying every progress line, per-file ⚠/🗑 diagnostic and the run's terminal summary.

🚨 The split exists because a deterministic id is a trap for anything written many times per run. When one poisoned row sat at import-{fingerprint} (memex Store, 2026-08-07: v166, 16 KB, unloadable by its own hub), every retry re-targeted that same node and died identically — each progress write burning the full 30 s no initial state arrived abort — and the only way out was deleting the row by hand in SQL. Minting the attempt id fresh means a faulted attempt can never be re-targeted; keeping the marker deterministic preserves the content-addressed short-circuit that is the whole point of it (issue #919).

Why startup, not the SQL migration

The SQL migration (Memex.Database.Migration) is a standalone process with no live mesh — it cannot post CreateOrUpdateNodeRequest or compute prerender through the pipeline. The portal boot has the live mesh, so the canonical pipeline is available there. The migration owns schema (DDL); static-repo content is owned by this startup import. The content-addressed activity is exactly what makes "runs in the portal" safe across replicas (not "runs N times").

(DocumentationBackfill — the old raw INSERT … content = NULL search-index write in the migration — is now redundant: the imported rows are content-bearing. It is harmless but its content-NULL rows, if present, are refilled by the import's upsert.)

Distributed serving (why this matters)

In the distributed (Orleans/PG) portal, routing does not consult the in-memory EmbeddedResourceStorageAdapter — so a partition that is only served from the embedded overlay 404s / hangs. The static-repo import is what makes built-in partitions (Doc/Agent/Model) served from the DB there. The monolith (in-process embedded routing) works either way, so the cutover is gated by Features:StaticRepoSync:Partitions (default ["Doc","Agent","Model"] for the distributed portal; monolith leaves it empty and keeps in-memory serving).

Returning to a previously imported source

A source can move B → A → B during a rollback or a change of sealed publication. The old import-{B} activity survives the A import because import history is outside the content prune. Its success is historical evidence. Before skipping, the importer reads the current import-manifest authoritatively and compares its content tokens with every requested non-empty node path and the root. A mismatch clears any Git-diff scope and evaluates the full source with the existing conflict policy. The manifest remains the same path-to-token map; older maps without a root entry receive one incremental full pass before they can authorize a skip. The root is always recorded as evaluated: EnsureRoot runs independently of the child Git-diff scope. Retaining its previous token after a scoped root change would incorrectly authorize the previous root's historical marker.

GitHubSyncService.ReconcileAtCommit likewise evaluates the whole source, without a Git comparison or a per-node manifest shortcut. A recorded B SHA makes the Git diff B..B empty even if the live mesh contains A. Reconciliation preserves two-way human edits and claims; it never sets Force. A subsequent ordinary import of a converged, unchanged source skips with no content writes.

Measured failure, memex-cloud, 2026-09-10 UTC. Store imported the new coupon/course-size source at 18:55 (Store/_Activity/9817ab34, commit 06d8187049f38aa8831a423ed64f32965af424cb). A sealed-source reconciliation then imported f4570459a34a64c4edd9b17a4a9b59a62a50165e at 19:10 (Store/_Activity/3f12d540), restoring the previous source and pruning six new nodes. At 21:07, attempts Store/_Activity/a24af2d7 and Store/_Activity/e348d6d8 targeted the new Store-equivalent commit 8ee8a1928d278aa74a162353dfa663ffac44c28c but explicitly skipped on historical marker Store/_Activity/import-1c252ab2c7141f07. _GitSync recorded that SHA as seen, while the actual Plugin source remained at fingerprint cc84832d2d7d3575 and the new nodes were absent. This is a bounded historical observation, not a claim about production after later imports or replica turnover.

ReturningSourceConvergesTest drives the real GitSync service and monolith importer, substituting only the GitHub I/O boundary. Its initial six-case matrix, before the scoped-root follow-up below, fails on unchanged core f1a945d5: B → A → B returns Skipped (including a root-only change); both same-SHA reconciliation cases restore the missing node but leave the existing source stale; an ordinary import whose recorded SHA is already B skips; and explicit reconciliation trusts a matching manifest despite measured live drift. The corrected regression also verifies unchanged repeats and a person's two-way edit with its conflict horizon held. These checks do not establish distributed serialization between replicas that are concurrently importing different selected publications; durable delivery still requires the intended publication and actual final source fingerprints to agree.

Initial local validation: those six cases are red on unchanged core f1a945d5 and green with the initial correction. All 19 targeted Hosting/GitSync cases and 51 existing Graph importer cases pass. Release builds of the touched projects and test dependencies report zero warnings and errors. The regression reads actual node content and checks written/pruned paths; it does not assert only on the sync SHA or on a success message. Production acceptance remains a separate release step.

Scoped-root follow-up: ScopedRootChange_CannotLeaveThePreviousRootsMarkerCurrent fails on 7aac375b: a full B import, followed by A with only index.json in the Git diff, writes root A but retains B's root token; returning to B skips and leaves the actual root at A. Recording the root's current token independently of the child scope corrects this. The test checks actual root content and the subsequent ordinary zero-content-write repeat. The new case is red on 7aac375b and green with the correction. This follow-up expands ReturningSourceConvergesTest from six to seven executed cases; all 20 targeted Hosting/GitSync cases pass on the corrected source. The initial six-case and later scoped-root red receipts describe separate validation runs.

Review controls, 2026-09-11: a Versioned source's revision changes the partition fingerprint; the manifest deliberately stores authored-content tokens. The new VersionOnlySourceRevision_DoesNotRewriteIdenticalContentOrTheOwnerClock case passes on unchanged f1ce3394: importing identical content at revisions 42 → 43 → 42 → 43 preserves both the content and the mesh owner's node version. A source revision alone must not manufacture a content write.

The separate EmptyPathIgnoredByTheFingerprint_DoesNotInvalidateTheCurrentManifest case exposes a real mismatch on that same baseline: the fingerprint and manifest writer omit empty paths, while the returning-source comparison included one. An otherwise unchanged repeat returns ImportedWithContentErrors instead of Skipped. Applying the same non-empty-path filter to the comparison keeps those three decisions consistent. Its comparison map and the test fixture's source list use immutable collections (correction commit 6dae9421). All nine cases then pass: the previous seven plus these two review controls. The 25 existing conflict-policy, scoped-marker and sync-mode cases also pass. Strict Release Hosting.Test and Graph.Test builds report zero warnings and errors. The distinct baseline and corrected receipts are /tmp/core3991-review-results/core3991-review-before.trx and /tmp/core3991-review-results/core3991-review-final.trx; the existing controls are in /tmp/core3991-review-results/core3991-review-graph.trx. The review baseline executes the two new controls: the version-only case passes and the empty-path case fails. The strict Documentation.Test build and 12 release-note, link and embed integrity cases also pass; their receipt is /tmp/core3991-review-results/core3991-review-doc.trx.

Removal review: adding the root to the manifest does not make it a prune candidate. Run reads existing descendants, and ComputePrunableNodes filters that existing set; manifest keys only narrow ownership in Additive mode. Claims, incomplete-listing refusal, protected local edits, held NodeTypes and their source subtrees retain their existing guards. Removing an entire compiled source partition still uses the independent Admin/_SourceOwnedCatalogs registry. Removing its root index file instead restores the standard synthesized root; it does not remove the partition.

A separate pre-existing boundary remains outside this fix: Run derives touched partitions solely from the current child nodes. If a source removes its last child, that list is empty and the existing-subtree read is empty, so old children are not presented to the prune. The root-inclusive manifest does not introduce that omission or repair it. This is a source-path finding; no new runtime test or production claim accompanies it here.

Integration review, 2026-09-10: main 89c914b64e6b5ff4c70f36de8604ac84c7c6e29b changes DataExtensions and read-visibility tests through #3976; PR #3981 head b7d011dd2cddb5c105e87c77420fc519d2575938 adds the content-workflow admission gate and adjusts its webhook fixtures. Neither changes this patch's importer/service/test files. The complete importer patch, including the scoped-root correction, applies cleanly to each tree and their clean combined tree 58d388500f458e2a51ba38d4314b7a1e83585be0, checked with a temporary Git index without changing any checkout. The two fixes govern different decisions: #3981 determines which CI completion may request an import; this fix determines whether the requested source is already present.

After the coordinated release hold is lifted, transplant both importer commits in order onto a fresh checkout of the then-current main; do not cherry-pick only 7aac375b and lose the scoped-root correction. Re-read main and the owner's final source first, preserve #3981's workflow-path fields in the existing webhook fixtures, and rerun the Hosting/GitSync regression selection, Graph importer selection and documentation/reactive guards on the combined tree. A clean patch application is not a compiled or tested integration receipt. Ship through the normal core/CD and sealed-publication workflow; verify the selected publication plus actual Store source/compiled fingerprints after normal reconciliation. No force import or conflict-policy bypass is part of the transplant.

🚨 A CONTENT verdict is final for its fingerprint; a transient failure is not

The marker at {Partition}/_Activity/import-{fingerprint} is content-addressed, so the fingerprint is the idempotence key. Re-running at the same fingerprint re-reads the same nodes and re-derives the same answer — which means re-running is only ever meaningful when something outside the content could have changed.

The skip arm used to recognise Succeeded and nothing else, so every non-green verdict re-imported in full on every trigger. Measured on memex.meshweaver.cloud (2026-09-02, #3146): 19 complete passes in 3 h, each ≈425 identical InvalidOperationExceptions — "A 'Space' owns its partition, so it must be top-level", "NodeType 'Northwind/Article' is not registered" — plus a NodeType compile of the same sample tree each time, on a portal already at 8/8 replicas. The trigger was a webhook per green core CI run.

So the outcome now distinguishes why a pass failed:

ImportedWithContentErrors is reached only when every failure in the pass was a content verdict (StaticRepoImporter.IsContentVerdict), and that is decided on the owner's structured NodeUpsertRejectionReason, never on the exception type:

🚨 The type cannot be used for this. Upsert wraps every failed CreateOrUpdateNodeResponse in one InvalidOperationException, and Unknown is where "Persistence read failed: …" and "Inner CreateNode faulted: …" land — a store that was briefly unreachable, wearing the same exception as a rule refusal. A first draft of this classifier keyed on the type and would have marked those final; that is #3101's freeze, re-introduced by the fix for #3146. Upsert therefore throws a typed UpsertRefusedException carrying the reason. One retryable failure among the pass's failures keeps the whole pass on the ordinary Warning arm.

Known residual: a validator that throws — including one whose own dependency is briefly unavailable — is mapped to ValidationFailed before the response is built, so it records as final. The importer cannot separate that from a rule refusal; the place to fix it is the validator contract (an unavailable dependency should surface as unavailable, not as a refusal). Pinned, so it is not rediscovered as a surprise, by FailedImportIsNotRetriedAtTheSameFingerprintTest.

🚨 Getting that backwards is the more expensive mistake, and it has a number: #3101, a Space frozen out of the mesh by a green marker nobody re-examined. That is why this is an allow-list of verdict-shaped faults rather than "anything that is not obviously transient".

The verdict is carried in ActivityLog.ReturnValue as {"outcome": "…"} — structured, not parsed out of the summary line. A skip decision that read a log message would start silently re-importing forever the first time somebody re-worded it. Force still re-runs regardless; a changed fingerprint re-runs by construction, because the fingerprint is the marker's id.

Import / export symmetry

There is no static-repo exporter in src/, and nothing gates the two directions against each other. No code writes a {Partition}/_Activity/export-… node, and no code checks for an in-flight import-* / export-* before starting — consistent with the marker not being a lock (above). Mesh → repo today is GitHub Sync, which is a separate mechanism with its own bookkeeping, not the mirror image of this importer. If you add an exporter, do not assume a mutual-exclusion gate exists to inherit; there is none to inherit.

Reuse map (do not re-invent)

ImportNodesRequest (in ImportDeleteRequests.cs) is dead/unimplemented — do not use it; this pattern supersedes it.

Status

Shipped and enabled for Doc / Agent / Model on the distributed portal. The migration backfill is superseded (content-NULL rows are refilled by the import).

Invariants (tested — StaticRepoImporterTests, PartitionSourceFingerprintTests)

• Fingerprint is order-independent and changes on add/remove/modify.
• Import materializes children with non-NULL content + PreRenderedHtml (round-tripped from PG) and a namespace="" Space root.
• Only the lowercased partition schema is provisioned — never a verbatim/capital ghost.
• A changed source re-imports and increments the Version of updated nodes (the canonical-upsert guarantee).
• An import over a content-NULL row refills its content (the migration-backfill shadow case).
• A node absent from the source is pruned in FullReplace; in Additive only a node the source previously owned is pruned (a user-added node survives); UpsertOnly prunes nothing (StaticRepoImporterSyncModeTest).
• A NodeType definition that still has instances is held, not pruned — kept, stamped pendingRetirement, counted as preserved (not converged) — and is pruned by the next import once the instances are gone; a type with no instances is pruned on the first pass (ImportDanglingNodeTypeTest).
• Re-run with an unchanged source is a no-op (fingerprint short-circuit).
• The import runs on the dedicated import/{meshHubId} hub, not the root mesh hub — the bulk create/upsert traffic never touches the router (verified end-to-end by OrleansStaticRepoImportTest / OrleansContentImportSyncTest, which complete only because the import hub is reachable).
• A single node failing does not abort the import; it logs a ⚠ line and the activity ends Warning, not a green Succeeded.
• A first import of a whole partition costs ⌈n ÷ 25⌉ write requests, not n — and a node the batch cannot carry (a satellite) or that a validator refuses is routed around it / re-run individually, so it is still named and everything else still lands (StaticRepoImportBulkWriteTest).