Storage Adapter Implementation

MeshWeaver's storage layer routes reads and writes across multiple backends without a central registry. There is no global map of "which partition owns which path" — routing is implicit, driven entirely by each adapter's return value.

This document explains how that routing works and how to implement the two contracts correctly.

PersistenceService dispatches via try-then-claim for reads/writes (first non-null wins) and fans out across all adapters for existence/listing queries.

How Routing Works

PersistenceService is the singleton that coordinates all storage operations. It does not consult a registry or predicate before dispatching — it simply sequences adapters and lets their return values speak.

The two contracts at a glance

Dispatch behaviour by operation

• Read — walks adapters sequentially (Observable.Concat); picks the first non-null result.
• Write — walks writable adapters (IsReadOnly == false) sequentially; first non-null wins. Throws "could not save" if every adapter returned null.
• Delete — containment-check and delete on every writable adapter; throws if no adapter held the path.
• Exists / FindBestPrefixMatch / ResolvePath / ListChildPaths — fan-out across all adapters (writable and read-only); aggregate results.

There is no Matches(path) predicate. The question "is this mine?" is answered entirely inside each adapter's Read and Write implementation. That predicate was removed; see Migration notes below.

Implementing IStorageAdapter

There are two archetypes. Choose the one that matches your adapter's role.

A. The adapter owns its own data

This covers InMemory, FileSystem, PostgreSQL, Cosmos, and Blob adapters — any adapter that is the terminal store for a set of paths.

public sealed class MyStorageAdapter : IStorageAdapter
{
    private readonly ConcurrentDictionary<string, MeshNode> _nodes = new();

    public IObservable<MeshNode?> Read(string path, JsonSerializerOptions opts)
        => Observable.Defer(() =>
        {
            _nodes.TryGetValue(Normalize(path), out var node);
            return Observable.Return(node);   // null if not present — caller's chain skips us
        });

    public IObservable<MeshNode?> Write(MeshNode node, JsonSerializerOptions opts)
        => Observable.Defer(() =>
        {
            // Decision point: does this adapter accept this path?
            if (!ShouldAccept(node.Path))
                return Observable.Return<MeshNode?>(null);   // try-then-claim falls through
            _nodes[Normalize(node.Path)] = node;
            return Observable.Return<MeshNode?>(node);
        });

    public IObservable<string> Delete(string path)
        => Observable.Defer(() =>
        {
            // Containment check happens here; if we don't own it we still
            // emit the path (PersistenceService.Delete reads back to decide
            // who actually deleted).
            _nodes.TryRemove(Normalize(path), out _);
            return Observable.Return(path);
        });
    // … remaining methods follow the same shape.
}

The key decision is in ShouldAccept. For an InMemory wildcard adapter that accepts everything, it is simply !string.IsNullOrEmpty(GetFirstSegment(path)). For a per-partition adapter scoped to one schema, it is path.StartsWith(_schema + "/"). There is no external routing layer that calls a Matches predicate — the decision is encapsulated here.

B. The adapter routes to other adapters

Examples: PostgreSqlPathRoutingAdapter (one schema-bound adapter per partition), VersionWritingStorageAdapter (a decorator that chains a version-write).

public sealed class MyRoutingAdapter : IStorageAdapter
{
    public IObservable<MeshNode?> Write(MeshNode node, JsonSerializerOptions opts)
    {
        // Map the first path segment to a target SYNCHRONOUSLY — no existence probe,
        // no cached partition state, no lazy CREATE SCHEMA on the write path.
        var target = ResolveTarget(GetFirstSegment(node.Path));
        return target is null
            ? Observable.Return<MeshNode?>(null)       // not mine — chain falls through
            : GetOrCreateAdapter(target).Write(node, opts);
    }
}

🚨 A routing adapter must NOT create the partition it is routing to. PostgreSqlPathRoutingAdapter's lazy EnsureSchemaForPartitionSync was deleted from both write paths (RouteWrite and CreateAdapterForTable): any unrecognised first segment — NodeType names, reserved words, request URLs — used to spawn a ghost schema in production. A write to an unprovisioned partition now faults 42P01 ("no partition, no write"). Provisioning is explicit, via IPartitionStorageProvider.EnsurePartitionProvisioned(namespace); see Partition Storage Routing.

Reads are the mirror image: they tolerate an absent schema (42P01 → empty result), never an error and never a slow tree walk.

Implementing IPartitionStorageProvider

The provider is the thin wrapper that wires an adapter into the PersistenceService chain.

public sealed class MyPartitionStorageProvider : IPartitionStorageProvider
{
    public string Name => "MyBackend";
    public bool IsReadOnly => false;          // false for InMemory/FS/PG/Cosmos/Blob
    public IStorageAdapter Adapter { get; }   // the actual adapter
    public PartitionDefinition? PartitionDefinition => null;  // null = backend-wide
}

Read-only providers (EmbeddedResource, StaticNode) set IsReadOnly = true. They still participate in reads — their adapter's Read method returns seed data — but PersistenceService.Write skips them entirely.

The "No Async Ever" Rule

Every method on IStorageAdapter returns IObservable<T>. There is no Task<T> on the public surface and no await between adapters and the routing layer.

🚨 Bridge async leaves through IIoPool — NEVER Observable.FromAsync. Observable.FromAsync is forbidden everywhere in src/ outside IoPool itself: it runs the function's synchronous prologue on the subscribing thread (the hub/grain scheduler, when the subscribe happens mid-handler) and applies no concurrency bound — exactly the deadlock-and-exhaustion class the pool exists to kill.

// ❌ FORBIDDEN
=> Observable.FromAsync(ct => ReadRowAsync(path, ct));

// ✅ Resolve an IIoPool from IoPoolRegistry (mesh-scoped singleton, never static)
=> _ioPool.Invoke(ct => ReadRowAsync(path, ct));            // Task<T> leaf: DB, blob, HTTP
=> _ioPool.InvokeBlocking(ct => File.ReadAllBytes(p));      // sync-blocking / CPU leaf
=> _ioPool.InvokeStream(ct => EnumerateRowsAsync(ct));      // IAsyncEnumerable<T> leaf

PostgreSQL pools are named pg:{adapter} and capped at 1, so the gate is the single Npgsql connection. Idempotent one-shots (schema provisioning) use the promise-cache — pool.Run(...) stashed in an instance ConcurrentDictionary. Full reference: Controlled I/O Pooling.

Postgres-Specific: there is no partition-state cache

PgPartitionCache and PgPartitionNotifyListener do not exist — the whole probe/TTL/invalidate machinery was removed (issue #15). The router does not cache or probe schema existence at all: it maps the first path segment to a schema name synchronously, and reads tolerate an absent schema (42P01 → empty). A partition created on another silo is therefore routable immediately, with no invalidation round-trip and nothing to go stale.

The only boot-time work is PostgreSqlPartitionSubscriptionHostedService, which does CREATE SCHEMA + table init for every framework partition explicitly advertised by an IStaticNodeProvider. It does not enumerate existing schemas.

Common Mistakes

Returning a thrown observable instead of null on decline. Observable.Throw<MeshNode>(...) propagates up to the caller and breaks the chain. Observable.Return<MeshNode?>(null) lets the next provider try. Decline means null; throw means a real error.

Doing the containment check in the routing layer. Adapters self-check. PersistenceService only sequences and aggregates — it never inspects path shapes itself.

Loading all partitions at startup. The routing layer never enumerates. Only explicitly-advertised framework partitions are seeded at boot; everything else routes by first path segment, with reads tolerating an absent schema.

Creating the partition on the write path. A routing adapter maps a path to a target; it never runs CREATE SCHEMA. Provisioning is an explicit, pooled, promise-cached EnsurePartitionProvisioned(...) subscription.

Calling a Matches() method. There is no Matches. That predicate was removed; routing is now driven entirely by Read/Write return values.

Reaching for Observable.FromAsync. Forbidden outside IoPool. Bridge every async / blocking leaf through IIoPool.

🚨 Storing the INSTANCE and forgetting what the serialization boundary does for free (#3816). An in-memory adapter that keeps the MeshNode object hands the same object back on Read — so everything a serializing backend does on the way through silently does not happen. Each such effect has to be reproduced by hand, and each one that is missed is a difference in behaviour between backends that only shows up as a test failing on one of them.

Two are known, and they arrived a year apart:

• MeshNode.HubConfiguration — an in-process delegate a durable store cannot hold; stripped explicitly, because FileSystem and Postgres drop it at the boundary.
• Content materialisation — a node whose Content is the as-written JsonObject DOM is returned as a JsonObject, so node.Content is TRecord (what every ordinary reader does) answers false on that backend and true on the others.

The second cost two full-project runs to find: AnInstanceReportsWhatItRunsTest failed 2 of 2 under load on an unmodified main and passed under --filter, because whether the last write left a typed record or the DOM is a matter of ordering. Deterministic in both directions is the tell that it is not a flake.

🚨 Reproduce the serializing backends, never improve on them. An unresolvable discriminator must still degrade to a JsonElement here, because that is what FileSystem and Postgres yield for the same content — an in-memory store that materialised what the others cannot would be a different lie in the other direction, and it would hide exactly the defect the degradation exists to surface.

Migration Notes from the Old Matches Design

References

• Change-Feed Isolation — what IStorageAdapter.Changes guarantees, and why an adapter must publish through IsolatedChangeFeed rather than a plain Subject<T>.
• Silent Completion — why Write's null decline sentinel must never be filtered away.
• src/MeshWeaver.Mesh.Contract/Services/IStorageAdapter.cs — the adapter contract.
• src/MeshWeaver.Mesh.Contract/Services/IPartitionStorageProvider.cs — the provider contract.
• src/MeshWeaver.Hosting/Persistence/PersistenceService.cs — try-then-claim write, fan-out read/delete.
• MeshWeaver.Plugins/src/MeshWeaver.Hosting.PostgreSql/PostgreSqlPathRoutingAdapter.cs — synchronous first-segment → schema/table routing.
• MeshWeaver.Plugins/src/MeshWeaver.Hosting.PostgreSql/PostgreSqlPartitionStorageProvider.cs — EnsurePartitionProvisioned (pooled promise-cache).
• src/MeshWeaver.Mesh.Contract/Threading/ + Controlled I/O Pooling — IIoPool, the only sanctioned async bridge.