A test that flips may be a platform skew, not a race

The rule. Before calling a satellite test non-deterministic, reproduce each verdict against a named core commit. A suite in this repo compiles against $(MeshWeaverRoot) — a local Systemorph/MeshWeaver checkout, at whatever commit and whatever bin/ state that checkout happens to be in — so "the same binary" is a claim about the test assembly, never about the platform assemblies it links. Two runs minutes apart can straddle a core commit.

What was reported (MeshWeaver.Plugins#2109)

src/MeshWeaver.Auth.Test/ApiTokenServiceTests.csDeleteToken_NonexistentPath_Completes answered differently on two consecutive runs:

Run Assertion in the binary Failure
1 — 16:00:10Z, full suite of 160 result.Should().BeFalse() Expected value to be false, but found True.
2 — 16:03:15Z, rebuilt result.Should().BeTrue() Expected value to be true, but found False.

The other 159 cases passed on both runs, and the test was identical to main. Read as one instrument that is the signature of a race: the service returns either verdict.

What it actually was

Both verdicts are deterministic, and they are the two sides of one core commit — 757c9d1ad4, "fix(auth): DeleteToken reports what it removed, not that it ran".

ApiTokenService lives in core (memex/Memex.Portal.Shared/Authentication/), not in this repo. Before that commit it read:

var primary = nodeFactory.DeleteNode(tokenNodePath)
    .Select(_ => true)                       // ← the constant
    .Catch<bool, Exception>(ex => { logger.LogWarning(...); return Observable.Return(false); });

That projection was harmless only while deleting an absent path faulted. MeshWeaver#4668 made it a SUCCESS that emits false (DeleteNodeResponse.AlreadyAbsent on the wire, IMeshService.DeleteNode returning !AlreadyAbsent), so from #4668 until 757c9d1ad4 the projection overwrote exactly the bit the new contract had added — and every absent path reported a removal that never happened. After 757c9d1ad4 the delete's own value passes through, and 505a416f10 then took the .Catch out too, so a REFUSAL faults instead of masquerading as "already gone".

The control (measured 2026-09-18, this machine)

One case on each side of the change, same worktree, same test source, only -p:MeshWeaverRoot= differing:

Platform under test Runs Result
core 757c9d1ad4^ (pre-fix) 5 5/5 red, Expected value to be false, but found True. — the issue's run-1 message, verbatim
core main (73e0519eda) 8 isolated + 3 × full suite (160) 11/11 green, and 480/480 cases green

So neither verdict is timing-dependent; the observed pair is fully explained by which Memex.Portal.Shared build the test host had loaded. No re-run, under any load, produced a second value from one platform.

The verdict DeleteToken returns, and why

false — it removed nothing. The delete verb's postcondition is "no node exists at that path", and an absent node already satisfies it, so the call SUCCEEDS; the bool is the honest half, saying which of the two successes happened. Core pins the same contract twice — test/MeshWeaver.Graph.Test/DeletingAnAlreadyDeletedNodeIsNotAnErrorTest at the framework level and test/Memex.Portal.Shared.Test/DeleteTokenReportsWhatItRemovedTest at the service level — so flipping this repo's assertion to BeTrue would have pinned a value core actively tests against.

What this repo changed

Not the service — core's is correct and covered. What was wrong here is that the test could not tell a contract from a constant, and so could not say which side it had measured:

How to read the next one

  1. Record the platform commit before you rerun anything, and rerun only against a NAMED one. It is the unpinned rerun that decides nothing — a second green says nothing about which platform produced the first red. Repeated runs are not the problem and steps 4-5 require them; an unstated platform is. Pinning also keeps the other verdict reachable: one fixed, named build that really is flaky shows up as a mixed result on that pin.
  2. Ask what the failing call's implementation is, and where it lives. If it is under $(MeshWeaverRoot), the suite's answer is the platform's answer.
  3. git log -S'<the projection or literal>' -- <that file> in core; the commit that introduced or removed it is the candidate boundary.
  4. Build the one test project twice — -p:MeshWeaverRoot=<worktree at BOUNDARY^> and -p:MeshWeaverRoot=<worktree at BOUNDARY> — and run it. Two deterministic, opposite results close the question; one flaky result means you have a real race and the repro is still owed.
  5. Say the denominator in the finding: N runs each side, not "it reproduces".

Related: AdvisoryFloorsInADependent.md · ATestFakeIsScopedToItsSubject.md · PlatformSourcePin.md

Reconnecting…
The server was updated. Reloading the page to pick up the latest version.