The package feed and its health
The store's catalog is a live feed, not a stored list. Every configured package source — a git
repo of package manifests, or another MeshWeaver instance's registry — is polled every
5 minutes, and the union of what they answer is what /Store shows and what
@Store/data/StorePackage returns.
The degradation is deliberate
A poll pass can fail: GitHub can time out, a registry can answer 503. When that happens the feed keeps the last good snapshot for that source. That is correct — a store must not empty itself because one source blinked, and a source that recovers is picked up on the next pass with no intervention.
…but it used to be invisible, and that was the defect
Before #1180 the feed could not tell three different things apart, because all three produced the same "nothing to report" marker:
| What actually happened | What the feed recorded |
|---|---|
| The head has not moved — nothing new to fetch | nothing to report |
| The pass blew up with a transient error | nothing to report |
| The pass blew up because the credential is rejected | nothing to report |
So on memex.meshweaver.cloud every git poll answered Octokit.AuthorizationException: Bad credentials, for the entire readable window, while @Store/data/StorePackage kept returning 132
packages and the store looked completely healthy. The catalog was frozen at whenever the
credential last worked; a package published after that simply did not appear, and the only trace
anywhere was a warn line every five minutes.
What the feed reports now
Each source carries a StoreFeedStatus row — published as a second virtual collection beside
the packages, from the same poll passes (never a second poll loop):
| Field | Meaning |
|---|---|
source / kind / target |
which source this row is about (git or registry) |
consecutiveFailures |
how many passes have failed in a row — zero after any success |
lastSuccess |
UTC instant of the last pass that ANSWERED (an unmoved head counts) |
lastFailure, error |
when it last failed, and a short bounded description |
fault |
None · Credential · Missing · Transient |
degraded |
whether this is a reported condition |
verdict |
one sentence: what is wrong, how stale the catalog is, and whether retrying can help |
The signal is PERSISTENCE, never one failure
A source becomes degraded after 3 consecutive failed passes (15 minutes). One failed poll is normal, and serving the last good snapshot is the right answer to it — a signal that fires on one bad minute is a signal people learn to ignore. The fault kind changes the wording and the log level; it never changes the threshold.
A permanent fault SAYS retrying will not help
Bad credentials, a 401, a non-rate-limited 403 and a 404 are permanent: every further pass
fails identically, and the verdict says so in those words. Telling somebody to "retry shortly" there
sends them round a loop that cannot close.
🚨 A rate-limited 403 is transient and is checked first. GitHub answers 403 both for "this identity may not read that" and for "you have asked too often" — and reading the second as a dead credential would send an operator to replace a credential that is perfectly fine, which is the one false positive that destroys trust in a signal like this.
Frozen is not the same as never loaded
If a pod restarted after the fault began, the source has no last-good list at all: its part of the catalog is missing, not stale. The verdict distinguishes the two, because "you are seeing an old list" and "you are seeing none of this source's packages" call for different reactions.
Where to read it
/Store/Catalog/FeedStatus— a table of every source, its state, its last successful poll (rendered in your time zone) and its verdict. Global admins only: it names repositories, registry URLs and raw failure text.- The store's own page — when a source is degraded, an admin opening
/Storegets a banner saying the catalog is stale, on the page that is serving the stale list. @Store/data/StoreFeedStatus— the same rows, for a health check.- The portal log — a failing pass below the threshold stays a
warn; a degraded source logs an error carrying the verdict, every pass, until it clears. Recovery is logged too.
What this does NOT do
It does not fix a broken credential, and it must not try to. Rotating the GitHub App installation token an instance polls with is an operator action on a running system.