Syncing a Space with GitHub

A Space can be connected to a GitHub repository and its content moved in both directions:

• Sync FROM GitHub (import) — read a repo to create a new Space, update an existing Space to the latest state of its branch, or re-import it at any branch or commit (bringing it to exactly that state).
• Sync TO GitHub (commit/export) — write the Space's nodes into the repo as a single commit. The repo mirrors the Space subtree. A sync is a commit.

On top of the two sync directions, the tab exposes the everyday Git operations you expect on a repo: create a branch, commit (a "Sync now"), checkout / update to latest, and open a pull request — drafted by AI, edited by you, then submitted, with its status read live from GitHub and a link to the PR. Every operation runs as a tracked activity, so you see progress and can cancel it.

Everything is configured in the Space's Settings → GitHub Sync tab. Your GitHub connection is personal: you authorize once with your own GitHub account, every commit and pull request is authored as you, and your token never leaves your account.

For the underlying, fingerprint-gated import-source model (platform content synced from a repo at a release tag), see DataSyncSetup.md.

1. Connect your GitHub account (once)

GitHub Sync authenticates with a long-standing OAuth credential via GitHub's authorization-code flow — no password, no pasted personal access token.

1. Open any Space → Settings → GitHub Sync.
2. Under Your GitHub account, click Connect GitHub →.
3. Your browser is redirected to GitHub; approve the authorization (authorize it for the org whose repos you'll sync). GitHub redirects back to the portal (/connect/github/callback), which stores the token and returns you to the Space.
4. The tab shows ✓ Connected as your-login. You can Disconnect anytime.

Your token is stored encrypted at rest (AES-256-GCM) on your own partition ({you}/_Provider/GitHub) and is reused for every sync — you only connect once. It is never written into exported content.

If you see "GitHub OAuth is not configured", the server has no OAuth App set up yet — see §5 Operator setup.

2. Point the Space at a repository

Under Repository:

Click Save repository settings.

Sync direction — unidirectional or bidirectional

Each source syncs in a configurable direction, enforced by GitHubSyncService on every operation (the GUI additionally hides what would be rejected):

Use ImportOnly for a Space that mirrors an upstream repository you don't own, and ExportOnly for a backup/publishing target that must never feed edits back.

Multiple sync sources

A Space can sync with more than one repository. The Repository section above edits the primary source ({space}/_GitSync); every additional source is its own config node at {space}/_GitSync/{sourceId} with its own repository, branch, subdirectory and direction. Manage them in the Additional sync sources section of the same settings tab (or, platform admins, on Global Settings → Administration → Partitions): add a source by name, edit its settings through the same data-bound editor, sync it with its own direction-aware buttons, and remove it when no longer needed. Programmatically: GitHubSyncService.AddSyncSource / WatchConfigNodes / RemoveSyncSource, and every sync operation takes an optional sourceId (null = the primary).

3. Sync TO GitHub — commit ("Sync now")

Click Sync now. The Space's content nodes are serialized and pushed as one commit — a sync is a commit:

• Markdown pages → *.md (with YAML frontmatter); typed nodes → *.json; code → *.cs.
• A node that has children is written as Name/index.md so its children live in the Name/ folder; the Space itself is the top-level index.json.
• Satellites and secrets are never exported — access assignments, threads, activities, notifications, pull requests, the GitHub credential, and the sync config are all skipped.
• Mirror semantics: within the configured subdirectory the repo is made to match the Space exactly — nodes you deleted are removed from the repo. Anything outside the subdirectory is untouched.
• Commits on the branch HEAD. The commit is parented on the current branch HEAD (read from GitHub at commit time), so it lands on top of whatever is already there — GitHub does the commit; we never overwrite history. Files outside the subdirectory are carried over by reference, so the rest of the repo is left intact.

When it finishes, the resulting commit SHA is stored on the Space and shown as "Last synced: … — commit …". (The stored SHA is a record of your last sync action, not a replica of the branch state — the branch's live HEAD always lives on GitHub.)

4. Sync FROM GitHub — import, update to latest, re-import

Create a new Space from a repository. Importing a repo into a brand-new Space provisions the partition, makes you its admin, and imports every node. (Programmatic entry point: GitHubSyncService.ImportFromGitHub(repoUrl, commitish, newSpaceId, name, subdirectory, userId).)

Update an existing Space to the latest. "Update to latest" re-fetches the configured branch HEAD and mirrors it into the Space (add / update / prune) — this is the checkout operation: it brings the working Space up to whatever is now on the branch.

Re-import at a chosen commit. Under Sync, the Commit or branch to import field is pre-filled with the last synced commit. Change it to any commit SHA or branch and click Re-import at this commit — the Space is mirrored to that exact state (added / updated / removed to match), and the new commit is recorded. This is how you roll a Space forward or back to a specific repository state.

Import reuses the platform's content-addressed import pipeline (fingerprint gate + activity lock + canonical upsert + prune) — see StaticRepoImport.md.

Two-way — never overwrite changes made on the server

By default import is git-first: an update overwrites (and prunes) the live node from the repo. That silently loses edits made on the server between syncs. Turn on Two-way (a checkbox on the sync source) to change the conflict rule to newest-writer-wins per node:

• A node whose live LastModified is newer than the last recorded sync (LastSyncedAt) was changed on the server since you last synced. An Update to latest keeps that node — it is neither overwritten nor pruned — so your local change is carried back to GitHub on the next Commit ("newer on the server wins → GitHub"). A node the server hasn't touched since the last sync is still updated from the repo as usual.
• Two-way only takes effect once a first sync has recorded LastSyncedAt; the initial import is git-first (there is nothing local to protect yet).

Force update is the escape hatch: it ignores two-way and overwrites/prunes from the repo regardless — use it to deliberately discard local changes back to the repository state. Via MCP: git_hub_sync(space, op:"update", force:true).

Take-over edits survive. A node you've edited and marked to exclude from sync is not overwritten or pruned by a re-import — that's how you "claim" content locally. Two-way generalizes this to every server-side edit (no explicit claim needed) as long as it post-dates the last sync.

The four facts on a sync source — and why they disagree on purpose

A sync source records four separate facts, and reading any one of them as the answer to another's question is how an investigation goes wrong. They are deliberately independent:

The horizon is the one with teeth. Everything newer than it counts as a pending server-side change and is protected from overwrite and from the prune, so advancing it past uncommitted work disarms exactly the protection two-way exists for — a later push would then delete that work. That is why the suppressions are there, and why the horizon must never be made to track "when did we last sync".

🚨 A frozen lastSyncedAt beside a fresh lastSyncCommitSha is not a bug. Measured on 2026-09-07, Edu/_GitSync read a lastSyncedAt of 2026-07-11 (memex) and 2026-08-07 (memex-cloud) beside a lastSyncCommitSha from that same morning. Both were correct: every sync in between had been a no-op at unchanged content, which advances the commit and holds the horizon. What was missing was the third fact — nothing recorded that a sync had run at all, so the only way to date one was to compare node timestamps against image tags in a container registry. lastSyncAttemptAt is that fact, and the settings tab now shows the three separately instead of printing the horizon under the words "Last synced".

🚨 And a frozen lastSyncAttemptAt is not necessarily a dead webhook. A source whose last attempt reached a FINAL verdict at a commit is deliberately skipped for every later delivery of that same commit, so its recency stamp stops moving until the repository produces a new one. The settings tab says so in as many words — "this commit has a final verdict — the next new commit re-attempts" — precisely so the stopped clock cannot be read as a stopped delivery.

Note that a node's own lastModified is not a substitute: stream.Update does not re-stamp it, so a node can be rewritten without its modification time moving.

Git is the source of truth — author in the repo, never only-live

A sync reconciles: Update to latest and Re-import mirror the branch into the Space with add / update / prune — so a node that exists live but is NOT in the repo is PRUNED. That is the model's whole point (the repo is authoritative and reproducible), but it has one sharp edge worth stating plainly:

⚠️ Content created or edited only live — never committed — is deleted by the next reconcile. A restart, a scheduled restore, or someone clicking Update to latest re-imports the repo baseline and prunes everything not in it. This is the single most common way live work is lost. Author in the repo.

The safe loop for anything you want to keep — the git-first discipline:

1. Edit in the repo — or, if you edited live, Sync now (op: commit) immediately to capture it in the repo; never let live-only state accumulate.
2. Commit / open a PR, review, merge.
3. Update to latest (op: update) — pull the merged state back into the Space.
4. Recycle any node whose type or configuration changed. Importing new content into a node that is already running does not swap its live views: a node that flipped Markdown → Deck, or whose NodeType source recompiled, keeps its old hub until you recycle it ({node}/Recycle, or post a DisposeRequest). Freshly created nodes get the right views immediately; only a type/config change on an existing node needs the recycle. A node whose content merely changed (same type) re-renders reactively — no recycle needed.

Export never silently drops a node

The export is the exact inverse of the import: every node serializes — through a per-type serializer (*.md / *.cs) or the universal JSON fallback (any content type → *.json, keyed on its $type, the inverse of the JSON import). If a node could ever fail to serialize, the export fails loudly rather than skipping it — a node dropped from the mirror would be pruned by the next import, i.e. silent data loss. Symmetrically, a repo file whose content is missing its $type is tolerated on import (the value is re-typed against the target at the read site), so a hand-edited file is not lost either. The only things left out of an export are the governance satellites (_Access, _Activity, _GitSync, … — see §8) and paths matched by the Space's gitignore-style ignore rules (SyncIgnore). Nothing else is excluded, and nothing is excluded silently.

5. Operations — branch, commit, checkout, pull request

The tab also surfaces the everyday repo operations:

Every operation runs as an activity — with progress and cancel

When you click Sync now (commit), Update to latest (checkout), Re-import, Check branch on GitHub, or Submit pull request, the operation runs as a tracked activity — not a fire-and-forget call:

• A progress panel appears showing the live log ("Committing on the branch HEAD…", "Committed a1b2c3d4 (3 written, 0 removed).") and a status badge (Running → Succeeded / Failed / Cancelled).
• A Cancel button is shown while the operation is running. Clicking it requests cancellation; the GitHub I/O is cancelled and the activity ends as Cancelled.
• The run is persisted at {space}/_Activity/{id}, so it also shows up in your normal activity feed — the same place every other operation in the platform records its history. You can revisit the log later.

Under the hood this is the platform's standard Activity Control Plane: the GitHub work runs off the message hub (so the portal stays responsive), progress streams onto the activity node, and Cancel flips RequestedStatus = Cancelled. Developers trigger the exact same activities through one unified IMessageHub API — hub.CommitToGitHub(...), hub.UpdateToLatestFromGitHub(...), hub.ReimportFromGitHub(...), hub.CreateBranchOnGitHub(...), hub.OpenPullRequestOnGitHub(...), hub.CheckBranchStateOnGitHub(...) — each returns the activity path to watch; the GUI and tests call these same methods.

Delegate to GitHub — don't replicate. Every Git operation is performed on GitHub (create branch, commit on HEAD, open PR, read PR status) — the Space never keeps a parallel copy of repository state that could drift. Live state (which branch, the branch HEAD, a PR's status) is asked from GitHub when you need it. The only things the Space persists are its own local state: the sync configuration, your last sync action's commit SHA, and a PR draft's title/body plus the immutable handle (number + URL) of a PR once opened. Conversely, content changes coming from Git only ever enter the Space through the import pipeline (import deltas — add / update / prune), never by ad-hoc node edits.

Open a pull request — AI drafts, you edit, then submit

This is a four-step flow, all in the Pull request section:

1. AI drafts it. Click Draft pull request with AI. The built-in PullRequestWriter agent is given the change context (the Space name + summary, the head and base branch) and returns a suggested title and markdown body. (If no model is configured, a sensible placeholder draft is created instead so you can still edit and submit.)
2. A draft is created. A PullRequest node is created at {space}/_PullRequest/{id} holding only local draft state — the suggested title/body and the head → base branches. It is not yet on GitHub.
3. You edit it. The title and body are shown in a data-bound editor wired directly to that node — your edits save as you type (no separate Save button). Tweak the wording, add detail, fix the branches.
4. You submit it. Click Submit pull request — this runs as an activity (progress + cancel, like every other operation above). The (edited) title/body are read from the node and a PR is opened on GitHub head → base. Only the immutable handle — the PR number and URL — is written back onto the node (that's how the Space later asks GitHub about this PR), and a clickable link (#N ↗) appears.

Pull request status is read live — never replicated

A PR's lifecycle status (Draft → Open → Merged / Closed) is owned by GitHub. The Space does not store it (a stored copy would drift). Click Check status on GitHub to ask GitHub for the PR's current state on demand — the answer comes straight from GitHub, so it can never be stale. The link (#N ↗) opens the pull request on GitHub. Before a PR is opened it is simply a local Draft.

6. Issues & pull requests (browse and act)

Beyond moving content, a Space can track and act on the repository's issues and pull requests — in the Settings → GitHub Issues & PRs tab. Structured lists are rendered with the framework's data grid (never hand-built HTML).

Issues — synced into the Space

Issues are the one GitHub object that is materialized into the mesh. Click Sync issues from GitHub and every issue is mirrored to a node at {space}/_Issue/{number} (NodeType GitHubIssue) and listed in a live table (number, title, state, author, labels, comments, updated). The table binds to a synced query, so it refreshes itself as issues land — from a sync, from Create issue (opens a new issue on GitHub and materializes its node), or from a webhook (below). You can create an issue and comment on one straight from the mesh — both act on GitHub, then refresh the affected node. Programmatic: IssueService.SyncIssues / SyncIssue / CreateIssue / CommentIssue / WatchIssueNodes, and the activity hub.SyncIssuesFromGitHub(...).

Pull requests — listed live, merge from the tab

The tab lists every pull request in the repo (number, title, author, status, draft, head → base, updated), read live from GitHub (never persisted — a stored copy would drift). Refresh pull requests re-reads them. You can merge an open PR — enter its number and pick Merge commit or Squash & merge — which runs as a tracked activity. Programmatic: PullRequestService.ListAll / GetDetail / Comment / Merge, and the activity hub.MergePullRequestOnGitHub(...). GetDetail additionally returns a checks roll-up (CI pass/fail/pending over the head commit) and a reviews roll-up (latest decision per reviewer).

The AI-drafted open a pull request flow (draft → edit → submit) lives in the GitHub Sync tab (§5). This tab is for browsing and acting on issues + PRs that already exist on GitHub.

Live updates via webhooks

So the synced issue nodes stay fresh without polling, register a GitHub webhook per repo pointing at https://{host}/webhooks/github (content-type application/json), subscribed to the Issues and Issue comments events, with the shared secret from GitHub:Webhook:Secret. Each delivery is HMAC-verified (X-Hub-Signature-256) and then applied: the event payload carries the full issue, so the receiver updates the {space}/_Issue/{number} node of every Space that syncs that repo without needing a token — the update runs under the system identity, merging in the new comment on a comment event. Pull-request events are ignored (PR state is read live, so there is no node to refresh). See GitHubWebhookProcessor.

🚨 A RENAMED repository still matches — and a delivery that matches nothing SHOUTS

Matching a delivery to the Spaces that sync it is a comparison between two strings: the repositoryUrl stored on each {space}/_GitSync, and the repository the payload is for. A GitHub rename breaks that comparison and nothing else. The old url 301-redirects, so git, gh, the REST API and every manual sync keep working — while a webhook payload always carries the repository's current name, which the stored old name can never equal. Casing is not the problem (education and Education are one repo, and always matched); education versus MeshWeaver.Education is.

So when the stored strings match nothing, the receiver asks GitHub what each stored url resolves to today (GET /repos/{owner}/{repo} follows the rename redirect and answers with the repository's current full_name) and matches on that instead. The answer is cached per repository for an hour, so this is a fallback and never a per-delivery network call: a repository that was never renamed matches on the free string path and costs nothing. A config matched this way is then repointed to the current url — keeping its scheme and host, so a GitHub Enterprise config is never moved to github.com — which makes the repair permanent instead of re-derived on every delivery.

And a delivery that still matches nothing is logged at Warning, naming both the incoming repository and every repository it was compared against. That level is the point: a zero-match means every Space that syncs that repository has just been skipped, which is a stale config, a rename, or a hook on the wrong repository — each of which wants a human. At Information it sits beside the routine "matched no sync source that needs updating" line and is indistinguishable from a healthy mesh with nothing to do, which is how ten course Spaces served four-day-stale content while every delivery reported success.

7. Operator setup (enabling the feature)

Server configuration for GitHub Sync — the first two are required, the rest optional:

1. A GitHub OAuth App per portal host. Register one under the GitHub organization (Settings → Developer settings → OAuth Apps → New). Set the Authorization callback URL to https://{host}/connect/github/callback. Copy the Client ID and generate a Client Secret. Request scope repo (read/write to private + public repos):

   // appsettings.json / env  (GitHub__OAuth__ClientId, GitHub__OAuth__ClientSecret)
   "GitHub": { "OAuth": { "ClientId": "Ov23li…", "ClientSecret": "<secret>", "Scopes": "repo" } }
   

   The ClientId is non-secret (env/values). Keep the ClientSecret in the Key Vault and surface it as the GitHub__OAuth__ClientSecret env via the SecretProviderClass (like the other secrets). Absent the client id + secret the Connect link is disabled and the rest of the tab still works for reading status.

2. An encryption master key so stored tokens are ciphertext at rest:

   "Ai": { "KeyProtection": { "MasterKey": "<base64 32-byte key>" } }
   

   This is the same key that protects AI provider credentials (see AccessControl.md). Without it, tokens are stored as plaintext (development only).

3. A webhook secret (optional — for live issue updates). To keep synced issue nodes fresh without polling, set a shared secret and register a webhook per repo (Issues + Issue comments events) at https://{host}/webhooks/github:

   "GitHub": { "Webhook": { "Secret": "<random shared secret>" } }
   

   Keep it in the Key Vault and surface it as GitHub__Webhook__Secret. Deliveries are HMAC-verified (X-Hub-Signature-256) against it; absent the secret the endpoint logs a warning and returns 503, and issue nodes refresh only on an explicit sync.

4. A GitHub App (optional — machine identity for server-side sync). Operations that run with no signed-in user — the plugin registry's sync of the plugins repo, boot imports — authenticate as a GitHub App installation rather than someone's personal OAuth token. The host binds GitHub:App next to GitHub:OAuth; left unconfigured, GitHubSyncService.ResolveAuth simply skips the App fallback and only user credentials work.

   "GitHub": { "App": {
     "ClientId": "Iv23li…",          // the App's client id
     "PrivateKey": "<PEM>",          // Key Vault → GitHub__App__PrivateKey
     "InstallationId": 12345678,     // or set InstallationOwner and let it resolve
     "InstallationOwner": "<org>"
   } }
   

   Note the transport split this enables: bulk push/fetch goes over the git protocol (GitProtocolRepoClient), because the REST path cost one request per file and a single large-repo sync exhausted the App installation's hourly rate budget. Refs, PRs and issues stay on the Octokit REST client.

   Installation tokens live one hour, and the cache refreshes at subscription time. GitHubAppTokenService.GetInstallationToken() returns a deferred observable: every subscription reads the current cached token, replays it while it is more than five minutes from expiry, and otherwise mints a replacement that concurrent subscribers share. That is the contract a long-lived consumer relies on — the Store's git poll loop holds ONE such observable for the life of its feed and subscribes once per pass. Before 2026-09-06 the promise was captured when the observable was built, so the refresh guard matched exactly once: the first expiry minted a new token, every later expiry compared against the stale capture and handed the expired token back. Two token lifetimes after boot every private source read as 401 Bad credentials until the process restarted (Systemorph/Memex#165 — measured on memex-cloud as the first failure 2 h 01 min after the container started). GitHubAppTokenRefreshTest in Memex.Portal.Shared.Test holds the invariant with an injected clock: the second and third refresh mint, a fresh token replays.

All GitHub HTTP and serialization run through the controlled I/O pool — see ControlledIoPooling.md.

8. What is and isn't synced

See also

• DataSyncSetup.md — the import-source model (platform content synced from a repo at a release tag).
• StaticRepoImport.md — the import mechanism reused here (fingerprint, activity lock, upsert, prune).
• What a Green Build Costs a Synced Space — what one webhook delivery costs, the single field that makes it free, and why a source that never converges re-clones its repository on every green build.
• When a Publication Seal Stops Advancing — why a green build can authorize an import that never happens, how to tell a HELD source from a settled one, and what a portal whose framework identity has stopped being published for looks like.
• ControlledIoPooling.md — why every GitHub HTTP call runs in the I/O pool.
• AccessControl.md — credential encryption + the master key.