devsyncwrld/backspace
1
0
Fork 0
Code Issues Pull Requests Actions 23 Packages Projects Releases Wiki Activity

docs(federation): document honest handshake contract; mark BUG-0/1/2/4/5 resolved

Browse Source
This commit is contained in:
Jannis Braun
2026-07-02 13:14:48 +02:00
parent cd28c0336c
commit 83ebc06759
4 changed files with 38 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/systems/federation.md
+30 -6
View File
@@ -46,7 +46,8 @@ Backspace federation is peer-to-peer with no central authority. Each instance ma
- Creates local peer record with `status='pending'`
- POSTs to `{remoteOrigin}/api/federation/peer/accept` with `{ sourceOrigin, challenge, hmacSecret }`
- Timeout: 10 seconds (`AbortSignal.timeout`)
- On remote 200 (accepted): updates local peer to `status='active'`, sets `lastSeenAt`, broadcasts `federation_peers_changed` to admin WS subscribers, returns 200 with `{ peer }`
- On remote 200 (accepted): **verifies before activating** — performs a signed `fetchPeerEpoch` (`POST /api/federation/epoch`) round-trip with the just-negotiated secret to PROVE the responder actually adopted it (a responder that returns 200 without adopting the secret is otherwise indistinguishable from a healthy fresh peering — this is the split-brain the honest contract prevents). If `fetchPeerEpoch` succeeds, updates local peer to `status='active'`, stores the **cryptographically-verified** epoch as `peer_instance_id` (authoritative over the unverified handshake-response body), sets `lastSeenAt`, broadcasts `federation_peers_changed`, and returns 200 with `{ peer, verified: true }`. If it returns null (secret doesn't authenticate / `/epoch` absent / unreachable), parks the peer in `status='needs_attention'` with `needs_attention_reason='repeer_incomplete'` and returns 200 with `{ peer, verified: false }` — never a silent false-active. See "Trust re-establishment contract" below.
- On remote 409 `PEER_EXISTS_RESET_REQUIRED` (responder already holds peering for us and honestly refuses to rekey): **deletes** its own pending row — keeping the local slot clean so the remote's own later Re-peer can land on a fresh responder slot — and returns 409 `{ code: 'PEER_EXISTS_RESET_REQUIRED', error }`. See "Trust re-establishment contract" below.
- On remote 202 (queued for remote admin approval): transitions local peer to `status='awaiting_approval'` (does **not** activate), broadcasts `federation_peers_changed`, returns 202 with `{ peer }`. Without this branch `response.ok` would be true and the local peer would flip to `active` while the remote had us pending — a transient local-active / remote-pending split that only self-healed when the remote admin approved. Mirrors the auto-peer 202 branch in `federationPeering.ts:performHandshake`.
- On remote 403 / other non-2xx: deletes pending peer, returns 502 with the remote's error message
- On network error / timeout: deletes pending peer, returns 502 (network) or 504 (timeout)
@@ -55,15 +56,16 @@ Backspace federation is peer-to-peer with no central authority. Each instance ma
- Auth: **none** (first contact -- no JWT, no HMAC)
- Rate-limited: 10 requests per minute per IP (in-memory sliding window, buckets cleaned every 60s)
- Validates `sourceOrigin`, `challenge`, `hmacSecret`, and (optional) `instanceName` from body
- Handles existing peers: active -> return 200 (idempotent), revoked -> return 403, pending -> update with new secret and activate
- Handles existing peers: **active or needs_attention -> return 409 `PEER_EXISTS_RESET_REQUIRED`** (honest refusal — the anti-hijack guard still does NOT adopt the caller's `hmac_secret`; only the reported status/body changed from the old false `200 {accepted:true}`), revoked -> return 403, pending -> update with new secret and activate
- Epoch-mismatch reset detection (`markPeerReset`) still fires on the active/needs_attention path **before** the 409 is returned — detection is layered on top of the unchanged anti-hijack guard
- New peer: creates record with provided `hmacSecret`, sets `status='active'`
- Returns `{ accepted: true, instanceName: <ourName | null>, instanceId: <ourEpoch> }` on success — see "Instance name & epoch exchange" below
- Returns `{ accepted: true, instanceName: <ourName | null>, instanceId: <ourEpoch> }` on true activation; on the existing-active/needs_attention refusal returns 409 `{ accepted: false, code: 'PEER_EXISTS_RESET_REQUIRED', error, instanceName, instanceId }` — see "Instance name & epoch exchange" and "Trust re-establishment contract" below
### Instance name & epoch exchange
The handshake is bidirectional for two pieces of metadata: the `instance_name` label rendered in the federation panel and in DM-call toasts (`peerLabel`), and the **instance epoch** (`instance_id`, this instance's persistent incarnation UUID minted by `ensureDefaults`, accessed via `getInstanceId()`). The epoch is the authenticated baseline used by the instance-epoch self-healing feature to detect a wipe-and-reinstall on the same domain (design: `docs/superpowers/specs/2026-07-01-federation-instance-epoch-self-healing-design.md`).
- **Initiator → responder:** the request body to `/peer/accept` carries `{ sourceOrigin, hmacSecret, instanceName, instanceId }`. The responder reads `instanceName` → `federation_peers.instance_name` and `instanceId` → `federation_peers.peer_instance_id` on every state-mutating activation path: `pending → active`, `awaiting_approval → active` (token-valid and autoAccept-fallback), `rejected → active` (override), and new-peer create. The idempotent early-return path for already-`active` and `needs_attention` peers does NOT overwrite — same security posture that already refuses to overwrite `hmac_secret` on these paths from an unauthenticated request. (The idempotent guard's *detection* of a changed epoch on that path is a later part of the self-healing feature; the handshake itself only writes the epoch on true activation.)
- **Initiator → responder:** the request body to `/peer/accept` carries `{ sourceOrigin, hmacSecret, instanceName, instanceId }`. The responder reads `instanceName` → `federation_peers.instance_name` and `instanceId` → `federation_peers.peer_instance_id` on every state-mutating activation path: `pending → active`, `awaiting_approval → active` (token-valid and autoAccept-fallback), `rejected → active` (override), and new-peer create. The existing-peer refusal path for already-`active` and `needs_attention` peers (now an honest `409 PEER_EXISTS_RESET_REQUIRED`, see "Trust re-establishment contract") does NOT overwrite — same security posture that already refuses to overwrite `hmac_secret` on these paths from an unauthenticated request. (The idempotent guard's *detection* of a changed epoch on that path is a later part of the self-healing feature; the handshake itself only writes the epoch on true activation.)
- **Responder → initiator:** the `/peer/accept` response body is `{ accepted: true, instanceName: <ourName | null>, instanceId: <ourEpoch> }`. The initiator (`performHandshake` in `utils/federationPeering.ts`, `/peer/initiate`, and both `/approval-requests/:id/approve` handlers in `routes/federation.ts`) parses `instanceName` and `instanceId` and persists them alongside the `status='active'` write (`peer_instance_id`). Older peers that omit either field are tolerated — the respective column stays `null` (backstopped later by the deterministic epoch-refresh and relay-envelope population). Non-JSON bodies are tolerated defensively.
@@ -368,7 +370,27 @@ Detection (`markPeerReset`) only snapshots + journals + notifies; it destroys no
**Reset-events admin surface (`GET /api/federation/reset-events`).** Admin-only, read-only. Returns the durable `federation_reset_events` journal joined with each origin's current orphaned real accounts (`federation_home_orphaned = 1`, `homeInstanceMatch`), each with `ownedSpaces`, `spaceMemberCount`, and authored-`messageCount` for disposition. Response type `FederationResetEventsResponse` (`{ events: FederationResetEvent[] }`, each event carrying `orphanedAccounts: FederationOrphanedAccount[]`). Disposition actions reuse existing endpoints — one-click Re-peer (`/peers/:id/reset` → `/peer/initiate`) and full-purge Remove (`DELETE /api/admin/users/:id`, owns-spaces → transfer first). See `admin.md` "FederationPanel" and `client-federation.md` §8.
**`needsAttentionReason` on the peer API.** `GET /api/federation/peers` returns `needsAttentionReason: 'auth_failures' | 'peer_reset_detected' | null` per peer, so the admin UI distinguishes a reset-detected peer (persistent Reset-cleanup banner + one-click Re-peer) from a generic auth-failure peer (plain "Reset Peering").
**`needsAttentionReason` on the peer API.** `GET /api/federation/peers` returns `needsAttentionReason: 'auth_failures' | 'peer_reset_detected' | 'repeer_incomplete' | null` per peer, so the admin UI distinguishes a reset-detected peer (persistent Reset-cleanup banner + one-click Re-peer) from a generic auth-failure peer (plain "Reset Peering") and from a peer whose Re-peer could not be cryptographically verified (`repeer_incomplete` — see "Trust re-establishment contract"). `repeer_incomplete` is a nullable-TEXT value only; no schema migration.
### Trust re-establishment contract
The peering handshake must never report success when trust was not actually re-established — a false success re-creates the permanent HMAC desync ("split-brain") the epoch feature exists to prevent. Two changes make the recovery handshake honest end-to-end, and the **anti-hijack guard is unchanged** (an unauthenticated `/peer/accept` never adopts a caller's secret over an existing `active`/`needs_attention` row; no path auto-rekeys).
**1. Responder honest refusal (`POST /api/federation/peer/accept`).** When a peer row already exists in `active` OR `needs_attention`, the endpoint returns `409 { accepted: false, code: 'PEER_EXISTS_RESET_REQUIRED', error, instanceName, instanceId }` instead of the old false `200 { accepted: true }`. It still does not adopt the caller's `hmac_secret` (identical guard behavior — only the reported status/body changed), and the epoch-mismatch `markPeerReset` detection still fires before the return. Legacy initiators that only read `response.ok` now fail loudly instead of silently desyncing; new initiators special-case the code.
**2. Initiator verify-before-activate (`POST /api/federation/peer/initiate`).**
- On remote `409 PEER_EXISTS_RESET_REQUIRED`: the initiator DELETES its own pending row (keeping its slot clean so the remote's own Re-peer can later land on a fresh responder slot) and returns `409 { code: 'PEER_EXISTS_RESET_REQUIRED', error }`.
- On remote 200: the initiator performs a signed `fetchPeerEpoch` (`POST /api/federation/epoch`) round-trip **before** activating. Null (secret doesn't authenticate / `/epoch` absent / unreachable) → the peer is parked in `needs_attention` with `needs_attention_reason='repeer_incomplete'` and the response is `200 { peer, verified: false }`. Success → the peer activates (storing the cryptographically-**verified** epoch as `peer_instance_id`) and the response is `200 { peer, verified: true }`.
**Recovery flows.**
- **Common reset case (one click from the survivor).** When the reset box makes contact, the survivor's detection (`markPeerReset`) moves the survivor's row to `needs_attention`. The survivor's **Re-peer** (reset local row → `/peer/initiate`) then lands on the reset box's clean responder slot → both sides re-key and the initiator verifies via `fetchPeerEpoch` → both active with a matching secret. If the reset box initiated first, the responder's new 409 refusal (change 1) means the reset box deletes its pending row rather than false-activating, so the survivor's subsequent Re-peer still finds a clean slot.
- **Bidirectional-stale case.** If both sides still hold a conflicting row, each side that holds one must reset once — the honest `409` / `verified:false` reporting tells the admin exactly that ("the remote still holds stale peering for you; its admin must reset their side, then Re-peer again"), instead of falsely reporting success on a dead peering.
**Backward-compat honesty (limitation, stated explicitly).** A genuinely pre-Phase-1 peer whose `/api/federation/epoch` returns 404 cannot be cryptographically verified, so `/peer/initiate` parks it in `needs_attention` (`repeer_incomplete`) rather than activating. This is intentional fail-safe: a legacy responder that returns 200 without adopting the secret is indistinguishable from a healthy fresh peering, so we refuse to trust it blind. All current Backspace instances ship `/epoch` (Phase 1), so this only affects truly pre-Phase-1 peers — it is an honest limitation, not a silent behavior change for healthy peers.
**Handshake `sourceOrigin` honors `PUBLIC_ORIGIN`.** `resolveLocalOrigin()` (`routes/federation.ts`) now delegates to `getOurOrigin()`, so the origin advertised in the handshake `sourceOrigin` is byte-identical to the `X-Federation-Origin` used for all authenticated S2S requests. Previously it used `https://${DOMAIN}` and ignored `PUBLIC_ORIGIN`, which silently desynced the responder's peer-row key from the auth origin on any instance where `PUBLIC_ORIGIN != https://DOMAIN` — producing permanent `403 Not peered`. See "Public Origin Override" below.
**Verification harness.** The self-contained two-instance integration harness (`packages/server/test/helpers/realHandshake.ts` + `packages/server/test/federation-handshake-desync.test.ts`) exercises the REAL cross-instance handshake — actual `/peer/initiate`→`/peer/accept`, the signed `/epoch` health probe (`s2sHealthy`), and `simulateReset` — using ephemeral in-process instances (each with `PUBLIC_ORIGIN` set to its `http://127.0.0.1:<port>` transport URL, its own temp DB and generated secrets, no live host). Case #1 is the clean-handshake control; #2 gates the responder-refusal fix (BUG-1); #4 gates one-click Re-peer recovery (BUG-2).
### S2S Identity Deletion (`DELETE /api/federation/identity`)
@@ -1645,7 +1667,9 @@ DM channel hard-delete cascades: reactions, embeds, attachments (DB rows + disk
### Public Origin Override
`PUBLIC_ORIGIN` env (read via `config.publicOrigin`, consumed by `getOurOrigin()` in `utils/federationAuth.ts`) overrides the federation transport URL verbatim, taking precedence over the default `https://${DOMAIN}`. When unset, behaviour is unchanged. Intended for reverse-proxy / dev-without-TLS deployments where the public origin must be advertised explicitly (typically `http://...`) and differs from the bare `DOMAIN` value used for federated identity. The integration test harness does NOT use this override — see `seedPeer.ts` for why localhost-port instances cannot collapse to a single peer row.
`PUBLIC_ORIGIN` env (read via `config.publicOrigin`, consumed by `getOurOrigin()` in `utils/federationAuth.ts`) overrides the federation transport URL verbatim, taking precedence over the default `https://${DOMAIN}`. When unset, behaviour is unchanged. Intended for reverse-proxy / dev-without-TLS deployments where the public origin must be advertised explicitly (typically `http://...`) and differs from the bare `DOMAIN` value used for federated identity. The seed-peer integration harness (`seedPeer.ts`) does NOT use this override — see it for why localhost-port instances cannot collapse to a single peer row. The **real-handshake** harness (`realHandshake.ts`) does the opposite: it sets `PUBLIC_ORIGIN` to each instance's ephemeral `http://127.0.0.1:<port>` so the advertised `sourceOrigin` matches the transport, exercising the same path as production.
**Handshake `sourceOrigin` honors this override.** `resolveLocalOrigin()` (`routes/federation.ts`) delegates to `getOurOrigin()`, so the origin advertised in the `/peer/accept` handshake body is identical to the `X-Federation-Origin` used for authenticated S2S requests. Using `https://${DOMAIN}` directly (the prior behavior) desynced the responder's peer-row key from the auth origin whenever `PUBLIC_ORIGIN != https://DOMAIN`, causing permanent `403 Not peered`. See "Trust re-establishment contract" (§1).
### Test-Only Routes