backspace/docs/systems/federation.md

# Federation System (Server-to-Server)

> **Companion spec:** This document covers **S2S (server-to-server)** federation — the relay protocol, HMAC auth, identity resolution, and background workers. For the **client-side** multi-instance architecture (how the web/desktop app connects to multiple instances, federated account creation, origin-aware routing), see [`client-federation.md`](client-federation.md). Both systems work together.

Source files:
- `packages/server/src/routes/federation.ts` -- **Barrel** for the federation route subsystem. Re-exports the public API (identity resolution, event processors, reconciliation, `validateOrigin`) so `from '.../routes/federation.js'` imports resolve unchanged, and composes the HTTP registrars into `federationRoutes()`. The implementation lives in `routes/federation/` (split out of the former single 7.6k-line file; see `docs/superpowers/specs/2026-07-10-federation-ts-split-design.md`):
  - `routes/federation/rateLimits.ts` -- In-memory sliding-window rate limiters (accept/relay/lookup/ensure) + replay-nonce store + eviction timers
  - `routes/federation/origin.ts` -- `validateOrigin`, `resolveLocalOrigin`, `sanitizePeer` (+ `SanitizedPeer` shape)
  - `routes/federation/identity.ts` -- Federated identity resolution: `extractDomain`, `getOurIdentityDomain`, `verifyAttribution`, `resolveLocalUser`, `findFederatedUser`, `resolveOrCreateReplicatedUser`, `backfillHomeUserId`
  - `routes/federation/dmChannels.ts` -- DM channel/message payload builders, `findOrCreateDmChannel`, `resolveLocalDmMessage`, `isUrlFromPeer`
  - `routes/federation/profile.ts` -- Replicated-profile hydration + asset download, `processProfileUpdateEvent`, `backfillReplicatedProfileAssets`
  - `routes/federation/reconciliation.ts` -- DM federated-id reconciliation + dead-incarnation artifact sweeps (worker-facing maintenance)
  - `routes/federation/events/*.ts` -- Inbound relay event processors, grouped by domain: `dmMessages`, `membership`, `friends`, `calls`, `dmState` (presence/read-state/close/reopen/file-rejected), and `dispatch` (`processRelayEvents`, the fan-out entry point shared by the HTTP relay handler and the initial-sync worker)
  - `routes/federation/handlers/*.ts` -- Fastify route registrars, grouped by endpoint concern: `peerHandshake` (initiate/accept/ensure/rotate/denied), `peerAdmin` (peer list/CRUD/reset/recheck/rotate), `approvals` (approval queue + peering subscriptions/notifications + approve/deny helpers), `relay` (identity delete, relay, epoch, sync), `lookup` (user lookups), `attach` (verify-attach-proof, `/api/users/@me/reattach`)
    - `routes/federation/handlers/s2sAuth.ts` -- `authenticateS2SPeer(request, reply, opts?)`: the shared inbound S2S-HMAC auth preamble (parse headers → resolve active peer → optional per-peer rate limit **before** signature → verify HMAC signature → nonce replay). Adopted by the six endpoints whose preamble is byte-identical: `DELETE /identity`, `POST /relay`, `POST /sync` (`relay.ts`), `POST /users/lookup`, `POST /users/by-home-id` (`lookup.ts`), and `POST /verify-attach-proof` (`attach.ts`). Returns `{ ok: true, peer, nonce }` or, having already sent the rejection reply, `{ ok: false }` (caller must `return`). **Intentional non-adopters** (each keeps a load-bearing gate the helper would flatten, documented in its own docstring/comment): `POST /epoch` (revoked-only gate for peer recovery, 400 on missing headers, no nonce check), `POST /peer/rotate` (active-only, no nonce check), `POST /peer/denied` (`awaiting_approval` gate, synthetic no-grace secret verify).
- `packages/server/src/utils/federationAuth.ts` -- HMAC signing, verification, header parsing, `getOurOrigin()`
- `packages/server/src/utils/federationOutbox.ts` -- Event queuing, coalescing, relay payload construction, mutation log, participant/target resolution
- `packages/server/src/utils/federationLookup.ts` -- HMAC-signed remote-user lookups: `lookupRemoteUser` (by username) and `lookupRemoteUserByHomeId` (reverse lookup, used by stub backfill)
- `packages/server/src/utils/federationPresence.ts` -- S2S presence relay: `queuePresenceRelay`, `snapshotPresenceForPeer` (relationship-scoped), `markPeerStubsOffline`
- `packages/server/src/utils/federationStubBackfill.ts` -- Heals legacy snowflake-named replicated-user stubs by reverse-looking-up the canonical username via the peer
- `packages/server/src/utils/federationWorker.ts` -- Background workers: outbox delivery, file download, health check, janitor, initial sync
- `packages/server/src/utils/storageJanitor.ts` -- Federation GC: outbox expiry, mutation log retention, file queue cleanup, DM channel purge
- `packages/server/src/routes/social.ts` -- Friend request/accept/cancel/remove endpoints that queue federation events
- `packages/server/src/routes/dm.ts` -- DM REST endpoints that queue federation events (message relay, group lifecycle)
- `packages/server/src/ws/events.ts` -- WebSocket event handlers that queue DM message/reaction relay events
- `packages/web/src/utils/profileSync.ts` -- Client-side profile sync via LWW timestamps (not S2S relay)
- `packages/web/src/utils/identity.ts` -- Client-side federated identity resolution helpers

DB tables: `federation_peers`, `federation_outbox`, `federation_file_queue`, `federation_mutation_log`, plus `users` (identity), `dm_channels`/`dm_members`/`dm_messages` (DM federation), `friends`/`friend_requests` (friend federation), `attachments` (file replication).
See `docs/systems/database.md` for full schemas.

---

## Architecture Overview

Backspace federation is peer-to-peer with no central authority. Each instance maintains its own copy of all data. Peers exchange real-time events for DMs and friendships via a signed relay protocol.

**Canonical identity:** The `(homeUserId, homeInstance)` pair is globally unique. Local users have `homeInstance = NULL` and `homeUserId = NULL`. Federated users are represented as **replicated user stubs** -- minimal user records with `passwordHash = '!federation-replicated'` (bcrypt never produces this value, so login is impossible).

**Trust model:** Symmetric shared-secret HMAC. Both peers share the same 256-bit secret. Events are attributed to users by `homeUserId + homeInstance` in the payload, with authority checks verifying the source instance matches the claimed origin of the acting user.

---

## 1. Peer Handshake & Discovery

### 2-Phase Flow

**Phase 1 -- Initiate** (`POST /api/federation/peer/initiate`)
- Auth: JWT + admin role required
- Validates `remoteOrigin` is a well-formed HTTP(S) URL via `validateOrigin()`
- Prevents self-peering (`localOrigin === remoteOrigin`)
- Handles existing peers: active -> return 200, pending -> return 409, revoked -> delete and re-initiate
- Generates HMAC secret: `generateHmacSecret()` -> `randomBytes(32).toString('hex')` (256-bit)
- Generates challenge: `randomBytes(16).toString('hex')` (128-bit, currently unused by acceptor)
- Creates local peer record with `status='pending'`
- POSTs to `{remoteOrigin}/api/federation/peer/accept` with `{ sourceOrigin, challenge, hmacSecret }`
- Timeout: 10 seconds (`AbortSignal.timeout`)
- On remote 200 (accepted): **verifies before activating** — performs a signed `fetchPeerEpoch` (`POST /api/federation/epoch`) round-trip with the just-negotiated secret to PROVE the responder actually adopted it (a responder that returns 200 without adopting the secret is otherwise indistinguishable from a healthy fresh peering — this is the split-brain the honest contract prevents). If `fetchPeerEpoch` succeeds, updates local peer to `status='active'`, stores the **cryptographically-verified** epoch as `peer_instance_id` (authoritative over the unverified handshake-response body), sets `lastSeenAt`, broadcasts `federation_peers_changed`, and returns 200 with `{ peer, verified: true }`. If it returns null (secret doesn't authenticate / `/epoch` absent / unreachable), parks the peer in `status='needs_attention'` with `needs_attention_reason='repeer_incomplete'` and returns 200 with `{ peer, verified: false }` — never a silent false-active. See "Trust re-establishment contract" below.
- On remote 409 `PEER_EXISTS_RESET_REQUIRED` (responder already holds peering for us and honestly refuses to rekey): **deletes** its own pending row — keeping the local slot clean so the remote's own later Re-peer can land on a fresh responder slot — and returns 409 `{ code: 'PEER_EXISTS_RESET_REQUIRED', error }`. See "Trust re-establishment contract" below.
- On remote 202 (queued for remote admin approval): transitions local peer to `status='awaiting_approval'` (does **not** activate), broadcasts `federation_peers_changed`, returns 202 with `{ peer }`. Without this branch `response.ok` would be true and the local peer would flip to `active` while the remote had us pending — a transient local-active / remote-pending split that only self-healed when the remote admin approved. Mirrors the auto-peer 202 branch in `federationPeering.ts:performHandshake`.
- On remote 403 / other non-2xx: deletes pending peer, returns 502 with the remote's error message
- On network error / timeout: deletes pending peer, returns 502 (network) or 504 (timeout)

**Phase 2 -- Accept** (`POST /api/federation/peer/accept`)
- Auth: **none** (first contact -- no JWT, no HMAC)
- Rate-limited: 10 requests per minute per IP (in-memory sliding window, buckets cleaned every 60s)
- Validates `sourceOrigin`, `challenge`, `hmacSecret`, and (optional) `instanceName` from body
- Handles existing peers: **active or needs_attention -> return 409 `PEER_EXISTS_RESET_REQUIRED`** (honest refusal — the anti-hijack guard still does NOT adopt the caller's `hmac_secret`; only the reported status/body changed from the old false `200 {accepted:true}`), revoked -> return 403, pending -> update with new secret and activate
- Epoch-mismatch reset detection (`markPeerReset`) still fires on the active/needs_attention path **before** the 409 is returned — detection is layered on top of the unchanged anti-hijack guard
- New peer: creates record with provided `hmacSecret`, sets `status='active'`
- Returns `{ accepted: true, instanceName: <ourName | null>, instanceId: <ourEpoch> }` on true activation; on the existing-active/needs_attention refusal returns 409 `{ accepted: false, code: 'PEER_EXISTS_RESET_REQUIRED', error, instanceName, instanceId }` — see "Instance name & epoch exchange" and "Trust re-establishment contract" below

### Instance name & epoch exchange

The handshake is bidirectional for two pieces of metadata: the `instance_name` label rendered in the federation panel and in DM-call toasts (`peerLabel`), and the **instance epoch** (`instance_id`, this instance's persistent incarnation UUID minted by `ensureDefaults`, accessed via `getInstanceId()`). The epoch is the authenticated baseline used by the instance-epoch self-healing feature to detect a wipe-and-reinstall on the same domain (design: `docs/superpowers/specs/2026-07-01-federation-instance-epoch-self-healing-design.md`).

- **Initiator → responder:** the request body to `/peer/accept` carries `{ sourceOrigin, hmacSecret, instanceName, instanceId }`. The responder reads `instanceName` → `federation_peers.instance_name` and `instanceId` → `federation_peers.peer_instance_id` on every state-mutating activation path: `pending → active`, `awaiting_approval → active` (token-valid and autoAccept-fallback), `rejected → active` (override), and new-peer create. The existing-peer refusal path for already-`active` and `needs_attention` peers (now an honest `409 PEER_EXISTS_RESET_REQUIRED`, see "Trust re-establishment contract") does NOT overwrite — same security posture that already refuses to overwrite `hmac_secret` on these paths from an unauthenticated request. (The idempotent guard's *detection* of a changed epoch on that path is a later part of the self-healing feature; the handshake itself only writes the epoch on true activation.)

- **Responder → initiator:** the `/peer/accept` response body is `{ accepted: true, instanceName: <ourName | null>, instanceId: <ourEpoch> }`. The initiator (`performHandshake` in `utils/federationPeering.ts`, `/peer/initiate`, and both `/approval-requests/:id/approve` handlers in `routes/federation.ts`) parses `instanceName` and `instanceId` and persists them alongside the `status='active'` write (`peer_instance_id`). Older peers that omit either field are tolerated — the respective column stays `null` (backstopped later by the deterministic epoch-refresh and relay-envelope population). Non-JSON bodies are tolerated defensively.

All four outbound `/peer/accept` senders (`performHandshake`, `/peer/initiate`, and the inbound + outbound `/approve` handlers) include `instanceId: getInstanceId()` in the request body, so a peer learns our epoch regardless of which path activated the relationship.

`instance_name` is cosmetic metadata, eventually-consistent. Anywhere `peerLabel` is rendered falls back to origin hostname when `instance_name IS NULL`. Instance renames do not currently re-broadcast — that's a separate, unimplemented feature. `peer_instance_id` is trust-consequential (only ever written from authenticated channels) — see the self-healing design spec for detection/heal semantics.

### Secret Storage & Rotation

Both instances store the **same** HMAC secret. The initiating instance generates it and sends it in the accept request.

**Rotation protocol:** Either peer (or automatically on a configurable interval, default 90 days) can trigger rotation:

1. Initiator generates a new secret, stores it as `pendingHmacSecret`, and POSTs to `{peer}/api/federation/peer/rotate` signed with the current secret
2. Acceptor verifies the HMAC, stores `pendingHmacSecret`, and returns `{ accepted: true }`
3. Both sides enter a 15-minute grace period where either secret is accepted for verification, while outbound requests are signed with the new secret
4. After the grace period, the health check worker promotes `pendingHmacSecret` → `hmacSecret` and clears the pending fields

**Conflict guard:** If `pendingHmacSecret` is already set, the rotate endpoint returns 409.

**Schema columns:** `pending_hmac_secret` (TEXT NULL), `secret_rotation_at` (INTEGER NULL), `secret_rotated_at` (INTEGER NULL), `auto_rotate_interval_days` (INTEGER NOT NULL DEFAULT 90).

### Peer Status Lifecycle

```
                     ensurePeered
  (none) ──────────► pending ──────────► active ──────► needs_attention
      ▲                │                   │ ▲              │
      │                │  remote 202       │ │              │ admin Reset
      │                ▼                   │ │              ▼
      │          awaiting_approval         │ │           (deleted)
      │                │                   │ │
      │    ┌───────────┼──────────┐        │ │ N consecutive
      │    │           │          │        │ │ auth failures (401/403)
      │  accept     denied     expired     │ │
      │  (fresh)    (admin)   (janitor)    │ │ delivery failures
      │    │           │          │        ▼ │
      │    ▼           ▼          ▼     unreachable
      │  active    rejected   rejected     │
      │                                    │ health check OK
      │  auto-peer rejected                ▼
      │  (403 PEERING_REQUIRES_APPROVAL) active
      └──────────────────────────── rejected
                                      │ admin revoke (active)
                                      ▼
                                   revoked
```

| Status | Outbox delivery | Health check | Relay accepts | Re-initiation | Admin clear |
|--------|----------------|--------------|---------------|---------------|-------------|
| `active` | Yes | No | Yes | No (returns existing) | N/A |
| `pending` | No | No | No | No (returns 409) | N/A |
| `awaiting_approval` | No | No | No | Returns pending; no re-handshake | Yes (admin deletes) |
| `unreachable` | No (entries wait) | Yes — demand-driven probe (backoff while mail queued; 15-min backstop when silent) | Yes (resets to active) | No | N/A |
| `needs_attention` | No (entries bounded by TTL) | No | Yes (200 no-update, same as active) | No (admin must Reset first) | Yes (admin Reset deletes record) |
| `revoked` | No (entries purged) | No | No (returns 403) | Yes (old record deleted) | N/A |
| `rejected` | No | No | No | Yes (admin deletes record, then re-initiates) | N/A |

### PEER_UNREACHABLE_THRESHOLD

Defined in `federationWorker.ts` as `10`. After 10 consecutive delivery failures for a peer, `handleOutboxDeliveryFailure` sets `status = 'unreachable'` and, on entry, resets the recovery pacing fields (`probe_attempts = 0`, `last_probe_at = NULL`) so the first probe fires immediately.

**Demand-driven recovery (`processRecoveryTick`)** — A dedicated recovery loop runs on a 5-second tick (`RECOVERY_TICK_INTERVAL_MS`). On each tick it scans `unreachable` peers and, for each, decides whether a probe is due:

- If the peer has ≥1 queued outbox row, the probe interval is `RECOVERY_BACKOFF_MS = [30s, 1m, 5m, 15m]` indexed (clamped) by `probe_attempts` — fast retries while mail is actually waiting.
- If the peer has no queued mail, it falls back to the `HEALTH_CHECK_INTERVAL_MS` (15-minute) backstop — there is nothing to deliver, so there is no urgency.
- A probe is due when `last_probe_at IS NULL` (immediate first probe on the unreachable transition) or `now - last_probe_at >= interval`.

When due, it calls `probePeerReachable(origin)` (`utils/federationRecovery.ts`) — an unauthenticated `GET /api/instance/info` with a 10-second timeout (no HMAC; reachability is not trust). On success it calls `markPeerRecovered(peerId)`, which reverts the peer to `active`, zeroes `consecutive_failures`, refreshes `last_seen_at`, resets `probe_attempts`/`last_probe_at`, and broadcasts `federation_peers_changed` (via `onPeerActivated`). On failure it increments `probe_attempts` and stamps `last_probe_at = now` to advance the backoff.

`processHealthCheckTick` (15-minute interval, matching `ROTATION_GRACE_PERIOD_MS`) no longer owns recovery — it now handles **secret-rotation grace-period finalization and auto-rotation only**.

### Auto-Peering

When the server needs to relay events to an instance it has not yet peered with, `ensurePeered()` in `federationPeering.ts` automatically initiates the handshake. This removes the requirement for an admin to manually initiate every peering relationship.

**Integration points:**
- **Outbox worker** (`federationWorker.ts`) — after delivering active peers, calls `ensurePeered()` for any pending-placeholder entries whose peer has not yet been activated.
- **Connection flow** (`POST /api/federation/peer/ensure`) — called by the client when establishing a cross-instance connection, ensuring the two instances are peered before any relay traffic is sent.

**`rejected` status** — Added to the peer lifecycle. Set when a remote instance explicitly rejects auto-peering with `403 PEERING_REQUIRES_APPROVAL`. This status is sticky: no automatic retry occurs. An admin can clear it by deleting the peer record (then re-initiating), or by manually initiating via `peer/initiate`. An incoming `peer/accept` from a remote admin can also override `rejected` → `active` (treated the same as a `pending` record).

**`autoAcceptPeering` instance setting** — Controls whether `POST /api/federation/peer/accept` accepts unsolicited peering requests. Default: `true`. When `false`, the endpoint returns `403 PEERING_REQUIRES_APPROVAL` for requests where no local `pending` record exists (i.e., a request the local admin did not initiate). The determination is made by checking the local peer table — not a client-provided flag.

### Peer Approval Queue

When `autoAcceptPeering` is `false` and an instance calls `POST /api/federation/peer/accept` without a matching local `pending` record, the endpoint returns `202 Accepted` and creates a row in `peer_approval_requests` instead of immediately peering. The requesting instance receives `202` (not an error), so it enters `awaiting_approval` status rather than `rejected`.

**`peer_approval_requests` table** — Holds incoming peering requests pending admin review:
- `id` — Snowflake PK
- `origin` — Requesting instance's origin URL (UNIQUE; only one pending request per origin)
- `instance_name` — Instance name sent by requester
- `hmac_secret` — Requester's HMAC secret; used to sign the denial notification
- `requested_at` / `expires_at` — Epoch ms; expiry is `requested_at + 30 days`
- `approval_token` — Single-use 64-hex-char random token issued in the 202 response and forwarded by `/approve`. See [Approval Token Verification](#approval-token-verification).

**Approval flow** — Admin approves via `POST /api/federation/approval-requests/:id/approve`. The handler dispatches on `peer_approval_requests.direction`:

*Inbound* (remote asked to peer with us):
1. A fresh `federationPeer` record is created (or existing `rejected`/`awaiting_approval` record is upserted) with status `pending`
2. A standard `peer/accept` handshake is sent to the requesting origin (forwarding the stored `approvalToken` per [Approval Token Verification](#approval-token-verification))
3. On 200 the local peer becomes `active`; on 202 the peer transitions to `awaiting_approval` (mutual-gate); the `peer_approval_requests` row is deleted in both cases

*Outbound* (local users asked us to peer with a remote — created by the gate in `ensurePeered`):
1. A fresh `federationPeer` row is created with status `pending` (HMAC generated at this moment — outbound queue rows store `hmac_secret = NULL`)
2. `peer/accept` is sent to the remote with no `approvalToken` (we are the initiator with no prior token from this remote)
3. On 200 the peer is promoted to `active`. `onPeerActivated` then fires `fanoutOutboundSubscribers` (see [Outbound peering gate](#outbound-peering-gate)) which writes `kind='approved'` notifications for each subscriber and cascade-deletes the parent `peer_approval_requests` row. The handler does NOT duplicate this cleanup.
4. On 202 the peer transitions to `awaiting_approval`, captures the returned `approvalToken`, and the queue row + subscribers are LEFT INTACT — they wait for the eventual remote-admin approval. `onPeerActivated` is NOT called on this path.
5. On 4xx/5xx/network error the peer row is deleted; the queue row is left intact so the admin can retry. Response status is `502`/`503`/`504` accordingly.
6. Response body shape: `{ success, peerStatus: 'active' | 'awaiting_approval', peer? }`. The `peerStatus` field is the outbound-only signal.

**Denial flow** — Admin denies via `POST /api/federation/approval-requests/:id/deny`. The handler dispatches on direction:

*Inbound*:
1. Server sends `POST {origin}/api/federation/peer/denied` signed with the requester's `hmac_secret` (from the approval request row)
2. Receiving instance transitions its local peer record from `awaiting_approval` → `rejected`
3. A local `federationPeer` record is upserted with status `rejected` to block future unsolicited requests from the same origin
4. The `peer_approval_requests` row is deleted

*Outbound*:
1. For each row in `peer_approval_subscribers`, a `kind='denied'` notification is inserted into `peer_approval_notifications` and a `peering_notification_received` WS event is sent to the user
2. The parent `peer_approval_requests` row is deleted (cascade clears subscribers)
3. No remote network call — the remote never knew we were considering this peer
4. `federation_peers_changed` is broadcast to admins so the queue UI refreshes

**Expiry** — The janitor (`federationJanitor.ts`) runs on its scheduled interval and deletes rows where `expires_at < now`. Expired requests do NOT create a `rejected` peer — the requesting instance can re-submit. Admin denial, by contrast, does create a `rejected` peer record, blocking re-requests until an admin clears it.

**Pre-handshake guard (`ensurePeered`)** — Before any outbound handshake, `ensurePeered(origin)` in `federationPeering.ts` refuses with `{ status: 'rejected', error: 'Local admin must resolve…' }` if an **inbound** `peer_approval_requests` row exists for that origin. The guard query is narrowed to `direction='inbound'` (commit `0d3d087`) so the gate's own outbound queue rows do not falsely block their own approval path. This blocks the auto-reconnect trigger: without the guard, any code path calling `ensurePeered` (e.g., the silent reconnect in `stores/instanceStore.ts`) could initiate a fresh outbound handshake to a peer that has a pending inbound approval request. The legitimate approve flow (`POST /api/federation/approval-requests/:id/approve`) does NOT call `ensurePeered` — it deletes the approval-request row and does its own direct `fetch` to `/peer/accept` — so the guard does not block legitimate approvals.

### Approval Token Verification

The pre-handshake guard above closes the most reliable trigger but cannot prevent every adversarial code path on a remote side from sending an inbound `/peer/accept` against a row in `awaiting_approval`. To close that broader class, the protocol adds a cryptographic single-use **approval token** verified on the receiver before promoting `awaiting_approval → active`. Spec: `docs/superpowers/specs/2026-04-26-peer-approval-token.md`.

**Issuance.** When `/peer/accept` returns 202 (queue-as-approval-request, `autoAcceptPeering=0`), the server generates a 32-byte random hex token via `crypto.randomBytes(32).toString('hex')`, stores it on the new `peer_approval_requests.approval_token` column, and returns it in the 202 body as `{ queued: true, message, approvalToken }`.

**Storage on the initiator.** When the initiator's outbound `/peer/accept` (from `performHandshake`, `/peer/initiate`, or `/approve`) receives 202, it parses `approvalToken` from the response body and stores it on the local `federation_peers.approval_token` column alongside `status='awaiting_approval'`.

**Forwarding from `/approve`.** When the local admin approves an inbound queued request, the `/approve` endpoint reads `approvalToken` from the queued `peer_approval_requests` row and includes it in its outbound `/peer/accept` body. No other code path forwards a token — the security property is "only `/approve` reads the stored token from the DB."

**Verification on the initiator's `/peer/accept` handler.** When an inbound request matches an existing `awaiting_approval` peer row, the handler verifies `existing.approval_token === request.body.approvalToken` (length-checked, plain `===` — see spec §3.1 on why constant-time comparison isn't required). On match, the row is promoted to `active`, the stored token is cleared (`approval_token=NULL`), and any stale `peer_approval_requests` row for the origin is deleted. On mismatch (or missing token), behavior depends on the receiver's `autoAcceptPeering`:
- `autoAcceptPeering=0` → `queueApprovalRequest()` is invoked: a new `peer_approval_requests` row is upserted with a fresh token, returns 202. The existing `awaiting_approval` row is left untouched. **No bypass.**
- `autoAcceptPeering=1` → fallback promote to `active` (no security regression vs. prior behavior — `autoAccept=1` would accept any inbound `/peer/accept` regardless).

**Single-use lifecycle.** The token is consumed on first successful match: cleared from `federation_peers.approval_token` at the receiver's promote step, cleared from the initiator's `federation_peers.approval_token` when its own outbound returns 200, and deleted along with the approval-request row when `/approve` completes. This bounds replay of a leaked token (DB snapshot, log capture) to the period before the legitimate `/approve` runs.

**Backward compatibility.** Both schema columns are nullable. Older peers that don't include `approvalToken` in the request body or 202 response result in `null` storage; the receiver's verification then falls through the `autoAcceptPeering` gate. Existing `active` peers and existing `awaiting_approval` rows in production at upgrade time are unaffected — the verification only runs on the receiver's `awaiting_approval` branch. Stalled legacy `awaiting_approval` rows (no stored token) cannot complete via inbound `/peer/accept` from a legacy initiator unless the receiver is `autoAccept=1`; admins should re-initiate them through the standard flow if needed.

**What this does NOT defend against** — a remote operator running custom code with full DB access can read their stored token and forge `/peer/accept`. That is the inherent trust radius of federation peering. The threat model is bug-prone code paths (auto-reconnect, voice-call peering races, future `ensurePeered` callers) on otherwise-honest peers, not adversarial operators. Sender-side outbound gating for `autoAcceptPeering=0` was tracked as the "Direction C" follow-up and is now closed by the [Outbound Peering Gate](#outbound-peering-gate) below (spec `docs/superpowers/specs/2026-04-26-outbound-peering-gate-design.md`, commits `ac2565f..c795d72`).

### Outbound Peering Gate

The receiver-side trust class is closed by the approval-token mechanism above. The sender-side trust class — *bug-prone or incidental code paths on the initiator that drag the local instance into peering relationships without local admin consent* — is closed by a centralized gate inside `ensurePeered`. After this change, `autoAcceptPeering=0` is symmetric: BOTH inbound and outbound new-peer establishment require local admin approval. Existing active peers keep working unchanged.

Spec: `docs/superpowers/specs/2026-04-26-outbound-peering-gate-design.md`.

**Gate location.** `ensurePeered` (`utils/federationPeering.ts`) is the single chokepoint every outbound new-peer attempt funnels through (friend-add, `/peer/ensure`, `sendCallRelay`'s no-active-peer branch, future callers). The gate runs ONLY when **no `federation_peers` row exists for the origin**. If any peer row exists in any status (`active`, `pending`, `awaiting_approval`, `rejected`, `revoked`, `unreachable`, `needs_attention`), the gate is a no-op — the existing branches in `ensurePeered` handle the row as today. This matches the threat model: the worry is automated paths creating *new* peerings without admin consent.

**Required caller intent.** Every call site MUST pass an explicit `EnsurePeeredCallerIntent` (declared in `packages/shared/src/types.ts`). The argument is required at the type level so a future caller cannot silently fall through to system behavior:

```ts
export type PeeringTriggerReason = 'friend_add' | 'space_join' | 'direct_message';

export type EnsurePeeredCallerIntent =
  | { kind: 'user_action'; userId: string; reason: PeeringTriggerReason; target: string }
  | { kind: 'system' };

export type EnsurePeeredResult =
  | { status: 'active'; peerId: string }
  | { status: 'rejected'; error: string }
  | { status: 'failed'; error: string }
  | { status: 'pending'; error: string }
  | { status: 'admin_required'; error: string };  // ← new variant
```

(Type was originally drafted as `TriggerReason` and renamed to `PeeringTriggerReason` in commit `f6487a2` to disambiguate from unrelated outbox/voice trigger enums.)

**Gate-but-don't-queue split.** When the gate fires (no peer row + `autoAcceptPeering=0`), behavior diverges on intent:

- **`intent.kind === 'user_action'`** — upsert an outbound `peer_approval_requests` row keyed on `(origin, direction='outbound')`, upsert a `peer_approval_subscribers` row keyed on `(request_id, user_id, trigger_reason, trigger_target)`, broadcast `federation_approval_request_received` to admins, broadcast `peering_subscription_changed` to the requesting user, return `{ status: 'admin_required', error: 'Awaiting your admin\'s approval to initiate peering' }`. The user's request is admin-approvable but no traffic reaches the wire yet.
- **`intent.kind === 'system'`** — no DB writes, no admin broadcast, no admin-visible queue clutter; return `{ status: 'admin_required', error: 'Outbound peering requires admin approval on this instance' }`. A stale outbox event or dead voice-call has no admin remedy worth surfacing.

**`'admin_required'` result semantics.** Each user-action caller maps the new variant to its own surface (e.g. friend-add returns `409 peer_pending_local_admin`; `sendCallRelay` returns `peer_admin_required` and threads through the existing exhaustive `CallRelayFailureReason` switch). See `social.md` §6 outbound flow for the friend-add error mapping.

**Lifecycle (centralized cleanup on `onPeerActivated`).** The most important correctness invariant: outbound subscriber cleanup hangs off **status transition to `active` (`onPeerActivated`)**, NOT off the local admin's approve-action handler. This holds across every activation path:

- queue approval (`/api/federation/approval-requests/:id/approve` 200 branch)
- admin-direct (`/peer/initiate`)
- autoAccept=1 remote (`/peer/accept` 200 from a remote that auto-accepts)
- mutual-token approval (the receiver's `/peer/accept` verifying `approvalToken` and promoting `awaiting_approval → active`)

When `federation_peers.status` transitions to `active`, `onPeerActivated` (`utils/federationPeerActivation.ts`) calls `fanoutOutboundSubscribers(origin)`: it queries the outbound `peer_approval_requests` row for the activated origin, inserts a `kind='approved'` row in `peer_approval_notifications` for each subscriber, broadcasts `peering_notification_received` per subscriber, then deletes the parent `peer_approval_requests` row (cascade clears subscribers). No per-action code knows about subscribers; the queue-approval handler does NOT duplicate this cleanup — it just performs the handshake and lets the resulting status transition trigger fanout.

The reason cleanup must NOT live in the approve-action handler: when the remote also has `autoAcceptPeering=0`, local admin approval transitions our peer row to `awaiting_approval`, not `active`. Subscribers must remain queued until full activation completes (which may be days later when the remote admin approves). Wiring cleanup to the approve-action handler instead would clear subscribers prematurely; users would retry against an `awaiting_approval` peer and hit `peer_pending_approval` 409, confused.

The other lifecycle exits write notifications and cascade-delete the parent at the trigger site:

- **Admin denies an outbound request** (`POST /api/federation/approval-requests/:id/deny`, `direction='outbound'` branch) — fans out `kind='denied'` notifications to subscribers, then deletes the parent (cascade clears subscribers). No remote network call — the remote never knew we were considering this peer. `federation_peers_changed` is broadcast to admins so the queue UI refreshes.
- **Last-subscriber cancel** (`DELETE /api/federation/peering-subscriptions/:id`) — deletes the subscriber row; if it was the last subscriber for the parent, cascade-deletes the parent. No notification created (the user took the action; they know).
- **Parent-row expiry** (`storageJanitor.ts` outbound branch) — fans out `kind='expired'` notifications to subscribers before deleting the parent. **Inbound expiry behavior is preserved unchanged** from pre-branch: the janitor still sends a signed `/peer/denied` to the inbound origin and only deletes the row on success (the Task 9 first-pass mistakenly removed this; commit `c4e7438` restored it).

**No status column on subscribers.** Existence = waiting; deletion = resolved (with the resolution mode encoded in the notification row created at deletion time, except for the canceller path).

**Gate composition with the trust-guard.** The pre-handshake trust-guard ("inbound `peer_approval_requests` exists → refuse outbound with `'rejected'`") is preserved unchanged and runs after the gate. The trust-guard query was narrowed to `direction='inbound'` so the gate's own outbound queue rows don't false-positive block their own approval path. The two layers compose cleanly: the gate fires earlier when there's no peer row at all; the trust-guard fires later when an inbound row exists.

**Caller summary** (intent each call site declares):

| Site | Intent | On `'admin_required'` |
|---|---|---|
| `routes/social.ts` (friend-add) | `user_action` reason `friend_add` target `name@domain` | 409 `peer_pending_local_admin` |
| `routes/federation.ts` (`/peer/ensure`) | `user_action` reason `friend_add` (default; client-supplied reason is ignored today) | response `peeringStatus: 'admin_required'` |
| `utils/federationOutbox.ts` (`sendCallRelay` no-active-peer) | `system` | `CallRelayFailureReason.peer_admin_required` |
| `utils/federationWorker.ts` (`resolvePendingPeers`) | n/a | operates only on already-existing pending rows; gate is unreachable from this path |

Admin-initiated paths (`/peer/initiate`, `/approve`) do NOT call `ensurePeered`. They issue their own `fetch` and run their own activation logic. The gate does not affect admin power; admins retain full ability to pre-peer or approve outbound regardless of the setting.

### Admin Endpoints

| Endpoint | Method | Auth | Purpose |
|----------|--------|------|---------|
| `/api/federation/peer/initiate` | POST | JWT + admin | Start peering handshake |
| `/api/federation/peer/accept` | POST | None (rate-limited) | Accept incoming handshake |
| `/api/federation/peer/ensure` | POST | JWT (any user), rate-limited 3/15min/user | Trigger auto-peering to a remote instance |
| `/api/federation/peers` | GET | JWT + admin | List all peers (secret excluded) |
| `/api/federation/peers/:id` | PATCH | JWT + admin | Update peer settings (auto-rotation interval) |
| `/api/federation/peers/:id` | DELETE | JWT + admin | Revoke peer, purge outbox |
| `/api/federation/peers/:id/permanent` | DELETE | JWT + admin | Hard-delete revoked peer record |
| `/api/federation/peers/:id/reset` | POST | JWT + admin | Delete peer record (cascade-deletes outbox). Only admissible in `needs_attention` state. |
| `/api/federation/peers/:id/rotate` | POST | JWT + admin | Trigger immediate secret rotation |
| `/api/federation/peers/:id/recheck` | POST | JWT + admin | Run an immediate reachability probe on an unreachable peer; 400 unless status='unreachable'; 200 `{ recovered, status }` |
| `/api/federation/approval-requests` | GET | JWT + admin | List pending peering approval requests (inbound + outbound). Outbound rows include `subscribers: ApprovalRequestSubscriberSummary[]` (possibly empty). Inbound rows omit `subscribers`. Each row carries `direction: 'inbound' \| 'outbound'`. |
| `/api/federation/approval-requests/:id/approve` | POST | JWT + admin | Approve request — direction-branched (see Approval flow above) |
| `/api/federation/approval-requests/:id/deny` | POST | JWT + admin | Deny request — direction-branched (see Denial flow above) |

**`POST /api/federation/peer/ensure`** — Wraps `ensurePeered()`. Accepts `{ remoteOrigin: string }` in body. Returns `{ peeringStatus, peerId?, error? }` where `peeringStatus` is one of `active`, `pending`, `awaiting_approval`, `rejected`, `unreachable`, or `revoked`.

### S2S Endpoints

| Endpoint | Method | Auth | Purpose |
|----------|--------|------|---------|
| `/api/federation/peer/rotate` | POST | HMAC | Accept secret rotation from peer |
| `/api/federation/peer/denied` | POST | HMAC | Receive denial notification for awaiting_approval peer |
| `/api/federation/identity` | DELETE | HMAC | Delete federated user identity (soft/full mode) |
| `/api/federation/users/lookup` | POST | HMAC, rate-limited 60/min/peer | Resolve a username on this instance to (homeUserId, profile snapshot) for cross-instance friend-request originators |
| `/api/federation/epoch` | POST | HMAC (signed request **and** signed response) | Return this instance's persistent epoch `{ instanceId }`; populates a peer's trusted epoch baseline (`peer_instance_id`) |
| `/api/federation/verify-attach-proof` | POST | HMAC (signed request **and** signed response), rate-limited 60/min/peer | Verify a one-time detached-account re-attach proof token; single-use, bound to the calling peer's domain (re-attach spec §3.1) |

### S2S Detached-Account Re-Attach Proof (`POST /api/federation/verify-attach-proof`)

The home-instance verifier for the detached-account re-attach flow (re-attach spec §3.1). A user who was detached on peer R (its home domain was reset; see the Detached-account guards above) proves control of the still-native home account H, mints a one-time proof token on H via `POST /api/auth/attach-proof` (`randomBytes(32).toString('hex')`, stored in `federation_attach_proofs`, bound to R's domain), and hands it to R. R then calls this endpoint on H to redeem it.

- **Request:** HMAC-signed (same boilerplate as `/users/by-home-id`: missing headers → 401, unknown/inactive peer → 403, rate-limited 60/min/peer → 429, bad signature → 401, nonce replay → 409/401). Body `{ token: string }`.
- **Peer-domain binding is server-side (anti-replay):** the token's `target_domain` must equal the domain of the **authenticated calling peer** (`extractDomain(peer.origin)`), never a value from the request body. A compromised peer cannot redeem a token minted for a different peer.
- **Single-use is atomic:** the claim is a raw `UPDATE federation_attach_proofs SET used_at=? WHERE token=? AND used_at IS NULL AND expires_at>? AND lower(target_domain)=? RETURNING home_user_id`. Only the first concurrent verification can flip `used_at` from NULL, so a token can never be redeemed twice.
- **Re-confirms native identity:** after the claim, the home user must still be native and live (`isDeleted=0 AND home_instance IS NULL`) — a user tombstoned or turned into a replicated stub after mint fails closed.
- **Signed response (epoch pattern):** the body is HMAC-signed with the peer's shared secret (`X-Federation-Signature/Timestamp/Nonce` response headers) so the caller can trust the identity it carries. `{ valid: true, homeUserId, username }` on success; every failure mode (unknown/expired/used/wrong-domain/malformed token, deleted/non-native home user) fails closed to a **still-signed** `{ valid: false }`.

### Peer-Side Re-Attach (`POST /api/users/@me/reattach`)

The owner-initiated exception to the detach invariant (re-attach spec §3.2), on peer R. **JWT-authenticated as the detached account, not S2S** — but registered in `routes/federation.ts` (not `users.ts`) because it consumes federation-internal machinery (`verifyAttachProofWithPeer`, `fetchHomeProfileByHomeId`, `downloadProfileAsset`, the peer HMAC channel). It re-binds the sovereign detached row back to the owner's new home identity, restoring live sync while keeping history. It links **only** when BOTH proofs hold: the session IS the detached account (local password authority, via `authenticate`) AND the one-time token verifies with the home peer over signed S2S (`verifyAttachProofWithPeer`). Identity is never guessed and never username-matched — the latter is exactly the tier-2 hijack the detach branch closed.

- **Body:** `{ token: string }` (64-char hex; else 400). **Response:** `{ success: true, user: User }` (sanitized self-view; `ReattachResponse`).
- **Guards, in order:** (1) session user is a **live detached** federated account (`home_instance` set, `federation_home_orphaned = 1`, `is_deleted = 0`) → else 403; a missing/tombstoned row → 404 (a tombstoned session is already 401'd at `authenticate`, so the handler's 404 is defense-in-depth for a concurrent-delete race — detached tombstones are not re-attachable). (2) The home domain is an **active peer** → else 409 (the proof is only as trustworthy as the S2S channel it verifies over). (3) `verifyAttachProofWithPeer` returns `valid:true` → else 401 (fails closed). (4) If the verified new identity already has a live local row for that domain, it MUST be a replicated stub (`password_hash = '!federation-replicated'`) → a real account holding it is state corruption, aborted with **409 + a `console.error`** (impossible while the detached row holds the username).
- **Effect (one `rawDb.transaction`):** merges any pre-existing stub for the new identity into the detached row (below), sets `home_user_id = <new homeUserId>`, `federation_home_orphaned = 0`, adopts the new home username base if it differs (existing collision-suffix `<base>_<n>@<domain>` scheme; base match keeps the current handle), and **nulls `profile_updated_at`** so the home's next `profile_update` (any version) tier-1 matches and applies. Group-DM authority the old identity held is migrated by `owner_home_user_id` (the S2S authority key, home-domain-normalized). After the transaction, a best-effort `fetchHomeProfileByHomeId` pull applies the current home profile (avatar/banner via `downloadProfileAsset`, fail-open); then `user_updated` is broadcast to friends/DM/space co-members + all self connections (`collectProfileBroadcastTargetIds`).
- **Guard re-enablement:** clearing `federation_home_orphaned` automatically re-enables normal federated-account semantics — login self-heal resumes, and the S2S `profile_update`/`presence_update`/tier-2/identity-delete guards correctly stop firing for this account (they only fire on `federation_home_orphaned = 1`). This is intended, not a guard regression.

**Stub merge (spec §3.3).** By the time the owner re-attaches, R may already hold a replicated stub for the new home identity (from ordinary DM/friend relay, e.g. `youruser_1@<domain>`). Two rows must not share `(homeUserId, homeInstance)`, so the stub is merged into the detached row inside the transaction: every `users.id` FK a replicated stub **can** populate is repointed, with collision rows deduped **before** repoint. Tables (audited against `schema.ts`): `dm_members` (dedupe on `dm_channel_id`), `dm_messages`, `messages`, `dm_reactions` (dedupe on `dm_message_id+emoji`), `reactions` (dedupe on `message_id+emoji`), `friends` (both columns + drop self-rows), `friend_requests` (both columns + drop self-rows), `read_states` (dedupe on `channel_id`), `dm_channels.owner_id` (plain-text column, no FK). Space-scoped FKs (`space_members`, `member_roles`, `*_overrides`, `bans`, `join_requests`, `voice_restrictions`, layouts/folders) and moderator/owner RESTRICT columns are **not** repointed — a DM/friend replica can never hold them. The stub row is then deleted. Only a `'!federation-replicated'` row is ever a merge source (guard 4).

**1-on-1 DM `federatedId` reconciliation (reattach-dm-reconcile spec §3.1–§3.2).** A 1-on-1 DM's identity is `computeFederatedId(homeUserIdA, homeUserIdB)` — a deterministic SHA-256 of the two sorted home user IDs. The re-bind changes the account's `home_user_id`, so **every** 1-on-1 DM it participates in now derives a different `federatedId`: pre-reattach history stays under the OLD-identity channel while post-reattach messages compute the NEW id and land in a parallel channel — one conversation surfaced twice. So, still inside the re-attach transaction (after the re-bind UPDATE), the endpoint enumerates the account's 1-on-1 channels (exactly 2 members, 1-on-1-shaped `federated_id`) and calls `reconcileDmChannelFederatedId(rawDb, channelId)` on each. That helper recomputes the expected id from the members' **current** home identities and, when it differs from the stored id, either **re-keys in place** (no channel already carries the new id) or **merges the drifted channel INTO the existing new-identity channel and deletes it** (`idx_dm_federated` is UNIQUE, so two rows can never share a `federated_id`). Merge moves `dm_messages` (globally-unique snowflake ids; `attachments`/`dm_reactions` follow by `dm_message_id`), dedupes `dm_members` and `read_states` on their composite PKs, then drops the source row. Group DMs (random-UUID `federatedId`) are **skipped** — their id is member-independent, so re-attach never drifts them. After commit, affected local members receive `dm_channel_closed` (merged source) + `dm_channel_created` (full surviving-channel payload) so the split collapses live without a reload.

**Why R-local reconciliation is complete, not partial (spec §2).** A 1-on-1 DM is stored on an instance only if that instance is the home of at least one participant. The detached account is homed at the reset domain, i.e. **not native to R** (the peer where it now lives) — so the *other* participant of every 1-on-1 DM it holds on R is necessarily R-native, and R is that channel's authoritative home. Re-keying locally therefore produces the globally-correct id (the same value the R-native counterpart and the new home-identity compute); the reset home instance holds no old-identity channel. There is no cross-instance residual to relay: R-local reconciliation covers 100% of the account's 1-on-1 DMs.

### S2S Epoch Refresh (`POST /api/federation/epoch`)

HMAC-authenticated in **both directions**: the request is signed (only a peer holding the shared secret may call it — unknown/revoked peers → 403, bad signature → 401, missing headers → 400) **and the response body `{ instanceId }` is HMAC-signed** with the same secret (`X-Federation-Signature/Timestamp/Nonce` response headers). The caller (`fetchPeerEpoch(peer)` in `utils/federationEpoch.ts`) verifies that response signature with the same secret before trusting the value, then writes it to `federation_peers.peer_instance_id`. Response-signing (not TLS-only) is deliberate: a poisoned baseline could drive a spurious data-heal on a live peer, so the newly-trusted epoch is authenticated (design §9). `fetchPeerEpoch` **fails safe** — a `404` from a not-yet-upgraded peer, an absent/invalid response signature, or a network/timeout error all return `null` (10s timeout via `AbortSignal.timeout`); the caller treats `null` as "retry on the next tick," never as an error to surface. This is the deterministic populator of the epoch baseline (the bounded periodic epoch-refresh, design §3.2), independent of organic relay traffic.

**Deterministic epoch-refresh driver (`refreshPeerEpochs()` in `utils/federationEpoch.ts`).** Selects every `active` peer whose `peer_instance_id IS NULL`, calls `fetchPeerEpoch(peer)` once each, and on a non-null result writes the epoch via `UPDATE ... SET peer_instance_id WHERE id = ? AND peer_instance_id IS NULL`. The trailing `IS NULL` guard makes it **populate-if-null only** — it can never overwrite a baseline another path (relay envelope, handshake) already established — and makes it **self-terminating**: once a peer's `peer_instance_id` is set, the `IS NULL` filter excludes it, so it is never fetched again. A `null` from `fetchPeerEpoch` (404 / bad-sig / network) is a benign `continue` with no error log-spam, retried next tick. Wired into the federation worker in two places: once at `startFederationWorkers()` startup and once at the end of `processHealthCheckTick()` (the existing 15-minute health-check tick), both as `refreshPeerEpochs().catch(() => {})`. This guarantees the trusted baseline is populated within one refresh cycle of an upgrade, independent of user/relay activity — the load-bearing populator that relay-only population cannot cover for idle peers.

### Reset Detection (`markPeerReset` — `utils/federationReset.ts`)

The epoch is a **detection signal only, never an authorization signal.** When a peer behind a known origin advertises an epoch that differs from the trusted baseline (`peer_instance_id`), the instance was wiped and a new incarnation stood up on the same domain. `markPeerReset(peerId, origin, deadEpoch, observedEpoch)` routes the peer for admin attention and snapshots the dead incarnation — but performs **NO rekey, NO tombstone, NO handle change, NO content deletion.** The actual data heal fires only later, from `onPeerActivated` after an admin-authenticated re-peer (design §6). Because detection grants no capability and destroys nothing, it is safe to fire on an unauthenticated signal: the worst a spoofed detection can do is flag a peer for admin review (admin-reversible nuisance).

In a single transaction, `markPeerReset`:
1. Sets the peer `status='needs_attention'`, `needs_attention_reason='peer_reset_detected'`, and `observed_peer_instance_id=observedEpoch`. **`peer_instance_id` (the trusted baseline) and `hmac_secret` are left untouched** — an unauthenticated observation never rekeys trust; the observed-but-untrusted epoch lives only in `observed_peer_instance_id`.
2. **Snapshots the dead incarnation:** sets `users.federation_heal_pending = 1` for every non-deleted user whose `home_instance` matches the origin. The match keys on `extractDomain(origin)` (bare domain, the canonical `home_instance` form) and defensively also matches the `https://`/`http://`-prefixed forms so any legacy full-URL straggler is caught (`homeInstanceMatch()`). Any stub created *after* detection (e.g. a friend-add reaching the new incarnation directly) is un-flagged and survives the heal.
3. **Journals the dead incarnation durably** by upserting a `federation_reset_events` row keyed by origin: `{ dead_epoch=deadEpoch, new_epoch=NULL, detected_at, resolved_at=NULL, stub_count, orphaned_account_count }`. `stub_count` counts flagged pure S2S stubs (`password_hash = '!federation-replicated'`); `orphaned_account_count` counts flagged real accounts. This row survives the peer-row deletion that Re-peer performs, preserving `dead_epoch` for the false-positive guard (design §6.1) and the admin surface.
4. After the transaction, broadcasts `federation_peers_changed` and `federation_peer_reset_detected {origin}` to admins.

**Idempotent / double-reset:** if an *unresolved* `federation_reset_events` row already exists for the origin (the peer reset again before an admin resolved the first), the original `dead_epoch` and `detected_at` are **preserved** (that is the incarnation whose users are already snapshotted) — only the summary counts are refreshed. `dead_epoch` is never overwritten on an unresolved row. A prior *resolved* reset starts a fresh journal entry.

**Detection sources (all three wired in this feature):**
- **Inbound handshake** — `/peer/accept` landing on an `active`/`needs_attention` row (`routes/federation.ts`): before the idempotent-200 return, `if (reqInstanceId && existing.peerInstanceId && reqInstanceId !== existing.peerInstanceId) markPeerReset(...)`. The guard still returns 200 and **still does not rekey** — the anti-hijack property is preserved verbatim; detection is layered on top.
- **Reachability probe (`unreachable` peers)** — `probePeerReachable()` (`utils/federationRecovery.ts`) now parses `instanceId` from the `/api/instance/info` response (returning `{ reachable, instanceId }`; a missing/unparseable epoch is `null`, never an error). The shared decision helper `recoverOrDetectReset(peer, result)` — used by both the background recovery tick (`processRecoveryTick`) and the manual recheck endpoint — routes to `markPeerReset` (returning `'reset_detected'`) when the peer has a non-null `peer_instance_id` and the probed epoch differs, and **does NOT call `markPeerRecovered`**. Rationale: a genuinely reset peer's HMAC secret is desynced, so flipping it back to `active` via a reachability probe would resume relay against a dead secret. Only when the probed epoch matches the baseline (or the baseline is null / epoch unknown) does the normal recovery-to-active path run. The manual recheck endpoint returns `{ recovered: false, status: 'needs_attention' }` on `'reset_detected'`.
- **`needs_attention`-peer reset probe** — closes design §4.1's remaining sub-case. A reset peer can reach `needs_attention` via the **auth-failure path** — its HTTP is up but returns 401/403 because the new incarnation has no peer row for us, so `consecutive_auth_failures` crosses `AUTH_FAILURE_THRESHOLD` — **without ever transitioning through `unreachable`**. The `unreachable`-only 5-second recovery probe therefore never observes its epoch change, so no journal is created; a later manual Re-peer would then run `healResetIncarnation` with no journal row → no heal → the split-brain persists. The shared per-peer unit is **`detectResetForPeer(peer)`** (`utils/federationRecovery.ts`): for a peer with a non-null baseline it runs one `probePeerReachable` (`/instance/info` GET) and calls `markPeerReset` on an observed epoch mismatch. It is invoked from **three** places, so detection latency is near-zero rather than up to a full health-check cycle:
  1. **Event-driven, at the transition** — the instant the outbox worker moves a peer to `needs_attention` on the auth-failure threshold (`federationWorker.ts`), it fires `detectResetForPeer` for that peer (fire-and-forget). This is the common live case: the moment the connection is declared broken, the epoch is checked and "Re-peer & heal" surfaces immediately.
  2. **Worker-startup sweep** — `startFederationWorkers()` runs `detectResetOnNeedsAttentionPeers()` once on boot, catching any peer already parked in `needs_attention` (reset while this instance was down, or transitioned before this probe shipped).
  3. **15-minute health-tick backstop** — `detectResetOnNeedsAttentionPeers()` also still runs at the end of `processHealthCheckTick` as the periodic safety net. It selects peers with `status='needs_attention'` AND `peer_instance_id IS NOT NULL` AND `needs_attention_reason` not already `peer_reset_detected` (those already carry a journal) and calls `detectResetForPeer` on each.
  **Detection only:** unlike `recoverOrDetectReset`, none of these ever flips a `needs_attention` peer to `active` (a match / unknown / unreachable result is a pure no-op) — that peer's secret is desynced and only an admin-authenticated re-peer restores trust. Neither `peer_instance_id` nor `hmac_secret` is touched.

Legacy peers advertise no epoch (`instanceId` null), so detection requires a non-null observed epoch differing from a non-null stored baseline — legacy peers never trigger it, and the existing `auth_failures → needs_attention → manual Reset` path continues unchanged for them.

### Limbo-window user error (`peer_reset_pending`)

Between reset detection (`markPeerReset`) and the admin's one-click Re-peer, the stale identity graph still exists and no heal has run (design §5.3). During this window a user re-adding a same-name friend, or creating a DM to that origin, would otherwise hit a confusing `already_friends` (stale friendship bound to the dead incarnation) or `peer_rejected` (the peer now sits in `needs_attention`, tripping `ensurePeered`). Both user-facing hot paths short-circuit with a clearer **409 `{ error: 'peer_reset_pending' }`**:

- **Friend-add** (`social.ts` `POST /api/social/requests`, federated branch): after `resolveOriginFromHostname` yields `peerOrigin` and **before** the peering/lookup/`already_friends` checks.
- **DM-create** (`dm.ts` `POST /api/dm`, `homeUserId + homeInstance` branch): before stub creation, so no un-flagged stub is left behind. The target origin is resolved with `resolveOriginFromHostname(new URL(canonicalizeHomeInstance(homeInstance)).host)`.

Both perform an **O(1) point lookup** on the `federation_reset_events` origin PRIMARY KEY (`origin = peerOrigin AND resolved_at IS NULL`). `peerOrigin` (from `resolveOriginFromHostname`, which returns the stored `federation_peers.origin` verbatim) is exactly the string `markPeerReset` journals, so the query is a single indexed hit/miss. The guard **only** short-circuits when an unresolved row exists; the common case — no reset in progress — is one indexed miss and the normal path proceeds byte-for-byte unchanged. Once the admin re-peers and `healResetIncarnation` resolves the journal (`resolved_at` set), the guard stops firing and the freshly-clean graph accepts the re-add.

### Data Self-Heal (`healResetIncarnation` — `utils/federationReset.ts`)

Detection (`markPeerReset`) only snapshots + journals + notifies; it destroys nothing. The actual heal is `healResetIncarnation(origin, newEpoch, reason)`, fired from `onPeerActivated` (`utils/federationPeerActivation.ts`) **after an admin-authenticated re-peer**, keyed to the confirmed epoch change (design §6). It runs **before** the mutation-log re-sync in `onPeerActivated` so re-sync repopulates onto a clean slate, and it runs **outside any transaction** (`tombstoneUser` opens its own; better-sqlite3 throws on a nested `BEGIN`).

**Two mandatory guards, in order:**

1. **Reason gate.** `onPeerActivated` fires on non-handshake paths too. Only genuine re-handshake reasons carry a freshly-exchanged, trustworthy epoch. `healResetIncarnation` returns immediately unless `reason` is in the allow-list `HANDSHAKE_ACTIVATION_REASONS` (typed `ReadonlySet<PeerActivationReason>`): `initiate_accepted`, `accept_new`, `accept_pending`, `accept_rejected_override`, `accept_awaiting_approval`, `accept_awaiting_approval_fallback`, `approval_handshake`, `ensure_peered`. The two EXCLUDED reasons — `health_check_recovery` (reachability flip in `markPeerRecovered`) and `startup_bootstrap` (boot re-scan) — flip a peer to `active` **without** a handshake, so their baseline is stale (still equals the journaled `dead_epoch`). Without the gate they would hit the `deadEpoch === newEpoch` false-alarm branch and silently resolve the journal + clear the flags WITHOUT healing, permanently burying the bug. Gated out, they leave the journal fully intact for a later genuine re-handshake to heal.

2. **Epoch comparison (false-positive guard).** For a gated-in reason, look up the UNRESOLVED `federation_reset_events` row for the origin (none → return):
   - **`journal.dead_epoch === newEpoch`** — the re-peer confirmed the SAME incarnation (spurious/spoofed detection, or an admin re-peer to a never-reset live peer). **NO tombstone** — the user-level snapshot flags alone must never authorize destruction; only a confirmed epoch change does. Clears `federation_heal_pending` for the origin and resolves the journal (`new_epoch`, `resolved_at`).
   - **`journal.dead_epoch !== newEpoch`** — a GENUINE new incarnation. Soft-tombstones the flagged **pure stubs** only, then clears their flags and resolves the journal.

**Soft-tombstone (pure stubs only).** For every user that is `federation_heal_pending = 1` AND `password_hash = '!federation-replicated'` (pure S2S stub sentinel) AND matches the origin (`homeInstanceMatch`), calls `tombstoneUser(uid, { purgeContent: false })`. The `purgeContent: false` is **non-negotiable** — the default (`true`) irreversibly deletes this box's reactions and authored space messages, violating the invariant that a remote's reset never destroys our non-re-syncable content. The soft tombstone clears exactly the relationship rows that cause the bug (`friends`, `friend_requests`, group `dm_members`, …) so stale friendships/DMs clear and re-adds work. A **1-on-1** `dm_members` row is now KEPT and anonymized so the survivor retains a read-only "Deleted User" thread — see `dm-system.md` § "DM Tombstone Semantics". Flags are then cleared **keyed by the stub id list** (not by re-querying the sentinel — `tombstoneUser` has already randomized `password_hash`).

**Live co-member update.** Around each stub, the heal loop now broadcasts a sanitized `user_updated` so survivors' clients flip the partner to "Deleted User" without a reload — aligning this path with the other three deletion callers (admin/self/identity-delete). For each stub it captures `collectDeletionBroadcastTargets(stub.id).targetUserIds` **before** `tombstoneUser` (which deletes the DM/friend/space rows that set is derived from), then re-reads the tombstoned row and calls `connectionManager.sendToUser(targetId, { type: 'user_updated', user: sanitizeUser(deletedRow) })` for each target.

**Real federated accounts — detach (design §6.3b, revised by the 2026-07-02 orphaned-account-detach spec).** A flagged user that is NOT a stub (`password_hash != '!federation-replicated'`) carries real, non-re-syncable local content and is **never** auto-tombstoned. In the genuine-reset branch, after the stub soft-tombstone loop, `healResetIncarnation` calls `quarantineOrphanedAccounts(origin)`:

- For every flagged real account (`federation_heal_pending = 1`, non-stub, `isDeleted = 0`, `homeInstanceMatch`): set `federation_home_orphaned = 1` and clear `federation_heal_pending`. **That is all** — this is a flag-only **detach**, not a freeze.
- **`federation_home_orphaned = 1` means "DETACHED / sovereign local account,"** not "frozen." The account was cut loose from its (now-reset) home instance and operates as a purely local account from here on: it logs in with its **local password** (`auth.ts` no longer blocks a detached account before password verify — only the self-heal branch is permanently disabled for it, see `auth.md` §4), and it gains local profile edit + local change-password (`users.ts`). There is **no login freeze**.
- **No rename.** The username is preserved — first-come-first-served on this instance. There is **no `!orphaned:{uid}@{domain}` handle-freeing** and **no space-owner special case**: all real accounts are treated uniformly and owners simply keep managing their spaces.
- **What closes the post-re-peer hijack** is NOT a login freeze. It is the combination of (a) the login self-heal being **permanently disabled** for detached accounts (`auth.ts` returns 401 in the failed-local-password branch without contacting the home domain — a new incarnation can never re-hash its way in) and (b) the S2S identity-binding guards, which **exclude detached rows** on every domain-keyed surface: `findFederatedUser` tier-2 (`federation_home_orphaned = 0` predicate, ~line 3522), the S2S `profile_update` handler (accept-and-skip, ~line 6162), the S2S `presence_update` handler (accept-and-skip, after the domain-collision check), the `hydrateReplicatedUserProfile` fill-empty path (no-op early return alongside the native-user skip), and the S2S `DELETE /api/federation/identity` guard (idempotent 200, no deletion, ~line 2357). Every domain-keyed **mutation** that a tier-1 (`homeUserId`) hit can reach is guarded at its own site — a tier-1 hit on a detached row is a legitimate historical read, but no write is applied. See "S2S Identity Deletion" above and design §4.3.
- Content (space messages, memberships, reactions) and usernames are preserved in all cases. No `user_updated` broadcast is emitted (nothing visible changes). The returned count refreshes the journal's `orphaned_account_count`.

**Login self-heal epoch guard (design §6.3a).** The federated password self-heal (`auth.ts` §4) gates re-hashing on the home instance's current epoch, read via the authenticated `fetchPeerEpoch(peer)` (HMAC-signed both ways): no baseline on record → allow (legacy); baseline differs from the fetched epoch → refuse; epoch can't be determined (`fetchPeerEpoch` null — 404/unreachable/bad-sig/desynced secret) → **fail closed/refuse**; match → allow. This closes the *pre*-re-peer hijack (a reset home accepting a new same-name user's password during the undetected-reset window). The *post*-re-peer window is closed by detach: once an account is detached, its self-heal path is permanently disabled and the S2S binding guards exclude it (above), so the new incarnation has zero influence over it. Full three-way in `auth.md` §4.

**Reset-events admin surface (`GET /api/federation/reset-events`).** Admin-only. Returns the durable `federation_reset_events` journal (each event carrying a nullable `acknowledgedAt`) joined with each origin's current detached real accounts (`federation_home_orphaned = 1`, `homeInstanceMatch`), each with `ownedSpaces`, `spaceMemberCount`, and authored-`messageCount` for disposition. Response type `FederationResetEventsResponse` (`{ events: FederationResetEvent[] }`, each event carrying `orphanedAccounts: FederationOrphanedAccount[]`). The endpoint returns **all** events (including acknowledged ones, for audit); the client filters to `acknowledgedAt === null`. Disposition actions — one-click Re-peer (`/peers/:id/reset` → `/peer/initiate`), full-purge Remove (`DELETE /api/admin/users/:id`, owns-spaces → transfer first) for genuinely-abandoned detached accounts, and a non-destructive **Dismiss** (`POST /api/federation/reset-events/acknowledge` with `{ origin }`, idempotent — sets `acknowledged_at`) that hides the card without touching the accounts (detached accounts keep working locally). See `admin.md` "FederationPanel" and `client-federation.md` §8.

**`needsAttentionReason` on the peer API.** `GET /api/federation/peers` returns `needsAttentionReason: 'auth_failures' | 'peer_reset_detected' | 'repeer_incomplete' | null` per peer, so the admin UI distinguishes a reset-detected peer (persistent Reset-cleanup banner + one-click Re-peer) from a generic auth-failure peer (plain "Reset Peering") and from a peer whose Re-peer could not be cryptographically verified (`repeer_incomplete` — see "Trust re-establishment contract"). `repeer_incomplete` is a nullable-TEXT value only; no schema migration.

### Trust re-establishment contract

The peering handshake must never report success when trust was not actually re-established — a false success re-creates the permanent HMAC desync ("split-brain") the epoch feature exists to prevent. Two changes make the recovery handshake honest end-to-end, and the **anti-hijack guard is unchanged** (an unauthenticated `/peer/accept` never adopts a caller's secret over an existing `active`/`needs_attention` row; no path auto-rekeys).

**1. Responder honest refusal (`POST /api/federation/peer/accept`).** When a peer row already exists in `active` OR `needs_attention`, the endpoint returns `409 { accepted: false, code: 'PEER_EXISTS_RESET_REQUIRED', error, instanceName, instanceId }` instead of the old false `200 { accepted: true }`. It still does not adopt the caller's `hmac_secret` (identical guard behavior — only the reported status/body changed), and the epoch-mismatch `markPeerReset` detection still fires before the return. Legacy initiators that only read `response.ok` now fail loudly instead of silently desyncing; new initiators special-case the code.

**2. Initiator verify-before-activate (`POST /api/federation/peer/initiate`).**
- On remote `409 PEER_EXISTS_RESET_REQUIRED`: the initiator DELETES its own pending row (keeping its slot clean so the remote's own Re-peer can later land on a fresh responder slot) and returns `409 { code: 'PEER_EXISTS_RESET_REQUIRED', error }`.
- On remote 200: the initiator performs a signed `fetchPeerEpoch` (`POST /api/federation/epoch`) round-trip **before** activating. Null (secret doesn't authenticate / `/epoch` absent / unreachable) → the peer is parked in `needs_attention` with `needs_attention_reason='repeer_incomplete'` and the response is `200 { peer, verified: false }`. Success → the peer activates (storing the cryptographically-**verified** epoch as `peer_instance_id`) and the response is `200 { peer, verified: true }`.

**Recovery flows.**
- **Common reset case (one click from the survivor).** When the reset box makes contact, the survivor's detection (`markPeerReset`) moves the survivor's row to `needs_attention`. The survivor's **Re-peer** (reset local row → `/peer/initiate`) then lands on the reset box's clean responder slot → both sides re-key and the initiator verifies via `fetchPeerEpoch` → both active with a matching secret. If the reset box initiated first, the responder's new 409 refusal (change 1) means the reset box deletes its pending row rather than false-activating, so the survivor's subsequent Re-peer still finds a clean slot.
- **Bidirectional-stale case.** If both sides still hold a conflicting row, each side that holds one must reset once — the honest `409` / `verified:false` reporting tells the admin exactly that ("the remote still holds stale peering for you; its admin must reset their side, then Re-peer again"), instead of falsely reporting success on a dead peering.

**Backward-compat honesty (limitation, stated explicitly).** A genuinely pre-Phase-1 peer whose `/api/federation/epoch` returns 404 cannot be cryptographically verified, so `/peer/initiate` parks it in `needs_attention` (`repeer_incomplete`) rather than activating. This is intentional fail-safe: a legacy responder that returns 200 without adopting the secret is indistinguishable from a healthy fresh peering, so we refuse to trust it blind. All current Backspace instances ship `/epoch` (Phase 1), so this only affects truly pre-Phase-1 peers — it is an honest limitation, not a silent behavior change for healthy peers.

**Handshake `sourceOrigin` honors `PUBLIC_ORIGIN`.** `resolveLocalOrigin()` (`routes/federation.ts`) now delegates to `getOurOrigin()`, so the origin advertised in the handshake `sourceOrigin` is byte-identical to the `X-Federation-Origin` used for all authenticated S2S requests. Previously it used `https://${DOMAIN}` and ignored `PUBLIC_ORIGIN`, which silently desynced the responder's peer-row key from the auth origin on any instance where `PUBLIC_ORIGIN != https://DOMAIN` — producing permanent `403 Not peered`. See "Public Origin Override" below.

**Verification harness.** The self-contained two-instance integration harness (`packages/server/test/helpers/realHandshake.ts` + `packages/server/test/federation-handshake-desync.test.ts`) exercises the REAL cross-instance handshake — actual `/peer/initiate`→`/peer/accept`, the signed `/epoch` health probe (`s2sHealthy`), and `simulateReset` — using ephemeral in-process instances (each with `PUBLIC_ORIGIN` set to its `http://127.0.0.1:<port>` transport URL, its own temp DB and generated secrets, no live host). Case #1 is the clean-handshake control; #2 gates the responder-refusal fix (BUG-1); #4 gates one-click Re-peer recovery (BUG-2).

### S2S Identity Deletion (`DELETE /api/federation/identity`)

Allows a home instance to remove a user's replicated identity from a remote instance.

**Request body:**
```json
{ "homeUserId": "<string>", "homeInstance": "<string>", "mode": "soft" | "full" }
```

**Behavior:**

- **Attribution guard:** Rejects with `403` if the user's `homeInstance` doesn't match the `X-Federation-Origin` of the signing peer. Prevents one instance from deleting another instance's users.
- **Detached-account guard:** After the attribution guard, if the resolved user has `federation_home_orphaned = 1` (detached — its home domain was reset and it is now a sovereign local account, see §4.2/§4.3 of the orphaned-account-detach design), returns an idempotent `200 { success: true }` **without deleting**. A new incarnation on the reset domain must never delete an established account by replaying its old `homeUserId`.
- **Idempotent:** Returns `{ success: true }` for already-deleted or nonexistent users (no error).
- **Owned spaces check:** Returns `409` with `{ ownedSpaces: string[] }` if the user owns any spaces on the remote. The user must transfer or delete those spaces before identity removal proceeds.
- **Mode `"soft"`:** Calls `tombstoneUser(uid, { purgeContent: false })` — anonymizes the user row and removes the user from spaces, friends, DM membership (`dm_members`), and read-states. The `purgeContent: false` flag skips only `reactions`, `dm_reactions`, and the user's space `messages` (with attachments + embeds); DM membership cleanup and orphaned-DM purge always run in both modes (per `userDeletion.ts:121-126, 169-202`) because zero-member DM channels are unreachable garbage regardless of authorship retention.
- **Mode `"full"`:** Calls `tombstoneUser(uid, { purgeContent: true })` — full tombstone including reactions and orphaned DM cleanup.
- **Post-deletion:** Broadcasts `member_left` WS events for all spaces the user belonged to before removal.

### S2S User Lookup (`POST /api/federation/users/lookup`)

HMAC-authenticated. Rate-limited to 60 requests/minute per peer. Used by the cross-instance friend-add flow to resolve a username on this instance before the sender's home server queues a `friend_request_create` event (see `social.md` §6 outbound flow).

**Request body:** `{ username: string }` — server trims and lowercases before lookup.

**Response 200:** `{ found: true, user: { homeUserId, username, profile: { displayName, avatar, avatarColor, banner, bio } } }` — returned for native, non-deleted users regardless of their `discoverable` setting.

**Response 404:** `{ found: false, code: 'user_not_found' }` — returned for tombstoned users (`isDeleted=1`), replicated stubs (`homeInstance IS NOT NULL`), or unknown usernames.

**Response 400:** Malformed input (missing, non-string, or empty-after-trim `username`).

**Response 429:** Per-peer rate limit (60/min) exceeded; `Retry-After` header set.

**Filter invariant:** `discoverable` is NOT consulted — exact-handle resolution must work for opted-out users. This mirrors the Direct-Add invariant from social.md commits 69d430b/c60ecc5/189cac4: discovery surfaces only opted-in users, but once you have a handle you can always send a request.

### WebSocket Events (Peering)

These S→C events are pushed to the acting user's connected clients by the federation subsystem.

| Event | Pushed when | Payload |
|-------|-------------|---------|
| `federation_peer_rejected` | Outbox worker receives `403 PEERING_REQUIRES_APPROVAL` from a remote instance during auto-peering | `{ peerId: string, origin: string }` |
| `federation_peer_active` | A previously `rejected` peer transitions to `active` (e.g., via manual `peer/initiate` or incoming `peer/accept`) | `{ peerId: string, origin: string }` |
| `federation_peer_reset_detected` | A peer's advertised instance epoch differs from the trusted baseline (wipe-and-reinstall on the same domain) — emitted by `markPeerReset` after routing the peer to `needs_attention` | `{ origin: string }` (admin-only, via `sendToAdmins`) |

---

## 2. HMAC Request Authentication

### Signing Format

```
HMAC-SHA256(secret, "${timestamp}.${requestBody}")
```

Where `timestamp` is `Date.now()` (Unix milliseconds) and `requestBody` is the JSON string.

### HTTP Headers

| Header | Format | Example |
|--------|--------|---------|
| `X-Federation-Signature` | `sha256=<hex>` | `sha256=a1b2c3...` |
| `X-Federation-Origin` | Full URL | `https://nova.ddns.net` |
| `X-Federation-Timestamp` | Unix ms string | `1711619400000` |
| `Content-Type` | `application/json` | -- |

### Verification (`federationAuth.ts:verifySignature`)

1. Validate inputs: reject empty/missing body, signature, or secret
2. **Timestamp window:** `Math.abs(Date.now() - timestamp) <= maxAgeMs` (default 15 minutes)
3. Recompute: `HMAC-SHA256(secret, "${timestamp}.${body}")`
4. **Constant-time comparison:** `crypto.timingSafeEqual` on hex-decoded buffers
5. Length check: mismatched buffer lengths are rejected before `timingSafeEqual`

### Replay Attack Prevention

Two layers of replay protection:

1. **Timestamp window:** Requests older than 15 minutes are rejected (`DEFAULT_MAX_AGE_MS`).
2. **Nonce:** Each request includes a `X-Federation-Nonce` header (UUID v4). The nonce is included in the HMAC payload (`${timestamp}.${nonce}.${body}`) so it cannot be stripped. The receiver stores seen nonces in memory (keyed by peer origin, evicted after 15 min) and rejects duplicates with `409 Conflict`.

**Auto-ratchet:** The `nonceSupported` column on `federation_peers` tracks whether a peer has ever sent a nonce. Once set to `1`, nonce-less requests from that peer are permanently rejected (`401`). This allows graceful rollout — new peers get nonce enforcement automatically, legacy peers are warned in logs until they upgrade.

### Inbound Verification Flow (`POST /api/federation/relay`)

1. `parseFederationHeaders()` extracts origin, timestamp, signature, and nonce from headers
2. Look up peer by `origin` in `federation_peers` -- must exist and be `status = 'active'`
3. Re-serialize request body to JSON: `JSON.stringify(request.body)`
4. `verifySignature(bodyString, signature, peer.hmacSecret, timestamp, nonce)` -- reject if false
5. Nonce enforcement: duplicate nonce → 409, missing nonce from ratcheted peer → 401, legacy peer → warn

**Important:** The body is re-serialized server-side. This means Fastify's JSON parsing and re-stringification must produce identical output to the sender's `JSON.stringify`. In practice this works because both sides use standard `JSON.stringify` with no custom replacers.

**Relay-envelope epoch (fast-path baseline population, design §3.2).** `FederationRelayRequest` carries `sourceInstanceId?: string` — the sender stamps its current epoch (`getInstanceId()`) when building the request in `federationWorker.ts`. Because the whole body is HMAC-verified above (step 4), a valid relay authentically carries the sender's current incarnation id. Immediately after the signature check passes (and only there — the authenticated boundary), the receiver runs **populate-if-null**: `if (sourceInstanceId && peer.peerInstanceId IS NULL) UPDATE federation_peers SET peer_instance_id = <claimed> WHERE id = ? AND peer_instance_id IS NULL`. This is the *fast-path* baseline populator — it fills the trusted epoch the instant organic traffic flows, usually before the deterministic 15-minute `refreshPeerEpochs` backstop fires. It **never overwrites** a non-null baseline: a differing incarnation implies a different HMAC secret that would have failed verification, so a valid relay can never carry an epoch differing from an established baseline. Runs independent of per-event processing and does not affect relay accept/reject. Backward-compatible: older peers omit `sourceInstanceId` → the update is skipped (no-op).

---

## 3. Identity Resolution

### Functions

**`extractDomain(homeInstance)`** -- `federation.ts`
- Extracts bare domain from a homeInstance value (full URL or bare domain)
- `"https://nova.ddns.net"` → `"nova.ddns.net"`, `"nova.ddns.net"` → `"nova.ddns.net"`
- **Use when:** Normalizing homeInstance for comparison or storage

**`findFederatedUser(homeUserId, homeInstance, db, hints?)`** -- `federation.ts`
- Three-tier lookup: homeUserId match → domain + username hint match → not found
- Tier 1: delegates to `resolveLocalUser` (fast path)
- Tier 2: uses `extractDomain(homeInstance)` + `hints.username` to match stubs created by the auth registration path (which may have a different homeUserId)
- **Tier 2 excludes detached accounts** (`federation_home_orphaned = 1`): a detached account is sovereign and must never be re-bound to the reset domain's new incarnation via username heuristics — that is exactly how a new same-name user would capture the established account. **Tier 1 (`homeUserId` match) is deliberately NOT excluded:** the new incarnation mints fresh `homeUserId`s, so a tier-1 hit on a detached row is a legitimate historical reference (e.g. an old group-DM attribution relayed by a third instance), not the new incarnation. Mutations are blocked at their own sites (profile_update handler, presence_update handler, `hydrateReplicatedUserProfile` fill-empty, S2S identity delete).
- Side-effect-free — does not modify any records
- When multiple candidates match in tier 2, prefers real accounts over stubs, then most profile data
- **Use when:** Read-only lookup that needs to find users created by either auth or relay path

**`resolveLocalUser(homeUserId, db)`** -- `federation.ts`
- Read-only lookup. Returns `undefined` if not found.
- Matches: `(users.homeUserId = homeUserId)` OR `(users.id = homeUserId AND homeInstance IS NULL)`
- Excludes deleted users (`isDeleted = 0`)
- When multiple candidates exist: prefers the one with `homeUserId` set (replicated stub) over a local ID match
- **Use when:** Optional lookups where null is acceptable (member_remove, reaction processing, friend_remove)

**`resolveOrCreateReplicatedUser(homeUserId, homeInstance, db, hints?)`** -- `federation.ts`
- Calls `findFederatedUser` first. If found, backfills `homeUserId` for future fast-path lookups and returns.
- Accepts optional `hints: { username?: string | null }` for tier-2 matching
- If not found, creates a stub with `homeInstance` normalized to bare domain via `extractDomain`
- Collision-safe: appends `_1`, `_2`, ..., `_10` suffix if username exists; after 10 attempts, uses `_<random hex>`
- **Self-homed guard:** an instance never creates a replicated stub homed at its own identity domain (`getOurIdentityDomain()`, DOMAIN-derived). A live self-reference resolves at tier 1; a self-domain identity reaching the create path is a dead incarnation and resolves to `null`. Wire snapshots may carry `deleted: true` — such identities also resolve to `null` at the create path (existing rows still resolve for historical attribution).
- **Use when:** You MUST have a valid user ID. Always pass `{ username: profile?.username }` when profile data is available.

**`hydrateReplicatedUserProfile(user, profile, db)`** -- `federation.ts:2041`
- Updates replicated stubs only (`homeInstance` must be set)
- Only updates null/empty fields (preserves manually-set local values)
- Exception: avatar/banner are overwritten if the current value is a bare filename (not an absolute URL)
- Resolves bare filenames to `{homeInstance}/api/uploads/{filename}` absolute URLs
- Sets `displayName` from `profile.displayName || profile.username` -- ensures federated users show a human-readable name instead of `user@instance`

### Critical Rule

Any code path that sets `ownerId`, creates a `dm_members` row, or inserts a message MUST use `resolveOrCreateReplicatedUser`. Using `resolveLocalUser` with a `?? null` fallback can cause data corruption (e.g., `ownerId` set to null for group DMs).

### Origin Normalization

**Two formats exist in the database:**

| Location | Format | Example |
|----------|--------|---------|
| `users.home_instance` | Bare domain (normalized) | `nova.ddns.net` |
| `federation_peers.origin` | Full URL | `https://nova.ddns.net` |
| `getOurOrigin()` return | Full URL | `https://orbit.ddns.net` |
| `resolveOrCreateReplicatedUser` stores | Bare domain (normalized via `extractDomain`) | `nova.ddns.net` |
| Auth registration stores | Bare domain | `nova.ddns.net` |

Both user creation paths now store bare domain. A self-healing migration in `migrate.ts` normalizes any existing full-URL `homeInstance` values to bare domain on startup.

**Normalization pattern used in code:**
```typescript
const normalized = homeInstance.startsWith('http') ? homeInstance : `https://${homeInstance}`;
```

Locations where normalization is applied:
- `getGroupDmTargetOrigins()` (`federationOutbox.ts:294`) -- normalizes before comparing to `ourOrigin`
- `dm.ts:655` -- `isLocalMember` broadcast filter checks both formats
- `dm.ts:743` -- normalizes target homeInstance before peer origin comparison

**Attribution verification (`verifyAttribution`):**
- `verifyAttribution(actingUserHomeInstance, sourceInstance)` normalizes both via `extractDomain` and compares
- Two valid cases:
  1. **Direct**: `authorDomain === sourceDomain` — standard S2S, peer sends events for its own users
  2. **Homeward relay**: `authorDomain === extractDomain(getOurOrigin())` — a client-federation user sent a message on a remote server, and the S2S relay forwards it back to the author's home instance
- Applied as the FIRST check in every relay event processor (13+ handlers) — before user resolution or DB writes
- Prevents malicious peers from forging events attributed to users on *unrelated* instances (neither source nor receiver)

All origin comparisons use `extractDomain()` or `getOurOrigin()` with normalization, handling both bare domains and full URLs consistently.
- `federation.ts:2447` -- same pattern in `processFriendRemoveEvent`

---

## 4. DM Message Relay

### 1-on-1 DMs

**Outbound (origin instance):**
1. Message created via REST (`POST /api/dm/:id/messages`) or WS (`dm_message_create`)
2. `queueDmRelay(message, channelId, 'create')` called from `dm.ts` / `events.ts`
3. `buildRelayPayload()` constructs the message portion with `homeUserId`, `homeInstance`, `content`, `replyToId`, `editedAt`, `createdAt`
4. `getDmParticipants(channelId)` resolves all members to `(homeUserId, homeInstance)` pairs with profile snapshots
5. `getGroupDmTargetOrigins(channelId)` returns `undefined` (no owner -> broadcast to all)
6. `queueOutboxEvent(messageId, channelId, 'create', payload, undefined)` -> queued to ALL active peers

**Channel creation (1-on-1 with federated user):**
- When `POST /api/dm` creates a channel where either participant has `homeInstance` set, the deterministic `federatedId = SHA256(sorted([homeUserIdA, homeUserIdB])).slice(0, 32)` is computed and stored immediately
- This ensures `findOrCreateDmChannel` on the receiving instance finds the existing channel when the S2S reply arrives, preventing duplicate channels

**Inbound (receiving instance -- `processCreateEvent`):**
1. Validate: `event.message` and `event.participants` (>= 2) required
2. Dedup: check `(sourceInstance, sourceMessageId)` -- reject if exists
3. Resolve ALL participants via `resolveOrCreateReplicatedUser`, hydrate profiles
4. No `event.federatedId` -> 1-on-1 path
5. Compute deterministic `federatedId = SHA256(sorted([homeUserIdA, homeUserIdB])).slice(0, 32)`
6. `findOrCreateDmChannel(federatedId, [localUserA.id, localUserB.id], db)`:
   - Find by `federatedId` in `dm_channels`
   - If exists: ensure both users are members (idempotent insert)
   - If not: create channel with `federatedId`, add both members
7. Insert `dm_messages` with `sourceInstance` and `sourceMessageId`
8. Process attachments (see File Replication)
9. Broadcast `dm_message_created` to local members, **skipping** members whose `homeInstance === sourceInstance` (they already have the original)

### Group DMs

**Outbound (origin instance):**
Same as 1-on-1 except:
- `getGroupDmTargetOrigins(channelId)` returns a list of peer origins that have at least one participant
- Normalizes `homeInstance` to full URL before comparison
- `queueOutboxEvent` receives `targetPeerOrigins` and only queues to those peers
- Payload includes `federatedId` (random UUID assigned at channel creation)

**Inbound (receiving instance -- `processCreateEvent`):**
1. `event.federatedId` present -> group DM path
2. Find channel by `federatedId` -- must already exist (bootstrapped by prior `member_add`)
3. If not found -> reject with `channel_not_found`
4. Insert message, broadcast to local members

### Federated ID Generation (`federationOutbox.ts:computeFederatedId`)

```typescript
// 1-on-1: deterministic 32-char hex hash
const sorted = [homeUserIdA, homeUserIdB].sort();
return sha256(sorted.join(':')).slice(0, 32);

// Group: random 36-char UUID with dashes
return crypto.randomUUID();
```

The format difference (32-char hash vs 36-char UUID) is used by the self-healing migration to detect channel type independently of `owner_id`.

### Message Deduplication

Every relayed message is stored with:
- `source_instance`: the relay request's `sourceInstance` header value
- `source_message_id`: the `event.messageId` (original message ID on source instance)

The `(source_instance, source_message_id)` pair is checked before insertion. Duplicates are rejected with reason `'duplicate'`. A unique partial index enforces this at the DB level: `idx_dm_messages_source_unique ON dm_messages(source_instance, source_message_id) WHERE source_instance IS NOT NULL`.

### Optional `message.type` field (added 2026-04-29)

`FederationRelayEvent.message.type?: 'user' | 'system'` is optional. When present and set to `'system'`, `processCreateEvent` writes the inserted `dm_messages.type` column accordingly; when absent, the receiving instance defaults to `'user'`.

This is a **forward- and backward-compatible** addition because the inbound relay endpoint (`/api/federation/relay`) validates only structural fields (`version`, `events` array shape, `sourceInstance`); unknown fields are passed through. Old peers that don't emit `type` produce relay events that get inserted as user messages on receiving peers (the existing default), and old peers receiving relay events from new peers ignore the field entirely. No protocol-version bump is required.

This permissiveness is **intentional** — the relay envelope is designed for additive evolution. Future optional fields should follow this same pattern (no schema bump, document the field here, defaults preserve old-peer behavior).

### Typing Indicator Relay

**Event types:** `dm_typing_start`, `dm_typing_stop`

**Model:** Fire-and-forget, same as call signaling. No outbox, no retry, no mutation log. Typing is ephemeral — lost packets are acceptable.

**Channel identification:** Uses `federatedId` (not instance-local `dmChannelId`) for cross-instance channel lookup, plus `participants` for resolution context.

**Outbound (`events.ts` / `dm.ts`):**
- `handleDmTypingStart()` → after local broadcast, calls `sendTypingRelay(dmChannelId, 'dm_typing_start', userId)`
- `broadcastDmMessage()` → after local `dm_typing_stop` broadcast, calls `sendTypingRelay(dmChannelId, 'dm_typing_stop', message.userId)`

**`sendTypingRelay()` (`federationOutbox.ts`):**
- Fetches channel's `federatedId` and `getDmParticipants()` for target resolution
- Builds `FederationRelayEvent` with `typing: { homeUserId, homeInstance, username }`
- Calls `sendCallRelay(origin, [event], { peeringTimeoutMs: 0 })` for each remote peer origin — non-active peers are skipped and a background `ensurePeered` warm-up is kicked off instead

**Inbound (`federation.ts`):**
- `processDmTypingStartEvent` → look up channel by `federatedId`, resolve user via `resolveLocalUser()` (no stub creation for ephemeral events), broadcast `dm_typing` to local members
- `processDmTypingStopEvent` → same, broadcast `dm_typing_stop` to local members
- **Implicit clear:** `processCreateEvent()` also emits `dm_typing_stop` for the message author after processing an inbound relay — primary typing clear mechanism for relayed messages

---

## 5. Outbox & Relay Pipeline

### Event Queuing (`federationOutbox.ts:queueOutboxEvent`)

```
Trigger (API/WS handler)
  -> isFederationRelayEnabled()? No -> return silently
  -> Fetch active peers from federation_peers
  -> Filter to targetPeerOrigins (if specified) -- EXACT string match against peer.origin
  -> If zero peers match -> logs warning and returns
  -> For each peer, in a transaction:
      -> Check for existing outbox entry by (peerId, entityId)
      -> COALESCE:
          - delete + existing create -> delete both (net: never relayed)
          - update + existing create -> update payload, keep 'create' eventType
          - update + existing update -> update payload and eventType
          - no existing -> insert new entry
      -> TTL: now + (relayTtlDays * 86400000)
```

### Coalescing Rules (per-peer, per-entity)

| Incoming | Existing | Result |
|----------|----------|--------|
| `delete` | `create` | Entry removed (message was never relayed) |
| `update` | `create` | Payload updated, keeps `create` type (peer gets full message) |
| `update` | `update` | Payload updated, type becomes latest |
| `delete` | `update` | Payload updated, type becomes `delete` |
| any | none | New entry inserted |

### Outbox Delivery Worker (`federationWorker.ts:processOutboxTick`)

**Interval:** 10 seconds (`OUTBOX_INTERVAL_MS`)
**Batch size:** 50 (`OUTBOX_BATCH_LIMIT`)
**Timeout:** 30 seconds per request (`OUTBOX_FETCH_TIMEOUT_MS`)

1. Query entries where `nextRetryAt <= now` joined with active peers, ordered by `createdAt ASC`, limit 50
2. Group by peer
3. For each peer, reconstruct `FederationRelayEvent[]` from stored payloads:
   - Parse JSON payload
   - Copy fields: `federatedId`, `participants`, `message`, `reactions`, `reaction`, `membership`, `ownership`, `group`, `friendship`, file_rejected fields
   - Set `eventType`, `contextType`, `messageId`, `dmChannelId`, `encryptionVersion`, `timestamp`
4. Build `FederationRelayRequest` with `version: 1`, `sourceInstance: ourOrigin`
5. Sign with `buildFederationHeaders(body, peerHmacSecret, ourOrigin)`
6. POST to `{peerOrigin}/api/federation/relay`
7. On success (200):
   - Compute the **terminal entity set** = accepted entries ∪ duplicate-rejected entries
   - Delete all terminal entries from outbox (matched by `entityId` -> `outboxId`)
   - Log remaining (non-duplicate) rejected entries at `console.warn` (they stay in outbox for retry)
   - Store `result.maxUploadSize` on peer record
   - Update peer: `lastSeenAt = now`, `consecutiveFailures = 0`
8. On failure (non-200 or network error):
   - `handleOutboxDeliveryFailure()`:
     - Increment `attempts` per entry, compute `nextRetryAt = now + backoff`
     - Increment peer `consecutiveFailures`, set `lastFailureAt`
     - If `consecutiveFailures >= PEER_UNREACHABLE_THRESHOLD (10)` -> mark peer `unreachable`

#### Terminal rejection reasons

`processOutboxTick` recognizes a configurable set of receiver-acknowledged **terminal rejection reasons** (constant `TERMINAL_REJECTION_REASONS` in `federationWorker.ts`): `duplicate`, `recipient_not_found`, `attribution_mismatch`, `unknown_event_type`, `self_target_invalid`. Outbox entries with these reasons are deleted with no retry.

- `duplicate` — the receiving instance already has the row (same `(sourceInstance, sourceMessageId)`); retrying will fail identically until TTL.
- `recipient_not_found`, `attribution_mismatch`, `unknown_event_type` — structural mismatches that cannot be resolved by retrying.
- `self_target_invalid` — emitted by `processFriendRequestCreateEvent` when an inbound `friend_request_create`'s `from`-identity equals its `to`-identity (after origin normalization). Defense-in-depth: the sender's local `cannot_friend_self` check should catch this, but the receiver does not trust upstream validation. Retrying will not change the payload. The friend-create rollback callback maps this to client-facing `peer_rejected`.

For non-`duplicate` terminals, the worker invokes a registered permanent-failure callback via `invokePermanentFailureCallback(eventType, messageId, reason)` from `utils/federationRollback.ts`. Currently registered: `friend_request_create` → `rollbackFriendRequestCreate` (deletes the local `friend_requests` row by `relay_message_id` and emits WS `friend_request_relay_failed` to the sender). Other event types may register their own callbacks. See `social.md` §6 "Failure Handling" for the friend-specific rollback contract.

**Ghost-row failure mode.** Rollback callbacks are best-effort: the registry catches and logs callback errors but does NOT re-throw. A botched rollback (e.g., DB write fails mid-rollback due to disk full or FK violation) can leave a ghost row that no longer corresponds to any in-flight relay. Acceptable vs. retry-forever blocking the outbox, but worth knowing when debugging stuck pending states.

**5xx and network failures are NOT terminal** — those use the existing exponential-backoff retry path. Pending operations stay pending indefinitely under sustained connectivity loss, matching the pre-existing DM relay behavior.

Logged at `console.log` ("outbox entry removed (terminal)") to distinguish from retained-for-retry `console.warn` messages.

### Retry Backoff Schedule

| Attempt | Delay |
|---------|-------|
| 1 | 30 seconds |
| 2 | 1 minute |
| 3 | 5 minutes |
| 4 | 15 minutes |
| 5 | 1 hour |
| 6 | 6 hours |
| 7+ | 24 hours (cap) |

The schedule above (`BACKOFF_SCHEDULE_MS`) paces per-entry retries. Peer-level recovery from `unreachable` is separate and demand-driven: `processRecoveryTick` probes unreachable peers on `RECOVERY_BACKOFF_MS = [30s, 1m, 5m, 15m]` while they have queued mail (15-min backstop when silent), paced by the per-peer `last_probe_at` / `probe_attempts` columns and the `probePeerReachable` helper in `utils/federationRecovery.ts`. See [PEER_UNREACHABLE_THRESHOLD](#peer_unreachable_threshold).

### Authentication-failure handling (401 / 403)

When a relay response is 401 (HMAC rejected) or 403 (remote's peer row is non-active or missing), the worker increments `consecutive_auth_failures` on the peer row, applies backoff to the queued outbox entries via the existing `BACKOFF_SCHEDULE_MS`, and preserves `hmac_secret`. After `AUTH_FAILURE_THRESHOLD = 5` consecutive auth failures (~21.5 min with the existing schedule), the peer transitions to `needs_attention`:

- Outbox delivery halts (the existing `status = 'active'` filter on the delivery query excludes `needs_attention`).
- `hmac_secret` is preserved (admin can inspect; no silent rotation).
- Affected local users receive `federation_peer_rejected` WS events with reason "Federation trust broken — admin must reset peering".
- Admins receive `federation_peers_changed`.

The worker NEVER re-handshakes via unauthenticated `/peer/accept` in response to a 401/403. The safeguard at `/peer/accept` (idempotent-200-no-update on `active` OR `needs_attention` peers) is what prevents silent HMAC rotation; the worker's job is to respect that signal and surface it to admins rather than loop. Recovery is via the admin "Reset peering" action, which deletes the local peer row and requires out-of-band re-peering.

Network failures (timeouts, non-401/403 non-2xx responses) are tracked separately via `consecutive_failures` and lead to `unreachable` at `PEER_UNREACHABLE_THRESHOLD = 10`. A successful delivery resets both counters.

### Relay Request/Response Format

**Request:**
```typescript
interface FederationRelayRequest {
  version: 1;
  sourceInstance: string;          // Full URL, e.g., "https://nova.ddns.net"
  events: FederationRelayEvent[];  // Max 50 per batch
}
```

**Response:**
```typescript
interface FederationRelayResponse {
  accepted: string[];              // messageIds successfully processed
  rejected: Array<{
    messageId: string;
    reason: string;                // e.g., 'duplicate', 'unknown_message', 'missing_participants'
  }>;
  undeliverable?: Array<{          // optional — omitted when empty; call-signaling only
    messageId: string;
    reason: string;                // e.g., 'no_recipient'
  }>;
  maxUploadSize: number;           // This instance's max upload size in bytes
}
```

#### `undeliverable` bucket (call-signaling only)

In addition to `accepted` and `rejected`, the relay response may include an
optional `undeliverable: Array<{messageId, reason}>`. Three-way classification,
non-overlapping: each messageId appears in exactly one of the three arrays.

| Bucket | Meaning | Retry? |
|---|---|---|
| `accepted` | Processed cleanly, ≥1 recipient reached. | No |
| `rejected` | Refused at data/protocol layer (schema, attribution, channel-not-found, etc.). | Terminal. |
| `undeliverable` | Processed cleanly, zero recipients reachable. | No — call-signaling specific. |

Currently used only for `dm_call_start`:
- **Path A** (local DM exists): if no local non-caller member has an active WS
  connection, the event is pushed to `undeliverable` with reason `no_recipient`
  instead of being silently accepted. No `FederatedCallEntry` is created.
- **Path B** (no local DM): the zero-participant-match early return pushes to
  `undeliverable` rather than `accepted`.

Other event types (messages, reactions, friend events, profile updates, etc.)
keep existing semantics — a message to an offline user is still `accepted`, since
messages persist and re-deliver on reconnect.

The field is optional on the wire. Old peers omit it; new peers include it only
when non-empty. Caller-side `sendCallRelay` parses the field (defaulting to an
empty array when missing), so upgrade skew is a no-op until both sides are on
new code.

### Inbound Relay Dispatch (`POST /api/federation/relay`)

Body limit: 10 MB. Max 50 events per batch. Rate-limited to 90 requests/min per peer (sliding window, keyed by `peer.origin`). Returns 429 when exceeded. Raised from 30 after FED-009 reduced the outbox worker interval from 10s to 1s — a busy sender can now hit 60 req/min during sustained traffic.

| eventType | Processor | contextType |
|-----------|-----------|-------------|
| `create` | `processCreateEvent` | dm |
| `update` | `processUpdateEvent` | dm |
| `delete` | `processDeleteEvent` | dm |
| `reaction_add` | `processReactionAddEvent` | dm |
| `reaction_remove` | `processReactionRemoveEvent` | dm |
| `member_add` | `processMemberAddEvent` | dm |
| `member_remove` | `processMemberRemoveEvent` | dm |
| `ownership_transfer` | `processOwnershipTransferEvent` | dm |
| `read_state_update` | `processReadStateUpdateEvent` | dm |
| `friend_request_create` | `processFriendRequestCreateEvent` | friend |
| `friend_request_update` | `processFriendRequestUpdateEvent` | friend |
| `friend_request_cancel` | `processFriendRequestCancelEvent` | friend |
| `friend_add` | `processFriendAddEvent` | friend |
| `friend_remove` | `processFriendRemoveEvent` | friend |
| `file_rejected` | `processFileRejectedEvent` | dm |
| `dm_typing_start` | `processDmTypingStartEvent` | dm (fire-and-forget, no outbox) |
| `dm_typing_stop` | `processDmTypingStopEvent` | dm (fire-and-forget, no outbox) |
| `dm_close` | `processDmCloseEvent` | dm |
| `dm_reopen` | `processDmReopenEvent` | dm |
| `group_metadata_update` | `processGroupMetadataUpdateEvent` | dm |

After processing all events, the relay endpoint updates the peer's `lastSeenAt` and resets `consecutiveFailures`, then returns accepted/rejected arrays plus `maxUploadSize`.

---

## 6. Group DM Lifecycle over Federation

### member_add (`processMemberAddEvent` -- `federation.ts:1618`)

**Required fields:** `event.federatedId`, `event.membership.user`

**Two paths:**

**Bootstrap path** (channel does not exist locally by `federatedId`):
1. Requires `event.group` metadata (owner + full member roster + group metadata snapshot)
2. Creates `dm_channels` row with `federatedId`, `ownerId` (resolved via `resolveOrCreateReplicatedUser`), `ownerHomeUserId`, `ownerHomeInstance`, plus the bootstrap `name`, `icon`, and `metadataUpdatedAt` from `event.group`
3. When `event.group.icon` is non-null, mirrors `processGroupMetadataUpdateEvent` and calls `downloadProfileAsset(icon, sourceInstance)` — stores the local bare filename on success or the absolute URL on failure
4. Adds ALL roster members from `event.group.members` (each resolved via `resolveOrCreateReplicatedUser`)
5. Sends `dm_channel_created` to **local-only members** (home instance matches `getOurOrigin()`, with normalization for bare domain)
6. Sets `bootstrapped = true` to skip redundant system messages and member_add broadcasts below

#### `FederationGroupPayload` (extended)

The `event.group` payload that bootstraps a brand-new replica carries the current metadata snapshot, so a peer that has never seen the channel materializes it with the right name and icon — no separate `group_metadata_update` round-trip needed:

```typescript
interface FederationGroupPayload {
  owner: FederationRelayParticipant;
  members: FederationRelayParticipant[];
  // Extended for the polish pass:
  name: string | null;            // explicit null = no custom name
  icon: string | null;            // absolute URL on the wire; null = no custom icon
  metadataUpdatedAt: number;      // 0 for legacy/unset rows
}
```

Older peers that omit these fields fall back to safe defaults (null name/icon, `metadataUpdatedAt = 0`). Receivers never re-relay these fields — only the owner's home instance authors `group_metadata_update` events.

**Incremental path** (channel already exists):
1. Validates authority: any HMAC-verified peer is accepted (relaxed — the `sourceInstance === channel.ownerHomeInstance` check was removed to support cross-instance access). The per-user attribution check (`verifyAttribution`) still applies.
2. Cancels soft-delete if channel was pending GC
3. Resolves added user via `resolveOrCreateReplicatedUser`
4. Enforces max 10 members
5. Inserts `dm_members` row (idempotent -- skip if exists)
6. Inserts system message, broadcasts `dm_member_added` to local WebSocket clients

### member_remove (`processMemberRemoveEvent` -- `federation.ts:1825`)

1. Find channel by `federatedId` -- if not found, accept idempotently
2. Validate authority: owner's instance for kicks (`reason !== 'leave'`), any instance for self-leave
3. Resolve user via `resolveLocalUser` -- if not found, accept idempotently
4. Insert system message (before deletion, so broadcast includes the leaving user)
5. Delete `dm_members` row, clean up `read_states`
6. Broadcast `dm_member_removed` to remaining local members
7. If zero members remain -> soft-delete channel (`deletedAt = now`)

### ownership_transfer (`processOwnershipTransferEvent` -- `federation.ts:1938`)

1. Find channel by `federatedId` -- if not found, accept idempotently
2. Validate authority: `normalizeOriginForCompare(sourceInstance) === normalizeOriginForCompare(channel.ownerHomeInstance)`. Both sides are normalized to handle the bare-vs-full storage convention (see `dm-system.md` historical bugs for why this matters).
3. Resolve new owner via `resolveOrCreateReplicatedUser` (**never** `resolveLocalUser` -- must guarantee valid ID)
4. Update `dm_channels`: `ownerId`, `ownerHomeUserId`, `ownerHomeInstance` (canonicalized to full URL on storage via `canonicalizeHomeInstance` so future authority checks stay stable)
5. Broadcast `dm_owner_updated` WebSocket event with `newOwnerHomeUserId` + `newOwnerHomeInstance` so local clients can refresh their owner-routing cache without reconnecting
6. Insert system message with previous owner as actor

Triggered by both auto-transfer-on-leave and the manual `POST /api/dm/:id/transfer` endpoint — the receiver path is the same.

### group_metadata_update (`processGroupMetadataUpdateEvent`)

Owner-authored update of a group DM's `name` and/or `icon`. Mirrors the `profile_update` shape — every payload carries both fields (`null` is unambiguously "cleared"), and a server-side version vector handles dedup; there is no partial-update wire form.

**Payload:** `FederationGroupMetadataPayload`:
- `name: string | null` — trimmed, length-checked at the owner instance and re-checked by the receiver
- `icon: string | null` — absolute http(s) URL on the wire (the owner instance normalizes its bare-filename storage via `normalizeIconForWire`); receivers download and store a bare filename, or fall back to the absolute URL on download failure
- `metadataUpdatedAt: number` — captured at the moment of the owner-instance DB write
- `actor: FederationRelayParticipant` — owner by authority invariant; used only for system-message rendering

**Authority:** `extractDomain(sourceInstance) === extractDomain(channel.ownerHomeInstance)`. Otherwise rejected as `attribution_mismatch`. Receivers never re-relay this event — the owner instance is the only emitter.

**Targeting:** `getGroupDmTargetOrigins(channelId)` — every peer that hosts a member of the channel.

**Coalescing:** queued via `queueGroupMetadataRelay`; rapid edits coalesce per peer with `entityId = channelId`.

**Receiver flow (`processGroupMetadataUpdateEvent`):**
1. Lookup channel by `event.federatedId`. If missing → accepted (idempotent — no replica to update).
2. Authority: domain match against `channel.ownerHomeInstance`. Mismatch → rejected as `attribution_mismatch`.
3. Receiver hardening (don't trust remote peers):
   - missing `event.metadata` → rejected as `missing_metadata_payload`
   - `payload.name` non-null with trimmed length outside `[GROUP_DM_NAME_MIN_LENGTH, GROUP_DM_NAME_MAX_LENGTH]` → rejected as `invalid_payload`
   - `payload.icon` non-null without `http://` or `https://` prefix → rejected as `invalid_payload`
4. Version check: `payload.metadataUpdatedAt > channel.metadataUpdatedAt`. Stale or duplicate → accepted silently (no side effects).
5. Diff against the stored row. If neither `name` nor `icon` actually changed → accepted; no system message; no broadcast.
6. Idempotency dedup pre-check on `(sourceInstance, sourceMessageId)` for both suffixes (see below). If every changed field already has its corresponding system row → accepted silently (outbox retry / initial-sync replay path).
7. Resolve icon: when `iconChanged` and `payload.icon !== null`, `downloadProfileAsset(payload.icon, sourceInstance)`. Local filename on success; absolute URL fallback on failure (mirrors `processProfileUpdateEvent`).
8. Resolve actor → local user id via `resolveOrCreateReplicatedUser`. On failure (e.g. tombstoned identity), fall back to `channel.ownerId`. If both are null → rejected as `actor_not_found`.
9. Single transaction: update `dm_channels.{name, icon, metadataUpdatedAt}`; insert one or two system messages tagged with `(sourceInstance, sourceMessageId)` using the suffix scheme `${event.messageId}:name` / `${event.messageId}:icon` so two changes in a single event yield two distinct dedup keys.
10. Broadcast `dm_channel_updated` to local members via `sendToDmMembers`.
11. Broadcast each new system message via `dm_message_created`.
12. Cleanup old local icon file when the icon changed away from a bare filename (matches the avatar precedent at `users.ts:463-466`).

**Wire dump (illustrative):**

```json
{
  "messageId": "1840000000000000123",
  "eventType": "group_metadata_update",
  "federatedId": "9b8c2f4e-1a2b-4c3d-9e8f-0a1b2c3d4e5f",
  "metadata": {
    "name": "weekend plans",
    "icon": "https://nova.ddns.net/api/uploads/abc123.png",
    "metadataUpdatedAt": 1746864000000,
    "actor": {
      "userId": "...",
      "homeUserId": "...",
      "homeInstance": "https://nova.ddns.net",
      "profile": { "username": "heidi" }
    }
  }
}
```

### Local-Only Broadcast Principle

Users connected to multiple instances must see each DM channel exactly once (from their home instance). All structural broadcasts (`dm_channel_created`, system messages) filter to **local members only**:

```typescript
const isLocalMember = (u: { homeInstance?: string | null }) =>
  !u.homeInstance || !domainOrigin ||
  u.homeInstance === domainOrigin ||
  `https://${u.homeInstance}` === domainOrigin;
```

**Does NOT apply to:** Regular DM messages (`dm_message_created` for user messages). These broadcast to all local `dm_members` regardless of home instance.

### System Messages

System messages (`type = 'system'` in `dm_messages`) are **instance-local** -- they are NOT relayed via federation. Each instance creates its own when processing events.

| Event | Content JSON | Actor (`userId`) |
|-------|-------------|-----------------|
| `member_added` | `{event, targetUserId, targetDisplayName}` | User who added them |
| `member_removed` | `{event, targetUserId, targetDisplayName, reason}` | User who left/was removed |
| `owner_changed` | `{event, newOwnerId, newOwnerDisplayName}` | Previous owner |
| `name_changed` | `{event, oldName, newName}` | Owner who renamed |
| `icon_changed` | `{event}` | Owner who set/cleared the icon |

The `name_changed` and `icon_changed` rows are created by both `PATCH /api/dm/:id` (origin instance) and `processGroupMetadataUpdateEvent` (receivers). On receivers they are dedup-tagged with `(sourceInstance, ${event.messageId}:name)` and `(sourceInstance, ${event.messageId}:icon)` so retries and initial-sync replay don't double-insert.

### Outbound Queuing (Origin Instance -- `dm.ts`)

When a group DM is created or modified locally, the origin instance queues federation events:

**Group DM creation** (`POST /api/dm/group`):
- Iterates each remote target user (those with `homeInstance !== domainOrigin`)
- Builds a `member_add` event per remote user, carrying the full roster in `event.group`
- Computes `finalTargets` by starting from `getGroupDmTargetOrigins()` and adding the new member's normalized homeInstance
- Calls `appendMutationLog` + `queueOutboxEvent` per event

**Add member to existing group** (`POST /api/dm/:id/members`):
- Same structure as creation -- builds `member_add` with full group metadata
- Normalizes new member's homeInstance to full URL before including in targets

**Leave group** (`DELETE /api/dm/:id/members`):
- Computes `fedTargetOrigins` **before** deleting the member (so the leaving user's peer is still included)
- Queues `member_remove` event with `reason: 'leave'`

**Ownership transfer** (auto-on-leave; manual via `POST /api/dm/:id/transfer`):
- Queues `ownership_transfer` event with `previousOwner` and `newOwner`

**Group metadata update** (`PATCH /api/dm/:id`):
- Owner-only. Queues `group_metadata_update` via `queueGroupMetadataRelay(channelId, { name, icon, metadataUpdatedAt, actor })` after the local DB write
- `targetOrigins = getGroupDmTargetOrigins(channelId)` — every peer hosting a member
- Receivers re-validate name length / icon URL scheme on inbound (see `processGroupMetadataUpdateEvent` above)

---

## 7. File Replication

### Outbound (origin instance)

When `queueDmRelay` constructs the relay payload, each attachment gets a `sourceUrl`:
```
sourceUrl: `${getOurOrigin()}/api/uploads/${attachment.filename}`
```
The payload also carries `playable` (video web-playability, computed by the origin from the probed codec — see uploads.md §3) so the receiving instance need not re-probe the file's codec.

### Inbound (receiving instance -- `processCreateEvent`)

1. For each attachment in `event.message.attachments`:
   - SSRF check: `isUrlFromPeer(sourceUrl, peerOrigin)` -- hostname of sourceUrl must match peer origin hostname
   - Create `attachments` row with `filename = sourceUrl` (remote URL as interim filename), carrying through `playable` from the relay payload
   - Queue `federation_file_queue` entry with `status = 'pending'`, `expiresAt = now + 30 days`
2. Initial WebSocket broadcast uses sourceUrl directly (frontend's `AttachmentRenderer` detects `http` prefix)

### File Download Worker (`federationWorker.ts:processFileQueueEntry`)

**Interval:** 30 seconds. **Batch:** 5 files. **Timeout:** 60 seconds per download.

1. SSRF protection: validate sourceUrl hostname matches peerOrigin hostname
2. Pre-download size check against `maxUploadSizeBytes` from instance settings
3. Download via `fetch` with streaming pipeline to disk (`Readable.fromWeb` -> `fs.createWriteStream`)
4. Post-download size verification (defense in depth)
5. Generate thumbnail via `sharp` (same as local upload flow)
6. Update `attachments` row: `filename = localFilename`, `size`, `thumbnailFilename`
7. Fallback: if no existing attachment row was found (legacy queue entry), insert a new one
8. Mark file queue entry as `completed` with `targetFilename`
9. Broadcast `dm_message_updated` to refresh client-side attachment display

### Size Rejection Flow (`handleSizeRejection`)

When a file exceeds the local instance's size limit:

1. Mark file queue entry as `rejected` with `reason = 'size_limit_exceeded'`
2. Update local attachment: `federationStatus = 'remote'`, `federationMeta` = source info JSON
3. Determine affected local users (native to this instance -- `!user.homeInstance || user.homeInstance === ourOrigin`)
4. Queue `file_rejected` reverse relay event to the sender's instance (`sourceInstance`)
5. Broadcast `dm_message_updated` locally so clients see the 'remote' badge

### Inbound file_rejected (`processFileRejectedEvent` -- `federation.ts:2458`)

When the origin instance receives a `file_rejected` event:

1. Find local message by `event.messageId` (the original local message ID)
2. Match attachment by `sourceFilename` or fallback to single attachment
3. Resolve `affectedUserIds` (homeUserIds) to local replicated user stubs
4. Merge rejection info into `federationMeta` (accumulates from multiple peers)
5. Set `federationStatus = 'remote_partial'`
6. Broadcast `dm_message_updated` + targeted `federation_file_rejected` toast to message author

### Federation Status on Attachments

| Status | Meaning |
|--------|---------|
| `null` | Local upload, no federation involvement |
| `'local'` | Successfully downloaded from peer |
| `'remote'` | Rejected (size limit), `federationMeta` has source instance info |
| `'remote_partial'` | Rejected by some peers, `federationMeta` has per-user rejection array |

### File Download Retry

Uses the same backoff schedule as outbox delivery. Max attempts: 10 (`MAX_FILE_ATTEMPTS`). After exceeding max attempts: `status = 'failed'`, `rejectionReason = 'max_attempts_exceeded'`.

### Boundary vs. tus Upload Migration (2026-04)

The `federation_file_queue` worker downloads files from peer instances via plain `fetch` + `Readable.fromWeb` to disk, post-completion. It consumes finished files at `${uploadDir}/${filename}`. Whether a file got there via the legacy multipart endpoint, the current tus endpoint at `/api/files/*`, or any future protocol is invisible to this worker -- its contract is purely with the on-disk filename. No changes were required when the upload protocol changed.

---

## 8. Read State Relay

### `read_state_update` Event

When a user marks a DM channel as read (`channel_ack`) or marks it unread (`mark_unread`), the read state is relayed to all peer instances so cross-instance sessions stay in sync.

**Outbound (`events.ts:handleChannelAck` / `handleMarkUnread`):**
- Fires after writing `read_states` locally
- Only triggers for DM channels with a `federatedId` (cross-instance DMs)
- Calls `queueReadStateRelay(channelId, messageId, userId)` in `federationOutbox.ts`
- Queued via the standard outbox pipeline — durable, retried by the background worker
- Entity key `read_state:{federatedId}:{userId}` enables coalescing (rapid acks collapse to latest)
- `mark_unread` with the `'0'` sentinel (delete read state entirely) is NOT relayed — it cannot be mapped to a message

**Event payload:**
```typescript
{
  eventType: 'read_state_update',
  dmChannelId: string,
  messageId: string,            // unique event ID: 'read_state:{userId}:{timestamp}'
  federatedId: string,          // DM channel's federatedId (cross-instance channel lookup)
  encryptionVersion: 0,
  timestamp: number,            // LWW tiebreaker
  readState: {
    user: { homeUserId: string; homeInstance: string },
    messageRef: { sourceInstance: string; sourceMessageId: string }
  }
}
```

`messageRef` identifies the acked message in federation coordinates. If the message originated on this instance, `sourceInstance` is our own origin and `sourceMessageId` is the local message ID. If the message was relayed here, `sourceInstance` and `sourceMessageId` come from the `dm_messages` row's `source_instance`/`source_message_id` columns.

**Inbound (`processReadStateUpdateEvent`):**
1. Resolve channel by `federatedId` — reject if not found
2. Resolve user via `resolveLocalUser` — reject if not found
3. Translate `messageRef` to local message ID:
   - If `sourceInstance` matches our origin: `sourceMessageId` IS our local ID
   - Otherwise: look up `dm_messages` by `source_instance + source_message_id`
4. If no local message found (relay hasn't arrived yet): silently accept (no-op)
5. Upsert `read_states` using timestamp-only LWW (`event.timestamp > existing.updatedAt`)
6. Echo `channel_ack` to the user's local WebSocket connections (multi-tab sync)

---

## 8b. DM Close/Reopen Relay

### Overview

When a user closes or reopens a DM on their home instance, the action is relayed to all peer instances that hold a copy of the channel. This keeps the visibility state of a DM consistent across all instances that participate in it.

Only DMs with a `federatedId` are eligible. Legacy local-only DMs (created before federation was added, with no `federatedId`) are silently skipped.

### Event Types

| Event | Trigger |
|-------|---------|
| `dm_close` | User calls `DELETE /api/dm/:id` (soft-close) |
| `dm_reopen` | User calls `POST /api/dm` and reopens a closed 1-on-1 DM |

### Payload

```typescript
{
  eventType: 'dm_close' | 'dm_reopen',
  dmChannelId: string,          // local channel ID (context only)
  federatedId: string,          // cross-instance channel lookup key
  messageId: string,            // unique event ID: 'dm_close:{federatedId}:{userId}:{ts}'
  encryptionVersion: 0,
  timestamp: number,
  dmCloseReopen: {
    homeUserId: string,         // acting user's home user ID
    homeInstance: string,       // acting user's home instance (full URL)
  }
}
```

### Outbound (`federationOutbox.ts:queueDmCloseRelay`)

Called from `dm.ts` after the local close or reopen is committed.

1. Fetch the channel's `federatedId` — if null (local-only DM), return silently
2. Fetch the acting user's `(homeUserId, homeInstance)` federation identity
3. Build `FederationRelayEvent` with `eventType` and `dmCloseReopen` payload
4. `getGroupDmTargetOrigins(dmChannelId)` resolves the delivery targets:
   - 1-on-1 DMs (`ownerId = NULL`): returns `undefined` → broadcast to ALL active peers
   - Group DMs: returns the set of peer origins that have at least one participant
5. Enqueue via `appendMutationLog` + `queueOutboxEvent`

### Inbound

**`processDmCloseEvent` (`federation.ts`):**
1. Look up channel by `federatedId` — if not found, accept silently (idempotent)
2. Resolve acting user via `resolveLocalUser` (lookup-only; no stub creation for close/reopen) — if not found, accept silently
3. If the user has no `dm_members` row in this channel, accept silently
4. Set `dm_members.closed = 1` for the resolved local user
5. Broadcast `dm_channel_closed` to the user's local WebSocket connections

**`processDmReopenEvent` (`federation.ts`):**
1. Look up channel by `federatedId` — if not found, accept silently
2. Resolve acting user via `resolveLocalUser` — if not found, accept silently
3. If the user has no `dm_members` row, accept silently
4. Set `dm_members.closed = 0`
5. Build full `DmChannel` payload and broadcast `dm_channel_created` to the user's local WebSocket connections (mirrors the automatic reopen path in `broadcastDmMessage`)

### Closed-State Reopen on Message Relay (Bug Fix)

`processCreateEvent` (inbound message relay) now mirrors the local `broadcastDmMessage` logic: before broadcasting `dm_message_created`, it checks each recipient's `dm_members.closed` flag. For any recipient with `closed = 1`:

1. Set `closed = 0`
2. Broadcast `dm_channel_created` (with the new message as `lastMessage`) to resurface the DM in the recipient's sidebar
3. Then broadcast `dm_message_created`

This fixes a gap where relayed messages bypassed the closed-state reopen logic, leaving the DM hidden for recipients whose `closed` flag was set on the receiving instance.

---

## 9. Friend Relay

### Event Flow (social.ts)

| User Action | Federation Event | Authority Check |
|-------------|-----------------|-----------------|
| Send friend request | `friend_request_create` | `from.homeInstance === sourceInstance` |
| Accept/decline request | `friend_request_update` | `to.homeInstance === sourceInstance` |
| Cancel outgoing request | `friend_request_cancel` | `from.homeInstance === sourceInstance` |
| Accept creates friendship | `friend_add` | `to.homeInstance === sourceInstance` |
| Remove friend | `friend_remove` | Either side's instance |

### Target Resolution (`getFriendEventTargets`)

Computes which peer origins need the event. Compares `fromHomeInstance` and `toHomeInstance` against `getOurOrigin()` with normalization applied, correctly handling both bare domain and full URL formats.

### Context ID

Friend events use a deterministic context ID: `friend:${sorted[homeUserIdA, homeUserIdB].join(':')}`.

### Outbound Payload Construction

Each friend endpoint builds a `FederationRelayEvent` with:
- `contextType: 'friend'`
- `friendship` payload containing `from` and `to` as `FederationRelayParticipant` objects
- `fromProfile` and/or `toProfile` snapshots (`FederationRelayProfileSnapshot`)
- `entityId` formatted as `friend_req:${sorted_ids}:${timestamp}` (for requests) or `friend_remove:${sorted_ids}:${timestamp}`

The full event payload is stored in both `appendMutationLog` (for sync) and `queueOutboxEvent` (for delivery).

### Inbound Processing

**`processFriendRequestCreateEvent` (`federation.ts:2082`):**
- Authority check: `from.homeInstance !== sourceInstance` -> reject
- Resolve sender via `resolveOrCreateReplicatedUser` + hydrate profile
- Resolve recipient via `resolveLocalUser` (must be native to this instance)
- Idempotency: if already friends or pending request exists, accept as no-op
- Create `friend_requests` row, broadcast `friend_request_received` to recipient

**`processFriendRequestUpdateEvent` (`federation.ts:2178`):**
- Authority check: `to.homeInstance !== sourceInstance` -> reject
- Resolve sender (original requester) via `resolveLocalUser` (must exist locally)
- Resolve recipient (acceptor/decliner) via `resolveOrCreateReplicatedUser`
- Find pending request, update status
- Broadcast `friend_request_accepted` or `friend_request_declined` to the original sender

**`processFriendRequestCancelEvent` (`federation.ts:2254`):**
- Authority check: `from.homeInstance !== sourceInstance` -> reject
- Both users must exist locally. If not, accept idempotently.
- Delete the pending friend request. Broadcast `friend_request_cancelled` to recipient.

**`processFriendAddEvent` (`federation.ts:2318`):**
- Authority check: `to.homeInstance !== sourceInstance` -> reject
- Resolve both users via `resolveOrCreateReplicatedUser` + hydrate profiles
- Insert `friends` row (idempotent)
- Auto-resolve any pending `friend_requests` to `'accepted'` (handles out-of-order delivery)
- Determine which user is local (`from.homeInstance === ourOrigin`) and broadcast `friend_request_accepted`

**`processFriendRemoveEvent` (`federation.ts:2404`):**
- Authority check: either `from.homeInstance` or `to.homeInstance` must be `sourceInstance`
- Both users resolved via `resolveLocalUser`. If not found, accept idempotently.
- Delete `friends` row in both directions
- Determine local user (whose `homeInstance` is NOT the source) and broadcast `friend_removed`

---

## 10. Profile Sync

Profile sync uses **two mechanisms** that operate independently:

### S2S Profile Hydration (Server-side)

When relay events carry `FederationRelayProfileSnapshot` data:
- `processCreateEvent`: hydrates participant profiles on message relay
- `processFriendRequestCreateEvent` / `processFriendAddEvent`: hydrates friend profiles
- `social.ts` federated friend-request handler: hydrates the looked-up stub

Snapshots for tombstoned users carry `deleted: true` and no profile fields — the internal `!deleted:<id>` username marker never leaves the instance (`getDmParticipants`, `buildProfileSnapshot`).

`hydrateReplicatedUserProfile` is **best-effort fill-empty only**: it never overwrites an existing avatar/banner/displayName/bio. This protects locally-downloaded bare filenames produced by `processProfileUpdateEvent` (which carries the monotonic `profileUpdatedAt` version) from being clobbered back to absolute URLs on the next DM/friend relay. Authoritative updates flow exclusively through the version-checked `profile_update` event.

**Detached-account guard:** Alongside the native-user skip (`!user.homeInstance → return`), the function also **no-ops on detached rows** (`federation_home_orphaned = 1 → return user`). A detached account retains its `homeInstance` for provenance (design §7), so the native-user skip alone would not catch it; without this guard a DM/friend relay from the reset domain's new incarnation, resolved via an old `homeUserId` tier-1 hit, could fill the sovereign account's empty fields. This is the same domain-keyed mutation class as `profile_update`/`presence_update`, guarded at its own site (design §4.3).

When the function does fill an empty avatar/banner, it calls `downloadProfileAsset` against the user's home instance and stores the resulting **local bare filename**. Only on download failure does it fall back to the absolute URL — matching the behavior of `processProfileUpdateEvent`.

#### Profile Sync (S2S)

Profile data is synced server-to-server. The home instance is authoritative — when a user updates their profile, the home server queues a `profile_update` relay event to all active peers via the outbox.

**Event:** `profile_update` (contextType: `profile`)

**Payload:** `FederationProfileUpdatePayload`:
- `homeUserId`, `homeInstance`, `profileUpdatedAt` (monotonic version)
- `username` — the home user's canonical handle (without `@domain`). Receivers apply `displayName ?? username` when writing the stub's displayName, so stubs whose home user has no displayName show the real handle instead of getting clobbered to null. Username itself is immutable on the home instance, so the receiver does NOT rewrite the stub's username column on profile_update.
- `displayName`, `avatar` (absolute URL or null), `banner` (absolute URL or null)
- `accentColor`, `avatarColor`, `bio`

**Targeting:** Broadcast to all active peers. Peers silently accept if they have no replica.

**Coalescing:** `entityId = homeUserId`. Rapid successive edits coalesce to one delivery per peer.

**Processing:** Remote overwrites all 6 mutable fields unconditionally; `displayName` falls back to `payload.displayName ?? payload.username`. Rejects if incoming `profileUpdatedAt ≤ stored`. Broadcasts `user_updated` to local WS clients.

**Detached-account guard:** After the domain-collision check and before the version check, if the resolved `localUser` has `federation_home_orphaned = 1` (detached — home domain was reset, now a sovereign local account), the event is **acked (messageId pushed to `accepted`) and skipped without applying**. The reset domain's new incarnation must never overwrite an established account's profile by replaying its old `homeUserId`. Ack rather than reject because the sender legitimately considers the identity theirs to update; from this side the update simply no-ops.

#### Profile Image File Replication

When a `profile_update` relay carries avatar or banner absolute URLs, the receiving instance downloads the image files locally rather than storing remote URLs. This eliminates cross-origin dependencies — avatars render from the local `/api/uploads/` endpoint.

**Flow:** `processProfileUpdateEvent` calls `downloadProfileAsset(url, sourceInstance)` for each of avatar/banner:
1. SSRF check (URL hostname must match authenticated source instance)
2. `fetch` with 10s timeout
3. Content-type validation (`image/*` only)
4. Stream to temp file (`temp_{snowflake}{ext}`), atomic rename to `{snowflake}{ext}`
5. Store local bare filename in user row

**Fallback:** On any download failure (timeout, HTTP error, non-image content, SSRF mismatch), the absolute URL is stored instead. This degrades to the pre-replication behavior — the avatar loads cross-origin from the home instance.

**File cleanup:** When avatar/banner changes, the old local file is deleted via `deleteUploadFile()`. The check `!oldValue.startsWith('http')` ensures only locally-downloaded files are deleted, not absolute URL strings.

**Migration / backfill:** `backfillReplicatedProfileAssets` runs on server start (kicked off by `startFederationWorkers`, non-blocking). It scans `users` rows where `home_instance` is set and `avatar`/`banner` start with `http`, calls `downloadProfileAsset` against the home instance for each, and rewrites successful downloads to local filenames. Idempotent: rows whose home is unreachable are left as URLs (they still render while the peer is up) and retried on the next start. This handles both legacy data from before file replication shipped and any rows whose home was offline during a previous attempt.

**Write protection:** Remote instances reject PATCH /users/@me profile field updates for replicated users (homeInstance set). Profile data is read-only on remote — only updated via S2S relay.

**Bootstrap:** When a new origin appears in a user's `replicatedInstances`, the home server queues a targeted `profile_update` to that peer.

**File handling:** Profile images are downloaded locally on relay receipt (see "Profile Image File Replication" above).

**Replaces:** Client-driven `profileSync.ts` (deleted). `hydrateReplicatedUserProfile` still bootstraps null fields during DM/friend relay — it now also runs the same local-download path so newly-created stubs end up with bare filenames, not URLs.

#### Presence Sync (S2S)

Native users' status (and optional rich activities) is projected to peers via the `presence_update` relay event. Closes the doc/code gap previously documented in `activity-presence.md` — replicated stubs now have their status maintained by the home instance over S2S, not derived from absent local WS state.

**Event:** `presence_update` (contextType: `profile`)

**Payload:** `FederationPresenceUpdatePayload`:
- `homeUserId`, `homeInstance`
- `status: 'online' | 'idle' | 'dnd' | 'offline'`
- `activities?: Activity[]` (omitted when empty)
- `ts: number` — emitter clock (last-write-wins per stub if needed)

**Outbox-only — never written to mutation log.** Presence is ephemeral. Replaying old presence on peer activation would be wrong (stale state). The outbox queues directly without `appendMutationLog`. Stale entries that fail delivery beyond retry budget are dropped.

**Sender call sites** (all in `utils/federationPresence.ts:queuePresenceRelay`):
- `ws/handler.ts` (auth path) — `online`
- `ws/handler.ts` (`finalizeDisconnect`) — `offline`
- `ws/events.ts` (`handlePresenceUpdate`) — manual `online`/`idle`/`dnd`
- `ws/events.ts` (`handleActivityUpdate`) — when activities change
- `routes/users.ts` (showActivity-toggle clear) — cleared activities

No-op for replicated users (we don't own their presence).

**Targeting:** broadcast to all active peers (mirrors `profile_update`). Peers without a stub silently no-op. Privacy: status is already public to anyone authorized to see the user via friend/DM/space relationships, so broadcast-fanout adds no new disclosure surface.

**Coalescing:** `entityId = userId`, `contextId = userId`. Rapid status flaps coalesce to the latest queued event per peer.

**Receiver:** `processPresenceUpdateEvent` (`routes/federation.ts`). Strict attribution — `payload.homeInstance` domain MUST equal source peer's domain. Resolves the local stub by `homeUserId`, validates the stub's `homeInstance` matches the payload's domain, updates the stub's `status`, and broadcasts a WS `presence_update` to local users via `collectProfileBroadcastTargetIds(stub.id)` — friends, DM members, and space co-members.

**Detached-account guard:** After the domain-collision check and before the status write, if the resolved stub has `federation_home_orphaned = 1` (detached — home domain was reset, now a sovereign local account), the event is **acked (messageId pushed to `accepted`) and skipped without applying**. The reset domain's new incarnation must never flip an established account's presence by replaying its old `homeUserId`. Ack (not reject) mirrors the `profile_update` guard rationale — the sender considers the identity theirs, so we no-op rather than trigger a retry loop.

**Peer lifecycle hooks** (`utils/federationPresence.ts`):
- **`onPeerActivated`** invokes `snapshotPresenceForPeer(origin)` — emits a `presence_update` only for online natives that have an S2S relationship with the peer (friend/DM with a peer-stub, or `replicatedInstances` opt-in for the peer origin). Snapshot work scales with relationship count, not native count.
- **`onPeerDeactivated`** invokes `markPeerStubsOffline(origin)` — flips every stub from that peer to `offline` and broadcasts a local `presence_update` so users see them go offline immediately.
- **Flap recovery semantics:** `onPeerActivated` re-runs on every transition into `active`, including the 15-minute health-check `unreachable → active` recovery. This is load-bearing for correctness: `markPeerStubsOffline` ran on the prior deactivation, presence is not in the mutation log, so a fresh snapshot is the only signal that re-establishes truth. The relationship-scoped query bounds the cost.

#### Stub Username Backfill

Legacy stubs (created before the realname scheme shipped) used `${homeUserId}@${domain}` as the local username. The current scheme uses `${realname}@${domain}` (`hints.username` in `resolveOrCreateReplicatedUser`). Backfill heals existing legacy rows.

**Worker:** `utils/federationStubBackfill.ts:backfillStubUsernamesForPeer(peerOrigin)`. Enumerates legacy-shaped stubs whose `home_instance` matches the peer's domain, asks the peer for the canonical username via `lookupRemoteUserByHomeId`, rewrites the stub's username and (if displayName is null) seeds displayName from the same fallback. Idempotent and collision-safe.

**Hook points:**
- `onPeerActivated` — runs per-peer on every transition to `active` (catches stubs whose home was unreachable on a prior pass).
- `startupBootstrapSync` — one-shot pass at boot for ALL currently-active peers (not just `lastSyncedAt = 0` first-time peers).

**New endpoint: `POST /api/federation/users/by-home-id`** (HMAC-authenticated, rate-limited 60/min/peer). Body: `{ homeUserId: string }`. Response: `{ found: false }` or `{ found: true, user: { homeUserId, username, profile: { displayName, avatar, avatarColor, banner, bio } } }`. Native non-deleted users only.

---

## 11. Reaction Relay

### Outbound

Reactions are queued by WS event handlers in `events.ts`:
- `dm_reaction_add` -> `queueOutboxEvent(reactionId, channelId, 'reaction_add', payload, targetOrigins)`
- `dm_reaction_remove` -> `queueOutboxEvent(messageId, channelId, 'reaction_remove', payload, targetOrigins)`

Payload includes `userId`, `homeUserId`, `emoji`, `createdAt`, plus `messageId` and `messageHomeInstance` for cross-instance message resolution.

The mutation log entry for reactions stores a simpler payload (no `messageId`/`messageHomeInstance`), while the outbox entry carries the full reaction payload including those fields.

### Inbound

**`processReactionAddEvent` (`federation.ts:1480`):**
1. Resolve message via `resolveLocalDmMessage(canonicalMessageId, messageHomeInstance, sourceInstance, db)`:
   - If `messageHomeInstance === getOurOrigin()` -> find by local ID (the message originated here)
   - Otherwise -> find by `(messageHomeInstance || sourceInstance, canonicalMessageId)` tracking -- uses `messageHomeInstance` when available (correct origin in 3-instance relay), falls back to `sourceInstance`
2. Resolve reacting user via `resolveLocalUser` (must already exist)
3. Dedup: check existing reaction by `(dmMessageId, userId, emoji)`
4. Insert `dm_reactions`, broadcast `reaction_added` to local clients

**`processReactionRemoveEvent` (`federation.ts:1561`):**
- Same resolution logic
- Delete matching reaction, broadcast `reaction_removed` if changes > 0

---

## 12. Initial Sync

### `startupBootstrapSync()` (`federationWorker.ts`)

Triggered once at server startup (async, non-blocking). Finds peers with `status = 'active'` and `lastSyncedAt = 0` and calls `onPeerActivated(peerId, 'startup_bootstrap')` for each. This preserves the original startup-sync semantics while unifying the code path with all other activation sites (see "Peer Activation Recovery" below).

### Peer Activation Recovery

Every transition of `federation_peers.status` to `active` invokes `onPeerActivated(peerId, reason)` — one handler wired at all transition sites. Two independent invariants, both unconditional:

1. **`resetOutboxBackoff`** — sets `nextRetryAt = now` and `attempts = 0` for every outbox entry belonging to the peer. Entries that accumulated exponential backoff before the peer went unreachable are immediately eligible again. Attempts counter is also reset so a freshly-healthy peer's next failure starts at `BACKOFF_SCHEDULE_MS[0]` (30s), not wherever the counter left off.
2. **`syncPeerMutationLog`** — pulls missed events from the peer's `/api/federation/sync` endpoint since `peer.lastSyncedAt`. Three passes: DM, friend, profile (in that order, each paginated). `lastSyncedAt` advances to `Date.now()` on full success; stays put on transient failure so the next activation retries the same window.

#### Call sites (must remain exhaustive)

| File | Context | Reason |
|---|---|---|
| `routes/federation.ts` | `/peer/initiate` 200 activation | `initiate_accepted` |
| `routes/federation.ts` | `/peer/accept` existing-rejected override | `accept_rejected_override` |
| `routes/federation.ts` | `/peer/accept` existing-awaiting_approval | `accept_awaiting_approval` |
| `routes/federation.ts` | `/peer/accept` existing-pending | `accept_pending` |
| `routes/federation.ts` | `/peer/accept` new-peer | `accept_new` |
| `routes/federation.ts` | `/approval-requests/:id/approve` success | `approval_handshake` |
| `utils/federationWorker.ts` | Health check unreachable → active | `health_check_recovery` |
| `utils/federationPeering.ts` | `ensurePeered/performHandshake` 200 | `ensure_peered` |
| `utils/federationWorker.ts` | Startup scan (status=active, lastSyncedAt=0) | `startup_bootstrap` |

HTTP handler sites dispatch fire-and-forget (`.catch(log)`) so the response is not blocked by sync-pull pagination. Worker-internal sites `await` since the worker tick is already async.

Concurrent activations for the same peer are deduplicated via an in-flight promise map keyed by `peerId`.

### onPeerDeactivated

Mirror of `onPeerActivated` for the transition *out* of `active`. Invoked wherever `federation_peers.status` is written to `unreachable`, `needs_attention`, `rejected`, or `revoked`. Responsibility: sweep `ConnectionManager.federatedCalls` for entries whose `federatedCallHost` matches the deactivated peer and evict them — emitting `dm_call_undeliverable { phase: 'host_unreachable', terminal: true }` to each entry's `ringedUserIds`. See `docs/systems/voice.md` for the client teardown contract.

**Call sites (exhaustive — grep `onPeerDeactivated(` to audit):**
- `utils/federationWorker.ts` handleOutboxDeliveryFailure when status flips to `unreachable`
- `utils/federationWorker.ts` auth-failure path when status flips to `needs_attention`
- `utils/federationWorker.ts` resolvePendingPeers case `'rejected'`
- `routes/federation.ts` admin revoke endpoint
- `utils/federationPeering.ts` performHandshake 403 `PEERING_REQUIRES_APPROVAL` path

Deduplicated by peerId using a **separate** `inFlightDeactivation` map (not shared with activation) so flapping peers retain clean activate-then-deactivate ordering.

A 30s periodic sentinel in `federationWorker.ts` (`runFederatedCallSentinelTick`) is the backstop — it scans active FederatedCallEntries, compares each host's current peer status against reality, and catches transitions missed by the hook sites.

#### Peer-state × outbox-enqueue × recovery matrix

| Status | `queueOutboxEvent` enqueue | Mutation log captures | Recovery on transition to `active` |
|---|---|---|---|
| `active` | Queue | Yes (for covered event types — see below) | N/A |
| `pending` | Queue | Yes | `onPeerActivated` |
| `unreachable` | Queue | Yes | `onPeerActivated` |
| `awaiting_approval` | **Drop, debug-log** | Yes | `onPeerActivated` |
| `needs_attention` | **Drop, debug-log** | Yes | `onPeerActivated` (fires when the row is re-created via admin Reset + re-peer) |
| `rejected` | Drop, debug-log | Yes | `onPeerActivated` |
| `revoked` | Drop, debug-log | Yes | `onPeerActivated` (fires when the row is re-created via hard-delete + re-initiate) |

`queueOutboxEvent` uses an exhaustive TypeScript `switch` on the narrowed peer-status union — adding a new status value without handling it fails compile-time typecheck (`const _exhaustive: never = status;`).

**Mid-call race catch:** when the initial peers SELECT filters out a peer because its status is non-deliverable, but the fallback loop observes the status has since flipped to `active`/`pending`/`unreachable`, the code re-fetches the peer row and appends it to `matchedPeers` so the outer enqueue loop includes it. Silent drops would lose real-time delivery under asymmetric failure (e.g., `/peer/accept` 200 response lost on the wire, health-check transition firing on only one side).

#### Mutation log coverage

Event types covered by `appendMutationLog` (replayed on sync-pull):

| Event type | `contextType` | Source |
|---|---|---|
| DM `create` / `update` / `delete` | `dm` | `federationOutbox.queueDmRelay`, `dm.ts` delete handler |
| `reaction_add` / `reaction_remove` | `dm` | `ws/events.ts` |
| `member_add` / `member_remove` / `ownership_transfer` | `dm` | `dm.ts` |
| `dm_close` / `dm_reopen` | `dm` | `federationOutbox.queueDmCloseRelay` |
| `read_state_update` | `dm` | `federationOutbox.queueReadStateRelay` |
| `file_rejected` | `dm` | `federationWorker.handleSizeRejection` |
| `friend_request_*` / `friend_add` / `friend_remove` | `friend` | `social.ts` |
| `profile_update` | `profile` | `routes/users.ts` (PATCH `/api/users/@me`) |

Ephemeral events (`dm_typing_*`, `dm_call_*`) are fire-and-forget by design and are NOT captured — missed typing/call-signaling packets are acceptable and carry no durable state.

#### `/api/federation/sync` contextType filter values

| Filter | Returns |
|---|---|
| (none) / omitted | DM events (including `dm_close`, `dm_reopen`, `read_state_update`, `file_rejected`) |
| `'friend'` | Friend events |
| `'profile'` | Profile update events |

#### Poison-Pill Event Handling

`syncPeerMutationLog` replays incoming events one at a time. If an event's inbound processor throws (e.g., UNIQUE conflict from a malformed payload, unexpected schema drift, a processor bug), the error is caught per-event, logged via `console.error` with the event's `eventType`, `messageId`, `timestamp`, peer origin, and error message, and the loop continues with the next event. `lastSyncedAt` is advanced past the failed event (using the event's own `timestamp`), so subsequent activations do not retry the poison pill.

The final `Sync-pull from <origin> replayed <N> events` log line is suffixed with `(<K> skipped due to errors)` when `K > 0`, surfacing the count to operators watching logs. Individual event failures are in the same logs under `Skipping poison-pill event` — grep `console.error` / `stderr` to recover them.

Trade-off: this policy prioritizes forward progress of the sync pipeline over strict at-least-once delivery of every mutation. An event that fails to process is silently lost to the receiving instance unless operators manually replay it (e.g., by resetting `lastSyncedAt` on the peer row or by reissuing the originating mutation on the sender). The alternative — refusing to advance on any error — caused the "stuck forever" state described in the pre-fix version of this section.

### Sync Endpoint (`POST /api/federation/sync`)

HMAC-authenticated. Returns events from the `federation_mutation_log`.

**Request:**
```typescript
{ sinceTimestamp: number, dmChannelId?: string, federatedId?: string, contextType?: 'dm'|'friend'|'profile', limit?: 1-500 }
```

**Response:**
```typescript
{ events: FederationRelayEvent[], hasMore: boolean, checkpoint: number }
```

**DM sync:**
- Queries `dm_channels` with non-null `federated_id` (not soft-deleted) that have ≥1 relevant member for the requesting peer (see relevance scoping below)
- Joins `federation_mutation_log` with `dm_messages` to reconstruct events
- Only returns locally-created messages (`source_instance IS NULL` via the LEFT JOIN)
- Handles delete mutations separately (message rows don't exist for deletes)
- For create/update: fetches current message state from DB, builds full relay event with attachments and participants
- Membership/friend mutations store the full event payload in the mutation log, so they are returned directly

**Friend sync:**
- Queries `federation_mutation_log WHERE context_type = 'friend'`
- Returns stored payloads directly (friend events carry their complete data)

**Relevance scoping (dead-incarnation filtering):** the DM branch only offers channels having ≥1 member whose local row is homed at the requesting peer's domain with `is_deleted = 0` AND `federation_home_orphaned = 0`. A freshly reset peer therefore receives an empty DM sync — its pre-reset history stays with the detached accounts on this side. The friend branch filters analogously in JS (an event qualifies iff one side is homed at the requester and does not resolve to a detached/tombstoned local row); `checkpoint`/`hasMore` are computed from pre-filter rows so filtered events still advance the cursor.

### Relay Event Processing

The event processing logic is extracted into `processRelayEvents()` (exported from `federation.ts`), shared by both the HTTP relay endpoint and the initial sync worker. This avoids a DNS hairpin issue where the server would HTTP-request itself through public DNS, which fails on networks without hairpin NAT.

---

## 13. DM Calls over Federation

DM calls work across federated instances. The caller's instance hosts the LiveKit room. Remote clients connect directly to the caller's LiveKit server using a token passed through S2S relay — no media is routed through the federation layer.

```
User A's client <--WS--> Instance 1 (hosts LiveKit) <--S2S HTTP--> Instance 2 <--WS--> User B's client
                              |                                                            |
                              +------------- LiveKit (direct client connection) -----------+
```

### S2S Event Types

Four relay event types are processed in `processRelayEvents()`:

| Event Type | Direction | Key Payload Fields |
|---|---|---|
| `dm_call_start` | Host → Peers | `federatedId`, `livekitUrl`, `tokens: Record<string, string>` (keyed by `homeUserId`), `caller: { homeUserId, homeInstance, displayName }`, `participants` |
| `dm_call_accept` | Participant → Host, then Host → All Peers | `federatedId`, `acceptor: { homeUserId, homeInstance }` |
| `dm_call_reject` | Participant → Host, then Host → All Peers | `federatedId`, `rejector: { homeUserId, homeInstance }` |
| `dm_call_end` | Any → Host (if not host), then Host → All Peers | `federatedId`, `endedBy: { homeUserId, homeInstance }` |

All events carry standard relay fields: `eventType`, `messageId`, `encryptionVersion: 0`, `timestamp`. All events pass through `verifyAttribution()` before any DB or state mutations.

### Direct Delivery (No Outbox)

**`sendCallRelay(targetPeerOrigin, events, opts?)`** (`federationOutbox.ts`):

- Latency-sensitive: returns `CallRelayResult = { ok: true } | { ok: false; reason: CallRelayFailureReason; error: string }`.
- Peering resolution:
  1. If the peer row is `active` or `unreachable`, POST directly (the health check restores `unreachable` peers; re-handshaking is wasteful).
  2. Otherwise race `ensurePeered` against `opts.peeringTimeoutMs` (default `CALL_PEERING_TIMEOUT_MS = 3_000` ms). The background handshake is **not** aborted on race loss — a warn-logged catch is attached so a late-rejecting background promise does not emit `unhandledRejection`.
- Peer-state → reason mapping is exhaustive over the `EnsurePeeredResult` union (`active` / `rejected` / `pending` / `failed`) plus the external `timeout` branch. TypeScript `never` check in the switch default catches future additions. Note: the `livekit_unavailable` reason in `DmCallUndeliverableReason` is emitted separately from `sendFederatedCallStart`'s LiveKit pre-flight in `ws/events.ts`, not from this switch — `sendCallRelay` only produces `CallRelayFailureReason` values (`peer_rejected` / `peer_awaiting_approval` / `peer_transient_failure` / `post_failed`).
- Non-blocking mode: `peeringTimeoutMs: 0` (used by typing) skips the POST for non-active peers, kicks off `ensurePeered` as a background warm-up, returns `peer_transient_failure` silently.

**`sendTypingRelay(dmChannelId, eventType, userId)`**:

- Fire-and-forget to each remote DM participant's home instance via `sendCallRelay(origin, [event], { peeringTimeoutMs: 0 })`. Typing is an ephemeral hint — lost packets are acceptable and there is no user-facing failure surface.

**Call-start failure surfacing.** `sendFederatedCallStart` aggregates targeted-peer results and emits `dm_call_undeliverable { phase: 'start' }` to the caller for failed targeted peers. See `docs/systems/voice.md` and `docs/systems/websocket.md` for the event contract.

**Accept / reject / end relay discipline.** Every `handleDmCall{Accept,Reject,End}` Path-2 branch awaits `sendCallRelay` and emits `dm_call_undeliverable { phase, terminal, failures }` to the originator on failure. Accept is pessimistic-rollback (terminal: true — local `FederatedCallEntry` cleared, optimistic `dm_call_accepted` walked back via `sendToFederatedCallUsers`); reject and end are optimistic (terminal: false — state already cleared, informational toast only). Path-1 fan-outs via `sendFederatedCallAccept` / `sendFederatedCallEnd` / `fanOutCallEvent` return `CallFanoutFailure[]` and the host-side caller receives `dm_call_undeliverable { terminal: false }` listing peers that were not reached.

**Ring-timeout fan-out.** `ConnectionManager.createDmRoom`'s 60 s ringing auto-clean now invokes a registered hook (`setRingTimeoutFanoutHook`, registered from `ws/events.ts:registerCallRelayHooks`) that fans `dm_call_end` out to remote peers, so stranded Path-A/B ringees on other instances exit their ring state instead of lingering until their own 60 s cleanup fires.

### Call Flows

**Start:** Host validates membership, generates LiveKit tokens for all DM members (local + remote), broadcasts `dm_call_incoming` to local WS clients, then sends `dm_call_start` S2S to each remote instance with per-user tokens.

**Accept:** Remote instance sends `dm_call_accept` S2S to host. Host transitions `ringing → active`, broadcasts `dm_call_accepted` locally, fans out `dm_call_accept` to all other remote instances.

**Reject:** Remote sends `dm_call_reject` to host. Host destroys room, sends `dm_call_end` to all peers. (For 1-on-1 DMs, reject = end.)

**End:** Initiating instance (host or not) routes through the host. Host destroys room, fans out `dm_call_end` to all remote instances.

**Timeout:** Both host and remote instances auto-clean stale ringing calls after 60 seconds.

### LiveKit Room Naming

Room name = `federatedId` (the cross-instance stable UUID), never the local `dmChannelId` (which differs per instance).

- 1-on-1 DMs: `federatedId` is a deterministic SHA-256 hash of the sorted `homeUserId` pair
- Group DMs: `federatedId` is a UUID assigned at creation

### LiveKit Token Generation

`generateFederatedCallToken(roomName, homeUserId, displayName)` generates tokens with:
- **TTL:** 5 minutes (short join window; local calls use 1 hour)
- **Room:** scoped to exact `federatedId`
- **Identity:** `${homeUserId}:${displayName}`
- **Permissions:** full DM grants (mic, camera, screen share, subscribe, data channel)

### Public LiveKit URL

The URL sent in S2S payloads is always `https://${DOMAIN}/livekit` (the Caddy-proxied public address). The internal `LIVEKIT_URL` env var (`http://livekit:7880`) is never sent to peers.

Instances without LiveKit configured can still receive federated calls — they pass the host's URL and token to the client, which does all the heavy lifting.

### In-Memory Call Registry

When a remote instance receives `dm_call_start`, it creates a `FederatedCallEntry` in memory:

```typescript
interface FederatedCallEntry {
  dmChannelId: string;          // local dmChannelId for this DM
  federatedId: string;          // cross-instance room identifier
  callerId: string;             // local userId of caller's stub
  callerHomeUserId: string;
  federatedCallHost: string;    // peer origin of the host instance
  livekitUrl: string;
  tokens: Map<string, string>;  // homeUserId → LiveKit token
  state: 'ringing' | 'active';
  startedAt: number;
}
```

This registry ensures tokens and `livekitUrl` survive browser refreshes via the `activeCalls` array in the `ready` WS payload. The server filters to the per-user token at payload assembly time.

---

## 14. Background Workers

All workers are started by `startFederationWorkers()` on server boot and stopped by `stopFederationWorkers()` on shutdown. Each worker uses `setTimeout` chains (not `setInterval`) with abort controllers for graceful shutdown.

| Worker | Interval | Batch | Timeout | Source |
|--------|----------|-------|---------|--------|
| Outbox delivery | 10s | 50 | 30s | `processOutboxTick` |
| File download | 30s | 5 | 60s | `processFileQueueTick` |
| Health check | 15min | all unreachable | 10s | `processHealthCheckTick` |
| Epoch-refresh baseline | Startup + 15min (end of health tick) | active peers w/ `peer_instance_id IS NULL` | 10s per peer | `refreshPeerEpochs` (populate-if-null, self-terminating) |
| Janitor | 1h | -- | -- | `runFederationJanitor` (sync) |
| Startup bootstrap sync | Once at startup | -- | 30s per page | `startupBootstrapSync` → `onPeerActivated` |
| Dead-incarnation sweep | Once at startup | -- | -- | `sweepDeadIncarnationArtifacts` (sync, idempotent) |
| DM `federatedId` reconciliation | Once at startup | -- | -- | `reconcileDriftedDmFederatedIds` (sync, idempotent) |

`sweepDeadIncarnationArtifacts` — startup, idempotent: deletes DM channels with no native member (with explicit child-row cleanup) and unreferenced replicated stubs homed at this instance's own domain; still-referenced stubs are skipped and logged. It does not rely on FK cascade (must not assume `PRAGMA foreign_keys` is ON): the channel delete explicitly clears its `dm_reactions`/`attachments`/`dm_messages`/`dm_members`/`read_states` rows, and the stub delete explicitly clears its `dm_reactions`/`reactions`/`read_states` rows — and the stub's deletable guard mirrors the non-cascading FKs to `users.id` by hand (a future non-cascading FK to `users.id` needs a matching NOT-EXISTS clause).

`reconcileDriftedDmFederatedIds` — startup, idempotent (reattach-dm-reconcile spec §3.3): in one transaction, iterates every 1-on-1 DM channel (exactly 2 members, 1-on-1-shaped `federated_id`) and calls `reconcileDmChannelFederatedId` on each, re-keying or merging any whose stored id has drifted from its members' current home identities. This **heals accounts re-attached before inline reconciliation shipped** (§3.2) — including the live split-conversation duplicate — without manual DB surgery. On a clean database every channel is a no-op (one hash per 1-on-1 channel); it logs a summary only when it changed something (`[federation] DM federatedId reconciliation: rekeyed N, merged M`). Wired next to `sweepDeadIncarnationArtifacts` in `startFederationWorkers`.

### Janitor Cleanup (`storageJanitor.ts:runFederationJanitor`)

| Target | Condition | Retention |
|--------|-----------|-----------|
| `federation_outbox` | `expiresAt < now` | Configurable via `federationRelayTtlDays` (default 30) |
| `federation_mutation_log` | `mutatedAt < (now - 90 days)` | 90 days |
| `federation_file_queue` (completed) | `createdAt < (now - 7 days)` | 7 days |
| `federation_file_queue` (any) | `expiresAt < now` | 30 days (set at queue time) |
| `dm_channels` (soft-deleted) | `deletedAt < (now - 24h)` | 24-hour grace period |

DM channel hard-delete cascades: reactions, embeds, attachments (DB rows + disk files), messages, members, outbox entries, mutation log entries, file queue entries.

### Worker Startup Gate

`startFederationWorkers()` in `utils/federationWorker.ts:1211` starts: outbox-delivery tick, federated-call sentinel, file-replication ticks, health-check ticks, and the storage janitor. The whole bundle is gated by `process.env.DISABLE_FEDERATION_WORKERS` matching `'1'` or `'true'` at `index.ts:171-176` (envBool semantics). Tests set `DISABLE_FEDERATION_WORKERS=1` to silence cross-instance background traffic during setup.

### Public Origin Override

`PUBLIC_ORIGIN` env (read via `config.publicOrigin`, consumed by `getOurOrigin()` in `utils/federationAuth.ts`) overrides the federation transport URL verbatim, taking precedence over the default `https://${DOMAIN}`. When unset, behaviour is unchanged. Intended for reverse-proxy / dev-without-TLS deployments where the public origin must be advertised explicitly (typically `http://...`) and differs from the bare `DOMAIN` value used for federated identity. The seed-peer integration harness (`seedPeer.ts`) does NOT use this override — see it for why localhost-port instances cannot collapse to a single peer row. The **real-handshake** harness (`realHandshake.ts`) does the opposite: it sets `PUBLIC_ORIGIN` to each instance's ephemeral `http://127.0.0.1:<port>` so the advertised `sourceOrigin` matches the transport, exercising the same path as production.

**Handshake `sourceOrigin` honors this override.** `resolveLocalOrigin()` (`routes/federation.ts`) delegates to `getOurOrigin()`, so the origin advertised in the `/peer/accept` handshake body is identical to the `X-Federation-Origin` used for authenticated S2S requests. Using `https://${DOMAIN}` directly (the prior behavior) desynced the responder's peer-row key from the auth origin whenever `PUBLIC_ORIGIN != https://DOMAIN`, causing permanent `403 Not peered`. See "Trust re-establishment contract" (§1).

### Test-Only Routes

`POST /api/admin/test/seed-peer` directly inserts a `federation_peers` row, skipping the multi-step peer handshake. Strictly gated: `NODE_ENV='test'` AND `ENABLE_TEST_ROUTES='1'` together; returns 404 in any other configuration. Used exclusively by the two-instance integration harness in `packages/server/test/`. Validates `origin` (must be http(s) URL), `hmacSecret` (≥32 chars), and `status` (must be one of `'active'`, `'pending'`, `'awaiting_approval'`, `'rejected'`, `'revoked'`, `'needs_attention'`, `'unreachable'`, `'accepted'`).

### Rate-limit bypass (test only)

`DISABLE_RATE_LIMITS=1` bypasses all `@fastify/rate-limit` enforcement at boot (envBool semantics: `'1'` or `'true'`). Used by integration test harnesses where many tests share the loopback IP and would otherwise exhaust per-IP buckets across unrelated tests. Defaults to enforced rate limits in production. The two-instance harness sets this in every spawned instance's env; tests that need to assert rate-limit behaviour (e.g. Test #15) use `bootTwoInstancesWithRateLimits()` which omits the env var.

---

## 15. Settings Cache

`federationOutbox.ts` caches `federationRelayEnabled` and `federationRelayTtlDays` from `instance_settings` for 30 seconds (`CACHE_TTL_MS`). This prevents repeated DB reads on every message send. The cache is invalidated by TTL only -- there is no explicit cache bust on settings change.

Relevant settings in `instance_settings`:

| Column | Default | Purpose |
|--------|---------|---------|
| `federation_relay_enabled` | 1 | Master toggle for all federation relay |
| `federation_relay_ttl_days` | 30 | Outbox entry TTL |
| `max_upload_size_bytes` | `null` (uses `config.maxUploadSize`) | File download size limit |

---

## 16. Client-Side Identity Helpers (`identity.ts`)

The frontend needs to resolve federated identities for display purposes:

**`parseFederatedUsername(username)`** -- splits `"erin@nova.ddns.net"` into `{baseName: "erin", domain: "nova.ddns.net"}`.

**`isSelf(user, homeUser)`** -- determines if a user object is the logged-in user or their replicated stub. Uses cascading checks: same ID, known self-ID set, homeInstance + baseName match.

**`canonicalUserMatch(a, b)`** -- federation-safe check for whether two user objects represent the same person. Cascades through: same local ID, `homeUserId` cross-match, username + homeInstance fallback.

**`resolveDisplayIdentity(user, homeUser)`** -- returns `homeUser` for display if `user` is a replicated stub of `homeUser`, enabling consistent avatars and display names across instances.

**Cross-instance self-ID registry:** `registerSelfId(id)` / `clearSelfIds()` track all Snowflake IDs belonging to the current user across connected instances, populated from WS `ready` events.

---

## 17. Self-Healing Migrations (`migrate.ts`)

The migration system includes several data integrity checks that run on every server startup:

**Group DM ownerId repair:**
Detects group DMs with UUID-format `federated_id` (length 36, matches `________-____-____-____-____________`) but `NULL owner_id`. Restores the owner from the first remaining member or from `owner_home_user_id`/`owner_home_instance`. Root cause: a bug in `processOwnershipTransferEvent` (fixed in cd7aff0) could set `ownerId = NULL` via `resolveLocalUser` fallback.

**Federated ID backfill:**
Finds 1-on-1 DM channels without `federated_id`, computes deterministic SHA-256 hash from home user IDs, and sets it. Also detects relay-created duplicate channels with the same `federated_id` and merges messages into the oldest channel.

**Duplicate channel merge:**
Finds `federated_id` values appearing on multiple channels and merges them into the oldest, moving messages, members, and cleaning up the duplicates.

**Mutation log backfill:**
If the `federation_mutation_log` table exists but is empty, populates it with `create` entries for all existing DM messages where `source_instance IS NULL` (locally-created messages).

---

## Known Issues

See `docs/federation-production-roadmap.md` for open items (FED-001 through FED-013).

- **Accept/reject/end relay failures now surfaced.** All three federated call-state transitions emit `dm_call_undeliverable { phase, terminal, failures }` to the originator on relay failure — accept rolls back optimistic state (terminal: true), reject/end keep the optimistic clear and emit an informational toast (terminal: false). See `docs/systems/voice.md` "Call relay failure surface" for the full contract. The host-side ring timeout also fans `dm_call_end` out to peers so stranded Path-A/B ringees exit the ring. One remaining edge documented in voice.md: non-host end-relay failure leaves the host's local `activeDmCall` marker until manual cleanup.