diff --git a/docs/systems/api.md b/docs/systems/api.md index 8ee8f998..b18a5df6 100644 --- a/docs/systems/api.md +++ b/docs/systems/api.md @@ -118,7 +118,7 @@ DELETE /dm/messages/:id → { succes ``` GET /social/friends → { friends[] } GET /social/requests → { requests[] } -POST /social/requests { username } → { request } +POST /social/requests { username } → { success, requestId } PATCH /social/requests/:id { status: 'accepted'|'declined' } → { request } DELETE /social/requests/:id → { success } (cancel, sender-only) DELETE /social/friends/:id → { success } @@ -126,6 +126,27 @@ GET /social/discover ?q=&limit=&offset= → { users[], total } GET /social/search ?q= → { users[] } ``` +### POST /api/social/requests — routing & error codes + +`body.username` may be `bare` (local), `bare@` (also routed local — server normalizes), or `bare@` (federated branch). The client sends the trimmed handle verbatim; all parsing, routing, peering, and remote lookup are server-side. + +| HTTP | error code | When | +|---|---|---| +| 200 | (success, idempotent) | Same-direction pending request already exists; returns existing `requestId` | +| 201 | (success, created) | New friend request created | +| 400 | `username_required` | Missing/empty/non-string username | +| 400 | `cannot_friend_self` | Looked-up identity matches sender | +| 400 | `invalid_target_domain` | Scheme resolution failed (e.g., non-localhost HTTP target when our scheme is HTTPS) | +| 403 | `peer_rejected` | Remote instance has rejected federation; admin must intervene | +| 403 | `not_authoritative_for_sender` | Caller is a federated (replicated) user; should not have reached here | +| 404 | `user_not_found` | Remote lookup returned 404 (no such user, or tombstoned) | +| 409 | `already_friends` | Friendship row already exists | +| 409 | `peer_pending_approval` | Remote admin needs to approve the peering relationship | +| 409 | `peer_pending` | Peer handshake in flight | +| 409 | `incoming_request_exists` | Opposite-direction pending request exists; response includes `requestId` for deep-link | +| 429 | `lookup_rate_limited` | Remote `/users/lookup` returned 429; `Retry-After` header forwarded | +| 503 | `peer_unreachable` | Remote instance unreachable (network/timeout/lookup-unreachable) | + ## Search (`routes/search.ts`) — auth required ``` GET /channels/:id/search ?q=&from=&has=&before=&after=&offset=&limit= → { results[], totalCount } [VIEW_CHANNEL] @@ -201,8 +222,11 @@ GET /federation/peers (admin) DELETE /federation/peers/:id (admin) → { success } + outbox cleanup POST /federation/relay (HMAC-signed S2S) FederationRelayRequest → { accepted[], rejected[] } POST /federation/sync (HMAC-signed S2S) { sinceTimestamp, limit?, dmChannelId?, federatedId?, contextType? } → { events[], hasMore, checkpoint } +POST /federation/users/lookup (HMAC-signed S2S, rate-limited 60/min/peer) { username } → { found, user? } ``` +**`POST /api/federation/users/lookup`** — HMAC-authenticated S2S endpoint. Resolves a username on this instance to its canonical `(homeUserId, profile snapshot)`. Used by the cross-instance friend-add flow on the sender's home server before queuing a `friend_request_create` event. Responds to native, non-deleted users only; ignores `discoverable`. Returns `{ found: false, code: 'user_not_found' }` for stubs, tombstoned users, or unknown handles. See `federation.md` §1 "S2S User Lookup" for the full contract. + ## Utilities (`routes/utils.ts`) — auth required ``` GET /utils/metadata ?url= → { title?, description?, image?, siteName? } diff --git a/docs/systems/client-federation.md b/docs/systems/client-federation.md index cc9b34b3..76fcc69a 100644 --- a/docs/systems/client-federation.md +++ b/docs/systems/client-federation.md @@ -15,6 +15,10 @@ Source files: ## Architecture Overview +**Client-side vs S2S federation by feature:** +- **Friend & DM relay are S2S.** Sending a friend request to `alice@orbit.tld` does not require having a federated account on `orbit.tld`; the sender's home server queues the relay (see `social.md` §6 outbound flow). DM messages are similarly relayed server-to-server once the initial channel exists. +- **Spaces are client-federated.** Joining a remote space still requires creating a federated account on that instance via the Connections UI. + Backspace supports **client-side federation**: a single app session (web or desktop — both are feature-identical) can connect to multiple Backspace instances simultaneously. The user has a **home instance** (their primary identity) and zero or more **connected remote instances**. ``` @@ -50,7 +54,9 @@ Backspace supports **client-side federation**: a single app session (web or desk ## 1. Federated Account Creation -When a user connects to a remote instance, the client creates (or logs into) an account on that instance. This is a **real account with a real bcrypt password** — not a replicated stub. +When a user adds a remote instance via the Connections settings to **join a space there**, the client creates (or logs into) an account on that instance. As of 2026-04-25, this is no longer required for friending or messaging users on a remote instance — those flows are fully S2S (see `social.md` §6 and `federation.md` §4 respectively). Federated accounts remain real loginable accounts and retain all capabilities (login, space membership, password sync, deletion). + +This is a **real account with a real bcrypt password** — not a replicated stub. ### Username Format @@ -80,6 +86,12 @@ When a user adds a remote instance via the Connections settings: The same password is used across all instances. Password changes on the home instance are synced to remote instances automatically. +### API Client Error Contract + +The shared API client (`packages/web/src/api/client.ts:298`) throws `new Error(body.error)` for non-2xx responses. The server's structured error code is on `err.message`; there is **no** `err.body` or `err.code` property. Catch handlers that need to map codes to UI messages should read `err.message` and pass it as both the code and the fallback to `mapServerErrorToMessage` (see `packages/web/src/utils/friendErrors.ts`). + +This was documented after T19/T20 catch blocks initially read the wrong shape and surfaced raw codes as toast text — fixed in commit `d207af4`. The same pattern applies to any new client-side code that catches API errors from the home or remote instances. + --- ## 2. Instance Store (`instanceStore.ts`) diff --git a/docs/systems/federation.md b/docs/systems/federation.md index e489f9c7..9ccfdc49 100644 --- a/docs/systems/federation.md +++ b/docs/systems/federation.md @@ -183,6 +183,7 @@ When `autoAcceptPeering` is `false` and an instance calls `POST /api/federation/ | `/api/federation/peer/rotate` | POST | HMAC | Accept secret rotation from peer | | `/api/federation/peer/denied` | POST | HMAC | Receive denial notification for awaiting_approval peer | | `/api/federation/identity` | DELETE | HMAC | Delete federated user identity (soft/full mode) | +| `/api/federation/users/lookup` | POST | HMAC, rate-limited 60/min/peer | Resolve a username on this instance to (homeUserId, profile snapshot) for cross-instance friend-request originators | ### S2S Identity Deletion (`DELETE /api/federation/identity`) @@ -202,6 +203,22 @@ Allows a home instance to remove a user's replicated identity from a remote inst - **Mode `"full"`:** Calls `tombstoneUser(uid, { purgeContent: true })` — full tombstone including reactions and orphaned DM cleanup. - **Post-deletion:** Broadcasts `member_left` WS events for all spaces the user belonged to before removal. +### S2S User Lookup (`POST /api/federation/users/lookup`) + +HMAC-authenticated. Rate-limited to 60 requests/minute per peer. Used by the cross-instance friend-add flow to resolve a username on this instance before the sender's home server queues a `friend_request_create` event (see `social.md` §6 outbound flow). + +**Request body:** `{ username: string }` — server trims and lowercases before lookup. + +**Response 200:** `{ found: true, user: { homeUserId, username, profile: { displayName, avatar, avatarColor, banner, bio } } }` — returned for native, non-deleted users regardless of their `discoverable` setting. + +**Response 404:** `{ found: false, code: 'user_not_found' }` — returned for tombstoned users (`isDeleted=1`), replicated stubs (`homeInstance IS NOT NULL`), or unknown usernames. + +**Response 400:** Malformed input (missing, non-string, or empty-after-trim `username`). + +**Response 429:** Per-peer rate limit (60/min) exceeded; `Retry-After` header set. + +**Filter invariant:** `discoverable` is NOT consulted — exact-handle resolution must work for opted-out users. This mirrors the Direct-Add invariant from social.md commits 69d430b/c60ecc5/189cac4: discovery surfaces only opted-in users, but once you have a handle you can always send a request. + ### WebSocket Events (Peering) These S→C events are pushed to the acting user's connected clients by the federation subsystem. @@ -487,14 +504,21 @@ Trigger (API/WS handler) - Increment peer `consecutiveFailures`, set `lastFailureAt` - If `consecutiveFailures >= PEER_UNREACHABLE_THRESHOLD (10)` -> mark peer `unreachable` -#### Terminal rejection: `duplicate` +#### Terminal rejection reasons -The outbox delivery worker treats a relay response of `{ rejected: [{ reason: 'duplicate', ... }] }` as effectively-accepted — the outbox entry is deleted rather than retained for retry. The `duplicate` reason is emitted by the receiving instance's inbound processors when a row with the same `(sourceInstance, sourceMessageId)` already exists; retrying will fail identically until TTL (30 days). Since the peer already has the message, terminal removal is the correct outcome. +As of 2026-04-25, `processOutboxTick` recognizes a configurable set of receiver-acknowledged **terminal rejection reasons** (constant `TERMINAL_REJECTION_REASONS` in `federationWorker.ts`): `duplicate`, `recipient_not_found`, `attribution_mismatch`, `unknown_event_type`. Outbox entries with these reasons are deleted with no retry. + +- `duplicate` — the receiving instance already has the row (same `(sourceInstance, sourceMessageId)`); retrying will fail identically until TTL. +- `recipient_not_found`, `attribution_mismatch`, `unknown_event_type` — structural mismatches that cannot be resolved by retrying. + +For non-`duplicate` terminals, the worker invokes a registered permanent-failure callback via `invokePermanentFailureCallback(eventType, messageId, reason)` from `utils/federationRollback.ts`. Currently registered: `friend_request_create` → `rollbackFriendRequestCreate` (deletes the local `friend_requests` row by `relay_message_id` and emits WS `friend_request_relay_failed` to the sender). Other event types may register their own callbacks. See `social.md` §6 "Failure Handling" for the friend-specific rollback contract. + +**Ghost-row failure mode.** Rollback callbacks are best-effort: the registry catches and logs callback errors but does NOT re-throw. A botched rollback (e.g., DB write fails mid-rollback due to disk full or FK violation) can leave a ghost row that no longer corresponds to any in-flight relay. Acceptable vs. retry-forever blocking the outbox, but worth knowing when debugging stuck pending states. + +**5xx and network failures are NOT terminal** — those use the existing exponential-backoff retry path. Pending operations stay pending indefinitely under sustained connectivity loss, matching the pre-existing DM relay behavior. Logged at `console.log` ("outbox entry removed (terminal)") to distinguish from retained-for-retry `console.warn` messages. -Other rejection reasons (`attribution_mismatch`, `missing_*_payload`, `unknown_event_type`, `unauthorized_source`, `channel_not_found`, `participant_not_found`, `processing_error`, …) remain on the retry path. Some are arguably terminal too; treating them as such is deferred until they are observed accumulating in practice. - ### Retry Backoff Schedule | Attempt | Delay | diff --git a/docs/systems/social.md b/docs/systems/social.md index 18c49469..516c5bca 100644 --- a/docs/systems/social.md +++ b/docs/systems/social.md @@ -287,13 +287,36 @@ The sorted join ensures the same pair always produces the same prefix regardless ### End-to-End Relay Flow: Friend Request Create -**Outbound (origin instance -- `social.ts:POST /api/social/requests`):** -1. Friend request created locally (DB insert + WS broadcast to local recipient) -2. Build identity objects for sender and recipient using `homeUserId || id` / `homeInstance || getOurOrigin()` -3. Call `getFriendEventTargets(from.homeInstance, to.homeInstance)` -- if both local, returns `[]` (no relay) -4. Build `FederationRelayEvent` with `eventType: 'friend_request_create'`, `contextType: 'friend'`, friendship payload including both identities and profile snapshots -5. Call `appendMutationLog()` -- records in `federation_mutation_log` for sync protocol -6. Call `queueOutboxEvent()` -- inserts into `federation_outbox` for each target peer +**Outbound (sender's home instance -- `social.ts:POST /api/social/requests`):** + +As of 2026-04-25, the sender's home server owns the entire federated friend-add flow. The client sends `{ username }` verbatim; all parsing, peering, remote lookup, and queueing happen server-side in this strict order: + +1. **Parse target.** If `body.username` contains no `@`, or the domain after `@` normalizes to this server's own host, fall through to the local-only path (unchanged). +2. **resolveOriginFromHostname(targetDomain)** — resolves the target peer's full origin URL. Prefers a stored `federation_peers` row matching the typed host; falls back to mirroring `getOurOrigin()`'s scheme. Returns null → 400 `invalid_target_domain`. +3. **Authority defense.** If the calling user's `homeInstance` is set and does not normalize to this server's own host (checked via `normalizeOriginForCompare`), return 403 `not_authoritative_for_sender`. Prevents replicated/federated users from queueing relay events the home server isn't authoritative for. Runs before peering to fail fast. +4. **ensurePeered(peerOrigin)** — blocks on the result. Status → HTTP mapping: + - `'active'` → continue + - `'pending'` (handshake in flight) → 409 `peer_pending` + - `'pending'` + peer row `awaiting_approval` (re-queried after the call) → 409 `peer_pending_approval` + - `'rejected'` → 403 `peer_rejected` + - `'failed'` → 503 `peer_unreachable` +5. **lookupRemoteUser(peerOrigin, baseName)** — POSTs HMAC-signed `{ username }` to `peerOrigin/api/federation/users/lookup`. Result mapping: + - `not_found` → 404 `user_not_found` + - `unreachable` → 503 `peer_unreachable` + - `rate_limited` → 429 `lookup_rate_limited` (with `Retry-After` header) +6. **Self-friend pre-check.** If the looked-up `(homeUserId, peerOrigin)` matches the sender's canonical identity (using `normalizeOriginForCompare` for host comparison) → 400 `cannot_friend_self`. +7. **resolveOrCreateReplicatedUser + hydrateReplicatedUserProfile** — creates or refreshes the local stub for the remote user. Tombstoned identities (resolveOrCreateReplicatedUser returns null) → 404 `user_not_found`. +8. **Direction-aware idempotency:** + - Same-direction pending request exists → 200 with existing `requestId` (idempotent). + - Opposite-direction pending request exists → 409 `incoming_request_exists` with existing `requestId` for client deep-link. + - Already friends → 409 `already_friends`. +9. **db.transaction(...)** (synchronous): inserts the `friend_requests` row with `relayMessageId = entityId`, calls `appendMutationLog`, calls `queueOutboxEvent` targeting `[peerOrigin]`. +10. **WS broadcast** `friend_request_sent` to the sender's other tabs/devices (multi-tab sync). +11. Returns `201 { success: true, requestId }`. + +The wire format of the queued event is identical to the pre-2026-04-25 flow; only the queueing instance has changed. The receiver's `processFriendRequestCreateEvent` is unchanged. Both peers' authority checks (`from.homeInstance === sourceInstance`) continue to pass because the sender's instance is now both source and queueing instance. + +> **Schema note:** The `friend_requests` table gained a `relayMessageId TEXT` column (added 2026-04-25, drizzle migration `0001_complex_screwball.sql`). It is `NULL` for local-only requests; for federated ones it carries the `entityId` of the originating relay event so the rollback hook can locate the row by message ID. **Delivery (federation worker -- `federationWorker.ts:processOutboxTick`):** 1. Worker polls outbox every 10 seconds @@ -311,6 +334,22 @@ The sorted join ensures the same pair always produces the same prefix regardless 7. **WS broadcast:** `friend_request_received` sent to local recipient with sender's sanitized profile 8. Push `event.messageId` to accepted array +### Failure Handling: Async Rollback + +When the outbox worker receives a relay response from the remote instance, it classifies each rejected entry. As of 2026-04-25, a configurable set of **terminal rejection reasons** (`TERMINAL_REJECTION_REASONS` in `federationWorker.ts`) causes an outbox entry to be deleted with no retry: `duplicate`, `recipient_not_found`, `attribution_mismatch`, `unknown_event_type`. + +For non-`duplicate` terminals, the worker invokes the registered permanent-failure callback via `invokePermanentFailureCallback(eventType, messageId, reason)` from `utils/federationRollback.ts`. For `friend_request_create`, this is **`rollbackFriendRequestCreate`**: + +1. Looks up the `friend_requests` row by `relayMessageId` (the stored `entityId`). +2. Deletes the row. +3. Emits WS `friend_request_relay_failed` to the sender's connections, with a client-facing reason: receiver `recipient_not_found` → `user_not_found`; everything else → `peer_rejected`. + +The client handler in `useWebSocket.ts` removes the row from `socialStore` and shows a warning toast. + +**5xx responses, network errors, and retry exhaustion are NOT terminal** — the outbox retries with exponential backoff. The sender sees indefinite "pending" under sustained connectivity loss, matching DM relay's behavior under the same conditions. + +**Ghost-row risk.** Rollback callbacks are best-effort: the registry catches and logs callback errors but does not re-throw. A failed rollback (e.g., DB write fails mid-rollback) leaves a ghost `friend_requests` row with no corresponding in-flight relay. Acceptable vs. retry-forever blocking the outbox, but worth knowing when debugging stuck pending requests. + ### End-to-End Relay Flow: Friend Request Update (Accept/Decline) **Outbound (`social.ts:PATCH /api/social/requests/:id`):** @@ -411,18 +450,11 @@ type TaggedUser = User & { _instanceOrigin: string }; Same `Promise.allSettled()` fan-out pattern as `loadFriends`. **Dedup by the other party's canonical identity** (`request.user.homeUserId ?? request.user.id`), preferring the record from the instance where the other party is native (`!request.user.homeInstance`). This is critical: a cross-instance request exists as two rows -- one on each instance -- and both sides return it, but only the record from the target's home instance has the canonical (non-stub) user ids and the correct `_instanceOrigin` tag. Matching those is what lets the Add Friend search card flip to "Request Pending" after sending. Normalizes assets for remote request user profiles. -### Sending Friend Requests (Federation Routing) +### Sending Friend Requests -`sendFriendRequest(username: string)` handles the `user@domain` routing: +`sendFriendRequest(username: string)` sends the trimmed handle verbatim to the home instance API (`POST /api/social/requests`). As of 2026-04-25, all routing, peering, and remote lookup happen server-side — the client no longer resolves the domain to a connected instance or throws `InstanceNotConnectedError`/`InstanceDisconnectedError`. The server returns a structured error code on any failure; the catch block in `socialStore` maps it via `mapServerErrorToMessage` from `packages/web/src/utils/friendErrors.ts` and surfaces it as a toast. -1. **No `@` in username:** Send to home instance API directly -2. **`@` present:** Extract `baseName` and `domain` via `lastIndexOf('@')` - - If domain matches `window.location.host`: strip domain, send to home API - - Otherwise: find matching connected instance by comparing `new URL(inst.origin).host === domain` - - **Not found:** Throw `InstanceNotConnectedError(domain)` (UI prompts to connect) - - **Found but disconnected:** Throw `InstanceDisconnectedError(domain)` (UI prompts to reconnect) - - **Found and connected:** Send to remote instance API with just the `baseName` (no domain suffix) -3. After success, reloads requests via `loadRequests()` +After success, reloads requests via `loadRequests()`. ### Cross-Instance Search (`searchUsers`) @@ -592,7 +624,9 @@ The Add Friend tab merges search and discovery into a single UI: 1. **Empty query:** Shows discover grid (from `discoverStore.fetchUsers()`, loaded on mount) 2. **Query entered:** Switches to search mode (debounced 300ms, uses `socialStore.searchUsers()`) -3. **Direct-Add row:** Shown whenever the search input is non-empty and resolves to a well-formed handle (`trimmed.length > 0 && (no @ || @ at non-edge position)`). Displays the resolved form: when the typed query has no `@`, the row shows `@` so the user sees which instance the request will hit; when `@` is present, displays the typed query verbatim. The submit button calls `sendFriendRequest(query.trim())` — the resolved form is display-only; routing to the correct API client (home-only for no-`@`, federation-aware for `@host` form) happens inside `sendFriendRequest`, which lowercases the parsed `@domain` before comparison so mixed-case hostnames route correctly. Server-side `POST /api/social/requests` lowercases the lookup input before matching, so mixed-case bare handles resolve too. A 404 from the server (`"User not found"`) surfaces as a warning toast via the catch-all in `handleDirectAdd`. +3. **Direct-Add row:** Shown whenever the search input is non-empty and resolves to a well-formed handle (`trimmed.length > 0 && (no @ || @ at non-edge position)`). Displays the resolved form: when the typed query has no `@`, the row shows `@` so the user sees which instance the request will hit; when `@` is present, displays the typed query verbatim. The submit button calls `sendFriendRequest(query.trim())` — the resolved form is display-only. All routing, peering, and remote lookup happen server-side on `POST /api/social/requests` (see §6 outbound flow). Server-side `POST /api/social/requests` lowercases the lookup input before matching, so mixed-case bare handles resolve too. + +**Error handling:** Server errors surface as toasts via `mapServerErrorToMessage` in `packages/web/src/utils/friendErrors.ts`. All structured error codes returned by the federated branch (`user_not_found`, `peer_pending`, `peer_rejected`, `incoming_request_exists`, etc.) are mapped to human-readable messages there. The `ConnectInstanceModal` component still exists in the codebase but is no longer triggered by friend-add — it is used only by the Connections settings panel and space-join flows. ### Search Result Enrichment @@ -612,7 +646,7 @@ Renders a card with banner, avatar, display name, username, bio, mutual counts, - `inbound_pending`: "Accept" / "Decline" buttons - `friends`: "Message" button -When sending a request to a remote user, constructs `baseName@originHost` format for the username. Handles `InstanceNotConnectedError` / `InstanceDisconnectedError` by showing `ConnectInstanceModal`. +When sending a request to a remote user, constructs `baseName@originHost` format for the username. Errors from the server are surfaced as toasts via `mapServerErrorToMessage` (see `packages/web/src/utils/friendErrors.ts`). --- @@ -657,8 +691,7 @@ All actions route through `socialStore` methods, which handle instance routing v - User profile is loaded via `getApiForOrigin(origin)` to fetch from the correct instance - Banner/avatar URLs resolved through the correct API client for remote users - Mutuals loaded via `loadFederatedMutuals()` with cross-instance fan-out -- Friend actions use `sendFriendRequest(user.username)` which handles `user@domain` routing -- `ConnectInstanceModal` shown when trying to add a friend on an unconnected instance +- Friend actions use `sendFriendRequest(user.username)` — all routing is server-side; errors surface as toasts via `mapServerErrorToMessage` ---