From 626a8ded501304ee22825005bedef907ed16bfb9 Mon Sep 17 00:00:00 2001 From: Jannis Braun <151788261+TheZwiss@users.noreply.github.com> Date: Tue, 7 Apr 2026 20:01:43 +0200 Subject: [PATCH] docs: update subsystem specs for cross-instance DM access Document read_state_update relay event, lifted DM gates, federatedId dedup, and relaxed authority check. --- docs/systems/client-federation.md | 8 ++--- docs/systems/dm-system.md | 16 +++++++++ docs/systems/federation.md | 59 +++++++++++++++++++++++++------ docs/systems/websocket.md | 4 +++ 4 files changed, 73 insertions(+), 14 deletions(-) diff --git a/docs/systems/client-federation.md b/docs/systems/client-federation.md index a3a098f9..3bed10d3 100644 --- a/docs/systems/client-federation.md +++ b/docs/systems/client-federation.md @@ -151,7 +151,7 @@ getChannelOrigin(channelId): string // Returns '' for home, origin URL for re Built during `populateFromReady()` when WS ready events arrive from each instance. -> **DM channels** are always mapped to `''` (home origin). `channelOriginMap` is only relevant for Space channels. DM operations always route to the home instance; S2S relay handles cross-instance distribution. +> **DM channels** are mapped to the origin of the instance that delivered them in the `ready` event. For 1-on-1 DMs created locally this is typically `''` (home), but federated DMs may arrive from any connected instance. DM read/write operations are routed to the channel's origin via `getApiForOrigin(getChannelOrigin(channelId))`. S2S relay then propagates changes to all other instances that have the same channel. ### API Client Resolution @@ -201,7 +201,7 @@ When a WS connection opens and authenticates, the server sends a `ready` event c 2. Merge into the unified space list (replacing stale data from same origin) 3. Build/update `channelOriginMap`, `channelToSpaceMap` 4. Normalize remote asset URLs to absolute paths -5. Skip DM channels from remote origins — DMs are managed exclusively by the home instance +5. Merge DM channels from all origins — DMs are accepted regardless of which instance sends the `ready` event. Channels are deduplicated by `federatedId`: if a DM channel with the same `federatedId` is already loaded (from a previous `ready` event on another connection), the second copy is skipped (first-loaded copy wins). Channels without a `federatedId` are always accepted. 6. Last-write-wins layout merge for sidebar order --- @@ -306,7 +306,7 @@ Client-side and S2S federation serve different purposes: | Aspect | Client-Side Federation | S2S Federation | |---|---|---| | **Purpose** | User interacts with multiple instances | Instances exchange data automatically | -| **Scope** | Spaces, friend discovery | DM relay, friend relay, file replication | +| **Scope** | Spaces, DM access, friend discovery | DM relay, friend relay, file replication, read state sync | | **Authentication** | Per-user JWT on each instance | Per-peer HMAC shared secret | | **Initiated by** | User (Connections settings) | Admin (peer handshake) | | **Connection** | Client → each server directly | Server → server via outbox | @@ -315,5 +315,5 @@ Client-side and S2S federation serve different purposes: 1. User adds a remote instance via Connections (client-side) 2. The client triggers S2S peering between the two servers (automatic) 3. User joins Spaces on the remote instance (client-side — API calls go directly to remote) -4. User sends DMs — all DM writes go to the home instance. S2S relay distributes messages, reactions, and membership changes to peer instances. +4. User sends DMs — DM writes go to whichever instance delivered the channel (determined by `channelOriginMap`). S2S relay distributes messages, reactions, read states, and membership changes to all peer instances. DM calls remain home-only (gated for federated users). 5. Friend requests and discovery work across instances (client loads friends from all connected instances, S2S relays friend events) diff --git a/docs/systems/dm-system.md b/docs/systems/dm-system.md index 91ecf4c3..538c7684 100644 --- a/docs/systems/dm-system.md +++ b/docs/systems/dm-system.md @@ -275,6 +275,8 @@ If a `member_add` federation event arrives for a soft-deleted channel (non-null **Request:** `{ content?: string, attachments?: string[], replyToId?: string }` +**Cross-instance access:** Federated users (those with `homeInstance` set) can send messages on any DM channel where they are a member, regardless of which instance serves the request. The `requireLocalUser` gate that previously blocked federated users from DM write endpoints has been removed. DM calls (`dm_call_*` WS events) remain gated — only local (non-replicated) users can initiate or receive calls. + **Validation:** - Caller must be a member (`isDmMember`) - Must have content or attachments (not both empty) @@ -402,6 +404,20 @@ The full event includes `participants` (all channel members with their federated 3. Clean up attachment files from disk 4. Broadcast `dm_message_deleted` to all local members +### Inbound: Read State Update + +**Function:** `federation.ts:processReadStateUpdateEvent()` + +Triggered by a `read_state_update` relay event sent when a user on another instance acknowledges a DM channel. + +1. Resolve channel by `federatedId` — reject if not found +2. Resolve user via `resolveLocalUser` — skip silently if not found +3. Resolve message by `messageRef` (local ID or `source_instance + source_message_id`) +4. Upsert `read_states` row for the resolved user and message +5. Broadcast `channel_ack` to the user's local WebSocket connections for multi-tab sync + +Not stored in the outbox or mutation log — fire-and-forget, missed deliveries are not retried. + ### Inbound: Reaction Add/Remove **Functions:** `federation.ts:processReactionAddEvent()`, `processReactionRemoveEvent()` diff --git a/docs/systems/federation.md b/docs/systems/federation.md index bb5ab751..2d97aa22 100644 --- a/docs/systems/federation.md +++ b/docs/systems/federation.md @@ -460,6 +460,7 @@ Body limit: 10 MB. Max 50 events per batch. Rate-limited to 90 requests/min per | `member_add` | `processMemberAddEvent` | dm | | `member_remove` | `processMemberRemoveEvent` | dm | | `ownership_transfer` | `processOwnershipTransferEvent` | dm | +| `read_state_update` | `processReadStateUpdateEvent` | dm | | `friend_request_create` | `processFriendRequestCreateEvent` | friend | | `friend_request_update` | `processFriendRequestUpdateEvent` | friend | | `friend_request_cancel` | `processFriendRequestCancelEvent` | friend | @@ -489,7 +490,7 @@ After processing all events, the relay endpoint updates the peer's `lastSeenAt` 5. Sets `bootstrapped = true` to skip redundant system messages and member_add broadcasts below **Incremental path** (channel already exists): -1. Validates authority: `sourceInstance === channel.ownerHomeInstance` (only owner's instance can add) +1. Validates authority: any HMAC-verified peer is accepted (relaxed — the `sourceInstance === channel.ownerHomeInstance` check was removed to support cross-instance access). The per-user attribution check (`verifyAttribution`) still applies. 2. Cancels soft-delete if channel was pending GC 3. Resolves added user via `resolveOrCreateReplicatedUser` 4. Enforces max 10 members @@ -628,7 +629,45 @@ Uses the same backoff schedule as outbox delivery. Max attempts: 10 (`MAX_FILE_A --- -## 8. Friend Relay +## 8. Read State Relay + +### `read_state_update` Event + +When a user marks a DM channel as read (via `channel_ack` WS event), the read state is relayed to all peer instances so cross-instance sessions stay in sync. + +**Outbound (`events.ts:handleChannelAck`):** +- Fires after writing `read_states` locally +- Only triggers for DM channels (channel ID found in `dm_members`) +- Calls `sendReadStateRelay(dmChannelId, userId, messageId)` in `federationOutbox.ts` +- Fire-and-forget — not queued via outbox, sent directly to all active peers (same pattern as typing relay) + +**Event payload:** +```typescript +{ + eventType: 'read_state_update', + contextType: 'dm', + federatedId: string, // DM channel's federatedId (cross-instance channel lookup) + readState: { + user: { homeUserId: string, homeInstance: string }, + messageRef: { messageId: string, sourceInstance: string | null } + } +} +``` + +`messageRef.sourceInstance` is non-null when the acknowledged message was relayed from another instance (i.e., it has a `source_instance` in `dm_messages`). The receiving instance uses this to locate the correct local message row. + +**Inbound (`processReadStateUpdateEvent`):** +1. Resolve channel by `federatedId` — reject if not found +2. Resolve user via `resolveLocalUser` — skip if not found (no-op, fire-and-forget) +3. Resolve the message by `messageRef` (local ID if `sourceInstance` is null, otherwise by `source_instance + source_message_id`) +4. Upsert `read_states` row for the resolved local user and message +5. Broadcast `channel_ack` to all of the user's local WebSocket connections (multi-tab/multi-device sync) + +**Not persisted in outbox or mutation log** — read state relay is ephemeral; missed deliveries are not retried. + +--- + +## 9. Friend Relay ### Event Flow (social.ts) @@ -694,7 +733,7 @@ The full event payload is stored in both `appendMutationLog` (for sync) and `que --- -## 9. Profile Sync +## 10. Profile Sync Profile sync uses **two mechanisms** that operate independently: @@ -733,7 +772,7 @@ Profile data is synced server-to-server. The home instance is authoritative — --- -## 10. Reaction Relay +## 11. Reaction Relay ### Outbound @@ -761,7 +800,7 @@ The mutation log entry for reactions stores a simpler payload (no `messageId`/`m --- -## 11. Initial Sync +## 12. Initial Sync ### `runInitialSyncForNewPeers()` (`federationWorker.ts:739`) @@ -806,7 +845,7 @@ The event processing logic is extracted into `processRelayEvents()` (exported fr --- -## 12. DM Calls over Federation +## 13. DM Calls over Federation DM calls work across federated instances. The caller's instance hosts the LiveKit room. Remote clients connect directly to the caller's LiveKit server using a token passed through S2S relay — no media is routed through the federation layer. @@ -888,7 +927,7 @@ This registry ensures tokens and `livekitUrl` survive browser refreshes via the --- -## 13. Background Workers +## 14. Background Workers All workers are started by `startFederationWorkers()` on server boot and stopped by `stopFederationWorkers()` on shutdown. Each worker uses `setTimeout` chains (not `setInterval`) with abort controllers for graceful shutdown. @@ -914,7 +953,7 @@ DM channel hard-delete cascades: reactions, embeds, attachments (DB rows + disk --- -## 14. Settings Cache +## 15. Settings Cache `federationOutbox.ts` caches `federationRelayEnabled` and `federationRelayTtlDays` from `instance_settings` for 30 seconds (`CACHE_TTL_MS`). This prevents repeated DB reads on every message send. The cache is invalidated by TTL only -- there is no explicit cache bust on settings change. @@ -928,7 +967,7 @@ Relevant settings in `instance_settings`: --- -## 15. Client-Side Identity Helpers (`identity.ts`) +## 16. Client-Side Identity Helpers (`identity.ts`) The frontend needs to resolve federated identities for display purposes: @@ -944,7 +983,7 @@ The frontend needs to resolve federated identities for display purposes: --- -## 16. Self-Healing Migrations (`migrate.ts`) +## 17. Self-Healing Migrations (`migrate.ts`) The migration system includes several data integrity checks that run on every server startup: diff --git a/docs/systems/websocket.md b/docs/systems/websocket.md index 943dd5ff..26b3dc49 100644 --- a/docs/systems/websocket.md +++ b/docs/systems/websocket.md @@ -192,6 +192,10 @@ reason: `'displaced'` (new tab) | `'session_closed'` |------|--------|-------| | `federation_file_rejected` | messageId, dmChannelId, attachmentId, affectedUsers[] | DM members | +**S2S relay-only event (not a direct client WS event):** + +`read_state_update` — sent peer-to-peer via the federation relay when a user acknowledges a DM channel on one instance. The receiving instance processes it, upserts the `read_states` row, and then emits a standard `channel_ack` event to the user's local WebSocket connections. The relay event itself is never forwarded to clients directly. + --- ## Ready Payload