docs: update subsystem specs for cross-instance DM access
Document read_state_update relay event, lifted DM gates, federatedId dedup, and relaxed authority check.
This commit is contained in:
Jannis Braun
@@ -151,7 +151,7 @@ getChannelOrigin(channelId): string // Returns '' for home, origin URL for re
|
||||
|
||||
Built during `populateFromReady()` when WS ready events arrive from each instance.
|
||||
|
||||
> **DM channels** are always mapped to `''` (home origin). `channelOriginMap` is only relevant for Space channels. DM operations always route to the home instance; S2S relay handles cross-instance distribution.
|
||||
> **DM channels** are mapped to the origin of the instance that delivered them in the `ready` event. For 1-on-1 DMs created locally this is typically `''` (home), but federated DMs may arrive from any connected instance. DM read/write operations are routed to the channel's origin via `getApiForOrigin(getChannelOrigin(channelId))`. S2S relay then propagates changes to all other instances that have the same channel.
|
||||
|
||||
### API Client Resolution
|
||||
|
||||
@@ -201,7 +201,7 @@ When a WS connection opens and authenticates, the server sends a `ready` event c
|
||||
2. Merge into the unified space list (replacing stale data from same origin)
|
||||
3. Build/update `channelOriginMap`, `channelToSpaceMap`
|
||||
4. Normalize remote asset URLs to absolute paths
|
||||
5. Skip DM channels from remote origins — DMs are managed exclusively by the home instance
|
||||
5. Merge DM channels from all origins — DMs are accepted regardless of which instance sends the `ready` event. Channels are deduplicated by `federatedId`: if a DM channel with the same `federatedId` is already loaded (from a previous `ready` event on another connection), the second copy is skipped (first-loaded copy wins). Channels without a `federatedId` are always accepted.
|
||||
6. Last-write-wins layout merge for sidebar order
|
||||
|
||||
---
|
||||
@@ -306,7 +306,7 @@ Client-side and S2S federation serve different purposes:
|
||||
| Aspect | Client-Side Federation | S2S Federation |
|
||||
|---|---|---|
|
||||
| **Purpose** | User interacts with multiple instances | Instances exchange data automatically |
|
||||
| **Scope** | Spaces, friend discovery | DM relay, friend relay, file replication |
|
||||
| **Scope** | Spaces, DM access, friend discovery | DM relay, friend relay, file replication, read state sync |
|
||||
| **Authentication** | Per-user JWT on each instance | Per-peer HMAC shared secret |
|
||||
| **Initiated by** | User (Connections settings) | Admin (peer handshake) |
|
||||
| **Connection** | Client → each server directly | Server → server via outbox |
|
||||
@@ -315,5 +315,5 @@ Client-side and S2S federation serve different purposes:
|
||||
1. User adds a remote instance via Connections (client-side)
|
||||
2. The client triggers S2S peering between the two servers (automatic)
|
||||
3. User joins Spaces on the remote instance (client-side — API calls go directly to remote)
|
||||
4. User sends DMs — all DM writes go to the home instance. S2S relay distributes messages, reactions, and membership changes to peer instances.
|
||||
4. User sends DMs — DM writes go to whichever instance delivered the channel (determined by `channelOriginMap`). S2S relay distributes messages, reactions, read states, and membership changes to all peer instances. DM calls remain home-only (gated for federated users).
|
||||
5. Friend requests and discovery work across instances (client loads friends from all connected instances, S2S relays friend events)
|
||||
|
||||
@@ -275,6 +275,8 @@ If a `member_add` federation event arrives for a soft-deleted channel (non-null
|
||||
|
||||
**Request:** `{ content?: string, attachments?: string[], replyToId?: string }`
|
||||
|
||||
**Cross-instance access:** Federated users (those with `homeInstance` set) can send messages on any DM channel where they are a member, regardless of which instance serves the request. The `requireLocalUser` gate that previously blocked federated users from DM write endpoints has been removed. DM calls (`dm_call_*` WS events) remain gated — only local (non-replicated) users can initiate or receive calls.
|
||||
|
||||
**Validation:**
|
||||
- Caller must be a member (`isDmMember`)
|
||||
- Must have content or attachments (not both empty)
|
||||
@@ -402,6 +404,20 @@ The full event includes `participants` (all channel members with their federated
|
||||
3. Clean up attachment files from disk
|
||||
4. Broadcast `dm_message_deleted` to all local members
|
||||
|
||||
### Inbound: Read State Update
|
||||
|
||||
**Function:** `federation.ts:processReadStateUpdateEvent()`
|
||||
|
||||
Triggered by a `read_state_update` relay event sent when a user on another instance acknowledges a DM channel.
|
||||
|
||||
1. Resolve channel by `federatedId` — reject if not found
|
||||
2. Resolve user via `resolveLocalUser` — skip silently if not found
|
||||
3. Resolve message by `messageRef` (local ID or `source_instance + source_message_id`)
|
||||
4. Upsert `read_states` row for the resolved user and message
|
||||
5. Broadcast `channel_ack` to the user's local WebSocket connections for multi-tab sync
|
||||
|
||||
Not stored in the outbox or mutation log — fire-and-forget, missed deliveries are not retried.
|
||||
|
||||
### Inbound: Reaction Add/Remove
|
||||
|
||||
**Functions:** `federation.ts:processReactionAddEvent()`, `processReactionRemoveEvent()`
|
||||
|
||||
+49
-10
@@ -460,6 +460,7 @@ Body limit: 10 MB. Max 50 events per batch. Rate-limited to 90 requests/min per
|
||||
| `member_add` | `processMemberAddEvent` | dm |
|
||||
| `member_remove` | `processMemberRemoveEvent` | dm |
|
||||
| `ownership_transfer` | `processOwnershipTransferEvent` | dm |
|
||||
| `read_state_update` | `processReadStateUpdateEvent` | dm |
|
||||
| `friend_request_create` | `processFriendRequestCreateEvent` | friend |
|
||||
| `friend_request_update` | `processFriendRequestUpdateEvent` | friend |
|
||||
| `friend_request_cancel` | `processFriendRequestCancelEvent` | friend |
|
||||
@@ -489,7 +490,7 @@ After processing all events, the relay endpoint updates the peer's `lastSeenAt`
|
||||
5. Sets `bootstrapped = true` to skip redundant system messages and member_add broadcasts below
|
||||
|
||||
**Incremental path** (channel already exists):
|
||||
1. Validates authority: `sourceInstance === channel.ownerHomeInstance` (only owner's instance can add)
|
||||
1. Validates authority: any HMAC-verified peer is accepted (relaxed — the `sourceInstance === channel.ownerHomeInstance` check was removed to support cross-instance access). The per-user attribution check (`verifyAttribution`) still applies.
|
||||
2. Cancels soft-delete if channel was pending GC
|
||||
3. Resolves added user via `resolveOrCreateReplicatedUser`
|
||||
4. Enforces max 10 members
|
||||
@@ -628,7 +629,45 @@ Uses the same backoff schedule as outbox delivery. Max attempts: 10 (`MAX_FILE_A
|
||||
|
||||
---
|
||||
|
||||
## 8. Friend Relay
|
||||
## 8. Read State Relay
|
||||
|
||||
### `read_state_update` Event
|
||||
|
||||
When a user marks a DM channel as read (via `channel_ack` WS event), the read state is relayed to all peer instances so cross-instance sessions stay in sync.
|
||||
|
||||
**Outbound (`events.ts:handleChannelAck`):**
|
||||
- Fires after writing `read_states` locally
|
||||
- Only triggers for DM channels (channel ID found in `dm_members`)
|
||||
- Calls `sendReadStateRelay(dmChannelId, userId, messageId)` in `federationOutbox.ts`
|
||||
- Fire-and-forget — not queued via outbox, sent directly to all active peers (same pattern as typing relay)
|
||||
|
||||
**Event payload:**
|
||||
```typescript
|
||||
{
|
||||
eventType: 'read_state_update',
|
||||
contextType: 'dm',
|
||||
federatedId: string, // DM channel's federatedId (cross-instance channel lookup)
|
||||
readState: {
|
||||
user: { homeUserId: string, homeInstance: string },
|
||||
messageRef: { messageId: string, sourceInstance: string | null }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`messageRef.sourceInstance` is non-null when the acknowledged message was relayed from another instance (i.e., it has a `source_instance` in `dm_messages`). The receiving instance uses this to locate the correct local message row.
|
||||
|
||||
**Inbound (`processReadStateUpdateEvent`):**
|
||||
1. Resolve channel by `federatedId` — reject if not found
|
||||
2. Resolve user via `resolveLocalUser` — skip if not found (no-op, fire-and-forget)
|
||||
3. Resolve the message by `messageRef` (local ID if `sourceInstance` is null, otherwise by `source_instance + source_message_id`)
|
||||
4. Upsert `read_states` row for the resolved local user and message
|
||||
5. Broadcast `channel_ack` to all of the user's local WebSocket connections (multi-tab/multi-device sync)
|
||||
|
||||
**Not persisted in outbox or mutation log** — read state relay is ephemeral; missed deliveries are not retried.
|
||||
|
||||
---
|
||||
|
||||
## 9. Friend Relay
|
||||
|
||||
### Event Flow (social.ts)
|
||||
|
||||
@@ -694,7 +733,7 @@ The full event payload is stored in both `appendMutationLog` (for sync) and `que
|
||||
|
||||
---
|
||||
|
||||
## 9. Profile Sync
|
||||
## 10. Profile Sync
|
||||
|
||||
Profile sync uses **two mechanisms** that operate independently:
|
||||
|
||||
@@ -733,7 +772,7 @@ Profile data is synced server-to-server. The home instance is authoritative —
|
||||
|
||||
---
|
||||
|
||||
## 10. Reaction Relay
|
||||
## 11. Reaction Relay
|
||||
|
||||
### Outbound
|
||||
|
||||
@@ -761,7 +800,7 @@ The mutation log entry for reactions stores a simpler payload (no `messageId`/`m
|
||||
|
||||
---
|
||||
|
||||
## 11. Initial Sync
|
||||
## 12. Initial Sync
|
||||
|
||||
### `runInitialSyncForNewPeers()` (`federationWorker.ts:739`)
|
||||
|
||||
@@ -806,7 +845,7 @@ The event processing logic is extracted into `processRelayEvents()` (exported fr
|
||||
|
||||
---
|
||||
|
||||
## 12. DM Calls over Federation
|
||||
## 13. DM Calls over Federation
|
||||
|
||||
DM calls work across federated instances. The caller's instance hosts the LiveKit room. Remote clients connect directly to the caller's LiveKit server using a token passed through S2S relay — no media is routed through the federation layer.
|
||||
|
||||
@@ -888,7 +927,7 @@ This registry ensures tokens and `livekitUrl` survive browser refreshes via the
|
||||
|
||||
---
|
||||
|
||||
## 13. Background Workers
|
||||
## 14. Background Workers
|
||||
|
||||
All workers are started by `startFederationWorkers()` on server boot and stopped by `stopFederationWorkers()` on shutdown. Each worker uses `setTimeout` chains (not `setInterval`) with abort controllers for graceful shutdown.
|
||||
|
||||
@@ -914,7 +953,7 @@ DM channel hard-delete cascades: reactions, embeds, attachments (DB rows + disk
|
||||
|
||||
---
|
||||
|
||||
## 14. Settings Cache
|
||||
## 15. Settings Cache
|
||||
|
||||
`federationOutbox.ts` caches `federationRelayEnabled` and `federationRelayTtlDays` from `instance_settings` for 30 seconds (`CACHE_TTL_MS`). This prevents repeated DB reads on every message send. The cache is invalidated by TTL only -- there is no explicit cache bust on settings change.
|
||||
|
||||
@@ -928,7 +967,7 @@ Relevant settings in `instance_settings`:
|
||||
|
||||
---
|
||||
|
||||
## 15. Client-Side Identity Helpers (`identity.ts`)
|
||||
## 16. Client-Side Identity Helpers (`identity.ts`)
|
||||
|
||||
The frontend needs to resolve federated identities for display purposes:
|
||||
|
||||
@@ -944,7 +983,7 @@ The frontend needs to resolve federated identities for display purposes:
|
||||
|
||||
---
|
||||
|
||||
## 16. Self-Healing Migrations (`migrate.ts`)
|
||||
## 17. Self-Healing Migrations (`migrate.ts`)
|
||||
|
||||
The migration system includes several data integrity checks that run on every server startup:
|
||||
|
||||
|
||||
@@ -192,6 +192,10 @@ reason: `'displaced'` (new tab) | `'session_closed'`
|
||||
|------|--------|-------|
|
||||
| `federation_file_rejected` | messageId, dmChannelId, attachmentId, affectedUsers[] | DM members |
|
||||
|
||||
**S2S relay-only event (not a direct client WS event):**
|
||||
|
||||
`read_state_update` — sent peer-to-peer via the federation relay when a user acknowledges a DM channel on one instance. The receiving instance processes it, upserts the `read_states` row, and then emits a standard `channel_ack` event to the user's local WebSocket connections. The relay event itself is never forwarded to clients directly.
|
||||
|
||||
---
|
||||
|
||||
## Ready Payload
|
||||
|
||||
Reference in New Issue
Block a user