Commit Graph
786 Commits
Author SHA1 Message Date
Jannis Braun 514d46c82e feat(federation): add participant-aware fan-out routing for group DMs 2026-03-26 20:10:51 +01:00
Jannis Braun 0742c37ca1 refactor(federation): rename canonicalDmPairId to computeFederatedId 2026-03-26 20:08:37 +01:00
Jannis Braun 6f901f4a7a feat(shared): add group DM federation event types and payloads
Extend FederationRelayEvent with member_add, member_remove, and
ownership_transfer event types, plus federatedId and the new optional
membership/ownership/group payload fields. Add FederationMembershipPayload,
FederationOwnershipPayload, and FederationGroupPayload interfaces.
Add federatedId to FederationSyncRequest for group DM sync filtering.
2026-03-26 20:05:47 +01:00
Jannis Braun 3b16d9e2ce fix(schema): fix migration ordering hazard and FK guard for table rebuild
Remove federated_id from addMissingColumns so it no longer races with
migrateDmChannelsFederatedId on installs that have canonical_pair_id.
Wrap the table rebuild in PRAGMA foreign_keys = OFF/ON with a transaction
so the DROP TABLE succeeds even when dm_members/dm_messages hold FK refs.
2026-03-26 20:04:34 +01:00
Jannis Braun d9dc33bd36 feat(schema): rename canonical_pair_id to federated_id, add group DM columns
Renames dm_channels.canonical_pair_id → federated_id and adds
owner_home_user_id, owner_home_instance, deleted_at columns to support
federated group DMs. Migration handles upgrade from existing installs
(full table rebuild to rename), partial states (ALTER TABLE), and fresh
installs (correct DDL from the start). Backfills federated_id for
existing 1-on-1 DM channels and owner federation identity for group DMs.
2026-03-26 19:59:44 +01:00
Jannis Braun 742d851aae fix(federation): branching message lookup in reaction relay processors
Uses messageHomeInstance to determine whether the target message
originated locally (lookup by ID) or was replicated from another
instance (lookup by sourceInstance + sourceMessageId). Supports
N-instance federation and falls back to relay-sender lookup for
backward compatibility with in-flight outbox entries.
2026-03-26 18:49:28 +01:00
Jannis Braun 92be236251 fix(federation): use canonical message identity in reaction outbox payloads
Resolves canonical messageId (sourceMessageId for replicated messages,
local ID for originated messages) and includes messageHomeInstance so
the receiver can deterministically find the target message regardless
of which instance created it.
2026-03-26 18:41:18 +01:00
Jannis Braun c3f342a411 feat(shared): add messageHomeInstance to FederationRelayReaction type
Enables the receiver to deterministically identify which instance
created the message being reacted to, supporting N-instance federation.
2026-03-26 18:40:05 +01:00
Jannis Braun 22664a6a14 refactor(federation): extract getOurOrigin() into shared federationAuth export
Consolidates 4 inline constructions of the instance origin URL into a
single shared function. Removes the private copy in federationWorker
and two ad-hoc domainOrigin variables in federationOutbox.
2026-03-26 18:39:42 +01:00
Jannis Braun 10d4ca0110 chore: remove unused broadcastDmMessage import from federation.ts 2026-03-26 17:58:24 +01:00
Jannis Braun 6bf2a2621c fix(federation): skip broadcasting relay messages to source-instance users
When Instance 2 receives a relay from Instance 1, it no longer
broadcasts dm_message_created to members whose home instance is Instance
1. Those users already have the original message via their home WS
connection. This prevents duplicate messages from appearing in the
sender's chat.
2026-03-26 17:57:07 +01:00
Jannis Braun 523cb0c4b3 fix(federation): deduplicate relay DM messages in chatStore
When a user has federated WS connections, they can receive both the
original message (from their home instance) and the relay copy (from the
remote instance). The relay copy has a different message ID but carries
sourceMessageId pointing to the original. The dedup now checks both
directions: incoming relay copy vs existing original, and incoming
original vs existing relay copy.
2026-03-26 17:55:58 +01:00
Jannis Braun 62f1d637ac feat(federation): surface sourceMessageId/sourceInstance in DM message payloads
Both REST and WebSocket DM message payloads now include the relay
identity fields. This enables client-side deduplication of relay copies.
2026-03-26 17:54:24 +01:00
Jannis Braun f00dfe6e27 feat(shared): add sourceMessageId and sourceInstance to DmMessage type
These fields already exist in the dm_messages DB table but were not
surfaced to clients. Needed for federation relay deduplication.
2026-03-26 17:54:17 +01:00
Jannis Braun 6ccf7fedbb fix(federation): stop destructive dm_messages migration from re-running on every deploy
SQLite stores self-referencing FK identifiers with quotes (REFERENCES
"dm_messages"), but the migration guard only checked for the unquoted
string. This caused the migration to re-run on every server startup,
which DROP TABLE dm_messages and triggered ON DELETE CASCADE on
attachments.dm_message_id — wiping every DM-linked attachment.
2026-03-26 17:54:07 +01:00
Jannis Braun f362f5530c fix(federation): prevent duplicate DM sidebar entries from relay broadcasts
When a relay-created dm_message_created arrives for an unknown channel,
check if an existing DM has the same member (by homeUserId). If so,
route the message to the existing channel instead of adding a duplicate
sidebar entry. Prevents the transient duplicate that required page
refresh to dedup.
2026-03-26 14:45:04 +01:00
Jannis Braun a21243256e fix(federation): fix reaction relay using wrong ID for message lookup
Reaction outbox events used reactionId (add) or a composite dedup key
(remove) as the event messageId. The receiver looked up dm_messages by
sourceMessageId = event.messageId, which never matched because it was
searching for a reactionId, not the actual message ID.

Fix: include the actual DM messageId in the reaction payload JSON.
The receiver now uses event.reaction.messageId for the lookup, with
fallback to event.messageId for backward compatibility.
2026-03-26 14:27:48 +01:00
Jannis Braun 086195aa7c fix(federation): generate thumbnails for downloaded federated attachments
After the file worker downloads a federated attachment, generate a
thumbnail using the same sharp pipeline as local uploads. This ensures
federated images use bandwidth-efficient thumbnails in chat view
instead of serving the full-size image.

Also fixes processCreateEvent to not copy the source instance's
thumbnailFilename (which doesn't exist locally).
2026-03-26 06:46:14 +01:00
Jannis Braun 4ed9b30ce9 fix(federation): don't copy source thumbnail filename to relay attachments
The source instance's thumbnail file doesn't exist locally, so copying
its thumbnailFilename causes a 404 in the chat view. Set to null so
AttachmentRenderer falls back to the full image URL.
2026-03-26 06:37:26 +01:00
Jannis Braun ee8abc1022 fix(federation): file worker UPDATEs existing attachment rows + broadcasts
The file worker now UPDATEs the attachment row created by processCreateEvent
(swapping sourceUrl interim filename to local path) instead of INSERTing a
duplicate. Falls back to INSERT for legacy queue entries.

After download, broadcasts dm_message_updated so clients see the attachment
transition from remote hotlink to local file in real-time.
2026-03-26 06:23:12 +01:00
Jannis Braun 50f7a398dc fix(federation): create attachment rows immediately in processCreateEvent
Insert attachment rows with sourceUrl as interim filename so the initial
WebSocket broadcast includes working remote URLs. AttachmentRenderer
already handles http:// filenames as direct URLs. The file worker will
UPDATE the filename to the local path after download completes.

This fixes the invisible bubble bug where attachment-only messages
(content: null, attachments: []) appeared as empty messages.
2026-03-26 06:21:01 +01:00
Jannis Braun ca591024cf refactor(federation): replace outbox hooks in events.ts with queueDmRelay() 2026-03-26 06:17:51 +01:00
Jannis Braun b49be65dcd refactor(federation): replace outbox hooks in dm.ts with queueDmRelay() 2026-03-26 06:15:13 +01:00
Jannis Braun 58e6dbabcd feat(federation): add queueDmRelay() to consolidate relay payload construction
Single function builds complete relay payload with attachments (including
sourceUrl) and participants. Eliminates 4-way duplication that caused
the attachments: [] hardcode bug.
2026-03-26 06:12:56 +01:00
Jannis Braun cf9fcb78ed fix(federation): add participants array to relay events and fix recipient resolution
The relay was failing because processCreateEvent relied on the friends
table to discover the DM recipient, but friendships aren't federated
across instances. Also, resolveLocalUser matched deleted replicated
users before active ones.

- Add participants[] to FederationRelayEvent with homeUserId/homeInstance
  for all DM channel members
- Add getDmParticipants() helper to look up member identities
- Include participants in outbox payloads (create/update) and sync events
- Rewrite processCreateEvent to resolve participants directly, compute
  canonicalDmPairId, and findOrCreateDmChannel — removing the entire
  friends-list fallback (60+ lines)
- Fix resolveLocalUser to filter out deleted users (is_deleted = 0)
  and prefer the replicated user match when multiple candidates exist
2026-03-26 05:36:41 +01:00
Jannis Braun cb9a70d7a6 fix(federation): use canonical_pair_id for sync scope instead of home_instance matching
The old query missed channels where both members appear as 'local'
(e.g., native Nova + native youruser on orbit). Using
canonical_pair_id is simpler and correct — any channel with a pair
ID is a federated DM that should be synced.
2026-03-26 04:41:00 +01:00
Jannis Braun 49f88c2552 fix(federation): actually remove source_instance IS NULL filter from sync SQL queries 2026-03-26 04:37:47 +01:00
Jannis Braun 277ff0e50c fix(federation): sync ALL locally-created messages, not just local-home-user messages
Messages by replicated users (e.g., Jannis browsing orbit) have
source_instance IS NULL because they were created on that instance.
The sync endpoint was incorrectly filtering these out, causing most
DM history to be missing from the sync response.
2026-03-26 04:34:13 +01:00
Jannis Braun ae004b72e1 feat(federation): trigger checkpoint sync on startup for peers that have never been synced 2026-03-26 04:30:54 +01:00
Jannis Braun a3d41fd568 fix(federation): handle existing canonical_pair_id during backfill by merging duplicate first 2026-03-26 04:25:03 +01:00
Jannis Braun 306e26a525 fix(federation): remove require('crypto') — already imported as ESM at top of file 2026-03-26 04:22:08 +01:00
Jannis Braun 960ca1c97f fix(federation): set correct homeInstance in relay payload and fix canonical backfill SQL
- buildRelayPayload now uses config.domain for local users instead of
  empty string, so the relay receiver can resolve the user
- Fixed canonical_pair_id backfill: SQLite NULL || ':' || x = NULL,
  so use COALESCE(home_user_id, id) instead of concatenation
2026-03-26 04:19:18 +01:00
Jannis Braun 4a109f71da fix(federation): backfill canonical_pair_id and merge duplicate DM channels
Existing 1-on-1 DM channels created before federation had canonical_pair_id = NULL,
causing the relay to create duplicate channels on receipt of a message. On server
startup, migrations now: (1) compute and set canonical_pair_id for all existing
1-on-1 channels using home_user_id (falling back to regular id for local users),
and (2) merge any duplicate channels that share the same canonical_pair_id by
moving all messages, read states, and members into the oldest channel and deleting
the stale duplicate. Both steps are non-fatal and idempotent.
2026-03-26 04:15:12 +01:00
Jannis Braun 73a4b69fe6 fix(federation): auto-peering on reconnect and DM deduplication across origins 2026-03-26 03:59:59 +01:00
Jannis Braun b34732e064 fix(federation): enable relay by default — it fixes a bug, not an optional feature
Federation relay was incorrectly defaulting to disabled, requiring
admins to discover a buried settings toggle. Since federation itself
is opt-in (you connect instances manually), relay should be on by
default. Also migrates existing instances from 0 → 1.
2026-03-26 03:54:20 +01:00
Jannis Braun c5188c1702 fix(federation): fix TypeScript strict mode error in parseFederationHeaders 2026-03-25 21:51:35 +01:00
Jannis Braun 46e6f8a7e2 feat(federation): add cleanup functions for outbox, mutation log, and file queue TTL 2026-03-25 21:48:46 +01:00
Jannis Braun 44de9b6f41 feat(federation): add client peering integration, transparency indicator, and admin federation settings
Task 13: Hook server-to-server peering into connectToRemote (non-fatal)
and add federation API namespace to the client (initiate, peers, revoke).

Task 14: Show a transparency notice in the DM welcome header when the
other user is on a remote instance, informing that messages are stored
on both home instances and are not end-to-end encrypted.

Task 15: Add Federation section to the instance settings General panel
with DM relay toggle, TTL config, and a live peer list with revoke
buttons. Also extends InstanceAdminSettings type and the server settings
route to support federationRelayEnabled / federationRelayTtlDays.
2026-03-25 21:47:10 +01:00
Jannis Braun 3898e4940a feat(federation): add outbox delivery, file queue download, and health check workers 2026-03-25 21:39:32 +01:00
Jannis Braun 7f20a4517c feat(federation): implement checkpoint sync endpoint using mutation log
Add POST /api/federation/sync for peer instances to catch up on missed
DM mutations after downtime. Uses HMAC-SHA256 authentication (same as
relay), determines shared channels between peers via raw SQL join on
dm_members/users, queries the federation_mutation_log with timestamp
pagination, and returns FederationRelayEvent payloads for each mutation
type (create, update, delete, reaction_add, reaction_remove).
2026-03-25 21:33:47 +01:00
Jannis Braun 5b9272e22b feat(federation): implement relay receive endpoint with dedup, channel mapping, SSRF validation 2026-03-25 21:27:28 +01:00
Jannis Braun fbdec82172 feat(federation): add peering handshake and admin peer management endpoints
Add federation routes module with four endpoints:
- POST /api/federation/peer/initiate — admin-only, starts peering handshake
  with a remote instance (validates origin, generates HMAC secret, sends
  server-to-server accept request with 10s timeout)
- POST /api/federation/peer/accept — server-to-server first contact,
  rate-limited to 10 req/min per IP, creates/activates peer record
- GET /api/federation/peers — admin-only, lists all peers (hmacSecret excluded)
- DELETE /api/federation/peers/:id — admin-only, revokes peer and cleans outbox
2026-03-25 21:19:47 +01:00
Jannis Braun 32a0c2e618 feat(federation): hook outbox and mutation log into DM message and reaction handlers
Wire appendMutationLog + queueOutboxEvent + buildRelayPayload into all
DM mutation paths so federation peers receive relay events:

- REST: POST /api/dm/:id/messages, PATCH /api/dm/messages/:id,
  DELETE /api/dm/messages/:id
- WebSocket: dm_message_create, dm_message_edit, dm_message_delete,
  reaction_add (DM path), reaction_remove (DM path)
- Fix buildRelayPayload parameter types to accept optional replyToId
  and editedAt (matching DmMessageWithUser's optional fields)
2026-03-25 21:14:34 +01:00
Jannis Braun 88ad1c676d feat(federation): add outbox coalescing writer, mutation log, and relay helpers 2026-03-25 21:07:16 +01:00
Jannis Braun 2ff53a3f9b feat(federation): add HMAC signing and verification utilities
Implements federationAuth.ts with generateHmacSecret, signRequest,
verifySignature (constant-time, 15-minute replay window), buildFederationHeaders,
and parseFederationHeaders for server-to-server federation authentication.
2026-03-25 21:03:03 +01:00
Jannis Braun 6afddab54e feat(federation): add shared types for relay events, sync, and peer management 2026-03-25 21:01:26 +01:00
Jannis Braun 4726eeb097 feat(federation): add migrations for federation tables, indexes, and mutation log backfill
Adds column migrations (dm_channels.canonical_pair_id, dm_messages.source_*,
attachments.source_url, instance_settings.federation_relay_*), CREATE TABLE
for federation_peers/outbox/file_queue/mutation_log, partial/filtered indexes,
and an idempotent backfill that seeds the mutation log from existing DM messages.

Also makes migrateDmMessagesReplyToFk and migrateAttachmentsDmMessageFk
column-aware so table recreation preserves any columns added by earlier
migrations (prevents dropping federation columns on upgrade).
2026-03-25 21:00:10 +01:00
Jannis Braun c80d565f7f feat(federation): add federation tables and columns to schema 2026-03-25 20:53:43 +01:00
Jannis Braun a1363a1ab3 fix: match GIF-only message size to standard media constraints
GIFs from Klipy were capped at 250×250px while all other visual media
(attachments, image embeds) used 400×300px. Unified to 400×300px.
2026-03-25 14:42:27 +01:00
Jannis Braun f7fa7371d6 fix: hide Open Original when sourceUrl is present to avoid redundancy 2026-03-25 05:13:14 +01:00