Commit Graph
174 Commits
Author SHA1 Message Date
Jannis Braun 017256162b feat(server): add homeInstance to authenticate, add requireLocalUser guard 2026-04-02 10:45:50 +02:00
Jannis Braun b3011fb3da feat(federation): add fire-and-forget S2S typing relay
sendTypingRelay() mirrors sendCallRelay() — direct POST to peers,
no outbox, no retry. Uses federatedId for cross-instance channel
identification. Wired into handleDmTypingStart() for typing_start
and broadcastDmMessage() for typing_stop.
2026-04-01 12:53:55 +02:00
Jannis Braun aff2a403d7 fix(federation): reduce outbox worker interval from 10s to 1s for lower relay latency 2026-03-31 23:42:31 +02:00
Jannis Braun 75b84c6ecb feat(federation): add sendCallRelay for direct S2S delivery (FED-009) 2026-03-31 23:22:06 +02:00
Jannis Braun abdaf99bb4 fix(federation): address code review findings for FED-011
- Fix race window: store pendingHmacSecret AFTER remote peer confirms,
  not before (admin endpoint + auto-rotation worker)
- Add hex validation on newSecret at /peer/rotate endpoint
- Use pending-secret-aware signing in initial sync worker
- Add test for corrupt state (pendingHmacSecret set, secretRotationAt null)
2026-03-31 21:01:02 +02:00
Jannis Braun 0c09a58d61 feat(federation): add grace period finalization and auto-rotation to health worker (FED-011) 2026-03-31 20:49:15 +02:00
Jannis Braun 1f6605b801 feat(federation): outbox worker signs with pending secret during rotation (FED-011) 2026-03-31 20:48:00 +02:00
Jannis Braun dcf0bc1857 feat(federation): add verifyPeerSignature with dual-secret grace period (FED-011) 2026-03-31 20:40:38 +02:00
Jannis Braun d0ed43cf58 fix(federation): add nonce length validation and fix verification flow docs (FED-008) 2026-03-31 18:18:50 +02:00
Jannis Braun d7c30f2c50 feat(federation): add nonce to HMAC sign/verify and header build/parse (FED-008)
- signRequest now accepts optional nonce; payload becomes `${timestamp}.${nonce}.${body}` when present, falling back to `${timestamp}.${body}` for legacy peers
- verifySignature accepts matching nonce parameter and passes it through
- buildFederationHeaders generates a UUID nonce per request and includes X-Federation-Nonce header
- parseFederationHeaders extracts X-Federation-Nonce into nonce field (null when absent)
2026-03-31 18:07:19 +02:00
Jannis Braun 2cac39a460 fix(federation): replace DNS hairpin self-POST with direct function call (FED-005)
Extract processRelayEvents() from the relay HTTP handler and call it
directly in runInitialSyncForNewPeers(), eliminating the HTTP round-trip
through public DNS that failed on networks without hairpin NAT.
2026-03-31 05:20:16 +02:00
Jannis Braun 70678ff198 fix(federation): batch A production readiness — normalization, logging, security
FED-001: normalize homeInstance in processCreateEvent member skip
FED-002: normalize homeInstance in getFriendEventTargets
FED-003: normalize homeInstance in handleSizeRejection
FED-004: add warning log when queueOutboxEvent drops events (zero peer match)
FED-012: remove unused challenge from peer handshake
FED-013: reject non-HTTPS origins in validateOrigin (except localhost)
2026-03-31 03:33:22 +02:00
Jannis Braun 44b6317c16 fix: federation relay for group DMs — origin normalization, missing federatedId, local-only broadcasts
Three stacked bugs prevented federation relay from working for group DMs:

1. Origin format mismatch: users.home_instance stores bare domains
   ("nova.ddns.net") but federation_peers.origin stores full URLs
   ("https://nova.ddns.net"). getGroupDmTargetOrigins() built target
   lists from bare domains, so queueOutboxEvent() never matched any
   peers — events were never queued into the outbox.

2. Missing federatedId in outbox reconstruction: the outbox worker
   rebuilt relay events from stored payloads but never copied the
   federatedId field. Receiving instances check this field and rejected
   all member_add/remove/ownership_transfer events with
   "missing_membership_payload".

3. Duplicate channels from cross-instance broadcasts: dm_channel_created
   was sent to ALL members including remote replicas. Users connected to
   multiple instances received the event twice (once per instance),
   creating duplicate group DMs in their sidebar. Fixed by only
   broadcasting to members whose home instance matches the current
   server — remote members receive the channel via federation bootstrap
   on their home instance.
2026-03-27 16:53:35 +01:00
Jannis Braun ceb0c9812d fix(federation): resolve code review issues in upload size mismatch
- Fix critical: outbox worker now copies file_rejected payload fields
  (attachmentId, sourceFilename, rejectionReason, rejectionLimit,
  affectedUserIds) so the reverse relay actually delivers them
- Fix: add sourceFilename to file_rejected event for reliable
  multi-attachment matching on the sender side
- Fix: change text-accent-warning to text-accent-amber (valid class)
- Add Array.isArray guard on federationMeta parse
2026-03-27 04:51:30 +01:00
Jannis Braun 1990b90022 feat(federation): send file_rejected reverse relay on size limit exceeded 2026-03-27 04:37:27 +01:00
Jannis Braun 7125c62a1f feat(federation): store remote peer upload size limit from relay response 2026-03-27 04:34:57 +01:00
Jannis Braun 0d6bb9dbd3 fix(federation): auto-create and hydrate user profiles during DM relay
Federated 1-on-1 DMs showed the raw snowflake ID as the display name
and no avatar when the remote user had no pre-existing local record.
processCreateEvent used resolveLocalUser (find-only) instead of
resolveOrCreateReplicatedUser, and relay events carried no profile data
for participants.

- Add profile snapshot (displayName, avatar, avatarColor) to
  FederationRelayParticipant and populate it in getDmParticipants
- Change processCreateEvent to auto-create replicated user stubs and
  hydrate them with profile data from the relay event
- Fix hydrateReplicatedUserProfile URL resolution for homeInstance
  values without protocol prefix
- Fix WelcomeHeader: return null while DM data is loading (eliminates
  "unknown" flash on reload), use displayName for @mention text
2026-03-27 03:43:45 +01:00
Jannis Braun 2a741a0dc7 refactor(federation): update worker and janitor for generalized outbox columns
Replace dmChannelId/messageId column references with contextId/entityId/contextType in
federationWorker outbox delivery, spread all payload fields (membership, ownership, group,
friendship), add friend-context initial sync pass, and fix storageJanitor DM purge queries.
2026-03-27 00:46:16 +01:00
Jannis Braun fe3d8b06d8 refactor(federation): generalize outbox function params and add friend relay helpers
Rename appendMutationLog and queueOutboxEvent params from DM-specific names
(dmMessageId/dmChannelId/messageId) to generic (entityId/contextId) with a new
contextType param defaulting to 'dm'. Update all internal schema column references
to match the renamed outbox/mutation-log schema columns. Add buildFriendContextId
and getFriendEventTargets helper functions for friend event relay routing.
2026-03-27 00:43:51 +01:00
Jannis Braun 4efa35f311 feat(federation): add soft-delete GC for empty group DMs with 24h grace period
Replace the hard-delete in the leave handler with a soft-delete (sets
deleted_at timestamp) when the last member leaves a group DM. A new
janitor sweep in the federation worker runs hourly and purges channels
whose grace period has expired, cascading through reactions, embeds,
attachments, messages, members, outbox/mutation-log/file-queue entries,
and finally the channel itself.

All client-facing dm_channels queries now filter on deleted_at IS NULL
to hide soft-deleted channels from the REST API and WebSocket ready
payload.
2026-03-26 20:35:55 +01:00
Jannis Braun f7ba391678 feat(federation): handle group DM messages via federatedId lookup in relay processor
In processCreateEvent, branch on event.federatedId: group DM messages now
look up the pre-bootstrapped local channel by federatedId instead of
computing a pair hash from two participants. In queueDmRelay, fetch the
channel's federatedId and ownerId and include federatedId in the outgoing
relay payload for group DMs so receiving instances can route correctly.
2026-03-26 20:26:23 +01:00
Jannis Braun 514d46c82e feat(federation): add participant-aware fan-out routing for group DMs 2026-03-26 20:10:51 +01:00
Jannis Braun 0742c37ca1 refactor(federation): rename canonicalDmPairId to computeFederatedId 2026-03-26 20:08:37 +01:00
Jannis Braun 22664a6a14 refactor(federation): extract getOurOrigin() into shared federationAuth export
Consolidates 4 inline constructions of the instance origin URL into a
single shared function. Removes the private copy in federationWorker
and two ad-hoc domainOrigin variables in federationOutbox.
2026-03-26 18:39:42 +01:00
Jannis Braun 086195aa7c fix(federation): generate thumbnails for downloaded federated attachments
After the file worker downloads a federated attachment, generate a
thumbnail using the same sharp pipeline as local uploads. This ensures
federated images use bandwidth-efficient thumbnails in chat view
instead of serving the full-size image.

Also fixes processCreateEvent to not copy the source instance's
thumbnailFilename (which doesn't exist locally).
2026-03-26 06:46:14 +01:00
Jannis Braun ee8abc1022 fix(federation): file worker UPDATEs existing attachment rows + broadcasts
The file worker now UPDATEs the attachment row created by processCreateEvent
(swapping sourceUrl interim filename to local path) instead of INSERTing a
duplicate. Falls back to INSERT for legacy queue entries.

After download, broadcasts dm_message_updated so clients see the attachment
transition from remote hotlink to local file in real-time.
2026-03-26 06:23:12 +01:00
Jannis Braun 58e6dbabcd feat(federation): add queueDmRelay() to consolidate relay payload construction
Single function builds complete relay payload with attachments (including
sourceUrl) and participants. Eliminates 4-way duplication that caused
the attachments: [] hardcode bug.
2026-03-26 06:12:56 +01:00
Jannis Braun cf9fcb78ed fix(federation): add participants array to relay events and fix recipient resolution
The relay was failing because processCreateEvent relied on the friends
table to discover the DM recipient, but friendships aren't federated
across instances. Also, resolveLocalUser matched deleted replicated
users before active ones.

- Add participants[] to FederationRelayEvent with homeUserId/homeInstance
  for all DM channel members
- Add getDmParticipants() helper to look up member identities
- Include participants in outbox payloads (create/update) and sync events
- Rewrite processCreateEvent to resolve participants directly, compute
  canonicalDmPairId, and findOrCreateDmChannel — removing the entire
  friends-list fallback (60+ lines)
- Fix resolveLocalUser to filter out deleted users (is_deleted = 0)
  and prefer the replicated user match when multiple candidates exist
2026-03-26 05:36:41 +01:00
Jannis Braun ae004b72e1 feat(federation): trigger checkpoint sync on startup for peers that have never been synced 2026-03-26 04:30:54 +01:00
Jannis Braun 960ca1c97f fix(federation): set correct homeInstance in relay payload and fix canonical backfill SQL
- buildRelayPayload now uses config.domain for local users instead of
  empty string, so the relay receiver can resolve the user
- Fixed canonical_pair_id backfill: SQLite NULL || ':' || x = NULL,
  so use COALESCE(home_user_id, id) instead of concatenation
2026-03-26 04:19:18 +01:00
Jannis Braun c5188c1702 fix(federation): fix TypeScript strict mode error in parseFederationHeaders 2026-03-25 21:51:35 +01:00
Jannis Braun 46e6f8a7e2 feat(federation): add cleanup functions for outbox, mutation log, and file queue TTL 2026-03-25 21:48:46 +01:00
Jannis Braun 3898e4940a feat(federation): add outbox delivery, file queue download, and health check workers 2026-03-25 21:39:32 +01:00
Jannis Braun 32a0c2e618 feat(federation): hook outbox and mutation log into DM message and reaction handlers
Wire appendMutationLog + queueOutboxEvent + buildRelayPayload into all
DM mutation paths so federation peers receive relay events:

- REST: POST /api/dm/:id/messages, PATCH /api/dm/messages/:id,
  DELETE /api/dm/messages/:id
- WebSocket: dm_message_create, dm_message_edit, dm_message_delete,
  reaction_add (DM path), reaction_remove (DM path)
- Fix buildRelayPayload parameter types to accept optional replyToId
  and editedAt (matching DmMessageWithUser's optional fields)
2026-03-25 21:14:34 +01:00
Jannis Braun 88ad1c676d feat(federation): add outbox coalescing writer, mutation log, and relay helpers 2026-03-25 21:07:16 +01:00
Jannis Braun 2ff53a3f9b feat(federation): add HMAC signing and verification utilities
Implements federationAuth.ts with generateHmacSecret, signRequest,
verifySignature (constant-time, 15-minute replay window), buildFederationHeaders,
and parseFederationHeaders for server-to-server federation authentication.
2026-03-25 21:03:03 +01:00
Jannis Braun 123e40bb9a feat(server): populate embed width/height from OG tags and image probing 2026-03-25 01:53:39 +01:00
Jannis Braun bbb2c8bd19 feat(server): extract og:image:width/height from HTML metadata 2026-03-25 01:51:31 +01:00
Jannis Braun 2f883c6f63 refactor(server): extract SSRF validation into shared ssrf.ts utility 2026-03-25 01:48:18 +01:00
Jannis Braun ee11408060 feat: add age-based media cleanup endpoint
Adds cleanupOldMedia() to the storage janitor and a new endpoint
POST /api/admin/storage/cleanup-media { maxAgeDays, dryRun }.
Deletes chat attachments older than the specified threshold while
preserving profile images.
2026-03-23 02:20:28 +01:00
Jannis Braun d687988ba8 fix: use explicit counter for danglingFilesOnDisk in referencedFiles calc
Matches the orphanedFiles counting pattern and avoids a subtle
discrepancy if two dangling records shared the same filename.
2026-03-23 01:53:25 +01:00
Jannis Braun b40999995b feat: detect and clean up dangling attachment records in storage janitor
Adds getDanglingAttachments() to find attachment records whose
messageId/dmMessageId points to a deleted message. Updates stats
to show dangling count and cleanup to delete them.
2026-03-23 01:49:41 +01:00
Jannis Braun bfe317f465 feat: interleaved category override resolution in permission engine 2026-03-21 18:22:22 +01:00
Jannis Braun d0f33446de feat: add video thumbnail generation and media metadata extraction 2026-03-21 17:16:40 +01:00
Jannis Braun bc408235c9 feat: add Activity type system, DB migration, and self-only showActivity in sanitizeUser
- Add Activity, ActivityType, ActivityTimestamps, ActivityAssets types to shared types
- Add activity_update client event and activities field on presence_update server event
- Add userActivities to ready payload and showActivity to User/UpdateUserRequest
- Create shared activities.ts with ACTIVITY_LIMITS, ACTIVITY_PRIORITY, getPrimaryActivity
- Add show_activity column to users table (schema + migration)
- Update sanitizeUser with isSelf parameter; only include showActivity for self
- Fix .map(sanitizeUser) calls to use arrow wrapper to prevent index-as-boolean bug
- Mark auth routes (register/login) as isSelf=true since they return own user data
2026-03-21 01:38:36 +01:00
Jannis Braun ce981ee5fc fix: prevent metadata fetch from overwriting fallback thumbnail with null
Use logical OR when merging fetched metadata so null values from
failed og: extraction don't destroy valid defaults (e.g. YouTube
thumbnail set before the fetch).
2026-03-21 00:35:44 +01:00
Jannis Braun 1c7d8557c6 fix: use predictable YouTube thumbnail URL as fallback for embed poster
YouTube may block og:image extraction via BackspaceBot user-agent.
Use img.youtube.com/vi/{ID}/hqdefault.jpg as a guaranteed thumbnail
that doesn't require metadata fetching. If og:image is successfully
fetched, it overwrites the fallback.
2026-03-21 00:31:59 +01:00
Jannis Braun c0133397e3 fix: resolve 4 media embed bugs from testing
1. Video aspect ratio: remove container border/overflow-hidden, use
   preload="metadata" so browser knows dimensions before play
2. YouTube Error 153: remove sandbox attr (too restrictive), add full
   allow permissions (encrypted-media, accelerometer, gyroscope, etc.)
3. Google Images not displaying: detect image Content-Type from HTTP
   response in metadataFetcher, override classifier to create image
   embed for URLs that serve image/* content
4. Audio seeking broken: add HTTP Range request support in uploads
   route (Accept-Ranges, Content-Range, 206 Partial Content)
2026-03-21 00:19:04 +01:00
Jannis Braun ff2decded5 fix: address code review issues for embeds implementation
- Fix ?? to || in metadataFetcher.ts to handle empty strings from Cheerio
- Fix stale embeds on message edit: delete old embeds synchronously before
  broadcast, then resolve new ones async (all 4 edit paths: REST+WS, msg+DM)
- Revert unrelated MessageList.tsx scroll threshold change (5000 not 150)
- Remove duplicate embed indexes from migrateAddIndexes (kept standalone ones)
2026-03-20 23:58:06 +01:00
Jannis Braun 01ee7ab67a feat: wire embeds into all message query and creation paths
Integrate embed infrastructure into the complete message flow:
- messages.ts: batch-fetch embeds in GET, resolve on POST, re-resolve on PATCH
- dm.ts: same pattern for DM messages with isDm=true
- search.ts: include embeds in all 4 search/around endpoints
- events.ts: embed resolution in WS message create/edit for both space and DM
- Fix embedClassifier.ts type errors (regex match undefined → null)
- Add embeds: [] to all inline MessageWithUser/DmMessageWithUser constructions
- Add embeds: [] to chatStore optimistic message
2026-03-20 23:41:53 +01:00