config.ts requires JWT_SECRET at module-load time. Since `.env` is gitignored
it doesn't propagate to git worktrees, fresh clones, or CI without secrets —
producing 200+ confusing cascade failures (vitest module-mock errors that
look unrelated to the real cause).
Add a vitest setupFile that sets JWT_SECRET to a fixed test-only value via
`??=`, so a real .env still wins where one exists. Tests are now hermetic:
clone the repo, `pnpm install`, `pnpm --filter server test` → 477/477 pass.
Two follow-on bugs from the initial S2S presence rollout:
(1) New friend stuck offline until they reload: presence_update fires only on
transitions, so a remote user already online when their stub is created
locally never receives a relay event seeding their actual status. The
stub defaulted to 'offline' at creation and stayed there until the next
transition. Fix: extend FederationRelayProfileSnapshot +
FederationUserLookupProfile with status. Sender-side buildProfileSnapshot,
getDmParticipants, and lookup endpoint responses populate it for native
users only (replicated stubs hold stale status owned elsewhere).
resolveOrCreateReplicatedUser uses hints.status to seed the new row's
status column. Threaded through every call site (DM participants, group
bootstrap, friend events, ownership transfer). Stub backfill worker also
heals existing rows whose status was stuck at 'offline' from creation.
(2) 'Online' text updates but green avatar dot stays grey on the same page:
spaceStore.updateMemberPresence patches members[] (which feeds space UIs)
but never patches userViews — the cache useCanonicalUserView reads from.
The Avatar in FriendItem reads canonical.status; the text reads
friend.status (socialStore). Two sources, one stale until full
user_updated arrives. Fix: updateMemberPresence now mirrors status into
matching userViews entries, so canonical-view consumers re-render with
fresh status the moment the WS event lands.
The outbox worker rebuilds FederationRelayEvent objects from stored JSON via
an allowlist of known fields. presenceUpdate was missed when presence_update
events shipped, so peers received events with eventType='presence_update' but
no payload — rejected with missing_presence_update_payload on every tick.
onPeerActivated re-emits a presence_update for every relationship-related
online native to the activating peer (relationship = friend with peer-stub /
DM-mate with peer-stub / replicatedInstances opt-in for peer origin). Scope
bounded by relationship count, not native count — flap recovery cost stays
proportional to actual interaction surface.
onPeerDeactivated flips every replicated stub from that peer to offline and
broadcasts a local presence_update so connected friends/DM-mates/space-co-members
see them disappear immediately, instead of seeing stale online until next signal.
Re-snapshot on every activation (incl. health-check unreachable→active flap)
is load-bearing for correctness — markPeerStubsOffline ran on the prior
deactivation and presence is not in the mutation log.
Six WS sites that previously broadcast presence_update to spaces only now use
collectProfileBroadcastTargetIds (the same recipient set as user_updated):
- ws/handler.ts finalizeDisconnect (offline)
- ws/handler.ts auth path (online)
- ws/events.ts handlePresenceUpdate (manual idle/dnd/online)
- ws/events.ts handleActivityUpdate (rich activity changes)
- routes/users.ts showActivity-toggle clear
- routes/users.ts status PATCH
Friends with no shared space + DM-only co-members now see each other's
online/offline transitions live, matching user_updated semantics. Federated
stub presence broadcasts (Task B3) use the same helper, so cross-instance
recipients are uniform.
Updates one assertion in social.federated.test.ts that asserted the old
snowflake-style stub username (now realname-based per A1).
processPresenceUpdateEvent updates the local stub's status and broadcasts a
WS presence_update to friends + DM members + space co-members of that stub
via collectProfileBroadcastTargetIds. Closes the doc/code drift in
activity-presence.md:147 — federated stubs now have their status projected
by the home instance as documented.
Strict attribution: payload.homeInstance domain must equal source peer
domain. Silently no-ops when no local replica exists (peer broadcast fanout
covers all peers, not all hold a stub).
New FederationPresenceUpdatePayload + queuePresenceRelay() helper. Five WS
sites now project the native user's status (and optional activities) to all
active peers via the outbox: WS auth-success, finalizeDisconnect,
manual presence_update, activity_update, showActivity-toggle clear.
Outbox-only (no mutation-log entry) — presence is ephemeral; the upcoming
peer-activation hook re-emits a fresh snapshot so peers recovering from
unreachable converge without history replay. No-op for replicated users.
Heals existing legacy stubs (created when resolveOrCreateReplicatedUser used
homeUserId@domain) by asking the peer for the canonical username via
lookupRemoteUserByHomeId and rewriting the local row. Idempotent and
collision-safe.
Wired into onPeerActivated (per-origin) so future peer flaps re-attempt for
stubs whose home was unreachable on a prior pass, and into a one-shot pass at
startupBootstrapSync for all currently-active peers (not just first-time
lastSyncedAt=0 peers).
HMAC-authenticated, rate-limited (60/min/peer) endpoint that resolves a
homeUserId on this instance to its canonical username + profile snapshot.
Native non-deleted users only. Mirrors /users/lookup's auth shape.
Adds lookupRemoteUserByHomeId to federationLookup.ts as the client-side
helper. Used by the upcoming stub-backfill worker on peers that hold legacy
snowflake-named replicas of users now visible by their real handle.
FederationProfileUpdatePayload gains `username`: the home user's canonical
handle. Receiver applies displayName ?? username so stubs whose home user has
no displayName show the real handle instead of getting clobbered to null.
Mirrors the existing fallback in hydrateReplicatedUserProfile. Username itself
is immutable on the home instance, so the receiver does not rewrite the stub's
username column on profile_update.
When friend_request_create / friend_add / DM relay carries a profile snapshot,
the canonical-username hint is now used as the stub's local-part. Stubs created
purely from S2S (no client-federation) now display the human-readable handle,
not the homeUserId snowflake. Falls back to the snowflake-id scheme only when
no hint is available.
spawnInstance now sets DISABLE_RATE_LIMITS=1 by default so unrelated tests
don't exhaust the shared 127.0.0.1 per-IP bucket. bootHomePlusRemotes/
bootTwoInstances accept an optional { enableRateLimits } that omits the env
for tests that need real enforcement, exposed via the explicit
bootTwoInstancesWithRateLimits() helper for Test #15 (rate-limit assertion).
Overloading NODE_ENV='test' to silently disable rate limiting layered a
second meaning onto an env that already gates the test-only seed-peer
route. A dedicated DISABLE_RATE_LIMITS env (envBool semantics, matching
DISABLE_FEDERATION_WORKERS) makes intent explicit, defaults off in
production, and leaves room for tests that need to assert real rate-limit
behaviour to opt back in by simply not setting the var.
Also fixes per-IP rate limit exhaustion in test environments: @fastify/rate-limit
v9 has no skip(); use allowList(() => NODE_ENV==='test') which propagates to
per-route overrides via mergeParams Object.assign merge.
The original peerInstances comment framed the two-row pattern as a
band-aid for getOurOrigin's https://${DOMAIN} default. After
investigating a clean collapse to one row (PUBLIC_ORIGIN override on
each spawned instance), the deeper coupling surfaces:
- extractDomain() strips port via new URL().hostname, so unique-port
localhost instances all share hostname '127.0.0.1' and the
receiver's attribution guard
extractDomain(user.homeInstance) === extractDomain(fedHeaders.origin)
becomes ambiguous in any multi-remote configuration.
- The homeInstance validator regex /^[a-zA-Z0-9._-]+$/ in auth.ts
rejects ':', so the port cannot be encoded into homeInstance to
disambiguate.
- Eliminating the second row would require a production refactor of
extractDomain (port-preserving), the attribution check (decoupled
from URL), or the homeInstance validator (allow ':') — all out of
scope.
So the harness DELIBERATELY keeps DOMAIN as a per-instance human label
('home.test.local' / 'remoteN.test.local') for stable identity, and the
two peer rows per direction (transport URL + getOurOrigin URL) are
structural to localhost-port test reality, not a band-aid. Comment
rewritten to reflect this. PUBLIC_ORIGIN remains available in
production code for reverse-proxy / dev-without-TLS deployments.
Adds an explicit override for the federation transport URL returned by
getOurOrigin(). When unset, behaviour is unchanged (https://${DOMAIN} ->
http://localhost:${PORT} dev fallback). Intended for reverse-proxy /
dev-without-TLS deployments where the public origin must be advertised
explicitly (typically http://...) and differs from the bare DOMAIN
value used for federated identity.
Wired via config.publicOrigin (envOptional('PUBLIC_ORIGIN')) so the
override flows through the existing config layer rather than scattering
process.env reads. Trailing slash is stripped for symmetry with
peer.origin storage.
docs/systems/federation.md gets a "Public Origin Override" subsection
under §14 Background Workers documenting the resolution order.
Adds setupFullDeletionFixture (federated user joins remote space, authors 2
messages with reactions, opens 1-on-1 DM with a live observer). Tests #3 (soft
mode: tombstone + dm_members cleared, messages/reactions retained) and #5 (full
mode: tombstone + messages/reactions purged, surviving 1-on-1 DM channel).
Also fixes seedPeer to install both the URL form (outbound lookup on sender)
and the DOMAIN-claim form (inbound auth on receiver) — required because the
test harness's ephemeral http://127.0.0.1 origin and DOMAIN-derived
getOurOrigin() return different strings, while production has them coincide.
This was latent: test #1 (leave mode) skips S2S, so #3 was the first test to
actually exercise the S2S delete path and surfaced the dual-origin gap.
Also fix dbInspect.ts UserRow column aliases: SELECT * returns snake_case
columns (is_deleted, display_name, etc.) but UserRow expected camelCase.
Switch to explicit aliased SELECT so all callers get the documented interface.
hydrateReplicatedUserProfile now calls downloadProfileAsset and stores bare local filenames, falling back to absolute URLs only on download failure. It also fills empty fields only — no longer clobbering local files written by processProfileUpdateEvent. Adds an idempotent startup backfill that converts existing http-prefixed avatar/banner rows on replicated users into local files, so federated profile pictures keep rendering when the home instance is offline.
Adds an admin-driven sweep on top of the existing 24h auto-expire so
operators can see and reap abandoned `.tus/` sessions without waiting.
- storageJanitor: extract `walkTusDir(predicate)` helper, add
`getStaleTusInfo` + `cleanupStaleTusSessions(thresholdMs, dryRun)`;
refactor `cleanupTusStragglers` to delegate while preserving its
janitor-tick `{ removed }` contract.
- StorageStats gains `staleTusSessions` + `staleTusSize` (fixed 1h
display threshold).
- New `POST /api/admin/storage/cleanup-tus` route with
`maxAgeHours` validation (positive finite number, default 1) and
`dryRun` support; admin-gated.
- StoragePanel: 6th overview card "Stale Uploads" + new cleanup
subsection mirroring the media-cleanup pattern (preview-then-clean
with shared result panel styling).
- Tests: 8 new janitor tests covering empty dir, threshold filtering,
dry-run vs live, oldest-mtime tracking, subdir skipping, and the
override path on the existing straggler sweep. New
`routes/admin.test.ts` covers auth/admin gates, validation (zero,
negative, NaN), default `maxAgeHours`, dry-run vs live unlink.
- Docs: `uploads.md` §Janitor expanded to the full lifecycle (cancel
DELETE, discard DELETE, auto-expire, straggler sweep, admin route);
`admin.md` Storage Management updated with the new endpoint and
StorageStats fields.
Migrates the remaining 5 profile/space upload sites (CreateSpace, AccountPanel
avatar+banner, OverviewPanel icon+banner) to transferStore.startUpload with
tray:false. Space sites pass _instanceOrigin so uploads route to the space's
home instance.
Removes upload/uploadWithProgress from api.uploads (and their private
uploadFile/uploadFileWithProgress helpers); api.uploads.url is preserved for
GET-path URL building. Deletes the server-side POST /api/uploads handler and
the now-unused @fastify/multipart plugin registration. GET /api/uploads/:filename
remains intact.
DmLastMessagePreview lacked a `type` field, so the sidebar rendered
`lastMessage.content` verbatim — surfacing JSON like
`{"event":"space_invite",...}` for space invites and member-add events.
Adds `type` to the preview payload (populated server-side from
`dm_messages.type`) and routes all sidebar call sites through a single
`formatDmSidebarPreview` helper that renders human-readable text for
each system event and skips the group `Sender:` prefix on system rows.
Empty-string (local) origin is now stored as the absolute home origin
so relayed DM space-invite cards carry the correct value to remote
recipients instead of resolving against the wrong instance.
Jannis Braun