macOS screen recordings are HEVC inside a .mov container, which Chromium,
Firefox and stock Electron can't decode. The file uploaded fine and a
server-side ffmpeg poster was generated, but inline <video> playback failed
silently — stuck at 0:00 with no error, since AttachmentRenderer had no error
handling. Root cause: the system had no concept of web-playability.
Server detects, client degrades:
- mediaPlayable.ts: classifyVideoPlayable(mimetype, codec) — tri-state
(false = known-undecodable e.g. HEVC/ProRes, true = web codec in web
container, null = unknown/optimistic). Never widens `false` beyond codecs
that fail everywhere, so ffmpeg-less instances keep prior behaviour.
- probeMediaMeta now captures the video codec_name; the upload finish hook
stores the verdict in the new attachments.playable column (migration 0007).
- Flag propagated through every serializer: space messages, DMs, WS, and
federation relay (outbound + inbound) — federation-compatible.
- VideoAttachment component: playable===false renders a download card (poster
+ "Can't play here — download" + name/duration/size) with no dead-player
flash; otherwise plays inline with an onError fallback to the same card.
Specs updated: uploads.md, database.md, federation.md.
Voice presence (voiceStates/voiceUserStates/spaceVoiceStates) was only ever
delivered in the WS `ready` payload — i.e. at connect/reload. A user joining a
space mid-session got `member_joined` (no voice data) and a bare space object;
`GET /api/spaces/:id` (the channel-sidebar hydrator) carries no voice state
either. So members already sitting in a voice channel stayed invisible in the
new member's sidebar until a full page reload.
Fix at the systemic root: ConnectionManager.addUserSpace — the single chokepoint
every join path funnels through (invite, public join, join-request approval),
and which is NOT used on reconnect (that path uses setUserSpaces) — now pushes a
scoped `space_voice_state` snapshot to the joining user. The snapshot is built by
a new buildSpaceVoiceState(spaceId, userId) helper that is also the single source
of truth feeding buildReadyPayload (refactored to use it), so the connect-time
and join-time paths can never drift.
Robustness:
- Delivered over the same ordered WebSocket as voice_state_update deltas — no
REST snapshot-vs-event-stream race.
- VIEW_CHANNEL-filtered via computePermissions exactly like `ready`: a joiner is
never told who occupies a voice channel they cannot see.
- Client applies it scoped to the space (utils/voiceStateSync.applySpaceVoiceState):
merges occupants/statuses and rebuilds only that space's restriction keys,
never disturbing voice state in other spaces.
- Skipped when the space has no active voice and no restrictions (e.g. space
creation).
Tests: server helper behavior, the join push, and private-channel exclusion;
client scoped-apply. Specs updated (websocket.md, voice.md, spaces.md).
The sidebar's visibleChannels filter is keyed on the channelPermissions
Map. Creating a channel raced two state updates: the optimistic create
(added to channels with no permission entry) and the channel_created WS
event (the only thing that set the permission). When the optimistic add
won the race, the WS handler hit its dedup guard, skipped setChannels,
and set the permission by mutating the Map in place — no new reference,
so visibleChannels never recomputed and the channel stayed hidden until
loadSpace rebuilt the maps (i.e. leaving and returning to the space).
Centralize the logic in a new upsertChannel store action that replaces
channels and channelPermissions with fresh references, used by both the
create path and the channel_created handler. Also return the creator's
computed myPermissions (and isPrivate) from POST so the channel renders
immediately from the response, independent of WS timing.
Adds spaceStore.upsertChannel.test.ts covering the reference-identity
regression and the optimistic-reconcile path.
Three layered bugs all manifesting as "the group name doesn't show / the
placeholder is a 40-character wall of names":
1. **WS ready payload was missing `name`/`icon`.** The handler serialized
DmChannel rows with only `id, federatedId, ownerId, createdAt, members,
lastMessage`. The optional metadata fields were silently dropped, so
the client store never received `dm.name` until a subsequent
`dm_channel_updated` event fired (i.e. only mid-session renames worked,
never the initial render). `ownerHomeUserId`, `ownerHomeInstance`, and
`metadataUpdatedAt` were also missing — added too because federated
routing depends on `ownerHomeInstance` (`getDmOwnerHomeInstance`).
2. **Header surfaces silently dropped `dm.name`.** `MainContent` (desktop
chat header) and `MobileChatScreen` always rendered the joined member
names, even when `dm.name` was set. Five other surfaces (`DmListItem`,
`MobileDmsScreen`, `MessageList` welcome hero, `MobileGroupDmInfo`,
`GroupDmSettings`) honored it correctly, so a renamed group showed
different titles depending on which surface you looked at.
3. **Message-input placeholder rendered joined names.** Once a group
has 4+ members "Message #Alice, Bob, Charlie, Dave" overflows the
textarea and obscures the call-to-action.
Consolidates the display-name logic behind two utilities in
`dmFormatters.ts`:
- `formatDmHeaderName(dm, currentUser)` — `dm.name` verbatim if set,
else joined names (excluding self); falls back to `'Group'` /
`'Direct Message'`. Used by all 5 header surfaces (was inlined
5 different ways).
- `formatDmInputLabel(dm, currentUser)` — `'#<name>'` if set,
`'the group'` for unnamed groups (collapses the unreadable
joined-names form), `'@<partner>'` for 1-on-1.
`MessageInput` accepts an optional `placeholder` prop that bypasses the
default `Message {#|@}<channelName>` derivation; DM call sites use it
to inject the `formatDmInputLabel`-based form. 1-on-1 DMs keep the
canonical-view lookup so replicated aliases still surface the home
account's displayName; the placeholder reuses the canonical `dmName`
so header + placeholder stay aligned even when raw partner ≠ canonical.
13 new unit tests covering `formatDmHeaderName` (8 cases: named, blank,
joined, federated-username base, empty group, 1-on-1, no-displayName,
no-partner) and `formatDmInputLabel` (4 cases: named, unnamed,
whitespace-only, 1-on-1). 365 → 377 web tests, 1053 server tests,
typecheck clean.
Manual ownership transfers between two federated instances diverged because
`dm_channels.ownerHomeInstance` was stored as a BARE host (`orbit.ddns.net`)
for federated owners — via `transferGroupDmOwnership` copying `users.homeInstance`
verbatim — while `sourceInstance` always arrives as a full URL on the wire.
`processOwnershipTransferEvent` and `processMemberRemoveEvent` then compared the
two with strict equality and rejected legitimate inbound events as
`unauthorized_source`, keeping ownership permanently divergent across peers.
Live DB inspection on the two test instances confirmed both rows (nova + orbit)
had a BARE `owner_home_instance`, matching the bug report exactly.
Three compounding fixes:
1. Receiver authority checks now compare via `normalizeOriginForCompare` so
legacy bare-vs-full rows accept legitimate transfers (and kicks).
2. New `canonicalizeHomeInstance` helper in `federationAuth.ts`; every write
site that persists `ownerHomeInstance` (`transferGroupDmOwnership`, group DM
creation, lazy federation in member-add, `processMemberAddEvent` bootstrap,
`processOwnershipTransferEvent` receiver storage) routes through it. Full URL
is the canonical storage form, matching how `sourceInstance` arrives.
3. `dm_owner_updated` WS event extended with optional `newOwnerHomeUserId` and
`newOwnerHomeInstance` fields. Client `updateDmOwner` writes them when
present and leaves existing values untouched otherwise (legacy-server safe).
Without this, `getOwnerInstanceForDm` returned the previous owner's home
after a successful WS broadcast, routing the next owner-only op to the wrong
instance.
Coverage: new `federation.ownershipTransfer.test.ts` (7 receiver tests including
the headline bare-vs-full regression and the dedup replay guard); new bare-vs-full
case in `federation.kick.test.ts`; two new client-side cases in
`groupDm.ownerRouting.test.ts` covering both the extended-payload write path and
the legacy-server passthrough. Tests: 1053 server + 364 web, all green.
Specs updated: `dm-system.md` historical bugs + frontend handler table + WS
state-change events table; `federation.md` `ownership_transfer` receiver flow;
`websocket.md` event-fields table.
Transferring ownership or kicking a member surfaced "Target user is not a
member of this DM channel" whenever the target was a federated user.
Root cause: the client passed `canonical.id` from `useCanonicalUserView`,
which returns the user's HOME id when the home view is in the userViews
cache. After owner-routing the request to the owner instance, that
instance's `dm_members.userId` (its own local replicated id) never
matched the home id, so `isDmMember` returned false. The same failure
mode applied across any cross-instance scenario where the
channel-serving instance and the owner-serving instance disagree on the
local replicated user id for the same federated user.
Fix: both endpoints now accept federated identification, mirroring the
existing pattern on `POST /api/dm/:id/members`:
- `POST /api/dm/:id/transfer` body: `{ newOwnerId? } | { homeUserId, homeInstance }`.
Federated args win when both are supplied (strictly more specific).
- `DELETE /api/dm/:id/members/:targetUserId` reads optional
`?homeInstance=<origin>` query; when present, the URL segment is
treated as a homeUserId and resolved via `resolveOrCreateReplicatedUser`.
Client `api.dm.kickMember` and `api.dm.transferOwnership` gain an
optional `federated` argument; `DmRosterPanel` and `MobileGroupDmInfo`
pass it whenever the target has `homeUserId` + `homeInstance` populated.
Adds 5 server tests (3 transfer + 2 kick) covering federated targets,
the federated-wins-over-local precedence rule, and federated-non-member
rejection. Updates 2 client routing tests and 2 DmRosterPanel test
assertions for the new signature. Updates `docs/systems/dm-system.md`
and `docs/systems/api.md`.
Server: 965 tests pass (was 960). Web: 362 tests pass (was 360).
getProfileReferencedFilenames() didn't include dm_channels.icon, so the
storage janitor deleted group DM icons within ~1 hour:
- Owner instance: PATCH /api/dm/:id leaves an attachments row with
messageId=null and dmMessageId=null. After 1h, getUnlinkedAttachments
flags it and cleanupStorage phase 2 deletes the file because the
filename isn't in profileReferenced.
- Receiver instance: downloadProfileAsset writes the icon directly to
uploadDir with no attachments row. cleanupStorage phase 1 treats it
as orphaned and deletes it.
Fix: include dm_channels.icon (non-null, not soft-deleted, not http://)
in the profile-referenced set. Soft-deleted DMs are excluded so their
files still get reaped by cleanupSoftDeletedDmChannels. Absolute URLs
are skipped because they live on a remote instance.
Also mirror the avatar precedent at the PATCH endpoint by deleting the
new icon's standalone attachment row — the file is now protected via
dm_channels.icon, matching users.ts:473.
Refactors the leave-DM destructive core into a shared `removeDmMember`
helper and adds a kick endpoint that reuses it. Both endpoints write the
`member_removed` system message, delete the dm_members + read_states
rows, broadcast `dm_member_removed`, and queue a federation
`member_remove` event with the appropriate `reason` ('leave' | 'kick').
Branching invariants preserved by the helper:
- Ownership transfer fires only on self-leave when the leaver was the
owner. Kicks cannot orphan a group (the owner is still present), so
the transfer block is skipped.
- Soft-delete on last-member-empty fires only on self-leave. Kicks are
guaranteed to leave the owner behind, so the channel can never be
empty after a kick.
Endpoint validation:
- 1-on-1 DM → 400 'Cannot kick from a 1-on-1 DM'
- Caller not the owner → 403 'Only the group owner can remove members'
- Self-target → 400 'Owners cannot kick themselves; use leave instead'
- Target not a member → 404
- Channel missing or soft-deleted → 404
The kicked user is also evicted from the DM voice room (mirroring the
self-leave path) and receives `dm_channel_closed` so the client closes
the channel locally.
Two follow-on bugs from the initial S2S presence rollout:
(1) New friend stuck offline until they reload: presence_update fires only on
transitions, so a remote user already online when their stub is created
locally never receives a relay event seeding their actual status. The
stub defaulted to 'offline' at creation and stayed there until the next
transition. Fix: extend FederationRelayProfileSnapshot +
FederationUserLookupProfile with status. Sender-side buildProfileSnapshot,
getDmParticipants, and lookup endpoint responses populate it for native
users only (replicated stubs hold stale status owned elsewhere).
resolveOrCreateReplicatedUser uses hints.status to seed the new row's
status column. Threaded through every call site (DM participants, group
bootstrap, friend events, ownership transfer). Stub backfill worker also
heals existing rows whose status was stuck at 'offline' from creation.
(2) 'Online' text updates but green avatar dot stays grey on the same page:
spaceStore.updateMemberPresence patches members[] (which feeds space UIs)
but never patches userViews — the cache useCanonicalUserView reads from.
The Avatar in FriendItem reads canonical.status; the text reads
friend.status (socialStore). Two sources, one stale until full
user_updated arrives. Fix: updateMemberPresence now mirrors status into
matching userViews entries, so canonical-view consumers re-render with
fresh status the moment the WS event lands.
The outbox worker rebuilds FederationRelayEvent objects from stored JSON via
an allowlist of known fields. presenceUpdate was missed when presence_update
events shipped, so peers received events with eventType='presence_update' but
no payload — rejected with missing_presence_update_payload on every tick.
onPeerActivated re-emits a presence_update for every relationship-related
online native to the activating peer (relationship = friend with peer-stub /
DM-mate with peer-stub / replicatedInstances opt-in for peer origin). Scope
bounded by relationship count, not native count — flap recovery cost stays
proportional to actual interaction surface.
onPeerDeactivated flips every replicated stub from that peer to offline and
broadcasts a local presence_update so connected friends/DM-mates/space-co-members
see them disappear immediately, instead of seeing stale online until next signal.
Re-snapshot on every activation (incl. health-check unreachable→active flap)
is load-bearing for correctness — markPeerStubsOffline ran on the prior
deactivation and presence is not in the mutation log.
Six WS sites that previously broadcast presence_update to spaces only now use
collectProfileBroadcastTargetIds (the same recipient set as user_updated):
- ws/handler.ts finalizeDisconnect (offline)
- ws/handler.ts auth path (online)
- ws/events.ts handlePresenceUpdate (manual idle/dnd/online)
- ws/events.ts handleActivityUpdate (rich activity changes)
- routes/users.ts showActivity-toggle clear
- routes/users.ts status PATCH
Friends with no shared space + DM-only co-members now see each other's
online/offline transitions live, matching user_updated semantics. Federated
stub presence broadcasts (Task B3) use the same helper, so cross-instance
recipients are uniform.
Updates one assertion in social.federated.test.ts that asserted the old
snowflake-style stub username (now realname-based per A1).
processPresenceUpdateEvent updates the local stub's status and broadcasts a
WS presence_update to friends + DM members + space co-members of that stub
via collectProfileBroadcastTargetIds. Closes the doc/code drift in
activity-presence.md:147 — federated stubs now have their status projected
by the home instance as documented.
Strict attribution: payload.homeInstance domain must equal source peer
domain. Silently no-ops when no local replica exists (peer broadcast fanout
covers all peers, not all hold a stub).
New FederationPresenceUpdatePayload + queuePresenceRelay() helper. Five WS
sites now project the native user's status (and optional activities) to all
active peers via the outbox: WS auth-success, finalizeDisconnect,
manual presence_update, activity_update, showActivity-toggle clear.
Outbox-only (no mutation-log entry) — presence is ephemeral; the upcoming
peer-activation hook re-emits a fresh snapshot so peers recovering from
unreachable converge without history replay. No-op for replicated users.
Heals existing legacy stubs (created when resolveOrCreateReplicatedUser used
homeUserId@domain) by asking the peer for the canonical username via
lookupRemoteUserByHomeId and rewriting the local row. Idempotent and
collision-safe.
Wired into onPeerActivated (per-origin) so future peer flaps re-attempt for
stubs whose home was unreachable on a prior pass, and into a one-shot pass at
startupBootstrapSync for all currently-active peers (not just first-time
lastSyncedAt=0 peers).
HMAC-authenticated, rate-limited (60/min/peer) endpoint that resolves a
homeUserId on this instance to its canonical username + profile snapshot.
Native non-deleted users only. Mirrors /users/lookup's auth shape.
Adds lookupRemoteUserByHomeId to federationLookup.ts as the client-side
helper. Used by the upcoming stub-backfill worker on peers that hold legacy
snowflake-named replicas of users now visible by their real handle.
FederationProfileUpdatePayload gains `username`: the home user's canonical
handle. Receiver applies displayName ?? username so stubs whose home user has
no displayName show the real handle instead of getting clobbered to null.
Mirrors the existing fallback in hydrateReplicatedUserProfile. Username itself
is immutable on the home instance, so the receiver does not rewrite the stub's
username column on profile_update.
When friend_request_create / friend_add / DM relay carries a profile snapshot,
the canonical-username hint is now used as the stub's local-part. Stubs created
purely from S2S (no client-federation) now display the human-readable handle,
not the homeUserId snowflake. Falls back to the snowflake-id scheme only when
no hint is available.
Overloading NODE_ENV='test' to silently disable rate limiting layered a
second meaning onto an env that already gates the test-only seed-peer
route. A dedicated DISABLE_RATE_LIMITS env (envBool semantics, matching
DISABLE_FEDERATION_WORKERS) makes intent explicit, defaults off in
production, and leaves room for tests that need to assert real rate-limit
behaviour to opt back in by simply not setting the var.
Also fixes per-IP rate limit exhaustion in test environments: @fastify/rate-limit
v9 has no skip(); use allowList(() => NODE_ENV==='test') which propagates to
per-route overrides via mergeParams Object.assign merge.
Adds an explicit override for the federation transport URL returned by
getOurOrigin(). When unset, behaviour is unchanged (https://${DOMAIN} ->
http://localhost:${PORT} dev fallback). Intended for reverse-proxy /
dev-without-TLS deployments where the public origin must be advertised
explicitly (typically http://...) and differs from the bare DOMAIN
value used for federated identity.
Wired via config.publicOrigin (envOptional('PUBLIC_ORIGIN')) so the
override flows through the existing config layer rather than scattering
process.env reads. Trailing slash is stripped for symmetry with
peer.origin storage.
docs/systems/federation.md gets a "Public Origin Override" subsection
under §14 Background Workers documenting the resolution order.
Jannis Braun