baselineExistingInstall, healInitialSchemaDrift, and healRenamedColumns
existed only because of pre-squash intermediate states: dev DBs drifted
against drizzle-kit's assumed 0000 baseline, or carried pre-rename
columns from the old manual migration system. Under a single squashed
baseline there are no intermediate states to drift against, and these
functions are unreachable.
initDatabase() is now: pragmas -> drizzle() -> migrate() -> ensureDefaults().
ensureDefaults stays (idempotent startup seeding for settings row, worker
ID, first-admin promotion).
Refs backlog #31 Phase 2.
Companion to #29. `healInitialSchemaDrift` can only ADD columns, so it
skips NOT NULL-without-default columns like `federation_outbox.entity_id`
/ `context_id` — which on some old pre-drizzle dev DBs carry the
pre-rename names `message_id` / `dm_channel_id` instead. The tables
load but the outbox worker fails every tick with "no such column:
federation_outbox.context_id" once the server is up.
Adds healRenamedColumns() — a second pass that runs right after
`healInitialSchemaDrift`. For each table whose physical column set is
*missing* columns declared by the current-migration-state snapshot AND
which holds zero rows, it DROPs the table and rebuilds it from the
snapshot's JSON: columns, defaults, foreign keys, composite PKs,
unique constraints, indexes.
Key design decisions:
- **Target is the current-migration-state snapshot, not the latest on
disk.** The current state is determined by the highest
`__drizzle_migrations.created_at` matched against `_journal.json`'s
`when` timestamps (with backward walk for idx values that lack a
snapshot, like the hand-written 0002). Rebuilding to a *future*
snapshot would introduce columns that drizzle's migrator is about
to add via ALTER TABLE ADD COLUMN, causing duplicate-column errors.
Rebuilding to the *current* snapshot preserves the invariant that
drizzle's pending migrations can run cleanly afterwards.
- **Missing-column gate, not extra-column.** Extra columns alone don't
break anything at runtime (the ORM ignores them); they're leftover
from pre-drizzle manual migrations and might matter to the operator.
Missing columns DO break runtime queries, so only those trigger
rebuild.
- **Empty-table gate.** Non-empty tables log a warning and skip —
data preservation wins over heal, and this path should only ever
hit a pre-drizzle dev DB that never exercised the affected tables
in the first place.
- **Transactional rebuild.** DROP + CREATE + index reinstatement wrap
in a single `db.transaction()` so a partial rebuild rolls back.
Verified against three scenarios via in-memory simulation:
(A) fresh install — heal no-op, drizzle creates everything; (B) pre-
drizzle dev DB with fed_outbox/fed_mutation_log rename drift —
tables rebuilt to 0000 snapshot, drizzle then applies 0001–0004
successfully to reach the current target schema; (C) post-migration-
correct (production-like) — heal no-op, drizzle no-op, schema
unchanged. Live boot on my actual dev DB: migrations complete
silently, server binds :3005, no outbox worker errors. Server tests
110/110, web 131/131, typecheck clean.
No migration files changed. Deployed Pi+VM instances are unaffected
(their schema matches the snapshot exactly — heal won't touch
anything).
Closes backlog #30.
baselineExistingInstall marks 0000_initial as applied when it detects
pre-existing tables, on the assumption the install's schema matches the
0000 baseline. That assumption is false for dev DBs created under the
pre-drizzle manual migrate.ts system that skipped or never ran some of
its idempotent ALTER TABLE steps — for example the b9e4c65 migration
that added federation_peers.remote_max_upload_size. On such DBs, 0000
is marked done without the column actually existing, and a later
migration that recreates the table (0004_cooing_black_knight)
subsequently crashes with "no such column: remote_max_upload_size"
while building its __new_federation_peers SELECT.
Adds healInitialSchemaDrift(): walks every table in 0000_snapshot.json,
and for each table that already exists, ADDs any columns the snapshot
declares but the physical table is missing. Runs immediately after
baselining, before drizzle's migrate() — so later migrations find the
schema they expect. Columns that SQLite's ALTER TABLE ADD COLUMN can't
safely express (PRIMARY KEY; NOT NULL without a default) are skipped
with a warning rather than corrupting data.
Idempotent: on fresh installs and correctly-migrated DBs every column
is already present, so the loop is a no-op. Production Pi+VM instances
are unaffected.
Verification: local dev DB that previously crashed on 0004 now boots
cleanly — federation_peers gained remote_max_upload_size, nonce_supported,
pending_hmac_secret, secret_rotation_at, secret_rotated_at, and
auto_rotate_interval_days; __drizzle_migrations advanced from 4 to 5
entries; server binds :3005. 110/110 server tests + 131/131 web tests
still pass.
Not covered: a deeper drift on federation_outbox /
federation_mutation_log where the physical tables retain pre-rename
column names (message_id / dm_channel_id) instead of the current
entity_id / context_id. Heal skips those (NOT NULL without default)
and the outbox worker emits SQLITE_ERROR ticks post-boot. Both tables
are empty on affected dev DBs, but a clean fix requires DROP +
RECREATE with index reinstatement which is out of #29's stated scope
("column existing"). Flagged for a follow-up.
Closes backlog #29.
Remove all manual migrations — Drizzle-kit now manages schema DDL.
Data-fix migrations have all completed on both instances.
Startup initialization (settings row, worker ID, first admin)
moves to idempotent ensureDefaults().
Three bugs that combined to corrupt DM identities during initial sync:
1. Sync endpoint omitted federatedId for group DMs, causing the receiver
to treat them as 1-on-1 DMs and compute wrong pair hashes — creating
phantom channels that collide with real ones.
2. backfillHomeUserId unconditionally overwrote existing homeUserIds,
so a single wrong match would permanently corrupt a user's identity
and cascade to all subsequent lookups.
3. Migration duplicate-stub Criterion 1 ("shared 1-on-1 DM membership")
incorrectly merged different users from the same domain who were
simply having a conversation, destroying one user's identity.
migrateFixOneOnOneOwnerIds was too aggressive — it set owner_id=NULL
on any 2-member channel, including group DMs that happened to have 2
members. The group DM repair then restored owner_id, creating noisy
logs every restart.
Now only targets channels with NULL or 32-char hex federatedId
(true 1-on-1 DMs), skipping UUID-format group DMs.
Adds nonce_supported INTEGER column to the federation_peers table in both
the Drizzle schema definition and via a safe ALTER TABLE migration, enabling
the auto-ratchet mechanism for replay attack protection.
Two real accounts from the same remote instance sharing a 1-on-1 DM
are different people, not duplicates. The shared DM is a legitimate
relay. Only merge when at least one user has passwordHash =
'!federation-replicated' (a relay-created stub).
Three fixes for group DM data integrity and display:
1. processOwnershipTransferEvent: use resolveOrCreateReplicatedUser instead
of resolveLocalUser to guarantee a valid ownerId. The previous ?? null
fallback converted group DMs into 1-on-1s when resolution failed.
2. Self-healing migration: detect group DMs with UUID-format federated_id
but NULL owner_id (corrupted by the old fallback) and restore owner from
the first remaining member. Found and repaired 7 across both instances.
3. Sidebar: group DMs with 0 other members (last person standing) now show
as "Empty Group" instead of being hidden. 1-on-1 DMs with 0 others are
still correctly filtered out.
Adds a `type` column (TEXT NOT NULL DEFAULT 'user') to the dm_messages
table via schema, migration, and type definition. Updates
buildDmMessageWithUser and the inline replyTo builder in the GET
messages handler to include the field in all DM message responses.
Renames DM-specific columns in federation_outbox (dm_channel_id → context_id, message_id → entity_id) and federation_mutation_log (dm_message_id → entity_id, dm_channel_id → context_id) to generic names, adding context_type = 'dm' for all existing rows so the outbox can carry friend events too.
Adds migrateResetFederationSyncForLegacyDms which resets last_synced_at=0
on all active federation peers so the S2S sync worker re-pulls all mutation
log entries (including newly-backfilled legacy DMs) on next server startup.
A legacy_dm_sync_done flag on instance_settings ensures this runs exactly once.
- Add migrateFixOneOnOneOwnerIds migration to NULL-out ownerId on all
existing 1-on-1 DMs (those with exactly 2 members)
- Fix POST /api/dm to create 1-on-1 channels with ownerId=null instead
of the creator's ID
- Guard POST /api/dm/:id/members: reject with 400 if channel has no
owner (i.e. is a 1-on-1), directing callers to POST /api/dm/group
- Guard DELETE /api/dm/:id/members: replace member-count check with
ownerId check; remove now-duplicate dmChannel query in that handler
- Add CreateGroupDmRequest type to shared types
Remove federated_id from addMissingColumns so it no longer races with
migrateDmChannelsFederatedId on installs that have canonical_pair_id.
Wrap the table rebuild in PRAGMA foreign_keys = OFF/ON with a transaction
so the DROP TABLE succeeds even when dm_members/dm_messages hold FK refs.
Renames dm_channels.canonical_pair_id → federated_id and adds
owner_home_user_id, owner_home_instance, deleted_at columns to support
federated group DMs. Migration handles upgrade from existing installs
(full table rebuild to rename), partial states (ALTER TABLE), and fresh
installs (correct DDL from the start). Backfills federated_id for
existing 1-on-1 DM channels and owner federation identity for group DMs.
SQLite stores self-referencing FK identifiers with quotes (REFERENCES
"dm_messages"), but the migration guard only checked for the unquoted
string. This caused the migration to re-run on every server startup,
which DROP TABLE dm_messages and triggered ON DELETE CASCADE on
attachments.dm_message_id — wiping every DM-linked attachment.
- buildRelayPayload now uses config.domain for local users instead of
empty string, so the relay receiver can resolve the user
- Fixed canonical_pair_id backfill: SQLite NULL || ':' || x = NULL,
so use COALESCE(home_user_id, id) instead of concatenation
Existing 1-on-1 DM channels created before federation had canonical_pair_id = NULL,
causing the relay to create duplicate channels on receipt of a message. On server
startup, migrations now: (1) compute and set canonical_pair_id for all existing
1-on-1 channels using home_user_id (falling back to regular id for local users),
and (2) merge any duplicate channels that share the same canonical_pair_id by
moving all messages, read states, and members into the oldest channel and deleting
the stale duplicate. Both steps are non-fatal and idempotent.
Federation relay was incorrectly defaulting to disabled, requiring
admins to discover a buried settings toggle. Since federation itself
is opt-in (you connect instances manually), relay should be on by
default. Also migrates existing instances from 0 → 1.
Adds column migrations (dm_channels.canonical_pair_id, dm_messages.source_*,
attachments.source_url, instance_settings.federation_relay_*), CREATE TABLE
for federation_peers/outbox/file_queue/mutation_log, partial/filtered indexes,
and an idempotent backfill that seeds the mutation log from existing DM messages.
Also makes migrateDmMessagesReplyToFk and migrateAttachmentsDmMessageFk
column-aware so table recreation preserves any columns added by earlier
migrations (prevents dropping federation columns on upgrade).
Rebuilds the attachments table with ON DELETE CASCADE on dm_message_id.
Dangling records are excluded during copy — their files become standard
disk orphans detectable by the storage janitor. Also updates the raw SQL
in index.ts to include width/height/duration columns for fresh installs.
- Add Activity, ActivityType, ActivityTimestamps, ActivityAssets types to shared types
- Add activity_update client event and activities field on presence_update server event
- Add userActivities to ready payload and showActivity to User/UpdateUserRequest
- Create shared activities.ts with ACTIVITY_LIMITS, ACTIVITY_PRIORITY, getPrimaryActivity
- Add show_activity column to users table (schema + migration)
- Update sanitizeUser with isSelf parameter; only include showActivity for self
- Fix .map(sanitizeUser) calls to use arrow wrapper to prevent index-as-boolean bug
- Mark auth routes (register/login) as isSelf=true since they return own user data
- Fix ?? to || in metadataFetcher.ts to handle empty strings from Cheerio
- Fix stale embeds on message edit: delete old embeds synchronously before
broadcast, then resolve new ones async (all 4 edit paths: REST+WS, msg+DM)
- Revert unrelated MessageList.tsx scroll threshold change (5000 not 150)
- Remove duplicate embed indexes from migrateAddIndexes (kept standalone ones)
- Resize avatars/icons to 256px and banners to 1280px (client crop + server safety net)
- Add server-side resizeProfileImage() for federation/API uploads without crop modal
- Fix unconstrained crop on RegisterPage and CreateSpace (was missing maxOutputDimension)
- PWA: switch to autoUpdate with skipWaiting/clientsClaim for seamless deploys
- Storage janitor: exclude profile images from unlinked cleanup, delete stale thumbnails
- Add deleteAttachmentByFilename() to clean orphaned attachment records for profile images
- Migration: one-time cleanup of stale profile image attachment records
- GeneralPanel: wrap in <form> to prevent implicit submission
- SSRF protection: DNS resolution + private IP blocking on metadata fetcher
- Upload security: CSP/X-Frame-Options headers, SVG forced download, nosniff
- Auth hardening: JWT secret min length, password min 8 chars, token revocation via password_changed_at
- Attachment ownership verification before linking to messages
- Message length limit (4000 chars) enforced on client and server
- Asset URL validation on avatar/banner updates
- Federation instance validation (domain regex, origin scheme, length limits)
- DB indexes on all FK columns for query performance
- Migrations: nullable moderator columns, dm_messages reply_to FK constraint
- File cleanup on avatar/banner replacement and space deletion
- Fastify trustProxy, AbortController on fetches, typing map size cap
- Add "Discover People" section to Add Friend tab with user cards, mutual counts, and inline actions
- Add discoverStore for fetching/searching discoverable users across local and federated instances
- Add PrivacyPanel to user settings with discoverability toggle
- Add is_discoverable column to users table with migration
- Fix "Send Friend Request" button vertical alignment using transform centering
One-time async migration runs after server is listening — generates 800px
WebP thumbnails for all existing image attachments that lack one. Gated by
a persistent flag in instance_settings so it never re-runs.
Avatars/banners now resize to max 512px/1920px and convert to WebP before
upload (zero server cost). Chat image uploads generate an 800px-wide WebP
thumbnail via Sharp; the feed shows the thumbnail, click opens the full-res
original. Adds lazy loading to avatars. Federation-compatible: remote
instances without this feature fall back gracefully.
- Rewrite ChannelSettingsModal with full tri-state permission override UI
for roles and members (allow/neutral/deny per permission bit)
- Switch font from Inter to self-hosted DM Sans (woff2 variable fonts)
- Add client-side VIEW_CHANNEL filtering in ChannelSidebar for private channels
- Broadcast isPrivate flag on channel override changes
- Fix voice permission bit migration: gate behind persistent flag to prevent
repeated re-runs that stripped STREAM from @everyone roles
- Add speakingUserIds set to voice store for efficient user-level lookups
- Clear current channel view when a channel is deleted
- Move .glass-strip to @layer utilities for proper CSS specificity
- Simplify avatar initials font size to proportional formula
Jannis Braun