- New 'admin_required' EnsurePeeredResult variant.
- Required intent argument (no optional default) prevents future
user-initiated callers from silently getting system behavior.
- Gate runs only when no peer row exists; toggling autoAccept later
does not retroactively gate established peer rows.
- queueOutboundApproval helper upserts parent + subscriber rows and
broadcasts to admins + user.
- Schema enum-narrows direction ('inbound' | 'outbound') and
notifications.kind ('approved' | 'denied' | 'expired') at the column
level; drizzle generate confirmed no migration delta.
- Existing call sites now fail typecheck — fixed in Task 5.
drizzle's snapshot does not encode SQL-level CHECK constraints. Without
this comment, a future migration that recreates the table for unrelated
reasons would silently drop the (direction='inbound' → hmac_secret
NOT NULL) invariant.
Nullable text columns on federation_peers and peer_approval_requests.
Existing rows degrade gracefully (NULL token) per spec §5; the verification
logic landing in subsequent commits routes legacy null-token state through
the existing autoAccept gate.
Nullable indexed column tracking the entityId of the originating relay
event for federated friend requests. Used by the rollback hook in
federationRollback.ts to locate and delete rows when a federated
friend_request_create is permanently rejected (spec §5).
baselineExistingInstall, healInitialSchemaDrift, and healRenamedColumns
existed only because of pre-squash intermediate states: dev DBs drifted
against drizzle-kit's assumed 0000 baseline, or carried pre-rename
columns from the old manual migration system. Under a single squashed
baseline there are no intermediate states to drift against, and these
functions are unreachable.
initDatabase() is now: pragmas -> drizzle() -> migrate() -> ensureDefaults().
ensureDefaults stays (idempotent startup seeding for settings row, worker
ID, first-admin promotion).
Refs backlog #31 Phase 2.
Companion to #29. `healInitialSchemaDrift` can only ADD columns, so it
skips NOT NULL-without-default columns like `federation_outbox.entity_id`
/ `context_id` — which on some old pre-drizzle dev DBs carry the
pre-rename names `message_id` / `dm_channel_id` instead. The tables
load but the outbox worker fails every tick with "no such column:
federation_outbox.context_id" once the server is up.
Adds healRenamedColumns() — a second pass that runs right after
`healInitialSchemaDrift`. For each table whose physical column set is
*missing* columns declared by the current-migration-state snapshot AND
which holds zero rows, it DROPs the table and rebuilds it from the
snapshot's JSON: columns, defaults, foreign keys, composite PKs,
unique constraints, indexes.
Key design decisions:
- **Target is the current-migration-state snapshot, not the latest on
disk.** The current state is determined by the highest
`__drizzle_migrations.created_at` matched against `_journal.json`'s
`when` timestamps (with backward walk for idx values that lack a
snapshot, like the hand-written 0002). Rebuilding to a *future*
snapshot would introduce columns that drizzle's migrator is about
to add via ALTER TABLE ADD COLUMN, causing duplicate-column errors.
Rebuilding to the *current* snapshot preserves the invariant that
drizzle's pending migrations can run cleanly afterwards.
- **Missing-column gate, not extra-column.** Extra columns alone don't
break anything at runtime (the ORM ignores them); they're leftover
from pre-drizzle manual migrations and might matter to the operator.
Missing columns DO break runtime queries, so only those trigger
rebuild.
- **Empty-table gate.** Non-empty tables log a warning and skip —
data preservation wins over heal, and this path should only ever
hit a pre-drizzle dev DB that never exercised the affected tables
in the first place.
- **Transactional rebuild.** DROP + CREATE + index reinstatement wrap
in a single `db.transaction()` so a partial rebuild rolls back.
Verified against three scenarios via in-memory simulation:
(A) fresh install — heal no-op, drizzle creates everything; (B) pre-
drizzle dev DB with fed_outbox/fed_mutation_log rename drift —
tables rebuilt to 0000 snapshot, drizzle then applies 0001–0004
successfully to reach the current target schema; (C) post-migration-
correct (production-like) — heal no-op, drizzle no-op, schema
unchanged. Live boot on my actual dev DB: migrations complete
silently, server binds :3005, no outbox worker errors. Server tests
110/110, web 131/131, typecheck clean.
No migration files changed. Deployed Pi+VM instances are unaffected
(their schema matches the snapshot exactly — heal won't touch
anything).
Closes backlog #30.
baselineExistingInstall marks 0000_initial as applied when it detects
pre-existing tables, on the assumption the install's schema matches the
0000 baseline. That assumption is false for dev DBs created under the
pre-drizzle manual migrate.ts system that skipped or never ran some of
its idempotent ALTER TABLE steps — for example the b9e4c65 migration
that added federation_peers.remote_max_upload_size. On such DBs, 0000
is marked done without the column actually existing, and a later
migration that recreates the table (0004_cooing_black_knight)
subsequently crashes with "no such column: remote_max_upload_size"
while building its __new_federation_peers SELECT.
Adds healInitialSchemaDrift(): walks every table in 0000_snapshot.json,
and for each table that already exists, ADDs any columns the snapshot
declares but the physical table is missing. Runs immediately after
baselining, before drizzle's migrate() — so later migrations find the
schema they expect. Columns that SQLite's ALTER TABLE ADD COLUMN can't
safely express (PRIMARY KEY; NOT NULL without a default) are skipped
with a warning rather than corrupting data.
Idempotent: on fresh installs and correctly-migrated DBs every column
is already present, so the loop is a no-op. Production Pi+VM instances
are unaffected.
Verification: local dev DB that previously crashed on 0004 now boots
cleanly — federation_peers gained remote_max_upload_size, nonce_supported,
pending_hmac_secret, secret_rotation_at, secret_rotated_at, and
auto_rotate_interval_days; __drizzle_migrations advanced from 4 to 5
entries; server binds :3005. 110/110 server tests + 131/131 web tests
still pass.
Not covered: a deeper drift on federation_outbox /
federation_mutation_log where the physical tables retain pre-rename
column names (message_id / dm_channel_id) instead of the current
entity_id / context_id. Heal skips those (NOT NULL without default)
and the outbox worker emits SQLITE_ERROR ticks post-boot. Both tables
are empty on affected dev DBs, but a clean fix requires DROP +
RECREATE with index reinstatement which is out of #29's stated scope
("column existing"). Flagged for a follow-up.
Closes backlog #29.
The column was `integer DEFAULT 0` (nullable) since the initial schema.
Counters should not be nullable — the semantics are a count, not an
optional measurement. `consecutive_auth_failures` (added later) was
correctly declared NOT NULL; tightening `consecutive_failures` to match
removes the drift and eliminates the "|null" burden everywhere the value
is read.
SQLite does not support in-place ALTER … SET NOT NULL, so drizzle-kit
cannot auto-generate this. The manual migration uses the standard
SQLite recreate pattern (new table + INSERT SELECT + DROP + RENAME +
recreate index) under `PRAGMA defer_foreign_keys = ON` so the existing
federation_outbox → federation_peers FK survives the swap. The COPY
step coalesces any hypothetical NULL to 0 defensively; live probes on
both test instances (nova, orbit) showed zero NULL rows so no
actual backfill is required.
Verified by applying the full migration chain against a copy of the VM's
live DB: column ends as `notnull=1 dflt=0`, the peer row is preserved,
the unique index on origin is recreated, NULL inserts are rejected, and
`PRAGMA foreign_key_check` reports no violations.
Server `SanitizedPeer.consecutiveFailures` tightened to `number` to
match the new drizzle inference and the shared `FederationPeer` shape.
Follow-up #22 from S2S DM unification backlog.
Tracks HMAC-failure count separately from consecutive_failures (network
errors). Auth failures and network failures have different resolution
paths; mixing them would let a single successful retry after a network
blip mask real secret desync.
Part of backlog #19 — outbox auth-failure recovery.
Note: drizzle-kit generated the 0003 SQL with an unexpected CREATE TABLE
for peer_approval_requests because the 0002 migration was authored
manually without a corresponding 0002_snapshot.json (see 14041e9). The
generated SQL has been trimmed to the single intended ALTER TABLE. The
regenerated 0003_snapshot.json correctly reflects the full current
schema, so future migrations will diff cleanly.
Adds `auto_accept_peering` integer column (default 1/true) to the
`instance_settings` singleton table, controlling whether this instance
auto-accepts incoming peering requests. Includes generated migration
`0001_clear_earthquake.sql` applied automatically on server boot.
Remove all manual migrations — Drizzle-kit now manages schema DDL.
Data-fix migrations have all completed on both instances.
Startup initialization (settings row, worker ID, first admin)
moves to idempotent ensureDefaults().
Three bugs that combined to corrupt DM identities during initial sync:
1. Sync endpoint omitted federatedId for group DMs, causing the receiver
to treat them as 1-on-1 DMs and compute wrong pair hashes — creating
phantom channels that collide with real ones.
2. backfillHomeUserId unconditionally overwrote existing homeUserIds,
so a single wrong match would permanently corrupt a user's identity
and cascade to all subsequent lookups.
3. Migration duplicate-stub Criterion 1 ("shared 1-on-1 DM membership")
incorrectly merged different users from the same domain who were
simply having a conversation, destroying one user's identity.
migrateFixOneOnOneOwnerIds was too aggressive — it set owner_id=NULL
on any 2-member channel, including group DMs that happened to have 2
members. The group DM repair then restored owner_id, creating noisy
logs every restart.
Now only targets channels with NULL or 32-char hex federatedId
(true 1-on-1 DMs), skipping UUID-format group DMs.
Adds nonce_supported INTEGER column to the federation_peers table in both
the Drizzle schema definition and via a safe ALTER TABLE migration, enabling
the auto-ratchet mechanism for replay attack protection.
Two real accounts from the same remote instance sharing a 1-on-1 DM
are different people, not duplicates. The shared DM is a legitimate
relay. Only merge when at least one user has passwordHash =
'!federation-replicated' (a relay-created stub).
Three fixes for group DM data integrity and display:
1. processOwnershipTransferEvent: use resolveOrCreateReplicatedUser instead
of resolveLocalUser to guarantee a valid ownerId. The previous ?? null
fallback converted group DMs into 1-on-1s when resolution failed.
2. Self-healing migration: detect group DMs with UUID-format federated_id
but NULL owner_id (corrupted by the old fallback) and restore owner from
the first remaining member. Found and repaired 7 across both instances.
3. Sidebar: group DMs with 0 other members (last person standing) now show
as "Empty Group" instead of being hidden. 1-on-1 DMs with 0 others are
still correctly filtered out.
Adds a `type` column (TEXT NOT NULL DEFAULT 'user') to the dm_messages
table via schema, migration, and type definition. Updates
buildDmMessageWithUser and the inline replyTo builder in the GET
messages handler to include the field in all DM message responses.
Renames DM-specific columns in federation_outbox (dm_channel_id → context_id, message_id → entity_id) and federation_mutation_log (dm_message_id → entity_id, dm_channel_id → context_id) to generic names, adding context_type = 'dm' for all existing rows so the outbox can carry friend events too.
Adds migrateResetFederationSyncForLegacyDms which resets last_synced_at=0
on all active federation peers so the S2S sync worker re-pulls all mutation
log entries (including newly-backfilled legacy DMs) on next server startup.
A legacy_dm_sync_done flag on instance_settings ensures this runs exactly once.
- Add migrateFixOneOnOneOwnerIds migration to NULL-out ownerId on all
existing 1-on-1 DMs (those with exactly 2 members)
- Fix POST /api/dm to create 1-on-1 channels with ownerId=null instead
of the creator's ID
- Guard POST /api/dm/:id/members: reject with 400 if channel has no
owner (i.e. is a 1-on-1), directing callers to POST /api/dm/group
- Guard DELETE /api/dm/:id/members: replace member-count check with
ownerId check; remove now-duplicate dmChannel query in that handler
- Add CreateGroupDmRequest type to shared types
The CREATE INDEX on federated_id in createTables runs before migrations.
On existing installs the column is still canonical_pair_id, causing a crash.
Wrap in try-catch so the migration handles it instead.
Remove federated_id from addMissingColumns so it no longer races with
migrateDmChannelsFederatedId on installs that have canonical_pair_id.
Wrap the table rebuild in PRAGMA foreign_keys = OFF/ON with a transaction
so the DROP TABLE succeeds even when dm_members/dm_messages hold FK refs.
Renames dm_channels.canonical_pair_id → federated_id and adds
owner_home_user_id, owner_home_instance, deleted_at columns to support
federated group DMs. Migration handles upgrade from existing installs
(full table rebuild to rename), partial states (ALTER TABLE), and fresh
installs (correct DDL from the start). Backfills federated_id for
existing 1-on-1 DM channels and owner federation identity for group DMs.
SQLite stores self-referencing FK identifiers with quotes (REFERENCES
"dm_messages"), but the migration guard only checked for the unquoted
string. This caused the migration to re-run on every server startup,
which DROP TABLE dm_messages and triggered ON DELETE CASCADE on
attachments.dm_message_id — wiping every DM-linked attachment.
- buildRelayPayload now uses config.domain for local users instead of
empty string, so the relay receiver can resolve the user
- Fixed canonical_pair_id backfill: SQLite NULL || ':' || x = NULL,
so use COALESCE(home_user_id, id) instead of concatenation
Existing 1-on-1 DM channels created before federation had canonical_pair_id = NULL,
causing the relay to create duplicate channels on receipt of a message. On server
startup, migrations now: (1) compute and set canonical_pair_id for all existing
1-on-1 channels using home_user_id (falling back to regular id for local users),
and (2) merge any duplicate channels that share the same canonical_pair_id by
moving all messages, read states, and members into the oldest channel and deleting
the stale duplicate. Both steps are non-fatal and idempotent.
Federation relay was incorrectly defaulting to disabled, requiring
admins to discover a buried settings toggle. Since federation itself
is opt-in (you connect instances manually), relay should be on by
default. Also migrates existing instances from 0 → 1.
Adds column migrations (dm_channels.canonical_pair_id, dm_messages.source_*,
attachments.source_url, instance_settings.federation_relay_*), CREATE TABLE
for federation_peers/outbox/file_queue/mutation_log, partial/filtered indexes,
and an idempotent backfill that seeds the mutation log from existing DM messages.
Also makes migrateDmMessagesReplyToFk and migrateAttachmentsDmMessageFk
column-aware so table recreation preserves any columns added by earlier
migrations (prevents dropping federation columns on upgrade).
Jannis Braun