feat(federation): peer_reset_pending guard during limbo window
This commit is contained in:
@@ -333,6 +333,15 @@ In a single transaction, `markPeerReset`:
|
||||
|
||||
Legacy peers advertise no epoch (`instanceId` null), so detection requires a non-null observed epoch differing from a non-null stored baseline — legacy peers never trigger it, and the existing `auth_failures → needs_attention → manual Reset` path continues unchanged for them.
|
||||
|
||||
### Limbo-window user error (`peer_reset_pending`)
|
||||
|
||||
Between reset detection (`markPeerReset`) and the admin's one-click Re-peer, the stale identity graph still exists and no heal has run (design §5.3). During this window a user re-adding a same-name friend, or creating a DM to that origin, would otherwise hit a confusing `already_friends` (stale friendship bound to the dead incarnation) or `peer_rejected` (the peer now sits in `needs_attention`, tripping `ensurePeered`). Both user-facing hot paths short-circuit with a clearer **409 `{ error: 'peer_reset_pending' }`**:
|
||||
|
||||
- **Friend-add** (`social.ts` `POST /api/social/requests`, federated branch): after `resolveOriginFromHostname` yields `peerOrigin` and **before** the peering/lookup/`already_friends` checks.
|
||||
- **DM-create** (`dm.ts` `POST /api/dm`, `homeUserId + homeInstance` branch): before stub creation, so no un-flagged stub is left behind. The target origin is resolved with `resolveOriginFromHostname(new URL(canonicalizeHomeInstance(homeInstance)).host)`.
|
||||
|
||||
Both perform an **O(1) point lookup** on the `federation_reset_events` origin PRIMARY KEY (`origin = peerOrigin AND resolved_at IS NULL`). `peerOrigin` (from `resolveOriginFromHostname`, which returns the stored `federation_peers.origin` verbatim) is exactly the string `markPeerReset` journals, so the query is a single indexed hit/miss. The guard **only** short-circuits when an unresolved row exists; the common case — no reset in progress — is one indexed miss and the normal path proceeds byte-for-byte unchanged. Once the admin re-peers and `healResetIncarnation` resolves the journal (`resolved_at` set), the guard stops firing and the freshly-clean graph accepts the re-add.
|
||||
|
||||
### Data Self-Heal (`healResetIncarnation` — `utils/federationReset.ts`)
|
||||
|
||||
Detection (`markPeerReset`) only snapshots + journals + notifies; it destroys nothing. The actual heal is `healResetIncarnation(origin, newEpoch, reason)`, fired from `onPeerActivated` (`utils/federationPeerActivation.ts`) **after an admin-authenticated re-peer**, keyed to the confirmed epoch change (design §6). It runs **before** the mutation-log re-sync in `onPeerActivated` so re-sync repopulates onto a clean slate, and it runs **outside any transaction** (`tombstoneUser` opens its own; better-sqlite3 throws on a nested `BEGIN`).
|
||||
|
||||
@@ -293,6 +293,7 @@ As of 2026-04-25, the sender's home server owns the entire federated friend-add
|
||||
|
||||
1. **Parse target.** If `body.username` contains no `@`, or the domain after `@` normalizes to this server's own host, fall through to the local-only path (unchanged).
|
||||
2. **resolveOriginFromHostname(targetDomain)** — resolves the target peer's full origin URL. Prefers a stored `federation_peers` row matching the typed host; falls back to mirroring `getOurOrigin()`'s scheme. Returns null → 400 `invalid_target_domain`.
|
||||
2a. **Limbo-window guard → 409 `peer_reset_pending`.** O(1) point lookup on the `federation_reset_events` origin PRIMARY KEY: if an **unresolved** row exists for `peerOrigin` (`origin = peerOrigin AND resolved_at IS NULL`), the peer was reset-detected (wipe-and-reinstall) but the admin has not yet re-peered — the local friendship/stub graph is still bound to the dead incarnation. Return 409 `peer_reset_pending` instead of the confusing `already_friends` (stale friendship) or `peer_rejected` (the `needs_attention` peer would otherwise trip `ensurePeered`). `peerOrigin` is the exact string `markPeerReset` journals (the peer's `federation_peers.origin`), so the match is a single indexed lookup; no reset in progress → one indexed miss → the normal path proceeds unchanged. See `docs/systems/federation.md` (instance-epoch self-healing) and the design spec §5.3. The equivalent guard runs on federated DM-create (`POST /api/dm`, `dm.ts`).
|
||||
3. **Authority defense.** If the calling user's `homeInstance` is set and does not normalize to this server's own host (checked via `normalizeOriginForCompare`), return 403 `not_authoritative_for_sender`. Prevents replicated/federated users from queueing relay events the home server isn't authoritative for. Runs before peering to fail fast.
|
||||
4. **ensurePeered(peerOrigin)** — blocks on the result. Status → HTTP mapping:
|
||||
- `'active'` → continue
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
import { describe, it, expect, beforeEach, vi } from 'vitest';
|
||||
import Fastify, { type FastifyInstance } from 'fastify';
|
||||
import Database from 'better-sqlite3';
|
||||
import { drizzle } from 'drizzle-orm/better-sqlite3';
|
||||
import { eq } from 'drizzle-orm';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import * as schema from '../db/schema.js';
|
||||
import { setWorkerId } from '../utils/snowflake.js';
|
||||
|
||||
setWorkerId(1);
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
|
||||
type TestDb = ReturnType<typeof drizzle<typeof schema>>;
|
||||
let sqlite: Database.Database;
|
||||
let testDb: TestDb;
|
||||
const currentUserId = 'user-A';
|
||||
|
||||
vi.mock('../db/index.js', () => ({
|
||||
getDb: () => testDb,
|
||||
schema,
|
||||
}));
|
||||
|
||||
vi.mock('../utils/auth.js', () => ({
|
||||
authenticate: async (req: { userId?: string }) => {
|
||||
req.userId = currentUserId;
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock('../ws/handler.js', () => ({
|
||||
connectionManager: {
|
||||
sendToUser: vi.fn(),
|
||||
sendToDmMembers: vi.fn(),
|
||||
sendToAdmins: vi.fn(),
|
||||
getAllOnlineUserIds: () => [],
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock('../utils/federationOutbox.js', async () => {
|
||||
const actual = await vi.importActual<typeof import('../utils/federationOutbox.js')>('../utils/federationOutbox.js');
|
||||
return {
|
||||
...actual,
|
||||
isFederationRelayEnabled: () => true,
|
||||
queueDmCloseRelay: vi.fn(),
|
||||
sendTypingRelay: vi.fn(),
|
||||
queueDmRelay: vi.fn(),
|
||||
queueOutboxEvent: vi.fn(),
|
||||
appendMutationLog: vi.fn(),
|
||||
};
|
||||
});
|
||||
|
||||
function applyMigrations(db: Database.Database): void {
|
||||
const migrationsDir = path.resolve(__dirname, '../../drizzle');
|
||||
const files = fs.readdirSync(migrationsDir).filter(f => f.endsWith('.sql')).sort();
|
||||
for (const f of files) {
|
||||
const sqlText = fs.readFileSync(path.join(migrationsDir, f), 'utf8');
|
||||
const statements = sqlText.split(/-->\s*statement-breakpoint/);
|
||||
for (const stmt of statements) {
|
||||
const clean = stmt.trim();
|
||||
if (clean) db.exec(clean);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function seedCaller(): void {
|
||||
testDb.insert(schema.users).values({
|
||||
id: 'user-A',
|
||||
username: 'alice',
|
||||
displayName: 'Alice',
|
||||
passwordHash: 'x',
|
||||
homeUserId: 'user-A',
|
||||
homeInstance: null,
|
||||
createdAt: Date.now(),
|
||||
}).run();
|
||||
}
|
||||
|
||||
/** The reset peer's persistent row (still present during the limbo window — only
|
||||
* deleted on admin Re-peer). Its `origin` is the exact string markPeerReset journals. */
|
||||
function seedPeer(): void {
|
||||
testDb.insert(schema.federationPeers).values({
|
||||
id: 'peer-remote',
|
||||
origin: 'https://remote.example',
|
||||
hmacSecret: 'secret',
|
||||
status: 'needs_attention',
|
||||
needsAttentionReason: 'peer_reset_detected',
|
||||
peerInstanceId: 'dead-epoch',
|
||||
observedPeerInstanceId: 'new-epoch',
|
||||
createdAt: Date.now(),
|
||||
}).run();
|
||||
}
|
||||
|
||||
function seedResetEvent(resolvedAt: number | null): void {
|
||||
testDb.insert(schema.federationResetEvents).values({
|
||||
origin: 'https://remote.example',
|
||||
deadEpoch: 'dead-epoch',
|
||||
newEpoch: resolvedAt === null ? null : 'new-epoch',
|
||||
detectedAt: Date.now(),
|
||||
resolvedAt,
|
||||
stubCount: 1,
|
||||
orphanedAccountCount: 0,
|
||||
}).run();
|
||||
}
|
||||
|
||||
async function buildApp(): Promise<FastifyInstance> {
|
||||
const app = Fastify({ logger: false });
|
||||
const { dmRoutes } = await import('./dm.js');
|
||||
await app.register(dmRoutes);
|
||||
await app.ready();
|
||||
return app;
|
||||
}
|
||||
|
||||
describe('POST /api/dm — limbo-window peer_reset_pending guard', () => {
|
||||
beforeEach(() => {
|
||||
sqlite = new Database(':memory:');
|
||||
testDb = drizzle(sqlite, { schema });
|
||||
applyMigrations(sqlite);
|
||||
seedCaller();
|
||||
seedPeer();
|
||||
});
|
||||
|
||||
it('returns 409 peer_reset_pending when creating a federated DM to a reset-pending origin', async () => {
|
||||
seedResetEvent(null);
|
||||
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/dm',
|
||||
payload: { homeUserId: 'remote-bob', homeInstance: 'https://remote.example' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(409);
|
||||
expect(res.json().error).toBe('peer_reset_pending');
|
||||
// No stub created and no DM channel created for the reset-pending peer.
|
||||
expect(testDb.select().from(schema.dmChannels).all()).toHaveLength(0);
|
||||
expect(testDb.select().from(schema.users).where(eq(schema.users.homeUserId, 'remote-bob')).all()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('proceeds normally when the reset event is RESOLVED', async () => {
|
||||
seedResetEvent(Date.now());
|
||||
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/dm',
|
||||
payload: { homeUserId: 'remote-bob', homeInstance: 'https://remote.example' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(201);
|
||||
expect(res.json().federatedId).toMatch(/^[a-f0-9]{32}$/);
|
||||
});
|
||||
|
||||
it('proceeds normally when NO reset event exists for the origin', async () => {
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/dm',
|
||||
payload: { homeUserId: 'remote-bob', homeInstance: 'https://remote.example' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(201);
|
||||
expect(res.json().federatedId).toMatch(/^[a-f0-9]{32}$/);
|
||||
});
|
||||
});
|
||||
@@ -47,6 +47,7 @@ import {
|
||||
normalizeIconForWire,
|
||||
} from '../utils/federationOutbox.js';
|
||||
import { getOurOrigin, canonicalizeHomeInstance } from '../utils/federationAuth.js';
|
||||
import { resolveOriginFromHostname } from '../utils/federationOriginResolve.js';
|
||||
import type { FederationRelayEvent } from '@backspace/shared';
|
||||
import { resolveLocalUser, resolveOrCreateReplicatedUser } from './federation.js';
|
||||
|
||||
@@ -928,6 +929,40 @@ export async function dmRoutes(app: FastifyInstance): Promise<void> {
|
||||
let targetUser: typeof schema.users.$inferSelect | undefined;
|
||||
|
||||
if (homeUserId && homeInstance) {
|
||||
// Limbo-window guard (federation instance-epoch self-healing §5.3).
|
||||
// If the target's home instance was reset-detected but the admin has not yet
|
||||
// re-peered, an UNRESOLVED `federation_reset_events` row exists for its origin.
|
||||
// Creating a DM now would bind to stale, dead-incarnation identity state, so
|
||||
// surface a clear `peer_reset_pending` instead of silently forming a doomed
|
||||
// channel. Checked BEFORE stub creation so no un-flagged stub is left behind.
|
||||
//
|
||||
// The journal is keyed by the peer's `federation_peers.origin` (the exact string
|
||||
// `markPeerReset` stores). `resolveOriginFromHostname` returns that stored origin
|
||||
// verbatim, giving an O(1) point lookup on the origin PRIMARY KEY; the common case
|
||||
// (no reset) is a single indexed miss and the normal path proceeds unchanged.
|
||||
const canon = canonicalizeHomeInstance(homeInstance);
|
||||
let peerOrigin: string | null = null;
|
||||
if (canon) {
|
||||
try {
|
||||
peerOrigin = resolveOriginFromHostname(new URL(canon).host);
|
||||
} catch {
|
||||
peerOrigin = null;
|
||||
}
|
||||
}
|
||||
if (peerOrigin) {
|
||||
const pendingReset = db
|
||||
.select({ origin: schema.federationResetEvents.origin })
|
||||
.from(schema.federationResetEvents)
|
||||
.where(and(
|
||||
eq(schema.federationResetEvents.origin, peerOrigin),
|
||||
isNull(schema.federationResetEvents.resolvedAt),
|
||||
))
|
||||
.get();
|
||||
if (pendingReset) {
|
||||
return reply.code(409).send({ error: 'peer_reset_pending', statusCode: 409 });
|
||||
}
|
||||
}
|
||||
|
||||
// Federated identity: resolve or create a replicated user stub
|
||||
targetUser = resolveOrCreateReplicatedUser(homeUserId, homeInstance, db) ?? undefined;
|
||||
} else if (userId && typeof userId === 'string') {
|
||||
|
||||
@@ -457,3 +457,100 @@ describe('POST /api/social/requests — federated branch (authority + self-frien
|
||||
expect(body.requestId).toBe('incoming-req');
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /api/social/requests — federated branch (limbo-window peer_reset_pending)', () => {
|
||||
beforeEach(() => {
|
||||
seedSelf();
|
||||
resolveOriginFromHostnameMock.mockReturnValue('https://orbit.test');
|
||||
});
|
||||
|
||||
function seedResetEvent(resolvedAt: number | null): void {
|
||||
testDb.insert(schema.federationResetEvents).values({
|
||||
origin: 'https://orbit.test',
|
||||
deadEpoch: 'dead-epoch',
|
||||
newEpoch: resolvedAt === null ? null : 'new-epoch',
|
||||
detectedAt: Date.now(),
|
||||
resolvedAt,
|
||||
stubCount: 1,
|
||||
orphanedAccountCount: 0,
|
||||
}).run();
|
||||
}
|
||||
|
||||
it('returns 409 peer_reset_pending when an UNRESOLVED reset event exists for the target origin', async () => {
|
||||
seedResetEvent(null);
|
||||
// Even a stale friendship must NOT surface as `already_friends` during the limbo window.
|
||||
testDb.insert(schema.users).values({
|
||||
id: 'stub-alice',
|
||||
username: 'remote-alice@orbit.test',
|
||||
displayName: 'Alice',
|
||||
passwordHash: '!federation-replicated',
|
||||
status: 'offline',
|
||||
isAdmin: 0,
|
||||
homeInstance: 'orbit.test',
|
||||
homeUserId: 'remote-alice-old',
|
||||
createdAt: Date.now(),
|
||||
}).run();
|
||||
testDb.insert(schema.friends).values({
|
||||
userId: CALLER_ID,
|
||||
friendId: 'stub-alice',
|
||||
createdAt: Date.now(),
|
||||
}).run();
|
||||
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/social/requests',
|
||||
payload: { username: 'alice@orbit.test' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(409);
|
||||
expect(JSON.parse(res.body).error).toBe('peer_reset_pending');
|
||||
// Short-circuits before peering/lookup — neither is consulted.
|
||||
expect(ensurePeeredMock).not.toHaveBeenCalled();
|
||||
expect(lookupRemoteUserMock).not.toHaveBeenCalled();
|
||||
// No new request row created.
|
||||
expect(testDb.select().from(schema.friendRequests).all()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('proceeds normally when the reset event is RESOLVED (resolved_at set)', async () => {
|
||||
seedResetEvent(Date.now());
|
||||
ensurePeeredMock.mockResolvedValue({ status: 'active', peerId: 'peer-1' });
|
||||
lookupRemoteUserMock.mockResolvedValue({
|
||||
ok: true,
|
||||
homeUserId: 'remote-alice',
|
||||
username: 'alice',
|
||||
profile: { displayName: 'Alice', avatar: null, avatarColor: 'mint', banner: null, bio: null },
|
||||
});
|
||||
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/social/requests',
|
||||
payload: { username: 'alice@orbit.test' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(201);
|
||||
expect(ensurePeeredMock).toHaveBeenCalled();
|
||||
expect(JSON.parse(res.body).success).toBe(true);
|
||||
});
|
||||
|
||||
it('proceeds normally when NO reset event exists for the origin', async () => {
|
||||
ensurePeeredMock.mockResolvedValue({ status: 'active', peerId: 'peer-1' });
|
||||
lookupRemoteUserMock.mockResolvedValue({
|
||||
ok: true,
|
||||
homeUserId: 'remote-alice',
|
||||
username: 'alice',
|
||||
profile: { displayName: 'Alice', avatar: null, avatarColor: 'mint', banner: null, bio: null },
|
||||
});
|
||||
|
||||
const app = await buildApp();
|
||||
const res = await app.inject({
|
||||
method: 'POST',
|
||||
url: '/api/social/requests',
|
||||
payload: { username: 'alice@orbit.test' },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(201);
|
||||
expect(ensurePeeredMock).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { FastifyInstance, FastifyRequest, FastifyReply } from 'fastify';
|
||||
import { eq, and, or, ne, like, sql, inArray } from 'drizzle-orm';
|
||||
import { eq, and, or, ne, like, sql, inArray, isNull } from 'drizzle-orm';
|
||||
import { getDb, getRawDb, schema } from '../db/index.js';
|
||||
import { authenticate } from '../utils/auth.js';
|
||||
import { generateSnowflake } from '../utils/snowflake.js';
|
||||
@@ -183,6 +183,32 @@ async function handleFederatedFriendRequest(
|
||||
return reply.code(400).send({ error: 'invalid_target_domain', statusCode: 400, domain: targetDomain });
|
||||
}
|
||||
|
||||
// 1a. Limbo-window guard (federation instance-epoch self-healing §5.3).
|
||||
// If this peer's home instance was reset-detected but the admin has not yet
|
||||
// re-peered, an UNRESOLVED `federation_reset_events` row exists for its origin
|
||||
// (the peer sits in `needs_attention`, its local friendship/stub graph still
|
||||
// bound to the dead incarnation). Without this guard the re-add would surface
|
||||
// a confusing `already_friends` (stale friendship) or `peer_rejected` (the
|
||||
// needs_attention peer) — neither of which tells the user what to do. Return a
|
||||
// clear `peer_reset_pending` instead.
|
||||
//
|
||||
// `resolveOriginFromHostname` returns the peer's stored `federation_peers.origin`
|
||||
// verbatim, which is exactly the string `markPeerReset` journals as
|
||||
// `federation_reset_events.origin` (its PRIMARY KEY), so this is an O(1) indexed
|
||||
// point lookup. The common case — no reset in progress — is a single indexed miss
|
||||
// and the normal path proceeds unchanged.
|
||||
const pendingReset = db
|
||||
.select({ origin: schema.federationResetEvents.origin })
|
||||
.from(schema.federationResetEvents)
|
||||
.where(and(
|
||||
eq(schema.federationResetEvents.origin, peerOrigin),
|
||||
isNull(schema.federationResetEvents.resolvedAt),
|
||||
))
|
||||
.get();
|
||||
if (pendingReset) {
|
||||
return reply.code(409).send({ error: 'peer_reset_pending', statusCode: 409 });
|
||||
}
|
||||
|
||||
// 2. ensurePeered — block until 'active', or surface peer status as error
|
||||
const peering = await ensurePeered(peerOrigin, {
|
||||
kind: 'user_action',
|
||||
|
||||
Reference in New Issue
Block a user