Merge branch 'feat/call-state-machine-hardening'
This commit is contained in:
@@ -283,7 +283,7 @@ If a `member_add` federation event arrives for a soft-deleted channel (non-null
|
|||||||
|
|
||||||
**Request:** `{ content?: string, attachments?: string[], replyToId?: string }`
|
**Request:** `{ content?: string, attachments?: string[], replyToId?: string }`
|
||||||
|
|
||||||
**Cross-instance access:** Federated users (those with `homeInstance` set) can send messages on any DM channel where they are a member, regardless of which instance serves the request. The `requireLocalUser` gate that previously blocked federated users from DM write endpoints has been removed. DM calls work across federated instances. The caller's instance hosts the LiveKit room; remote clients connect directly. Call signaling is relayed to all active federation peers via synchronous HTTP POST (not the outbox worker). See `docs/systems/voice.md` for the full federated call architecture.
|
**Cross-instance access:** Federated users (those with `homeInstance` set) can send messages on any DM channel where they are a member, regardless of which instance serves the request. The `requireLocalUser` gate that previously blocked federated users from DM write endpoints has been removed. DM calls work across federated instances. The caller's instance hosts the LiveKit room; remote clients connect directly. Call signaling is relayed to all active federation peers via synchronous HTTP POST (not the outbox worker). Relay failures at any call state transition emit `dm_call_undeliverable { phase, terminal, failures }` to the originator — see `docs/systems/voice.md` for the full call state machine and failure surface.
|
||||||
|
|
||||||
**Validation:**
|
**Validation:**
|
||||||
- Caller must be a member (`isDmMember`)
|
- Caller must be a member (`isDmMember`)
|
||||||
|
|||||||
@@ -1134,7 +1134,11 @@ All events carry standard relay fields: `eventType`, `messageId`, `encryptionVer
|
|||||||
|
|
||||||
- Fire-and-forget to each remote DM participant's home instance via `sendCallRelay(origin, [event], { peeringTimeoutMs: 0 })`. Typing is an ephemeral hint — lost packets are acceptable and there is no user-facing failure surface.
|
- Fire-and-forget to each remote DM participant's home instance via `sendCallRelay(origin, [event], { peeringTimeoutMs: 0 })`. Typing is an ephemeral hint — lost packets are acceptable and there is no user-facing failure surface.
|
||||||
|
|
||||||
**Call-start failure surfacing.** `sendFederatedCallStart` aggregates targeted-peer results and emits `dm_call_undeliverable` to the caller for failed targeted peers. See `docs/systems/voice.md` and `docs/systems/websocket.md` for the event contract.
|
**Call-start failure surfacing.** `sendFederatedCallStart` aggregates targeted-peer results and emits `dm_call_undeliverable { phase: 'start' }` to the caller for failed targeted peers. See `docs/systems/voice.md` and `docs/systems/websocket.md` for the event contract.
|
||||||
|
|
||||||
|
**Accept / reject / end relay discipline.** Every `handleDmCall{Accept,Reject,End}` Path-2 branch awaits `sendCallRelay` and emits `dm_call_undeliverable { phase, terminal, failures }` to the originator on failure. Accept is pessimistic-rollback (terminal: true — local `FederatedCallEntry` cleared, optimistic `dm_call_accepted` walked back via `sendToFederatedCallUsers`); reject and end are optimistic (terminal: false — state already cleared, informational toast only). Path-1 fan-outs via `sendFederatedCallAccept` / `sendFederatedCallEnd` / `fanOutCallEvent` return `CallFanoutFailure[]` and the host-side caller receives `dm_call_undeliverable { terminal: false }` listing peers that were not reached.
|
||||||
|
|
||||||
|
**Ring-timeout fan-out.** `ConnectionManager.createDmRoom`'s 60 s ringing auto-clean now invokes a registered hook (`setRingTimeoutFanoutHook`, registered from `ws/events.ts:registerCallRelayHooks`) that fans `dm_call_end` out to remote peers, so stranded Path-A/B ringees on other instances exit their ring state instead of lingering until their own 60 s cleanup fires.
|
||||||
|
|
||||||
### Call Flows
|
### Call Flows
|
||||||
|
|
||||||
@@ -1269,6 +1273,4 @@ If the `federation_mutation_log` table exists but is empty, populates it with `c
|
|||||||
|
|
||||||
See `docs/federation-production-roadmap.md` for open items (FED-001 through FED-013).
|
See `docs/federation-production-roadmap.md` for open items (FED-001 through FED-013).
|
||||||
|
|
||||||
- **Accept-relay failure dead end.** If Bob on B accepts a call from Alice on A and the `dm_call_accept` S2S relay back to A fails, Alice's client does not exit the `outgoingCall` state until the 60 s ring timeout fires `dm_call_ended`. The client clears `outgoingCall` only on `dm_call_accepted | rejected | ended` (`useWebSocket.ts`), with no LiveKit participant-join fallback. Surfacing this requires a B-side event and call-state rollback; deferred.
|
- **Accept/reject/end relay failures now surfaced.** All three federated call-state transitions emit `dm_call_undeliverable { phase, terminal, failures }` to the originator on relay failure — accept rolls back optimistic state (terminal: true), reject/end keep the optimistic clear and emit an informational toast (terminal: false). See `docs/systems/voice.md` "Call relay failure surface" for the full contract. The host-side ring timeout also fans `dm_call_end` out to peers so stranded Path-A/B ringees exit the ring. One remaining edge documented in voice.md: non-host end-relay failure leaves the host's local `activeDmCall` marker until manual cleanup.
|
||||||
- **End-relay failure dead end.** Similar mechanism, lower severity because LiveKit `ParticipantDisconnected` typically unwinds the voice UI on the host side; local DM call state still lingers to the 60 s timeout. Deferred.
|
|
||||||
- **Path-B reject-relay failure dead end.** Third-instance user (Carol on C) rings for a call hosted on A via Path B; if her `dm_call_reject` C→A relay fails, A never deducts her from the pending ringees, so her name persists in the caller's "still ringing" set until the 60 s timeout. Same class as accept-failure; deferred.
|
|
||||||
|
|||||||
+17
-2
@@ -60,9 +60,24 @@ All `dm_call_*` signaling events (`start`, `accept`, `reject`, `end`) are relaye
|
|||||||
|
|
||||||
**Auto-peering at send time.** If the target origin has no active peer record, `sendCallRelay` races an `ensurePeered` handshake against a 3 s deadline (`CALL_PEERING_TIMEOUT_MS`). On success the relay POSTs normally; on timeout it returns `peer_transient_failure` without aborting the background handshake, so a subsequent attempt typically succeeds. Typing (`sendTypingRelay`) passes `peeringTimeoutMs: 0` — the POST is skipped for non-active peers and a warm-up `ensurePeered` runs in the background.
|
**Auto-peering at send time.** If the target origin has no active peer record, `sendCallRelay` races an `ensurePeered` handshake against a 3 s deadline (`CALL_PEERING_TIMEOUT_MS`). On success the relay POSTs normally; on timeout it returns `peer_transient_failure` without aborting the background handshake, so a subsequent attempt typically succeeds. Typing (`sendTypingRelay`) passes `peeringTimeoutMs: 0` — the POST is skipped for non-active peers and a warm-up `ensurePeered` runs in the background.
|
||||||
|
|
||||||
**Call-start failure surface.** `sendFederatedCallStart` aggregates results from its targeted-peer relays (peers whose origin matches a DM member's `homeInstance`) and emits a single `dm_call_undeliverable` event to the caller when any targeted peer fails. `terminal: true` (no successful targeted relay AND no connected local non-caller member) also destroys the local ring room so the caller's UI clears immediately; `terminal: false` leaves the call ringing for reachable recipients. The all-peers broadcast (Path-B fallback to non-member-hosting peers) logs failures only — they are not surfaced.
|
**Call relay failure surface.** Every `dm_call_{start,accept,reject,end}` relay is failure-aware. On failure the originating server emits a `dm_call_undeliverable` event with a `phase` discriminator identifying which action failed. Client copy is phase-specific; state rollback depends on the phase.
|
||||||
|
|
||||||
Accept and end relay failures are NOT surfaced today; see the federation doc's "Known issues" section for the deferred accept-failure dead-end.
|
| `phase` | `terminal` | Emitted when | Client action |
|
||||||
|
|---------|------------|--------------|---------------|
|
||||||
|
| `start` | true | No plausible recipient after targeted-peer fan-out; ring room destroyed. | Clear `outgoingCall`, disconnect LK, warning toast. |
|
||||||
|
| `start` | false | Some targeted peers failed but reachable recipients remain; ring continues. | Keep state; info toast. |
|
||||||
|
| `accept` | true | Acceptor's B→host relay failed; optimistic state is rolled back on B. | Clear `activeDmCall` + `incomingCall`, disconnect LK, warning toast. |
|
||||||
|
| `accept` | false | Host → peer fan-out of accept failed; local host call continues. | No state change; info toast. |
|
||||||
|
| `reject` | false | Rejector's relay to host failed OR host's fan-out after a local reject failed; state already cleared. | No state change; info toast. |
|
||||||
|
| `end` | false | Ender's relay to host failed OR host's fan-out after a local end failed; state already cleared. | No state change; info toast. |
|
||||||
|
|
||||||
|
**Accept-rollback semantics.** `handleDmCallAccept` Path 2 transitions the `FederatedCallEntry` to active and broadcasts `dm_call_accepted` optimistically so the acceptor's UI flips immediately. If the B→host relay fails, the server clears the entry, fans `dm_call_undeliverable { phase: 'accept', terminal: true }` out to all ringed users on B (via `sendToFederatedCallUsers`), and the client tears its call state back down.
|
||||||
|
|
||||||
|
**Reject / end are optimistic.** Local state is cleared before the relay is awaited because the user's intent is to terminate. If the relay fails, the originator receives an informational `dm_call_undeliverable { terminal: false }` so they know remote peers may briefly display stale state; no local rollback.
|
||||||
|
|
||||||
|
**Ring-timeout fan-out.** When the host's 60 s ringing timeout fires without an accept, `dm_call_end` is fanned out to all remote peers so stranded Path-A/B ringees on other instances exit their ring state instead of lingering. Registered via `connectionManager.setRingTimeoutFanoutHook` from the WS events module.
|
||||||
|
|
||||||
|
**Remaining edge.** When a non-host participant (Bob on B) ends an active call and the relay to the host (Alice on A) fails, Alice's `activeDmCall` marker lingers until she manually ends — LK `ParticipantDisconnected` tears down her voice UI but does not clear the DM-call marker. This is a host-side cleanup concern, tracked separately.
|
||||||
|
|
||||||
### Dual-Path Processing
|
### Dual-Path Processing
|
||||||
|
|
||||||
|
|||||||
@@ -170,7 +170,7 @@ reason: `'displaced'` (new tab) | `'session_closed'`
|
|||||||
| `dm_call_accepted` | dmChannelId?, federatedCallId? | DM members |
|
| `dm_call_accepted` | dmChannelId?, federatedCallId? | DM members |
|
||||||
| `dm_call_rejected` | dmChannelId?, federatedCallId? | DM members |
|
| `dm_call_rejected` | dmChannelId?, federatedCallId? | DM members |
|
||||||
| `dm_call_ended` | dmChannelId?, federatedCallId? | DM members |
|
| `dm_call_ended` | dmChannelId?, federatedCallId? | DM members |
|
||||||
| `dm_call_undeliverable` | Sent to caller when a call-start relay to one or more targeted peers fails. `failures[]` enumerates each failed peer with a `reason` (`peer_rejected` / `peer_awaiting_approval` / `peer_transient_failure` / `livekit_unavailable`). `terminal: true` means the local ring was destroyed; `false` means the call continues for reachable recipients. | caller only |
|
| `dm_call_undeliverable` | Sent to the originator when a call relay (start / accept / reject / end) to one or more peers fails. Includes `phase: 'start' \| 'accept' \| 'reject' \| 'end'` identifying the action; `failures[]` enumerates failed peers with a `reason` (`peer_rejected` / `peer_awaiting_approval` / `peer_transient_failure` / `livekit_unavailable`). `terminal: true` means local call state should be (or has been) torn down; `terminal: false` is informational. See `docs/systems/voice.md` for the full phase × terminal matrix. | originator (caller / acceptor / rejector / ender) |
|
||||||
|
|
||||||
### Social
|
### Social
|
||||||
| type | fields | scope |
|
| type | fields | scope |
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ import { adminRoutes } from './routes/admin.js';
|
|||||||
import { gifRoutes } from './routes/gif.js';
|
import { gifRoutes } from './routes/gif.js';
|
||||||
import { federationRoutes } from './routes/federation.js';
|
import { federationRoutes } from './routes/federation.js';
|
||||||
import { startFederationWorkers, stopFederationWorkers } from './utils/federationWorker.js';
|
import { startFederationWorkers, stopFederationWorkers } from './utils/federationWorker.js';
|
||||||
|
import { registerCallRelayHooks } from './ws/events.js';
|
||||||
|
|
||||||
import { registerWebSocket } from './ws/handler.js';
|
import { registerWebSocket } from './ws/handler.js';
|
||||||
import path from 'path';
|
import path from 'path';
|
||||||
@@ -124,6 +125,9 @@ async function main(): Promise<void> {
|
|||||||
// Log ffmpeg availability at startup (so admins see the warning immediately)
|
// Log ffmpeg availability at startup (so admins see the warning immediately)
|
||||||
checkFfmpeg();
|
checkFfmpeg();
|
||||||
|
|
||||||
|
// Register WS-layer call relay hooks (ring-timeout fan-out).
|
||||||
|
registerCallRelayHooks();
|
||||||
|
|
||||||
// Start federation background workers (outbox delivery, file download, health check)
|
// Start federation background workers (outbox delivery, file download, health check)
|
||||||
startFederationWorkers();
|
startFederationWorkers();
|
||||||
|
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ import { getDb, getRawDb, schema } from '../db/index.js';
|
|||||||
import { config } from '../config.js';
|
import { config } from '../config.js';
|
||||||
import { connectionManager } from '../ws/handler.js';
|
import { connectionManager } from '../ws/handler.js';
|
||||||
import type { FederatedCallEntry, DmRoomMeta } from '../ws/handler.js';
|
import type { FederatedCallEntry, DmRoomMeta } from '../ws/handler.js';
|
||||||
|
import { mapCallReasonToEventReason, type CallFanoutFailure } from '../utils/federationOutbox.js';
|
||||||
|
import type { DmCallUndeliverableFailure } from '@backspace/shared';
|
||||||
import { sanitizeUser } from '../utils/sanitize.js';
|
import { sanitizeUser } from '../utils/sanitize.js';
|
||||||
import { deleteAttachmentFiles, deleteUploadFile } from '../utils/fileCleanup.js';
|
import { deleteAttachmentFiles, deleteUploadFile } from '../utils/fileCleanup.js';
|
||||||
import { tombstoneUser, collectDeletionBroadcastTargets, collectProfileBroadcastTargetIds } from '../utils/userDeletion.js';
|
import { tombstoneUser, collectDeletionBroadcastTargets, collectProfileBroadcastTargetIds } from '../utils/userDeletion.js';
|
||||||
@@ -4464,10 +4466,14 @@ function processDmCallAcceptEvent(
|
|||||||
|
|
||||||
// Fan out to ALL other remote instances (exclude the one that sent the accept)
|
// Fan out to ALL other remote instances (exclude the one that sent the accept)
|
||||||
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
||||||
fanOutCallEvent(dmChannelId!, event.federatedId, 'dm_call_accept', {
|
const hostCallerId = (room.metadata as DmRoomMeta).callerId;
|
||||||
|
const localDmId = dmChannelId!;
|
||||||
|
void fanOutCallEvent(localDmId, event.federatedId, 'dm_call_accept', {
|
||||||
call: { acceptor: event.call.acceptor },
|
call: { acceptor: event.call.acceptor },
|
||||||
}, normalizedSource, db).catch(err =>
|
}, normalizedSource, db).then(failures => {
|
||||||
console.error('[federation] Fan-out dm_call_accept failed:', err)
|
emitHostFanoutUndeliverable(hostCallerId, localDmId, event.federatedId!, 'accept', failures);
|
||||||
|
}).catch(err =>
|
||||||
|
console.error('[federation] Fan-out dm_call_accept threw:', err),
|
||||||
);
|
);
|
||||||
} else {
|
} else {
|
||||||
// We're a REMOTE instance receiving fan-out — transition local state
|
// We're a REMOTE instance receiving fan-out — transition local state
|
||||||
@@ -4517,19 +4523,23 @@ function processDmCallRejectEvent(
|
|||||||
const room = dmChannelId ? connectionManager.getRoom(dmChannelId) : undefined;
|
const room = dmChannelId ? connectionManager.getRoom(dmChannelId) : undefined;
|
||||||
if (room && room.roomType === 'dm') {
|
if (room && room.roomType === 'dm') {
|
||||||
const meta = room.metadata as DmRoomMeta;
|
const meta = room.metadata as DmRoomMeta;
|
||||||
|
const hostCallerId = meta.callerId;
|
||||||
|
const localDmId = dmChannelId!;
|
||||||
connectionManager.clearVoiceWs(meta.callerId);
|
connectionManager.clearVoiceWs(meta.callerId);
|
||||||
connectionManager.destroyRoom(dmChannelId!);
|
connectionManager.destroyRoom(localDmId);
|
||||||
|
|
||||||
connectionManager.sendToDmMembers(dmChannelId!, {
|
connectionManager.sendToDmMembers(localDmId, {
|
||||||
type: 'dm_call_rejected',
|
type: 'dm_call_rejected',
|
||||||
dmChannelId: dmChannelId!,
|
dmChannelId: localDmId,
|
||||||
});
|
});
|
||||||
|
|
||||||
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
||||||
fanOutCallEvent(dmChannelId!, event.federatedId, 'dm_call_end', {
|
void fanOutCallEvent(localDmId, event.federatedId, 'dm_call_end', {
|
||||||
call: { endedBy: event.call.rejector },
|
call: { endedBy: event.call.rejector },
|
||||||
}, normalizedSource, db).catch(err =>
|
}, normalizedSource, db).then(failures => {
|
||||||
console.error('[federation] Fan-out dm_call_end (reject) failed:', err)
|
emitHostFanoutUndeliverable(hostCallerId, localDmId, event.federatedId!, 'reject', failures);
|
||||||
|
}).catch(err =>
|
||||||
|
console.error('[federation] Fan-out dm_call_end (reject) threw:', err),
|
||||||
);
|
);
|
||||||
} else {
|
} else {
|
||||||
const fedCall = connectionManager.getFederatedCall(event.federatedId);
|
const fedCall = connectionManager.getFederatedCall(event.federatedId);
|
||||||
@@ -4572,23 +4582,27 @@ function processDmCallEndEvent(
|
|||||||
const room = dmChannelId ? connectionManager.getRoom(dmChannelId) : undefined;
|
const room = dmChannelId ? connectionManager.getRoom(dmChannelId) : undefined;
|
||||||
if (room && room.roomType === 'dm') {
|
if (room && room.roomType === 'dm') {
|
||||||
const meta = room.metadata as DmRoomMeta;
|
const meta = room.metadata as DmRoomMeta;
|
||||||
|
const hostCallerId = meta.callerId;
|
||||||
|
const localDmId = dmChannelId!;
|
||||||
connectionManager.clearVoiceWs(meta.callerId);
|
connectionManager.clearVoiceWs(meta.callerId);
|
||||||
for (const pid of room.participants) {
|
for (const pid of room.participants) {
|
||||||
connectionManager.clearVoiceUserStatus(pid);
|
connectionManager.clearVoiceUserStatus(pid);
|
||||||
connectionManager.clearVoiceWs(pid);
|
connectionManager.clearVoiceWs(pid);
|
||||||
}
|
}
|
||||||
connectionManager.destroyRoom(dmChannelId!);
|
connectionManager.destroyRoom(localDmId);
|
||||||
|
|
||||||
connectionManager.sendToDmMembers(dmChannelId!, {
|
connectionManager.sendToDmMembers(localDmId, {
|
||||||
type: 'dm_call_ended',
|
type: 'dm_call_ended',
|
||||||
dmChannelId: dmChannelId!,
|
dmChannelId: localDmId,
|
||||||
});
|
});
|
||||||
|
|
||||||
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
const normalizedSource = sourceInstance.startsWith('http') ? sourceInstance : `https://${sourceInstance}`;
|
||||||
fanOutCallEvent(dmChannelId!, event.federatedId, 'dm_call_end', {
|
void fanOutCallEvent(localDmId, event.federatedId, 'dm_call_end', {
|
||||||
call: { endedBy: event.call.endedBy },
|
call: { endedBy: event.call.endedBy },
|
||||||
}, normalizedSource, db).catch(err =>
|
}, normalizedSource, db).then(failures => {
|
||||||
console.error('[federation] Fan-out dm_call_end failed:', err)
|
emitHostFanoutUndeliverable(hostCallerId, localDmId, event.federatedId!, 'end', failures);
|
||||||
|
}).catch(err =>
|
||||||
|
console.error('[federation] Fan-out dm_call_end threw:', err),
|
||||||
);
|
);
|
||||||
} else {
|
} else {
|
||||||
const fedCall = connectionManager.getFederatedCall(event.federatedId);
|
const fedCall = connectionManager.getFederatedCall(event.federatedId);
|
||||||
@@ -5144,7 +5158,7 @@ async function fanOutCallEvent(
|
|||||||
extraFields: Partial<FederationRelayEvent>,
|
extraFields: Partial<FederationRelayEvent>,
|
||||||
excludeOrigin: string | undefined,
|
excludeOrigin: string | undefined,
|
||||||
db: ReturnType<typeof getDb>,
|
db: ReturnType<typeof getDb>,
|
||||||
): Promise<void> {
|
): Promise<CallFanoutFailure[]> {
|
||||||
const members = db.select({ homeInstance: schema.users.homeInstance })
|
const members = db.select({ homeInstance: schema.users.homeInstance })
|
||||||
.from(schema.dmMembers)
|
.from(schema.dmMembers)
|
||||||
.innerJoin(schema.users, eq(schema.dmMembers.userId, schema.users.id))
|
.innerJoin(schema.users, eq(schema.dmMembers.userId, schema.users.id))
|
||||||
@@ -5161,7 +5175,7 @@ async function fanOutCallEvent(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (targets.size === 0) return;
|
if (targets.size === 0) return [];
|
||||||
|
|
||||||
const relayEvent: FederationRelayEvent = {
|
const relayEvent: FederationRelayEvent = {
|
||||||
eventType,
|
eventType,
|
||||||
@@ -5172,11 +5186,51 @@ async function fanOutCallEvent(
|
|||||||
...extraFields,
|
...extraFields,
|
||||||
} as FederationRelayEvent;
|
} as FederationRelayEvent;
|
||||||
|
|
||||||
await Promise.all(
|
const labelByOrigin = new Map<string, string | null>();
|
||||||
Array.from(targets).map(origin =>
|
for (const r of db.select({ origin: schema.federationPeers.origin, instanceName: schema.federationPeers.instanceName })
|
||||||
sendCallRelay(origin, [relayEvent]).catch(err =>
|
.from(schema.federationPeers)
|
||||||
console.error(`[federation] Fan-out ${eventType} to ${origin} failed:`, err)
|
.all()) {
|
||||||
)
|
labelByOrigin.set(r.origin, r.instanceName ?? null);
|
||||||
)
|
}
|
||||||
|
|
||||||
|
const results = await Promise.all(
|
||||||
|
Array.from(targets).map(async origin => ({ origin, result: await sendCallRelay(origin, [relayEvent]) })),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const failures: CallFanoutFailure[] = [];
|
||||||
|
for (const { origin, result } of results) {
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] Fan-out ${eventType} to ${origin} failed (${result.reason}): ${result.error}`);
|
||||||
|
failures.push({
|
||||||
|
origin,
|
||||||
|
peerLabel: labelByOrigin.get(origin) ?? undefined,
|
||||||
|
reason: mapCallReasonToEventReason(result.reason),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return failures;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Emit a non-terminal dm_call_undeliverable for a host-side fan-out failure. */
|
||||||
|
function emitHostFanoutUndeliverable(
|
||||||
|
userId: string,
|
||||||
|
dmChannelId: string,
|
||||||
|
federatedId: string,
|
||||||
|
phase: 'accept' | 'reject' | 'end',
|
||||||
|
fanoutFailures: CallFanoutFailure[],
|
||||||
|
): void {
|
||||||
|
if (fanoutFailures.length === 0) return;
|
||||||
|
const failures: DmCallUndeliverableFailure[] = fanoutFailures.map(f => ({
|
||||||
|
reason: f.reason,
|
||||||
|
peerOrigin: f.origin,
|
||||||
|
peerLabel: f.peerLabel,
|
||||||
|
}));
|
||||||
|
connectionManager.sendToUser(userId, {
|
||||||
|
type: 'dm_call_undeliverable',
|
||||||
|
dmChannelId,
|
||||||
|
federatedCallId: federatedId,
|
||||||
|
terminal: false,
|
||||||
|
phase,
|
||||||
|
failures,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ import * as schema from '../db/schema.js';
|
|||||||
import { eq, and, inArray } from 'drizzle-orm';
|
import { eq, and, inArray } from 'drizzle-orm';
|
||||||
import { generateSnowflake } from './snowflake.js';
|
import { generateSnowflake } from './snowflake.js';
|
||||||
import crypto from 'node:crypto';
|
import crypto from 'node:crypto';
|
||||||
import type { FederationRelayEvent, FederationRelayParticipant, FederationRelayAttachment, DmMessageWithUser, FederationRelayRequest } from '@backspace/shared';
|
import type { FederationRelayEvent, FederationRelayParticipant, FederationRelayAttachment, DmMessageWithUser, FederationRelayRequest, DmCallUndeliverableReason } from '@backspace/shared';
|
||||||
import { getOurOrigin, buildFederationHeaders, generateHmacSecret } from './federationAuth.js';
|
import { getOurOrigin, buildFederationHeaders, generateHmacSecret } from './federationAuth.js';
|
||||||
import { extractDomain } from '../routes/federation.js';
|
import { extractDomain } from '../routes/federation.js';
|
||||||
import { racePeering, ensurePeered } from './federationPeering.js';
|
import { racePeering, ensurePeered } from './federationPeering.js';
|
||||||
@@ -527,6 +527,23 @@ export type CallRelayResult =
|
|||||||
| { ok: true }
|
| { ok: true }
|
||||||
| { ok: false; reason: CallRelayFailureReason; error: string };
|
| { ok: false; reason: CallRelayFailureReason; error: string };
|
||||||
|
|
||||||
|
/** Per-peer failure record returned by Path-1 call fan-out helpers. */
|
||||||
|
export interface CallFanoutFailure {
|
||||||
|
origin: string;
|
||||||
|
peerLabel?: string;
|
||||||
|
reason: DmCallUndeliverableReason;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Map a sendCallRelay reason to the dm_call_undeliverable event-surface reason. */
|
||||||
|
export function mapCallReasonToEventReason(reason: CallRelayFailureReason): DmCallUndeliverableReason {
|
||||||
|
switch (reason) {
|
||||||
|
case 'peer_rejected': return 'peer_rejected';
|
||||||
|
case 'peer_awaiting_approval': return 'peer_awaiting_approval';
|
||||||
|
case 'peer_transient_failure': return 'peer_transient_failure';
|
||||||
|
case 'post_failed': return 'peer_transient_failure'; // 4xx looks transient to users
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Send call signaling events directly to a remote peer (bypasses outbox).
|
* Send call signaling events directly to a remote peer (bypasses outbox).
|
||||||
* Latency-sensitive: if no active peer exists, race an ensurePeered handshake
|
* Latency-sensitive: if no active peer exists, race an ensurePeered handshake
|
||||||
|
|||||||
@@ -0,0 +1,300 @@
|
|||||||
|
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||||
|
import Database from 'better-sqlite3';
|
||||||
|
import { drizzle } from 'drizzle-orm/better-sqlite3';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import * as schema from '../db/schema.js';
|
||||||
|
import { setWorkerId } from '../utils/snowflake.js';
|
||||||
|
|
||||||
|
setWorkerId(1);
|
||||||
|
|
||||||
|
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||||
|
type TestDb = ReturnType<typeof drizzle<typeof schema>>;
|
||||||
|
|
||||||
|
let testDb: TestDb;
|
||||||
|
|
||||||
|
vi.mock('../db/index.js', () => ({
|
||||||
|
getDb: () => testDb,
|
||||||
|
schema,
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock('../utils/federationAuth.js', () => ({
|
||||||
|
getOurOrigin: () => 'https://local.example',
|
||||||
|
buildFederationHeaders: () => ({}),
|
||||||
|
generateHmacSecret: () => 'test-secret',
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Mock sendCallRelay so tests can control relay-result per case.
|
||||||
|
const sendCallRelayMock = vi.fn();
|
||||||
|
vi.mock('../utils/federationOutbox.js', async () => {
|
||||||
|
const actual = await vi.importActual<typeof import('../utils/federationOutbox.js')>('../utils/federationOutbox.js');
|
||||||
|
return {
|
||||||
|
...actual,
|
||||||
|
sendCallRelay: (...args: unknown[]) => sendCallRelayMock(...args),
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
function applyMigrations(db: Database.Database): void {
|
||||||
|
const migrationsDir = path.resolve(__dirname, '../../drizzle');
|
||||||
|
const files = fs.readdirSync(migrationsDir).filter(f => f.endsWith('.sql')).sort();
|
||||||
|
for (const f of files) {
|
||||||
|
const sql = fs.readFileSync(path.join(migrationsDir, f), 'utf8');
|
||||||
|
const statements = sql.split(/-->\s*statement-breakpoint/);
|
||||||
|
for (const stmt of statements) {
|
||||||
|
const clean = stmt.trim();
|
||||||
|
if (clean) db.exec(clean);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function seedPeerLabel(origin: string, instanceName: string): void {
|
||||||
|
testDb.insert(schema.federationPeers).values({
|
||||||
|
id: `peer-${origin}`,
|
||||||
|
origin,
|
||||||
|
hmacSecret: 'secret',
|
||||||
|
status: 'active',
|
||||||
|
instanceName,
|
||||||
|
lastSyncedAt: 0,
|
||||||
|
createdAt: Date.now(),
|
||||||
|
}).run();
|
||||||
|
}
|
||||||
|
|
||||||
|
function seedLocalUser(id: string, homeUserId: string | null): void {
|
||||||
|
testDb.insert(schema.users).values({
|
||||||
|
id,
|
||||||
|
username: id,
|
||||||
|
passwordHash: 'test',
|
||||||
|
homeUserId,
|
||||||
|
homeInstance: null,
|
||||||
|
createdAt: Date.now(),
|
||||||
|
}).run();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function importSUT() {
|
||||||
|
// Late import so the module picks up the mocked dependencies.
|
||||||
|
return await import('./events.js');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function importManager() {
|
||||||
|
const mod = await import('./handler.js');
|
||||||
|
return mod.connectionManager;
|
||||||
|
}
|
||||||
|
|
||||||
|
type FedCallEntry = import('./handler.js').FederatedCallEntry;
|
||||||
|
|
||||||
|
function makeFedCall(partial: Partial<FedCallEntry> = {}): FedCallEntry {
|
||||||
|
return {
|
||||||
|
dmChannelId: 'dm-1',
|
||||||
|
federatedId: `fed-${Math.random().toString(36).slice(2, 10)}`,
|
||||||
|
callerId: 'caller-user',
|
||||||
|
callerHomeUserId: 'caller@pi',
|
||||||
|
federatedCallHost: 'https://pi.example',
|
||||||
|
livekitUrl: 'wss://pi.example/lk',
|
||||||
|
tokens: new Map([['acceptor@vm', 'token']]),
|
||||||
|
ringedUserIds: ['acceptor-user'],
|
||||||
|
state: 'ringing',
|
||||||
|
startedAt: Date.now(),
|
||||||
|
...partial,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
const sqlite = new Database(':memory:');
|
||||||
|
testDb = drizzle(sqlite, { schema });
|
||||||
|
applyMigrations(sqlite);
|
||||||
|
seedPeerLabel('https://pi.example', 'Pi-Instance');
|
||||||
|
seedLocalUser('acceptor-user', 'acceptor@vm');
|
||||||
|
seedLocalUser('caller-user', 'caller@pi');
|
||||||
|
sendCallRelayMock.mockReset();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('handleDmCallEnd Path-2 relay failure', () => {
|
||||||
|
it('emits dm_call_undeliverable { phase:"end", terminal:false } when the relay fails', async () => {
|
||||||
|
const { handleDmCallEndForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall({ state: 'active' });
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: false, reason: 'peer_transient_failure', error: 'timeout' });
|
||||||
|
|
||||||
|
await handleDmCallEndForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
);
|
||||||
|
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(1);
|
||||||
|
const ev = undelivCalls[0]![1] as { phase: string; terminal: boolean };
|
||||||
|
expect(ev.phase).toBe('end');
|
||||||
|
expect(ev.terminal).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not emit undeliverable on relay success', async () => {
|
||||||
|
const { handleDmCallEndForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall({ state: 'active' });
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: true });
|
||||||
|
|
||||||
|
await handleDmCallEndForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
);
|
||||||
|
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('handleDmCallReject Path-2 relay failure', () => {
|
||||||
|
it('emits dm_call_undeliverable { phase:"reject", terminal:false } to the rejector when the relay fails', async () => {
|
||||||
|
const { handleDmCallRejectForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall();
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: false, reason: 'peer_rejected', error: 'rejected' });
|
||||||
|
|
||||||
|
await handleDmCallRejectForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
);
|
||||||
|
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(1);
|
||||||
|
const ev = undelivCalls[0]![1] as { phase: string; terminal: boolean };
|
||||||
|
expect(ev.phase).toBe('reject');
|
||||||
|
expect(ev.terminal).toBe(false);
|
||||||
|
// Local state was cleared before the relay even fired.
|
||||||
|
expect(connectionManager.getFederatedCall(fedCall.federatedId)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not emit undeliverable on relay success', async () => {
|
||||||
|
const { handleDmCallRejectForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall();
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: true });
|
||||||
|
|
||||||
|
await handleDmCallRejectForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
);
|
||||||
|
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('ring-timeout fan-out hook', () => {
|
||||||
|
it('invokes the fan-out hook when the ring timer fires', async () => {
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const hook = vi.fn(async () => {});
|
||||||
|
connectionManager.setRingTimeoutFanoutHook(hook);
|
||||||
|
|
||||||
|
vi.useFakeTimers();
|
||||||
|
try {
|
||||||
|
connectionManager.createDmRoom('dm-ringout-test', 'caller-ringout');
|
||||||
|
vi.advanceTimersByTime(60_000 + 10);
|
||||||
|
} finally {
|
||||||
|
vi.useRealTimers();
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(hook).toHaveBeenCalledWith('dm-ringout-test', 'caller-ringout');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('registerCallRelayHooks wires up the hook', async () => {
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
const { registerCallRelayHooks } = await importSUT();
|
||||||
|
|
||||||
|
const setSpy = vi.spyOn(connectionManager, 'setRingTimeoutFanoutHook');
|
||||||
|
registerCallRelayHooks();
|
||||||
|
expect(setSpy).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('handleDmCallAccept Path-2 relay failure', () => {
|
||||||
|
it('emits dm_call_undeliverable { phase:"accept", terminal:true } to the acceptor ONLY and clears the fedCall when the relay fails', async () => {
|
||||||
|
const { handleDmCallAcceptForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall({
|
||||||
|
// Two ringed users — a group DM where only one accepts. Only the acceptor
|
||||||
|
// should receive the terminal undeliverable; the other ringee must stay
|
||||||
|
// in ring state so their own timeout/reject path governs teardown.
|
||||||
|
ringedUserIds: ['acceptor-user', 'other-ringee'],
|
||||||
|
});
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
const sendToCallUsersSpy = vi.spyOn(connectionManager, 'sendToFederatedCallUsers');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: false, reason: 'peer_transient_failure', error: 'timeout' });
|
||||||
|
|
||||||
|
await handleDmCallAcceptForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
{} as never,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(connectionManager.getFederatedCall(fedCall.federatedId)).toBeUndefined();
|
||||||
|
|
||||||
|
// Terminal undeliverable went to the acceptor only.
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(1);
|
||||||
|
expect(undelivCalls[0]![0]).toBe('acceptor-user');
|
||||||
|
const undeliverable = undelivCalls[0]![1] as { phase: string; terminal: boolean; failures: unknown[] };
|
||||||
|
expect(undeliverable.phase).toBe('accept');
|
||||||
|
expect(undeliverable.terminal).toBe(true);
|
||||||
|
expect(undeliverable.failures).toHaveLength(1);
|
||||||
|
|
||||||
|
// No undeliverable broadcast to all ringed users.
|
||||||
|
const callUsersCalls = sendToCallUsersSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(callUsersCalls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not emit undeliverable on relay success', async () => {
|
||||||
|
const { handleDmCallAcceptForTest } = await importSUT();
|
||||||
|
const connectionManager = await importManager();
|
||||||
|
|
||||||
|
const fedCall = makeFedCall();
|
||||||
|
connectionManager.createFederatedCall(fedCall);
|
||||||
|
const sendToUserSpy = vi.spyOn(connectionManager, 'sendToUser');
|
||||||
|
sendCallRelayMock.mockResolvedValue({ ok: true });
|
||||||
|
|
||||||
|
await handleDmCallAcceptForTest(
|
||||||
|
{ federatedCallId: fedCall.federatedId },
|
||||||
|
fedCall.ringedUserIds[0]!,
|
||||||
|
{} as never,
|
||||||
|
);
|
||||||
|
|
||||||
|
const undelivCalls = sendToUserSpy.mock.calls.filter(([, ev]) =>
|
||||||
|
(ev as { type: string }).type === 'dm_call_undeliverable',
|
||||||
|
);
|
||||||
|
expect(undelivCalls).toHaveLength(0);
|
||||||
|
expect(connectionManager.getFederatedCall(fedCall.federatedId)?.state).toBe('active');
|
||||||
|
connectionManager.clearFederatedCall(fedCall.federatedId);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -7,7 +7,8 @@ import type { VoiceRoom, DmRoomMeta, SpaceRoomMeta } from './handler.js';
|
|||||||
import { isMember, getChannelSpaceId, isDmMember, hasPermission, computePermissions, PermissionBits } from '../utils/permissions.js';
|
import { isMember, getChannelSpaceId, isDmMember, hasPermission, computePermissions, PermissionBits } from '../utils/permissions.js';
|
||||||
import { broadcastDmMessage, getDmMessageWithUser } from '../routes/dm.js';
|
import { broadcastDmMessage, getDmMessageWithUser } from '../routes/dm.js';
|
||||||
import { MAX_MESSAGE_LENGTH, type MessageWithUser, type Attachment, type DmMessageWithUser, type Embed, type Activity, type ActivityType, type ActivityTimestamps, type ActivityAssets, type ServerEvent, type DmCallUndeliverableFailure, type DmCallUndeliverableReason } from '@backspace/shared';
|
import { MAX_MESSAGE_LENGTH, type MessageWithUser, type Attachment, type DmMessageWithUser, type Embed, type Activity, type ActivityType, type ActivityTimestamps, type ActivityAssets, type ServerEvent, type DmCallUndeliverableFailure, type DmCallUndeliverableReason } from '@backspace/shared';
|
||||||
import type { CallRelayFailureReason } from '../utils/federationOutbox.js';
|
import type { CallRelayResult, CallFanoutFailure } from '../utils/federationOutbox.js';
|
||||||
|
import { mapCallReasonToEventReason } from '../utils/federationOutbox.js';
|
||||||
import { ACTIVITY_LIMITS } from '@backspace/shared/src/activities.js';
|
import { ACTIVITY_LIMITS } from '@backspace/shared/src/activities.js';
|
||||||
import { sanitizeUser } from '../utils/sanitize.js';
|
import { sanitizeUser } from '../utils/sanitize.js';
|
||||||
import { deleteAttachmentFiles } from '../utils/fileCleanup.js';
|
import { deleteAttachmentFiles } from '../utils/fileCleanup.js';
|
||||||
@@ -196,13 +197,19 @@ export function handleClientEvent(
|
|||||||
handleDmCallStart(event, userId, username, ws);
|
handleDmCallStart(event, userId, username, ws);
|
||||||
break;
|
break;
|
||||||
case 'dm_call_accept':
|
case 'dm_call_accept':
|
||||||
handleDmCallAccept(event, userId, ws);
|
handleDmCallAccept(event, userId, ws).catch(err =>
|
||||||
|
console.error('[ws] handleDmCallAccept error:', err),
|
||||||
|
);
|
||||||
break;
|
break;
|
||||||
case 'dm_call_reject':
|
case 'dm_call_reject':
|
||||||
handleDmCallReject(event, userId);
|
handleDmCallReject(event, userId).catch(err =>
|
||||||
|
console.error('[ws] handleDmCallReject error:', err),
|
||||||
|
);
|
||||||
break;
|
break;
|
||||||
case 'dm_call_end':
|
case 'dm_call_end':
|
||||||
handleDmCallEnd(event, userId);
|
handleDmCallEnd(event, userId).catch(err =>
|
||||||
|
console.error('[ws] handleDmCallEnd error:', err),
|
||||||
|
);
|
||||||
break;
|
break;
|
||||||
case 'voice_status':
|
case 'voice_status':
|
||||||
handleVoiceStatus(event, userId);
|
handleVoiceStatus(event, userId);
|
||||||
@@ -1442,7 +1449,7 @@ function handleDmCallStart(event: Record<string, unknown>, userId: string, usern
|
|||||||
.catch(err => console.error('[federation] sendFederatedCallStart error:', err));
|
.catch(err => console.error('[federation] sendFederatedCallStart error:', err));
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws: WebSocket): void {
|
async function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws: WebSocket): Promise<void> {
|
||||||
let dmChannelId = (event.dmChannelId as string) || null;
|
let dmChannelId = (event.dmChannelId as string) || null;
|
||||||
const federatedCallId = (event.federatedCallId as string) || null;
|
const federatedCallId = (event.federatedCallId as string) || null;
|
||||||
if (!dmChannelId && !federatedCallId) {
|
if (!dmChannelId && !federatedCallId) {
|
||||||
@@ -1506,8 +1513,14 @@ function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws:
|
|||||||
userId,
|
userId,
|
||||||
action: 'join',
|
action: 'join',
|
||||||
});
|
});
|
||||||
sendFederatedCallAccept(dmChannelId, userId)
|
const acceptFanoutFailures = await sendFederatedCallAccept(dmChannelId, userId);
|
||||||
.catch(err => console.error('[federation] sendFederatedCallAccept error:', err));
|
emitFanoutUndeliverable(
|
||||||
|
userId,
|
||||||
|
dmChannelId,
|
||||||
|
fedIdRow?.federatedId ?? null,
|
||||||
|
'accept',
|
||||||
|
acceptFanoutFailures,
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1520,6 +1533,8 @@ function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws:
|
|||||||
: undefined;
|
: undefined;
|
||||||
|
|
||||||
if (fedCall) {
|
if (fedCall) {
|
||||||
|
// Optimistic transition so the acceptor's client flips to active immediately.
|
||||||
|
// Rolled back below if the relay to host fails.
|
||||||
connectionManager.activateFederatedCall(fedCall.federatedId);
|
connectionManager.activateFederatedCall(fedCall.federatedId);
|
||||||
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
||||||
type: 'dm_call_accepted',
|
type: 'dm_call_accepted',
|
||||||
@@ -1534,7 +1549,7 @@ function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws:
|
|||||||
.get();
|
.get();
|
||||||
const homeUserId = user?.homeUserId || userId;
|
const homeUserId = user?.homeUserId || userId;
|
||||||
|
|
||||||
sendCallRelay(fedCall.federatedCallHost, [{
|
const result = await sendCallRelay(fedCall.federatedCallHost, [{
|
||||||
eventType: 'dm_call_accept',
|
eventType: 'dm_call_accept',
|
||||||
messageId: generateSnowflake(),
|
messageId: generateSnowflake(),
|
||||||
encryptionVersion: 0,
|
encryptionVersion: 0,
|
||||||
@@ -1543,14 +1558,32 @@ function handleDmCallAccept(event: Record<string, unknown>, userId: string, ws:
|
|||||||
call: {
|
call: {
|
||||||
acceptor: { homeUserId, homeInstance: getOurOrigin() },
|
acceptor: { homeUserId, homeInstance: getOurOrigin() },
|
||||||
},
|
},
|
||||||
}]).catch(err => console.error('[federation] Failed to send dm_call_accept:', err));
|
}]);
|
||||||
|
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] dm_call_accept relay to ${fedCall.federatedCallHost} failed (${result.reason}): ${result.error}`);
|
||||||
|
const failure = buildFailureFromResult(result, fedCall.federatedCallHost, db);
|
||||||
|
// Clear first so a concurrent end-handler sees a cleared entry (idempotent).
|
||||||
|
connectionManager.clearFederatedCall(fedCall.federatedId);
|
||||||
|
// Terminal targets ONLY the acceptor — other ringed users (group DM) didn't
|
||||||
|
// accept and should stay in their ring state; their own dm_call_end / timeout
|
||||||
|
// paths govern their teardown.
|
||||||
|
connectionManager.sendToUser(userId, {
|
||||||
|
type: 'dm_call_undeliverable',
|
||||||
|
dmChannelId: fedCall.dmChannelId,
|
||||||
|
federatedCallId: fedCall.federatedId,
|
||||||
|
terminal: true,
|
||||||
|
phase: 'accept',
|
||||||
|
failures: [failure],
|
||||||
|
});
|
||||||
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
connectionManager.sendToUser(userId, { type: 'error', message: 'No active call in this DM channel' });
|
connectionManager.sendToUser(userId, { type: 'error', message: 'No active call in this DM channel' });
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleDmCallReject(event: Record<string, unknown>, userId: string): void {
|
async function handleDmCallReject(event: Record<string, unknown>, userId: string): Promise<void> {
|
||||||
let dmChannelId = (event.dmChannelId as string) || null;
|
let dmChannelId = (event.dmChannelId as string) || null;
|
||||||
const federatedCallId = (event.federatedCallId as string) || null;
|
const federatedCallId = (event.federatedCallId as string) || null;
|
||||||
|
|
||||||
@@ -1576,8 +1609,16 @@ function handleDmCallReject(event: Record<string, unknown>, userId: string): voi
|
|||||||
connectionManager.clearVoiceWs(meta.callerId);
|
connectionManager.clearVoiceWs(meta.callerId);
|
||||||
connectionManager.destroyRoom(dmChannelId);
|
connectionManager.destroyRoom(dmChannelId);
|
||||||
connectionManager.sendToDmMembers(dmChannelId, { type: 'dm_call_rejected', dmChannelId });
|
connectionManager.sendToDmMembers(dmChannelId, { type: 'dm_call_rejected', dmChannelId });
|
||||||
sendFederatedCallEnd(dmChannelId, userId)
|
const fedIdRejectRow = getDb().select({ federatedId: schema.dmChannels.federatedId })
|
||||||
.catch(err => console.error('[federation] sendFederatedCallEnd error:', err));
|
.from(schema.dmChannels).where(eq(schema.dmChannels.id, dmChannelId)).get();
|
||||||
|
const rejectFanoutFailures = await sendFederatedCallEnd(dmChannelId, userId);
|
||||||
|
emitFanoutUndeliverable(
|
||||||
|
userId,
|
||||||
|
dmChannelId,
|
||||||
|
fedIdRejectRow?.federatedId ?? null,
|
||||||
|
'reject',
|
||||||
|
rejectFanoutFailures,
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1590,13 +1631,16 @@ function handleDmCallReject(event: Record<string, unknown>, userId: string): voi
|
|||||||
: undefined;
|
: undefined;
|
||||||
|
|
||||||
if (fedCall) {
|
if (fedCall) {
|
||||||
// Exclude the rejecting user — they already handled their own state
|
// Optimistic local clear — user intent is to reject.
|
||||||
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
||||||
type: 'dm_call_rejected',
|
type: 'dm_call_rejected',
|
||||||
dmChannelId: fedCall.dmChannelId,
|
dmChannelId: fedCall.dmChannelId,
|
||||||
federatedCallId: fedCall.federatedId,
|
federatedCallId: fedCall.federatedId,
|
||||||
} as ServerEvent, userId);
|
} as ServerEvent, userId);
|
||||||
connectionManager.clearFederatedCall(fedCall.federatedId);
|
const host = fedCall.federatedCallHost;
|
||||||
|
const fedId = fedCall.federatedId;
|
||||||
|
const dmId = fedCall.dmChannelId;
|
||||||
|
connectionManager.clearFederatedCall(fedId);
|
||||||
|
|
||||||
const db = getDb();
|
const db = getDb();
|
||||||
const user = db.select({ homeUserId: schema.users.homeUserId })
|
const user = db.select({ homeUserId: schema.users.homeUserId })
|
||||||
@@ -1605,20 +1649,33 @@ function handleDmCallReject(event: Record<string, unknown>, userId: string): voi
|
|||||||
.get();
|
.get();
|
||||||
const homeUserId = user?.homeUserId || userId;
|
const homeUserId = user?.homeUserId || userId;
|
||||||
|
|
||||||
sendCallRelay(fedCall.federatedCallHost, [{
|
const result = await sendCallRelay(host, [{
|
||||||
eventType: 'dm_call_reject',
|
eventType: 'dm_call_reject',
|
||||||
messageId: generateSnowflake(),
|
messageId: generateSnowflake(),
|
||||||
encryptionVersion: 0,
|
encryptionVersion: 0,
|
||||||
timestamp: Date.now(),
|
timestamp: Date.now(),
|
||||||
federatedId: fedCall.federatedId,
|
federatedId: fedId,
|
||||||
call: {
|
call: {
|
||||||
rejector: { homeUserId, homeInstance: getOurOrigin() },
|
rejector: { homeUserId, homeInstance: getOurOrigin() },
|
||||||
},
|
},
|
||||||
}]).catch(err => console.error('[federation] Failed to send dm_call_reject:', err));
|
}]);
|
||||||
|
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] dm_call_reject relay to ${host} failed (${result.reason}): ${result.error}`);
|
||||||
|
const failure = buildFailureFromResult(result, host, db);
|
||||||
|
connectionManager.sendToUser(userId, {
|
||||||
|
type: 'dm_call_undeliverable',
|
||||||
|
dmChannelId: dmId,
|
||||||
|
federatedCallId: fedId,
|
||||||
|
terminal: false,
|
||||||
|
phase: 'reject',
|
||||||
|
failures: [failure],
|
||||||
|
});
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleDmCallEnd(event: Record<string, unknown>, userId: string): void {
|
async function handleDmCallEnd(event: Record<string, unknown>, userId: string): Promise<void> {
|
||||||
let dmChannelId = (event.dmChannelId as string) || null;
|
let dmChannelId = (event.dmChannelId as string) || null;
|
||||||
const federatedCallId = (event.federatedCallId as string) || null;
|
const federatedCallId = (event.federatedCallId as string) || null;
|
||||||
|
|
||||||
@@ -1646,10 +1703,18 @@ function handleDmCallEnd(event: Record<string, unknown>, userId: string): void {
|
|||||||
connectionManager.clearVoiceUserStatus(participantId);
|
connectionManager.clearVoiceUserStatus(participantId);
|
||||||
connectionManager.clearVoiceWs(participantId);
|
connectionManager.clearVoiceWs(participantId);
|
||||||
}
|
}
|
||||||
|
const fedIdEndRow = getDb().select({ federatedId: schema.dmChannels.federatedId })
|
||||||
|
.from(schema.dmChannels).where(eq(schema.dmChannels.id, dmChannelId)).get();
|
||||||
connectionManager.destroyRoom(dmChannelId);
|
connectionManager.destroyRoom(dmChannelId);
|
||||||
connectionManager.sendToDmMembers(dmChannelId, { type: 'dm_call_ended', dmChannelId });
|
connectionManager.sendToDmMembers(dmChannelId, { type: 'dm_call_ended', dmChannelId });
|
||||||
sendFederatedCallEnd(dmChannelId, userId)
|
const endFanoutFailures = await sendFederatedCallEnd(dmChannelId, userId);
|
||||||
.catch(err => console.error('[federation] sendFederatedCallEnd error:', err));
|
emitFanoutUndeliverable(
|
||||||
|
userId,
|
||||||
|
dmChannelId,
|
||||||
|
fedIdEndRow?.federatedId ?? null,
|
||||||
|
'end',
|
||||||
|
endFanoutFailures,
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1662,14 +1727,16 @@ function handleDmCallEnd(event: Record<string, unknown>, userId: string): void {
|
|||||||
: undefined;
|
: undefined;
|
||||||
|
|
||||||
if (fedCall) {
|
if (fedCall) {
|
||||||
// Exclude the user who ended the call — they already disconnected in their click handler.
|
// Exclude the user who ended the call — they already disconnected client-side.
|
||||||
// Sending dm_call_ended back to them causes redundant disconnectFn() and double sounds.
|
|
||||||
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
connectionManager.sendToFederatedCallUsers(fedCall.federatedId, {
|
||||||
type: 'dm_call_ended',
|
type: 'dm_call_ended',
|
||||||
dmChannelId: fedCall.dmChannelId,
|
dmChannelId: fedCall.dmChannelId,
|
||||||
federatedCallId: fedCall.federatedId,
|
federatedCallId: fedCall.federatedId,
|
||||||
} as ServerEvent, userId);
|
} as ServerEvent, userId);
|
||||||
connectionManager.clearFederatedCall(fedCall.federatedId);
|
const host = fedCall.federatedCallHost;
|
||||||
|
const fedId = fedCall.federatedId;
|
||||||
|
const dmId = fedCall.dmChannelId;
|
||||||
|
connectionManager.clearFederatedCall(fedId);
|
||||||
|
|
||||||
const db = getDb();
|
const db = getDb();
|
||||||
const user = db.select({ homeUserId: schema.users.homeUserId })
|
const user = db.select({ homeUserId: schema.users.homeUserId })
|
||||||
@@ -1678,16 +1745,29 @@ function handleDmCallEnd(event: Record<string, unknown>, userId: string): void {
|
|||||||
.get();
|
.get();
|
||||||
const homeUserId = user?.homeUserId || userId;
|
const homeUserId = user?.homeUserId || userId;
|
||||||
|
|
||||||
sendCallRelay(fedCall.federatedCallHost, [{
|
const result = await sendCallRelay(host, [{
|
||||||
eventType: 'dm_call_end',
|
eventType: 'dm_call_end',
|
||||||
messageId: generateSnowflake(),
|
messageId: generateSnowflake(),
|
||||||
encryptionVersion: 0,
|
encryptionVersion: 0,
|
||||||
timestamp: Date.now(),
|
timestamp: Date.now(),
|
||||||
federatedId: fedCall.federatedId,
|
federatedId: fedId,
|
||||||
call: {
|
call: {
|
||||||
endedBy: { homeUserId, homeInstance: getOurOrigin() },
|
endedBy: { homeUserId, homeInstance: getOurOrigin() },
|
||||||
},
|
},
|
||||||
}]).catch(err => console.error('[federation] Failed to send dm_call_end:', err));
|
}]);
|
||||||
|
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] dm_call_end relay to ${host} failed (${result.reason}): ${result.error}`);
|
||||||
|
const failure = buildFailureFromResult(result, host, db);
|
||||||
|
connectionManager.sendToUser(userId, {
|
||||||
|
type: 'dm_call_undeliverable',
|
||||||
|
dmChannelId: dmId,
|
||||||
|
federatedCallId: fedId,
|
||||||
|
terminal: false,
|
||||||
|
phase: 'end',
|
||||||
|
failures: [failure],
|
||||||
|
});
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1896,16 +1976,24 @@ async function sendFederatedCallStart(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Map a sendCallRelay reason to the event-surface reason. */
|
/** Build a DmCallUndeliverableFailure from a failed CallRelayResult, enriching with peer label. */
|
||||||
function mapCallReasonToEventReason(reason: CallRelayFailureReason): DmCallUndeliverableReason {
|
function buildFailureFromResult(
|
||||||
switch (reason) {
|
result: Extract<CallRelayResult, { ok: false }>,
|
||||||
case 'peer_rejected': return 'peer_rejected';
|
peerOrigin: string,
|
||||||
case 'peer_awaiting_approval': return 'peer_awaiting_approval';
|
db: ReturnType<typeof getDb>,
|
||||||
case 'peer_transient_failure': return 'peer_transient_failure';
|
): DmCallUndeliverableFailure {
|
||||||
case 'post_failed': return 'peer_transient_failure'; // 4xx looks transient to users
|
const label = db.select({ instanceName: schema.federationPeers.instanceName })
|
||||||
}
|
.from(schema.federationPeers)
|
||||||
|
.where(eq(schema.federationPeers.origin, peerOrigin))
|
||||||
|
.get()?.instanceName;
|
||||||
|
return {
|
||||||
|
reason: mapCallReasonToEventReason(result.reason),
|
||||||
|
peerOrigin,
|
||||||
|
peerLabel: label ?? undefined,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Emit dm_call_undeliverable to the caller. If terminal, also destroy the
|
* Emit dm_call_undeliverable to the caller. If terminal, also destroy the
|
||||||
* local ring room (which clears the ringing timer and voice WS binding)
|
* local ring room (which clears the ringing timer and voice WS binding)
|
||||||
@@ -1944,17 +2032,49 @@ function emitUndeliverableAndMaybeDestroy(args: {
|
|||||||
dmChannelId,
|
dmChannelId,
|
||||||
federatedCallId: federatedId,
|
federatedCallId: federatedId,
|
||||||
terminal,
|
terminal,
|
||||||
|
phase: 'start',
|
||||||
failures,
|
failures,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async function sendFederatedCallAccept(dmChannelId: string, acceptorUserId: string): Promise<void> {
|
/**
|
||||||
|
* Emit a non-terminal dm_call_undeliverable to the local user whose Path-1 action
|
||||||
|
* (accept / reject / end) had one or more fan-out failures reach peers.
|
||||||
|
* No-op when there were no failures or no federatedId to reference.
|
||||||
|
*/
|
||||||
|
function emitFanoutUndeliverable(
|
||||||
|
userId: string,
|
||||||
|
dmChannelId: string | null,
|
||||||
|
federatedId: string | null | undefined,
|
||||||
|
phase: 'accept' | 'reject' | 'end',
|
||||||
|
fanoutFailures: CallFanoutFailure[],
|
||||||
|
): void {
|
||||||
|
if (fanoutFailures.length === 0 || !federatedId) return;
|
||||||
|
const failures: DmCallUndeliverableFailure[] = fanoutFailures.map(f => ({
|
||||||
|
reason: f.reason,
|
||||||
|
peerOrigin: f.origin,
|
||||||
|
peerLabel: f.peerLabel,
|
||||||
|
}));
|
||||||
|
connectionManager.sendToUser(userId, {
|
||||||
|
type: 'dm_call_undeliverable',
|
||||||
|
dmChannelId,
|
||||||
|
federatedCallId: federatedId,
|
||||||
|
terminal: false,
|
||||||
|
phase,
|
||||||
|
failures,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function sendFederatedCallAccept(
|
||||||
|
dmChannelId: string,
|
||||||
|
acceptorUserId: string,
|
||||||
|
): Promise<CallFanoutFailure[]> {
|
||||||
const db = getDb();
|
const db = getDb();
|
||||||
const channel = db.select({ federatedId: schema.dmChannels.federatedId })
|
const channel = db.select({ federatedId: schema.dmChannels.federatedId })
|
||||||
.from(schema.dmChannels)
|
.from(schema.dmChannels)
|
||||||
.where(eq(schema.dmChannels.id, dmChannelId))
|
.where(eq(schema.dmChannels.id, dmChannelId))
|
||||||
.get();
|
.get();
|
||||||
if (!channel?.federatedId) return;
|
if (!channel?.federatedId) return [];
|
||||||
|
|
||||||
const members = db.select({ homeInstance: schema.users.homeInstance })
|
const members = db.select({ homeInstance: schema.users.homeInstance })
|
||||||
.from(schema.dmMembers)
|
.from(schema.dmMembers)
|
||||||
@@ -1970,7 +2090,7 @@ async function sendFederatedCallAccept(dmChannelId: string, acceptorUserId: stri
|
|||||||
if (normalized !== ourOrigin) targets.add(normalized);
|
if (normalized !== ourOrigin) targets.add(normalized);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (targets.size === 0) return;
|
if (targets.size === 0) return [];
|
||||||
|
|
||||||
const user = db.select({ homeUserId: schema.users.homeUserId })
|
const user = db.select({ homeUserId: schema.users.homeUserId })
|
||||||
.from(schema.users)
|
.from(schema.users)
|
||||||
@@ -1989,22 +2109,41 @@ async function sendFederatedCallAccept(dmChannelId: string, acceptorUserId: stri
|
|||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
await Promise.all(
|
const labelByOrigin = new Map<string, string | null>();
|
||||||
Array.from(targets).map(origin =>
|
for (const r of db.select({ origin: schema.federationPeers.origin, instanceName: schema.federationPeers.instanceName })
|
||||||
sendCallRelay(origin, [event]).catch(err =>
|
.from(schema.federationPeers)
|
||||||
console.error(`[federation] Failed to send dm_call_accept to ${origin}:`, err)
|
.all()) {
|
||||||
)
|
labelByOrigin.set(r.origin, r.instanceName ?? null);
|
||||||
)
|
}
|
||||||
|
|
||||||
|
const results = await Promise.all(
|
||||||
|
Array.from(targets).map(async origin => ({ origin, result: await sendCallRelay(origin, [event]) })),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const failures: CallFanoutFailure[] = [];
|
||||||
|
for (const { origin, result } of results) {
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] dm_call_accept fanout to ${origin} failed (${result.reason}): ${result.error}`);
|
||||||
|
failures.push({
|
||||||
|
origin,
|
||||||
|
peerLabel: labelByOrigin.get(origin) ?? undefined,
|
||||||
|
reason: mapCallReasonToEventReason(result.reason),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return failures;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function sendFederatedCallEnd(dmChannelId: string, endedByUserId: string): Promise<void> {
|
async function sendFederatedCallEnd(
|
||||||
|
dmChannelId: string,
|
||||||
|
endedByUserId: string,
|
||||||
|
): Promise<CallFanoutFailure[]> {
|
||||||
const db = getDb();
|
const db = getDb();
|
||||||
const channel = db.select({ federatedId: schema.dmChannels.federatedId })
|
const channel = db.select({ federatedId: schema.dmChannels.federatedId })
|
||||||
.from(schema.dmChannels)
|
.from(schema.dmChannels)
|
||||||
.where(eq(schema.dmChannels.id, dmChannelId))
|
.where(eq(schema.dmChannels.id, dmChannelId))
|
||||||
.get();
|
.get();
|
||||||
if (!channel?.federatedId) return;
|
if (!channel?.federatedId) return [];
|
||||||
|
|
||||||
const members = db.select({ homeInstance: schema.users.homeInstance })
|
const members = db.select({ homeInstance: schema.users.homeInstance })
|
||||||
.from(schema.dmMembers)
|
.from(schema.dmMembers)
|
||||||
@@ -2020,9 +2159,8 @@ async function sendFederatedCallEnd(dmChannelId: string, endedByUserId: string):
|
|||||||
if (normalized !== ourOrigin) targets.add(normalized);
|
if (normalized !== ourOrigin) targets.add(normalized);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (targets.size === 0) return;
|
if (targets.size === 0) return [];
|
||||||
|
|
||||||
// Resolve actual homeUserId from DB
|
|
||||||
const endUser = db.select({ homeUserId: schema.users.homeUserId })
|
const endUser = db.select({ homeUserId: schema.users.homeUserId })
|
||||||
.from(schema.users)
|
.from(schema.users)
|
||||||
.where(eq(schema.users.id, endedByUserId))
|
.where(eq(schema.users.id, endedByUserId))
|
||||||
@@ -2040,13 +2178,29 @@ async function sendFederatedCallEnd(dmChannelId: string, endedByUserId: string):
|
|||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
await Promise.all(
|
const labelByOrigin = new Map<string, string | null>();
|
||||||
Array.from(targets).map(origin =>
|
for (const r of db.select({ origin: schema.federationPeers.origin, instanceName: schema.federationPeers.instanceName })
|
||||||
sendCallRelay(origin, [event]).catch(err =>
|
.from(schema.federationPeers)
|
||||||
console.error(`[federation] Failed to send dm_call_end to ${origin}:`, err)
|
.all()) {
|
||||||
)
|
labelByOrigin.set(r.origin, r.instanceName ?? null);
|
||||||
)
|
}
|
||||||
|
|
||||||
|
const results = await Promise.all(
|
||||||
|
Array.from(targets).map(async origin => ({ origin, result: await sendCallRelay(origin, [event]) })),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const failures: CallFanoutFailure[] = [];
|
||||||
|
for (const { origin, result } of results) {
|
||||||
|
if (!result.ok) {
|
||||||
|
console.error(`[federation] dm_call_end fanout to ${origin} failed (${result.reason}): ${result.error}`);
|
||||||
|
failures.push({
|
||||||
|
origin,
|
||||||
|
peerLabel: labelByOrigin.get(origin) ?? undefined,
|
||||||
|
reason: mapCallReasonToEventReason(result.reason),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return failures;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Voice Moderation Handlers ──────────────────────────────────────────────
|
// ─── Voice Moderation Handlers ──────────────────────────────────────────────
|
||||||
@@ -2308,3 +2462,24 @@ function handleVoiceDisconnect(event: Record<string, unknown>, userId: string):
|
|||||||
channelId,
|
channelId,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register the ring-timeout fan-out so a host-side 60s auto-clean notifies remote peers.
|
||||||
|
* Called from server startup; split from module-load to keep test isolation clean
|
||||||
|
* (tests that exercise ring timeouts can register their own stub via
|
||||||
|
* `connectionManager.setRingTimeoutFanoutHook`).
|
||||||
|
*/
|
||||||
|
export function registerCallRelayHooks(): void {
|
||||||
|
connectionManager.setRingTimeoutFanoutHook(async (dmChannelId, callerId) => {
|
||||||
|
const failures = await sendFederatedCallEnd(dmChannelId, callerId);
|
||||||
|
if (failures.length > 0) {
|
||||||
|
console.warn('[federation] Ring-timeout fan-out had failures:', failures);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Test-only exports ──────────────────────────────────────────────────────
|
||||||
|
/** Direct export for unit tests — do not use in production code paths. */
|
||||||
|
export const handleDmCallAcceptForTest = handleDmCallAccept;
|
||||||
|
export const handleDmCallRejectForTest = handleDmCallReject;
|
||||||
|
export const handleDmCallEndForTest = handleDmCallEnd;
|
||||||
|
|||||||
@@ -99,6 +99,9 @@ class ConnectionManager {
|
|||||||
private pendingOfflineTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
private pendingOfflineTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
||||||
// roomId → Timeout for ringing DM rooms (60s auto-cleanup)
|
// roomId → Timeout for ringing DM rooms (60s auto-cleanup)
|
||||||
private ringingTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
private ringingTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
||||||
|
// Callback registered by events.ts to fan dm_call_end out to peers on ring timeout.
|
||||||
|
// Null during startup — ring timeouts that fire before registration simply no-op (there are no peers to notify before boot completes).
|
||||||
|
private ringTimeoutFanoutHook: ((dmChannelId: string, callerId: string) => Promise<void>) | null = null;
|
||||||
/** Federated calls where this instance is NOT the host. Keyed by federatedId. */
|
/** Federated calls where this instance is NOT the host. Keyed by federatedId. */
|
||||||
private federatedCalls: Map<string, FederatedCallEntry> = new Map();
|
private federatedCalls: Map<string, FederatedCallEntry> = new Map();
|
||||||
private federatedCallTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
private federatedCallTimeouts: Map<string, NodeJS.Timeout> = new Map();
|
||||||
@@ -401,6 +404,11 @@ class ConnectionManager {
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Register a fan-out callback invoked when a ringing DM room hits its 60s timeout. */
|
||||||
|
setRingTimeoutFanoutHook(fn: (dmChannelId: string, callerId: string) => Promise<void>): void {
|
||||||
|
this.ringTimeoutFanoutHook = fn;
|
||||||
|
}
|
||||||
|
|
||||||
/** Create a DM room in ringing state with 60s auto-cleanup. */
|
/** Create a DM room in ringing state with 60s auto-cleanup. */
|
||||||
createDmRoom(dmChannelId: string, callerId: string): boolean {
|
createDmRoom(dmChannelId: string, callerId: string): boolean {
|
||||||
const created = this.createRoom(dmChannelId, 'dm', {
|
const created = this.createRoom(dmChannelId, 'dm', {
|
||||||
@@ -415,11 +423,19 @@ class ConnectionManager {
|
|||||||
this.ringingTimeouts.delete(dmChannelId);
|
this.ringingTimeouts.delete(dmChannelId);
|
||||||
const room = this.voiceRooms.get(dmChannelId);
|
const room = this.voiceRooms.get(dmChannelId);
|
||||||
if (room && room.roomType === 'dm' && (room.metadata as DmRoomMeta).state === 'ringing') {
|
if (room && room.roomType === 'dm' && (room.metadata as DmRoomMeta).state === 'ringing') {
|
||||||
|
const ringedCallerId = (room.metadata as DmRoomMeta).callerId;
|
||||||
this.destroyRoom(dmChannelId);
|
this.destroyRoom(dmChannelId);
|
||||||
this.sendToDmMembers(dmChannelId, {
|
this.sendToDmMembers(dmChannelId, {
|
||||||
type: 'dm_call_ended',
|
type: 'dm_call_ended',
|
||||||
dmChannelId,
|
dmChannelId,
|
||||||
});
|
});
|
||||||
|
// Fan dm_call_end out to remote peers so stranded Path-A/B ringees exit the ring.
|
||||||
|
// Without this, an accept-relay failure → Alice's 60s auto-clean leaves Bob's FederatedCallEntry lingering with no terminal event.
|
||||||
|
if (this.ringTimeoutFanoutHook) {
|
||||||
|
this.ringTimeoutFanoutHook(dmChannelId, ringedCallerId).catch(err =>
|
||||||
|
console.error('[ws] ring-timeout fan-out error:', err),
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}, 60_000);
|
}, 60_000);
|
||||||
this.ringingTimeouts.set(dmChannelId, timeout);
|
this.ringingTimeouts.set(dmChannelId, timeout);
|
||||||
|
|||||||
@@ -364,6 +364,8 @@ export type DmCallUndeliverableReason =
|
|||||||
| 'peer_transient_failure'
|
| 'peer_transient_failure'
|
||||||
| 'livekit_unavailable';
|
| 'livekit_unavailable';
|
||||||
|
|
||||||
|
export type DmCallPhase = 'start' | 'accept' | 'reject' | 'end';
|
||||||
|
|
||||||
export interface DmCallUndeliverableFailure {
|
export interface DmCallUndeliverableFailure {
|
||||||
reason: DmCallUndeliverableReason;
|
reason: DmCallUndeliverableReason;
|
||||||
peerOrigin?: string;
|
peerOrigin?: string;
|
||||||
@@ -426,7 +428,7 @@ export type ServerEvent =
|
|||||||
| { type: 'dm_call_accepted'; dmChannelId: string | null; federatedCallId?: string }
|
| { type: 'dm_call_accepted'; dmChannelId: string | null; federatedCallId?: string }
|
||||||
| { type: 'dm_call_rejected'; dmChannelId: string }
|
| { type: 'dm_call_rejected'; dmChannelId: string }
|
||||||
| { type: 'dm_call_ended'; dmChannelId: string }
|
| { type: 'dm_call_ended'; dmChannelId: string }
|
||||||
| { type: 'dm_call_undeliverable'; dmChannelId: string | null; federatedCallId: string; terminal: boolean; failures: DmCallUndeliverableFailure[] }
|
| { type: 'dm_call_undeliverable'; dmChannelId: string | null; federatedCallId: string; terminal: boolean; phase: DmCallPhase; failures: DmCallUndeliverableFailure[] }
|
||||||
| { type: 'voice_status_update'; userId: string; channelId: string; isMuted: boolean; isDeafened: boolean; isCameraOn: boolean; isScreenSharing: boolean }
|
| { type: 'voice_status_update'; userId: string; channelId: string; isMuted: boolean; isDeafened: boolean; isCameraOn: boolean; isScreenSharing: boolean }
|
||||||
| { type: 'dm_channel_created'; dmChannel: DmChannel }
|
| { type: 'dm_channel_created'; dmChannel: DmChannel }
|
||||||
| { type: 'dm_channel_closed'; dmChannelId: string }
|
| { type: 'dm_channel_closed'; dmChannelId: string }
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
import { describe, it, expect } from 'vitest';
|
||||||
|
import { buildCallUndeliverableToast } from '../../utils/callUndeliverableToast.js';
|
||||||
|
|
||||||
|
describe('buildCallUndeliverableToast', () => {
|
||||||
|
const fail = (overrides: Partial<{ reason: string; peerOrigin?: string; peerLabel?: string }> = {}) => ({
|
||||||
|
reason: 'peer_transient_failure',
|
||||||
|
peerLabel: 'nova',
|
||||||
|
...overrides,
|
||||||
|
});
|
||||||
|
|
||||||
|
it('start + terminal single failure: existing copy', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], true, 'start')).toMatch(/Could not reach nova/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('start + non-terminal: "some participants" copy', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], false, 'start')).toMatch(/Some participants could not be reached/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('accept + terminal: tear-down copy', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], true, 'accept'))
|
||||||
|
.toMatch(/Couldn't confirm your accept with nova/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reject + non-terminal: info copy', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], false, 'reject'))
|
||||||
|
.toMatch(/Couldn't notify nova that you declined/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('end + non-terminal: info copy', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], false, 'end'))
|
||||||
|
.toMatch(/Couldn't notify nova that you hung up/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('legacy two-arg signature still works', () => {
|
||||||
|
expect(buildCallUndeliverableToast([fail()], true)).toMatch(/Could not reach nova/);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -116,40 +116,9 @@ function buildWsUrl(origin: string): string {
|
|||||||
|
|
||||||
// ─── Call relay helpers ───────────────────────────────────────────────────────
|
// ─── Call relay helpers ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
function buildCallUndeliverableToast(
|
import { buildCallUndeliverableToast } from '../utils/callUndeliverableToast';
|
||||||
failures: Array<{ reason: string; peerOrigin?: string; peerLabel?: string }>,
|
|
||||||
terminal: boolean,
|
|
||||||
): string {
|
|
||||||
const primary = failures[0];
|
|
||||||
const labelFor = (f: { peerLabel?: string; peerOrigin?: string }) =>
|
|
||||||
f.peerLabel ?? f.peerOrigin?.replace(/^https?:\/\//, '') ?? 'the remote instance';
|
|
||||||
|
|
||||||
if (!terminal) {
|
export { buildCallUndeliverableToast };
|
||||||
const labels = failures.map(labelFor).join(', ');
|
|
||||||
return `Some participants could not be reached: ${labels}.`;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (failures.length > 1) {
|
|
||||||
const labels = failures.map(labelFor).join(', ');
|
|
||||||
return `Could not reach ${failures.length} instances: ${labels}.`;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!primary) return 'Call could not be placed.';
|
|
||||||
|
|
||||||
const label = labelFor(primary);
|
|
||||||
switch (primary.reason) {
|
|
||||||
case 'peer_rejected':
|
|
||||||
return `Cannot reach ${label} — this instance requires manual peering approval.`;
|
|
||||||
case 'peer_awaiting_approval':
|
|
||||||
return `Waiting for ${label} admin to approve your instance. Calls will work once approved.`;
|
|
||||||
case 'peer_transient_failure':
|
|
||||||
return `Could not reach ${label}. Try again in a moment.`;
|
|
||||||
case 'livekit_unavailable':
|
|
||||||
return 'Voice is not configured on this instance.';
|
|
||||||
default:
|
|
||||||
return `Call to ${label} could not be placed.`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Event handling ───────────────────────────────────────────────────────────
|
// ─── Event handling ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -1030,7 +999,7 @@ function handleEvent(origin: string, event: ServerEvent): void {
|
|||||||
if (disconnectFn) disconnectFn();
|
if (disconnectFn) disconnectFn();
|
||||||
}
|
}
|
||||||
|
|
||||||
const msg = buildCallUndeliverableToast(event.failures, event.terminal);
|
const msg = buildCallUndeliverableToast(event.failures, event.terminal, event.phase);
|
||||||
addToast(msg, event.terminal ? 'warning' : 'info', 8_000);
|
addToast(msg, event.terminal ? 'warning' : 'info', 8_000);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
/**
|
||||||
|
* Builds a user-facing toast message from a `dm_call_undeliverable` event.
|
||||||
|
*
|
||||||
|
* Copy is phase-aware:
|
||||||
|
* - `start`: call-start delivery; terminal means the ring was destroyed, non-terminal
|
||||||
|
* means the call continues for other reachable recipients.
|
||||||
|
* - `accept`: the acceptor's B→host relay failed; terminal means their optimistic
|
||||||
|
* active-call state was rolled back.
|
||||||
|
* - `reject`: the rejector's relay to the host failed; state was already cleared
|
||||||
|
* locally, so non-terminal info toast only.
|
||||||
|
* - `end`: the ender's relay to the host failed; state was already cleared locally.
|
||||||
|
*
|
||||||
|
* Extracted from `useWebSocket.ts` so it can be unit-tested without pulling in
|
||||||
|
* the full WS handler graph (livekit / audio deps).
|
||||||
|
*/
|
||||||
|
export function buildCallUndeliverableToast(
|
||||||
|
failures: Array<{ reason: string; peerOrigin?: string; peerLabel?: string }>,
|
||||||
|
terminal: boolean,
|
||||||
|
phase: 'start' | 'accept' | 'reject' | 'end' = 'start',
|
||||||
|
): string {
|
||||||
|
const primary = failures[0];
|
||||||
|
const labelFor = (f: { peerLabel?: string; peerOrigin?: string }) =>
|
||||||
|
f.peerLabel ?? f.peerOrigin?.replace(/^https?:\/\//, '') ?? 'the remote instance';
|
||||||
|
|
||||||
|
if (phase === 'accept' && terminal) {
|
||||||
|
const label = primary ? labelFor(primary) : 'the host instance';
|
||||||
|
return `Couldn't confirm your accept with ${label} — the call was dropped.`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (phase === 'reject') {
|
||||||
|
const labels = failures.map(labelFor).join(', ') || 'the host instance';
|
||||||
|
return `Couldn't notify ${labels} that you declined. Caller may still see you as ringing briefly.`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (phase === 'end') {
|
||||||
|
const labels = failures.map(labelFor).join(', ') || 'the host instance';
|
||||||
|
return `Couldn't notify ${labels} that you hung up. Remote participants may see the call for up to 60 seconds.`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase === 'start' (default + legacy)
|
||||||
|
if (!terminal) {
|
||||||
|
const labels = failures.map(labelFor).join(', ');
|
||||||
|
return `Some participants could not be reached: ${labels}.`;
|
||||||
|
}
|
||||||
|
if (failures.length > 1) {
|
||||||
|
const labels = failures.map(labelFor).join(', ');
|
||||||
|
return `Could not reach ${failures.length} instances: ${labels}.`;
|
||||||
|
}
|
||||||
|
if (!primary) return 'Call could not be placed.';
|
||||||
|
|
||||||
|
const label = labelFor(primary);
|
||||||
|
switch (primary.reason) {
|
||||||
|
case 'peer_rejected':
|
||||||
|
return `Cannot reach ${label} — this instance requires manual peering approval.`;
|
||||||
|
case 'peer_awaiting_approval':
|
||||||
|
return `Waiting for ${label} admin to approve your instance. Calls will work once approved.`;
|
||||||
|
case 'peer_transient_failure':
|
||||||
|
return `Could not reach ${label}. Try again in a moment.`;
|
||||||
|
case 'livekit_unavailable':
|
||||||
|
return 'Voice is not configured on this instance.';
|
||||||
|
default:
|
||||||
|
return `Call to ${label} could not be placed.`;
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user