Public-release prep: ELv2 license, README/CLA/NOTICE, SSRF safeFetch, identifier genericization, export tooling
This commit is contained in:
Jannis Braun
@@ -573,8 +573,8 @@ Called by `useAuth()` hook when token exists but user object is null:
|
||||
|
||||
Splits a potentially federated username:
|
||||
```
|
||||
"youruser@nova.ddns.net" -> { baseName: "youruser", domain: "nova.ddns.net" }
|
||||
"youruser" -> { baseName: "youruser", domain: null }
|
||||
"erin@nova.ddns.net" -> { baseName: "erin", domain: "nova.ddns.net" }
|
||||
"erin" -> { baseName: "erin", domain: null }
|
||||
```
|
||||
|
||||
### Self-ID Registry
|
||||
|
||||
@@ -62,9 +62,9 @@ This is a **real account with a real bcrypt password** — not a replicated stub
|
||||
|
||||
| Account type | Username | passwordHash | homeInstance | Can log in? |
|
||||
|---|---|---|---|---|
|
||||
| Local (native) | `youruser` | bcrypt hash | `NULL` | Yes |
|
||||
| Federated (client-created) | `youruser@nova.ddns.net` | bcrypt hash | `nova.ddns.net` | Yes |
|
||||
| Replicated stub (S2S-created) | `youruser@nova.ddns.net` | `!federation-replicated` | `nova.ddns.net` | No |
|
||||
| Local (native) | `erin` | bcrypt hash | `NULL` | Yes |
|
||||
| Federated (client-created) | `erin@nova.ddns.net` | bcrypt hash | `nova.ddns.net` | Yes |
|
||||
| Replicated stub (S2S-created) | `erin@nova.ddns.net` | `!federation-replicated` | `nova.ddns.net` | No |
|
||||
|
||||
Key distinction: **Federated accounts** and **replicated stubs** can have the same username format (`user@instance`), but federated accounts have real passwords and can log in. Replicated stubs are server-created placeholders for identity resolution and cannot log in.
|
||||
|
||||
@@ -75,9 +75,9 @@ The merge migration in `migrate.ts` detects when both exist for the same remote
|
||||
When a user adds a remote instance via the Connections settings:
|
||||
|
||||
1. **Verify home password** — client confirms the user's password against the home instance
|
||||
2. **Compute federated username** — `{bareUsername}@{homeHost}` (e.g., `youruser@nova.ddns.net`)
|
||||
2. **Compute federated username** — `{bareUsername}@{homeHost}` (e.g., `erin@nova.ddns.net`)
|
||||
3. **Try registration** on remote instance with:
|
||||
- Username: `youruser@nova.ddns.net`
|
||||
- Username: `erin@nova.ddns.net`
|
||||
- Password: same as home instance password
|
||||
- `homeInstance`: `nova.ddns.net` (bare domain)
|
||||
- `homeUserId`: user's Snowflake ID on home instance
|
||||
@@ -106,7 +106,7 @@ interface ConnectedInstance {
|
||||
label: string; // Instance display name
|
||||
token: string; // JWT for this instance
|
||||
user: User; // User record on this instance
|
||||
username: string; // e.g., 'youruser@nova.ddns.net'
|
||||
username: string; // e.g., 'erin@nova.ddns.net'
|
||||
status: 'connected' | 'connecting' | 'disconnected' | 'error';
|
||||
error?: string;
|
||||
api: BackspaceApiClient; // Authenticated API client
|
||||
|
||||
@@ -7,7 +7,7 @@ Source files:
|
||||
- `docker-compose.yml` -- `backspace` + `caddy` (+ optional `livekit`) services, healthcheck
|
||||
- `Caddyfile` -- reverse proxy / auto-HTTPS config
|
||||
- `install.sh` -- interactive first-time setup
|
||||
- `deploy.sh` -- rsync + rebuild to Jannis's two boxes
|
||||
- `deploy.sh` -- rsync + rebuild to Heidi's two boxes
|
||||
- `backup.sh` / `restore.sh` -- manual snapshot + restore tooling (host side)
|
||||
- `packages/server/src/config.ts` -- `config.backup.*` env parsing
|
||||
- `packages/server/src/utils/backup.ts` -- `createSnapshot` (VACUUM INTO), `listSnapshots`, `pruneSnapshots`, off-box hook
|
||||
@@ -62,7 +62,7 @@ Caddy provisions and renews TLS certificates automatically for `DOMAIN`; the per
|
||||
|
||||
### Redeploy: `deploy.sh [pi|vm|all]`
|
||||
|
||||
`./deploy.sh` is Jannis's redeploy helper for the two live instances — `nova.ddns.net` (Raspberry Pi) and `orbit.ddns.net` (VM). It does **not** build locally; it `rsync`s the working tree to the target (excluding `node_modules`, `.env`, `data/`, build output, and a list of local-only paths) and then runs `docker compose up -d --build` on the remote so the image is rebuilt in place. Targets:
|
||||
`./deploy.sh` is Heidi's redeploy helper for the two live instances — `nova.ddns.net` (Raspberry Pi) and `orbit.ddns.net` (VM). It does **not** build locally; it `rsync`s the working tree to the target (excluding `node_modules`, `.env`, `data/`, build output, and a list of local-only paths) and then runs `docker compose up -d --build` on the remote so the image is rebuilt in place. Targets:
|
||||
|
||||
| Arg | Target |
|
||||
|-----|--------|
|
||||
|
||||
@@ -891,7 +891,7 @@ For full wire formats, see `docs/systems/websocket.md`.
|
||||
| Missing federatedId in outbox | All membership events rejected by peer | Outbox worker reconstruction omitted `federatedId` | Copy `parsed.federatedId` during reconstruction |
|
||||
| Cross-instance duplicate channels | Duplicate sidebar entries | `dm_channel_created` broadcast to ALL members including remote | Local-only broadcast principle |
|
||||
| Bootstrap vs incremental confusion | N/A (design note) | `bootstrapped` flag is function-local; batch events work correctly because bootstrap adds ALL roster members | No fix needed -- documented as correct behavior |
|
||||
| Duplicated membership system messages across restarts | 4× "Jannis added youruser" in group DM, channel keeps flipping to unread after each deploy | Membership event processors inserted system messages unconditionally. Each approval-flow re-peering reset peer `last_synced_at = 0`, so initial sync replayed every historical `member_add` / `member_remove` / `ownership_transfer` on next boot. Each replay's new snowflake ID exceeded the user's `read_states.last_read_message_id`, flipping unread. | Dedup by `(sourceInstance, event.messageId)` on the inserted system message. Both bootstrap and incremental paths in `processMemberAddEvent` now persist these fields so replay is a no-op. |
|
||||
| Duplicated membership system messages across restarts | 4× "Heidi added erin" in group DM, channel keeps flipping to unread after each deploy | Membership event processors inserted system messages unconditionally. Each approval-flow re-peering reset peer `last_synced_at = 0`, so initial sync replayed every historical `member_add` / `member_remove` / `ownership_transfer` on next boot. Each replay's new snowflake ID exceeded the user's `read_states.last_read_message_id`, flipping unread. | Dedup by `(sourceInstance, event.messageId)` on the inserted system message. Both bootstrap and incremental paths in `processMemberAddEvent` now persist these fields so replay is a no-op. |
|
||||
| Raw JSON in DM sidebar previews | DM sidebar showed `{"event":"space_invite",...}` / `{"event":"member_added",...}` as the last-message preview | `DmLastMessagePreview` shape omitted `type`, so the client could not distinguish system from user messages and rendered `lastMessage.content` verbatim. | Added `type` to `DmLastMessagePreview`, populated it from `dm_messages.type` in every server emission site, and routed the sidebar through a single `formatDmSidebarPreview` helper that renders human-readable text for each system event. |
|
||||
| Owner-only requests routed to wrong instance after manual transfer (latent) | After `POST /api/dm/:id/transfer` moved ownership to a member whose `homeInstance` differed from the channel's pinned serving origin, owner-only client calls (`updateMetadata`, `kickMember`, `transferOwnership`) routed via `getChannelOrigin` would emit outbox events with `sourceInstance !== ownerHomeInstance`, and all peers would reject them as `attribution_mismatch`. Latent only because pre-polish there was no kick endpoint and no metadata edit; auto-transfer-on-leave masked the issue (the leaver IS the actor, and `member_remove reason='leave'` accepts any source). | Added `getOwnerInstanceForDm(channelId)` exported next to `getChannelOrigin`. All four owner-only API client methods (`updateMetadata`, `kickMember`, `transferOwnership` — and any future owner-only routes) call `getApiForOrigin(getOwnerInstanceForDm(channelId))` instead of channel origin. Non-owner operations are unchanged. |
|
||||
| Kick / transfer to federated member always failed with "user not a member" | `DELETE /api/dm/:id/members/:targetUserId` and `POST /api/dm/:id/transfer` accepted only a local user id. The client passed `canonical.id` from `useCanonicalUserView`, which returns the user's HOME id when the home view is cached. After owner-routing the request to the owner instance, the owner instance's `dm_members.userId` (its own local replicated id) never matched the home id, so `isDmMember` returned false. | Both endpoints now accept federated identification (`homeUserId` + `homeInstance`) — the transfer endpoint takes them in the body, the kick endpoint reads `homeInstance` from a query string and treats the URL segment as a homeUserId. Server resolves via `resolveOrCreateReplicatedUser` before membership check. Mirrors the `addDmMember` pattern. Client `kickMember` / `transferOwnership` accept an optional `federated` arg and pass it when the target has `homeUserId` + `homeInstance` populated. |
|
||||
|
||||
@@ -137,9 +137,9 @@ Invalid URLs (fail `new URL()` parsing): same as fallthrough.
|
||||
|
||||
## 3. SSRF Protection
|
||||
|
||||
`ssrf.ts:validateExternalUrl()`
|
||||
`ssrf.ts:validateExternalUrl()` and `ssrf.ts:safeFetch()`
|
||||
|
||||
Called before every outbound fetch (metadata fetching and image dimension probing). Throws on any violation.
|
||||
All outbound fetches to user- or peer-supplied URLs go through `safeFetch()`, which validates the target with `validateExternalUrl()` (below) and re-validates the destination of every redirect hop. `validateExternalUrl()` throws on any violation.
|
||||
|
||||
### Validation Steps
|
||||
|
||||
@@ -166,7 +166,9 @@ Called before every outbound fetch (metadata fetching and image dimension probin
|
||||
|
||||
### Redirect Handling
|
||||
|
||||
Both `fetchUrlMetadata` and `probeRemoteImageDimensions` use `redirect: 'follow'` in their `fetch()` calls. SSRF validation is performed on the **original** URL before fetch, but the native `fetch` follows redirects without re-validating intermediate URLs. This means a redirect from a public IP to a private IP would not be caught by the current implementation.
|
||||
Outbound fetches use `safeFetch()` (`ssrf.ts`), which follows redirects **manually** (`redirect: 'manual'`) and runs `validateExternalUrl()` against every hop's destination before following it, capped at 5 redirects. A redirect from a public host to a private/internal address (loopback, link-local, RFC1918) is therefore blocked, closing the redirect-based SSRF bypass. Callers — `fetchUrlMetadata`, `probeRemoteImageDimensions`, and `fetchSpaceInviteSnapshot` — all route through `safeFetch` rather than calling `validateExternalUrl` + `fetch` separately.
|
||||
|
||||
**Residual:** `validateExternalUrl` resolves DNS and `fetch` resolves again, leaving a narrow DNS-rebinding TOCTOU window. Closing it fully requires pinning the resolved IP at connect time via a custom dispatcher; the redirect re-validation closes the practical, attacker-controlled bypass.
|
||||
|
||||
---
|
||||
|
||||
@@ -176,8 +178,8 @@ Both `fetchUrlMetadata` and `probeRemoteImageDimensions` use `redirect: 'follow'
|
||||
|
||||
### Flow
|
||||
|
||||
1. `validateExternalUrl(url)` — SSRF check, returns `null` on failure
|
||||
2. `fetch(url)` with `User-Agent: BackspaceBot/1.0`, 5-second timeout via `AbortController`
|
||||
1. `safeFetch(url)` — SSRF-validated fetch (initial URL + every redirect hop); throws on block, caught to return `null`
|
||||
2. Request sent with `User-Agent: BackspaceBot/1.0`, 5-second timeout via `AbortController`
|
||||
3. **Content-Type detection** — if response is `image/*`, `video/*`, or `audio/*`, returns early with `contentType` field set (no HTML parsing)
|
||||
4. **Size guard** — rejects responses with `Content-Length > 512KB`
|
||||
5. **Stream-read with hard limit** — reads body via `ReadableStream`, stops at 512KB even for chunked (unknown-length) responses
|
||||
@@ -283,7 +285,7 @@ Used for direct image URLs where dimensions are unknown (not provided by OG tags
|
||||
|
||||
### Mechanism
|
||||
|
||||
1. **SSRF validation** — `validateExternalUrl(url)`, returns `null` on block
|
||||
1. **SSRF-validated fetch** — `safeFetch(url)` validates the URL and every redirect hop; a block throws and is caught to return `null`
|
||||
2. **Range request** — fetches first 32KB (`PROBE_BYTES = 32_768`) with header `Range: bytes=0-32767`
|
||||
3. **Timeout** — 3-second abort (`PROBE_TIMEOUT_MS = 3_000`)
|
||||
4. **Graceful body read** — reads up to `PROBE_BYTES` via `ReadableStream`, then aggressively cancels the connection via `reader.cancel()`
|
||||
|
||||
@@ -859,7 +859,7 @@ Owner-authored update of a group DM's `name` and/or `icon`. Mirrors the `profile
|
||||
"userId": "...",
|
||||
"homeUserId": "...",
|
||||
"homeInstance": "https://nova.ddns.net",
|
||||
"profile": { "username": "jannis" }
|
||||
"profile": { "username": "heidi" }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1593,7 +1593,7 @@ Relevant settings in `instance_settings`:
|
||||
|
||||
The frontend needs to resolve federated identities for display purposes:
|
||||
|
||||
**`parseFederatedUsername(username)`** -- splits `"youruser@nova.ddns.net"` into `{baseName: "youruser", domain: "nova.ddns.net"}`.
|
||||
**`parseFederatedUsername(username)`** -- splits `"erin@nova.ddns.net"` into `{baseName: "erin", domain: "nova.ddns.net"}`.
|
||||
|
||||
**`isSelf(user, homeUser)`** -- determines if a user object is the logged-in user or their replicated stub. Uses cascading checks: same ID, known self-ID set, homeInstance + baseName match.
|
||||
|
||||
|
||||
@@ -590,7 +590,7 @@ Source: `packages/web/src/utils/identity.ts`
|
||||
|
||||
### `parseFederatedUsername(username)`
|
||||
|
||||
Splits `"youruser@nova.ddns.net"` into `{ baseName: "youruser", domain: "nova.ddns.net" }`. Uses `indexOf('@')` (first occurrence). Returns `{ baseName: username, domain: null }` for non-federated usernames.
|
||||
Splits `"erin@nova.ddns.net"` into `{ baseName: "erin", domain: "nova.ddns.net" }`. Uses `indexOf('@')` (first occurrence). Returns `{ baseName: username, domain: null }` for non-federated usernames.
|
||||
|
||||
### `isSelf(user, homeUser)`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user