diff --git a/docs/systems/database.md b/docs/systems/database.md index 9539248d..1a244143 100644 --- a/docs/systems/database.md +++ b/docs/systems/database.md @@ -105,6 +105,7 @@ PK: (spaceId, userId) | width | integer | | Image/video pixel width | | height | integer | | Image/video pixel height | | duration | real | | Audio/video seconds | +| playable | integer (boolean) | NULL | Video web-playability tri-state: 1 = browser-decodable, 0 = known-undecodable (e.g. HEVC .mov), NULL = unknown/non-video. See uploads.md §3. | | sourceUrl | text | | Remote URL (federation) | | federationStatus | text | | local/remote/remote_partial | | federationMeta | text | | JSON rejection info | diff --git a/docs/systems/federation.md b/docs/systems/federation.md index 47356b03..d7d1127a 100644 --- a/docs/systems/federation.md +++ b/docs/systems/federation.md @@ -941,12 +941,13 @@ When `queueDmRelay` constructs the relay payload, each attachment gets a `source ``` sourceUrl: `${getOurOrigin()}/api/uploads/${attachment.filename}` ``` +The payload also carries `playable` (video web-playability, computed by the origin from the probed codec — see uploads.md §3) so the receiving instance need not re-probe the file's codec. ### Inbound (receiving instance -- `processCreateEvent`) 1. For each attachment in `event.message.attachments`: - SSRF check: `isUrlFromPeer(sourceUrl, peerOrigin)` -- hostname of sourceUrl must match peer origin hostname - - Create `attachments` row with `filename = sourceUrl` (remote URL as interim filename) + - Create `attachments` row with `filename = sourceUrl` (remote URL as interim filename), carrying through `playable` from the relay payload - Queue `federation_file_queue` entry with `status = 'pending'`, `expiresAt = now + 30 days` 2. Initial WebSocket broadcast uses sourceUrl directly (frontend's `AttachmentRenderer` detects `http` prefix) diff --git a/docs/systems/uploads.md b/docs/systems/uploads.md index bf339f81..7b45da7d 100644 --- a/docs/systems/uploads.md +++ b/docs/systems/uploads.md @@ -3,7 +3,8 @@ Source files: - `packages/server/src/routes/uploads.ts` -- File serving (cache, security, Range) - `packages/server/src/routes/files.ts` -- tus protocol endpoints (`/api/files/*`), PRE_CREATE / PRE_PATCH / POST_FINISH hooks, janitor helpers -- `packages/server/src/utils/thumbnail.ts` -- Image thumbnail generation (sharp), video thumbnail extraction (ffmpeg), image dimension probing, profile image resizing, media metadata extraction +- `packages/server/src/utils/thumbnail.ts` -- Image thumbnail generation (sharp), video thumbnail extraction (ffmpeg), image dimension probing, profile image resizing, media metadata extraction (incl. video codec) +- `packages/server/src/utils/mediaPlayable.ts` -- `classifyVideoPlayable(mimetype, codec)` web-playability classifier (drives `attachments.playable`) - `packages/server/src/utils/fileCleanup.ts` -- File deletion helpers (disk + thumbnail + attachment record cleanup) - `packages/server/src/utils/storageJanitor.ts` -- Storage stats, orphan detection, cleanup routines (orphaned files, unlinked attachments, dangling references, old media), federation GC, soft-deleted DM channel purge - `packages/web/src/stores/transferStore.ts` -- Client transfer manager (uploads via tus-js-client, downloads via fetch + FS Access) @@ -149,10 +150,20 @@ Processing occurs inline during upload, before the response is sent. All process - Returns `{ thumbnailFilename, width, height }` (dimensions are from the original frame, not the thumbnail) 2. **Metadata probing** (`thumbnail.ts:probeMediaMeta`) - - ffprobe for dimensions: `-select_streams v:0 -show_entries stream=width,height -of json` + - ffprobe for dimensions + codec: `-select_streams v:0 -show_entries stream=width,height,codec_name -of json` - ffprobe for duration: `-show_entries format=duration -of json` - Duration rounded to 2 decimal places - If thumbnail extraction failed, dimensions fall back to ffprobe values + - Returns `codec` (the primary video stream's `codec_name`) when available + +3. **Web-playability classification** (`mediaPlayable.ts:classifyVideoPlayable`) + - The finish hook calls `classifyVideoPlayable(mimetype, codec)` and stores the result in `attachments.playable` (tri-state, see below). + - The browser `