fix(uploads): graceful fallback for browser-unplayable video (HEVC .mov)
macOS screen recordings are HEVC inside a .mov container, which Chromium, Firefox and stock Electron can't decode. The file uploaded fine and a server-side ffmpeg poster was generated, but inline <video> playback failed silently — stuck at 0:00 with no error, since AttachmentRenderer had no error handling. Root cause: the system had no concept of web-playability. Server detects, client degrades: - mediaPlayable.ts: classifyVideoPlayable(mimetype, codec) — tri-state (false = known-undecodable e.g. HEVC/ProRes, true = web codec in web container, null = unknown/optimistic). Never widens `false` beyond codecs that fail everywhere, so ffmpeg-less instances keep prior behaviour. - probeMediaMeta now captures the video codec_name; the upload finish hook stores the verdict in the new attachments.playable column (migration 0007). - Flag propagated through every serializer: space messages, DMs, WS, and federation relay (outbound + inbound) — federation-compatible. - VideoAttachment component: playable===false renders a download card (poster + "Can't play here — download" + name/duration/size) with no dead-player flash; otherwise plays inline with an onError fallback to the same card. Specs updated: uploads.md, database.md, federation.md.
This commit is contained in:
@@ -105,6 +105,7 @@ PK: (spaceId, userId)
|
||||
| width | integer | | Image/video pixel width |
|
||||
| height | integer | | Image/video pixel height |
|
||||
| duration | real | | Audio/video seconds |
|
||||
| playable | integer (boolean) | NULL | Video web-playability tri-state: 1 = browser-decodable, 0 = known-undecodable (e.g. HEVC .mov), NULL = unknown/non-video. See uploads.md §3. |
|
||||
| sourceUrl | text | | Remote URL (federation) |
|
||||
| federationStatus | text | | local/remote/remote_partial |
|
||||
| federationMeta | text | | JSON rejection info |
|
||||
|
||||
@@ -941,12 +941,13 @@ When `queueDmRelay` constructs the relay payload, each attachment gets a `source
|
||||
```
|
||||
sourceUrl: `${getOurOrigin()}/api/uploads/${attachment.filename}`
|
||||
```
|
||||
The payload also carries `playable` (video web-playability, computed by the origin from the probed codec — see uploads.md §3) so the receiving instance need not re-probe the file's codec.
|
||||
|
||||
### Inbound (receiving instance -- `processCreateEvent`)
|
||||
|
||||
1. For each attachment in `event.message.attachments`:
|
||||
- SSRF check: `isUrlFromPeer(sourceUrl, peerOrigin)` -- hostname of sourceUrl must match peer origin hostname
|
||||
- Create `attachments` row with `filename = sourceUrl` (remote URL as interim filename)
|
||||
- Create `attachments` row with `filename = sourceUrl` (remote URL as interim filename), carrying through `playable` from the relay payload
|
||||
- Queue `federation_file_queue` entry with `status = 'pending'`, `expiresAt = now + 30 days`
|
||||
2. Initial WebSocket broadcast uses sourceUrl directly (frontend's `AttachmentRenderer` detects `http` prefix)
|
||||
|
||||
|
||||
+14
-3
@@ -3,7 +3,8 @@
|
||||
Source files:
|
||||
- `packages/server/src/routes/uploads.ts` -- File serving (cache, security, Range)
|
||||
- `packages/server/src/routes/files.ts` -- tus protocol endpoints (`/api/files/*`), PRE_CREATE / PRE_PATCH / POST_FINISH hooks, janitor helpers
|
||||
- `packages/server/src/utils/thumbnail.ts` -- Image thumbnail generation (sharp), video thumbnail extraction (ffmpeg), image dimension probing, profile image resizing, media metadata extraction
|
||||
- `packages/server/src/utils/thumbnail.ts` -- Image thumbnail generation (sharp), video thumbnail extraction (ffmpeg), image dimension probing, profile image resizing, media metadata extraction (incl. video codec)
|
||||
- `packages/server/src/utils/mediaPlayable.ts` -- `classifyVideoPlayable(mimetype, codec)` web-playability classifier (drives `attachments.playable`)
|
||||
- `packages/server/src/utils/fileCleanup.ts` -- File deletion helpers (disk + thumbnail + attachment record cleanup)
|
||||
- `packages/server/src/utils/storageJanitor.ts` -- Storage stats, orphan detection, cleanup routines (orphaned files, unlinked attachments, dangling references, old media), federation GC, soft-deleted DM channel purge
|
||||
- `packages/web/src/stores/transferStore.ts` -- Client transfer manager (uploads via tus-js-client, downloads via fetch + FS Access)
|
||||
@@ -149,10 +150,20 @@ Processing occurs inline during upload, before the response is sent. All process
|
||||
- Returns `{ thumbnailFilename, width, height }` (dimensions are from the original frame, not the thumbnail)
|
||||
|
||||
2. **Metadata probing** (`thumbnail.ts:probeMediaMeta`)
|
||||
- ffprobe for dimensions: `-select_streams v:0 -show_entries stream=width,height -of json`
|
||||
- ffprobe for dimensions + codec: `-select_streams v:0 -show_entries stream=width,height,codec_name -of json`
|
||||
- ffprobe for duration: `-show_entries format=duration -of json`
|
||||
- Duration rounded to 2 decimal places
|
||||
- If thumbnail extraction failed, dimensions fall back to ffprobe values
|
||||
- Returns `codec` (the primary video stream's `codec_name`) when available
|
||||
|
||||
3. **Web-playability classification** (`mediaPlayable.ts:classifyVideoPlayable`)
|
||||
- The finish hook calls `classifyVideoPlayable(mimetype, codec)` and stores the result in `attachments.playable` (tri-state, see below).
|
||||
- The browser `<video>` element can't decode every uploaded format. The dominant failure case is a macOS screen recording — a `video/quicktime` (.mov) container holding an **HEVC (H.265)** stream — which Chromium, Firefox and stock Electron can't decode. The file uploads fine and a server-side ffmpeg poster is generated, but inline playback silently fails (stuck at 0:00 with no error).
|
||||
- `attachments.playable` is a deliberate tri-state:
|
||||
- `0` / `false` — codec is confidently undecodable in mainstream browsers (HEVC, ProRes, WMV, MPEG-1/2, etc.). The client renders a download fallback card directly, no flash of a dead player.
|
||||
- `1` / `true` — web-standard codec (H.264/AVC, VP8/VP9, AV1, Theora) in a web container (`video/mp4`, `video/webm`, `video/ogg`). Plays inline.
|
||||
- `NULL` — unknown / optimistic. Codec couldn't be probed (ffmpeg absent or probe failed), or it's a web-safe codec in a container with inconsistent cross-browser support (H.264 in .mov). The client attempts inline playback and degrades via the `<video>` `onError` handler.
|
||||
- `false` is never widened beyond codecs known to fail everywhere, so an instance without ffmpeg keeps prior behaviour (attempt playback) rather than regressing every video to "unplayable".
|
||||
|
||||
### Audio Processing
|
||||
|
||||
@@ -521,7 +532,7 @@ URL resolution for attachment/thumbnail:
|
||||
| MIME category | Rendering |
|
||||
|---------------|-----------|
|
||||
| `image/*` | `<img>` with click-to-preview, lazy loading, aspect ratio from width/height, max 400x300px, uses thumbnail if available |
|
||||
| `video/*` | `<video>` with native controls, poster from thumbnail, preload `none` (with dimensions) or `metadata` (without), max 400px wide / 300px tall |
|
||||
| `video/*` | `VideoAttachment` sub-component. Playable (`playable !== false`): `<video src>` with native controls, poster from thumbnail, preload `none` (with dimensions) or `metadata` (without), max 400px wide / 300px tall, with an `onError` handler that falls back to the download card. Unplayable (`playable === false`, e.g. HEVC .mov): renders the download card directly — poster (if any) under a "Can't play here — download" overlay, plus filename, duration, size and a one-tap download. Never a silently broken player. |
|
||||
| `audio/*` | Audio card with icon, filename, size, `<audio>` with native controls, preload `metadata`, max 420px wide |
|
||||
| Other | Download link card with file icon, filename (link-styled), size |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user