fix(uploads): graceful fallback for browser-unplayable video (HEVC .mov)

macOS screen recordings are HEVC inside a .mov container, which Chromium,
Firefox and stock Electron can't decode. The file uploaded fine and a
server-side ffmpeg poster was generated, but inline <video> playback failed
silently — stuck at 0:00 with no error, since AttachmentRenderer had no error
handling. Root cause: the system had no concept of web-playability.

Server detects, client degrades:
- mediaPlayable.ts: classifyVideoPlayable(mimetype, codec) — tri-state
  (false = known-undecodable e.g. HEVC/ProRes, true = web codec in web
  container, null = unknown/optimistic). Never widens `false` beyond codecs
  that fail everywhere, so ffmpeg-less instances keep prior behaviour.
- probeMediaMeta now captures the video codec_name; the upload finish hook
  stores the verdict in the new attachments.playable column (migration 0007).
- Flag propagated through every serializer: space messages, DMs, WS, and
  federation relay (outbound + inbound) — federation-compatible.
- VideoAttachment component: playable===false renders a download card (poster
  + "Can't play here — download" + name/duration/size) with no dead-player
  flash; otherwise plays inline with an onError fallback to the same card.

Specs updated: uploads.md, database.md, federation.md.
This commit is contained in:
Jannis Braun
2026-06-30 17:38:11 +02:00
parent e84daf57aa
commit 209aef7e9d
18 changed files with 3999 additions and 30 deletions
@@ -0,0 +1,77 @@
/**
* Web-playability classification for video attachments.
*
* The browser `<video>` element can only decode a subset of the formats users
* upload. The dominant failure case is a macOS screen recording — a
* `video/quicktime` (.mov) container holding an HEVC (H.265) stream — which
* Chromium, Firefox and stock Electron cannot decode. The file uploads fine,
* a server-side ffmpeg poster is generated, but inline playback silently fails
* (stuck at 0:00 with no error). We persist this classification per attachment
* so the client can render a download fallback instead of a dead player.
*
* The result is a deliberate tri-state:
* - `false` — confidently undecodable in mainstream browsers (e.g. HEVC).
* The client renders the fallback card directly, no flash.
* - `true` — confidently decodable (web codec in a web container).
* - `null` — unknown / optimistic. The codec couldn't be probed (ffmpeg
* absent or probe failed), or it's a web-safe codec in a
* container with inconsistent cross-browser support (e.g. H.264
* in .mov). The client attempts playback and degrades to the
* fallback via the `<video>` `onError` handler.
*
* We never widen `false` beyond codecs we are certain fail everywhere, so an
* instance without ffmpeg (codec always undefined) keeps today's behaviour
* (attempt playback) rather than regressing every video to "unplayable".
*/
/** ffprobe `codec_name` (or mp4/mov box tag) values that no mainstream browser decodes. */
const UNPLAYABLE_VIDEO_CODECS = new Set([
// HEVC / H.265 — codec_name is `hevc`; box tags are `hvc1` / `hev1`.
'hevc', 'hvc1', 'hev1', 'h265',
// Apple ProRes — editing/intermediate codec, never web-decodable.
'prores',
// Windows Media Video.
'wmv1', 'wmv2', 'wmv3', 'vc1',
// Legacy / capture codecs.
'mpeg1video', 'mpeg2video', 'dnxhd', 'mjpeg', 'vp6', 'vp6f',
]);
/** Codecs every modern browser can decode when in a web-standard container. */
const PLAYABLE_VIDEO_CODECS = new Set([
'h264', 'avc1', // H.264 / AVC
'vp8', 'vp9',
'av1', 'av01',
'theora',
]);
/** Containers with reliable cross-browser `<video>` support. */
const PLAYABLE_VIDEO_CONTAINERS = new Set([
'video/mp4',
'video/webm',
'video/ogg',
]);
/**
* Classify whether a video attachment can be played inline in a browser
* `<video>` element, given its container mimetype and the probed video codec.
*
* @param mimetype Container mimetype (e.g. `video/quicktime`).
* @param codec ffprobe `codec_name` of the primary video stream, if known.
* @returns `false` (known-unplayable), `true` (known-playable) or `null` (unknown/optimistic).
*/
export function classifyVideoPlayable(
mimetype: string,
codec: string | null | undefined,
): boolean | null {
if (!mimetype.startsWith('video/')) return null;
if (!codec) return null;
const c = codec.toLowerCase();
if (UNPLAYABLE_VIDEO_CODECS.has(c)) return false;
if (PLAYABLE_VIDEO_CONTAINERS.has(mimetype) && PLAYABLE_VIDEO_CODECS.has(c)) return true;
// Web-safe codec in a shaky container (H.264 in .mov), or an unrecognised
// codec we can't vouch for: stay optimistic and let the client's onError
// handler catch a genuine playback failure.
return null;
}