feat(settings/mobile): Voice & Video adaption + chat header username normalization

MobileChatScreen DM header now applies parseFederatedUsername and
useCanonicalUserView, matching MobileDmsScreen so federated users render
as 'realname' rather than 'realname@domain'.

Mobile Voice & Audio renamed to Voice & Video across MobileSettingsScreen
and MobileYouScreen — parity with desktop's VoicePanel title.

AudioInputSection and VideoSection picker click-outside now listens for
touchstart alongside mousedown so a single tap dismisses on iOS Safari
(which doesn't synthesize mousedown reliably from touch).

AudioOutputSection feature-detects HTMLMediaElement.setSinkId at module
load and returns null on unsupported platforms (notably iOS Safari).
Split into outer gate + inner body to keep Rules of Hooks intact;
VoicePanel's space-y-5 collapses cleanly with no visible hole.

VideoSection 'Stop preview' CTA gets responsive sizing (mobile:
always-visible 44 px tap target; desktop: original hover-reveal pill via
md: prefixes). The preview <video> gains autoPlay so iOS Safari starts
the stream when srcObject is assigned — manual videoEl.play() after an
awaited getUserMedia loses the user-gesture context on iOS and was
silently rejected by .catch(()=>{}), causing the black-preview bug.

voice.md updated for the iOS hide-when-unsupported policy and the
touch-close contract on device pickers.
This commit is contained in:
Jannis Braun
2026-05-05 23:24:03 +02:00
parent fc9a07523f
commit 899dc8b601
7 changed files with 104 additions and 25 deletions
+6 -1
View File
@@ -309,7 +309,12 @@ Both surfaces are backed by the shared `useAudioDevices()` hook. The store field
- Returns `inputLabels` / `outputLabels` maps with disambiguation suffixes for duplicate names (e.g. `"USB Audio (1)"`, `"USB Audio (2)"`). - Returns `inputLabels` / `outputLabels` maps with disambiguation suffixes for duplicate names (e.g. `"USB Audio (1)"`, `"USB Audio (2)"`).
### Output routing — `AudioContext.setSinkId` ### Output routing — `AudioContext.setSinkId`
All audio (remote voice, screen-share audio, sound effects) flows through `AudioManager`'s master bus → `AudioContext.destination`. Output device switching is therefore done via `AudioContext.setSinkId(deviceId)`, NOT via LiveKit's `switchActiveDevice('audiooutput')` (which targets `<audio>` elements that are killed by `AppLayout`'s MutationObserver). Safari < 17 lacks `setSinkId` on AudioContext — `AudioOutputSection` detects this and falls back to OS default with an explanatory note. All audio (remote voice, screen-share audio, sound effects) flows through `AudioManager`'s master bus → `AudioContext.destination`. Output device switching is therefore done via `AudioContext.setSinkId(deviceId)`, NOT via LiveKit's `switchActiveDevice('audiooutput')` (which targets `<audio>` elements that are killed by `AppLayout`'s MutationObserver). Safari < 17 lacks `setSinkId` on AudioContext — `AudioOutputSection` detects this (once a real context exists) and falls back to OS default with an explanatory note.
**Mobile platforms with no per-element output routing (iOS Safari):** `AudioOutputSection` feature-detects `'setSinkId' in HTMLMediaElement.prototype` at module load (cached). When false, the entire section is hidden — no header, no fallback copy. iOS users adjust audio routing via OS controls (Bluetooth menu, Control Center) and do not expect per-app output selection. Android Chrome ≥ 110 supports `setSinkId` and renders the picker normally. The detection runs before any hooks via an outer wrapper (`AudioOutputSection` → early-return-`null``AudioOutputSectionInner`) so the inner component's hook order remains stable.
### Touch-close on device pickers
The Audio Input, Audio Output, and Video device dropdowns (`AudioInputSection.tsx`, `AudioOutputSection.tsx`, `VideoSection.tsx`) all listen for both `mousedown` AND `touchstart` (`{ passive: true }`) when implementing click-outside-to-close. iOS Safari does not synthesize `mousedown` reliably from a single tap; without the `touchstart` listener, mobile users would have to tap twice to dismiss an open popover.
### Input pipeline — republish, never `switchActiveDevice` ### Input pipeline — republish, never `switchActiveDevice`
Input device changes flow through `AudioManager.setInputDevice(deviceId)` (serialized chain). The `useLiveKit syncMic` effect detects the bumped stream generation and unpublishes/republishes via `getFreshTrack()`. This asymmetry vs. the camera (which uses `room.switchActiveDevice('videoinput', …)`) is intentional and documented under "Architectural asymmetry" below — the published mic track is the output of a Web Audio graph (RNNoise, gain, AEC), not a raw `getUserMedia` track. Input device changes flow through `AudioManager.setInputDevice(deviceId)` (serialized chain). The `useLiveKit syncMic` effect detects the bumped stream generation and unpublishes/republishes via `getFreshTrack()`. This asymmetry vs. the camera (which uses `room.switchActiveDevice('videoinput', …)`) is intentional and documented under "Architectural asymmetry" below — the published mic track is the output of a Web Audio graph (RNNoise, gain, AEC), not a raw `getUserMedia` track.
@@ -7,6 +7,11 @@ import { MessageList } from '../chat/MessageList';
import { MessageInput } from '../chat/MessageInput'; import { MessageInput } from '../chat/MessageInput';
import { TypingIndicator } from '../chat/TypingIndicator'; import { TypingIndicator } from '../chat/TypingIndicator';
import { TransferIndicator } from './TransferIndicator'; import { TransferIndicator } from './TransferIndicator';
import { parseFederatedUsername } from '../../utils/identity';
import { useCanonicalUserView } from '../../utils/userViewLookup';
import type { User } from '@backspace/shared';
const FALLBACK_USER = { id: '', username: '', createdAt: 0, isAdmin: false, replicatedInstances: [] } as unknown as User;
interface MobileChatScreenProps { interface MobileChatScreenProps {
params?: Record<string, string>; params?: Record<string, string>;
@@ -32,16 +37,31 @@ export function MobileChatScreen({ params }: MobileChatScreenProps) {
loadMessages(channelId); loadMessages(channelId);
}, [channelId, setCurrentChannel, loadMessages]); }, [channelId, setCurrentChannel, loadMessages]);
// Resolve the "main other member" of a 1:1 DM up-front so we can route it
// through useCanonicalUserView (hook, must run unconditionally). Group DMs
// don't get the cache treatment in the header — the comma-joined title falls
// back to per-member parseFederatedUsername normalization, matching the
// pattern in MobileDmsScreen.
const dm = isDm && channelId ? dmChannels.find(d => d.id === channelId) : undefined;
const otherMembers = dm ? dm.members.filter(m => m.id !== authUser?.id) : [];
const isGroup = !!dm?.ownerId;
const rawMainOther = !isGroup ? otherMembers[0] : undefined;
const canonicalMainOther = useCanonicalUserView((rawMainOther as unknown as User) ?? FALLBACK_USER);
// Resolve channel/DM name // Resolve channel/DM name
let channelName = 'Channel'; let channelName = 'Channel';
if (isDm && channelId) { if (isDm && dm) {
const dm = dmChannels.find(d => d.id === channelId); if (isGroup) {
if (dm) { channelName = otherMembers
const otherMembers = dm.members.filter(m => m.id !== authUser?.id); .map(m => m.displayName ?? parseFederatedUsername(m.username).baseName)
const isGroup = !!dm.ownerId; .join(', ');
channelName = isGroup } else if (rawMainOther) {
? otherMembers.map(m => m.displayName ?? m.username).join(', ') channelName =
: otherMembers[0]?.displayName ?? otherMembers[0]?.username ?? 'Direct Message'; canonicalMainOther.displayName ??
parseFederatedUsername(canonicalMainOther.username).baseName ??
'Direct Message';
} else {
channelName = 'Direct Message';
} }
} else if (!isDm && channelId) { } else if (!isDm && channelId) {
const ch = channels.find(c => c.id === channelId); const ch = channels.find(c => c.id === channelId);
@@ -12,7 +12,7 @@ interface MobileSettingsScreenProps {
const panelConfig: Record<string, { title: string; component: React.ReactNode }> = { const panelConfig: Record<string, { title: string; component: React.ReactNode }> = {
account: { title: 'Account', component: <AccountPanel /> }, account: { title: 'Account', component: <AccountPanel /> },
voice: { title: 'Voice & Audio', component: <VoicePanel /> }, voice: { title: 'Voice & Video', component: <VoicePanel /> },
privacy: { title: 'Privacy', component: <PrivacyPanel /> }, privacy: { title: 'Privacy', component: <PrivacyPanel /> },
connections: { title: 'Connections', component: <ConnectionsPanel /> }, connections: { title: 'Connections', component: <ConnectionsPanel /> },
}; };
@@ -76,7 +76,7 @@ export function MobileSettingsScreen({ initialPanel }: MobileSettingsScreenProps
// Settings section list // Settings section list
const sections = [ const sections = [
{ id: 'account', label: 'Account' }, { id: 'account', label: 'Account' },
{ id: 'voice', label: 'Voice & Audio' }, { id: 'voice', label: 'Voice & Video' },
{ id: 'privacy', label: 'Privacy' }, { id: 'privacy', label: 'Privacy' },
{ id: 'connections', label: 'Connections' }, { id: 'connections', label: 'Connections' },
...(isAdmin ? [{ id: 'instance', label: 'Instance' }] : []), ...(isAdmin ? [{ id: 'instance', label: 'Instance' }] : []),
@@ -45,7 +45,7 @@ export function MobileYouScreen() {
action: () => pushMobileScreen('settings-connections'), action: () => pushMobileScreen('settings-connections'),
}, },
{ {
label: 'Voice & Audio', label: 'Voice & Video',
icon: ( icon: (
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}> <svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
<path strokeLinecap="round" strokeLinejoin="round" d="M12 18.75a6 6 0 006-6v-1.5m-6 7.5a6 6 0 01-6-6v-1.5m6 7.5v3.75m-3.75 0h7.5M12 15.75a3 3 0 01-3-3V4.5a3 3 0 116 0v8.25a3 3 0 01-3 3z" /> <path strokeLinecap="round" strokeLinejoin="round" d="M12 18.75a6 6 0 006-6v-1.5m-6 7.5a6 6 0 01-6-6v-1.5m6 7.5v3.75m-3.75 0h7.5M12 15.75a3 3 0 01-3-3V4.5a3 3 0 116 0v8.25a3 3 0 01-3 3z" />
@@ -23,16 +23,22 @@ export function AudioInputSection() {
const dropdownRef = useRef<HTMLDivElement>(null); const dropdownRef = useRef<HTMLDivElement>(null);
const animFrameRef = useRef<number>(0); const animFrameRef = useRef<number>(0);
// Click-outside-to-close. // Click-outside-to-close. iOS Safari does not synthesize `mousedown` from
// touch reliably, so we listen for `touchstart` alongside `mousedown` to
// make the popover dismissable with a single tap on touch devices.
useEffect(() => { useEffect(() => {
if (!listOpen) return; if (!listOpen) return;
const onMouseDown = (e: MouseEvent) => { const onPointerDown = (e: MouseEvent | TouchEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) { if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) {
setListOpen(false); setListOpen(false);
} }
}; };
document.addEventListener('mousedown', onMouseDown); document.addEventListener('mousedown', onPointerDown);
return () => document.removeEventListener('mousedown', onMouseDown); document.addEventListener('touchstart', onPointerDown, { passive: true });
return () => {
document.removeEventListener('mousedown', onPointerDown);
document.removeEventListener('touchstart', onPointerDown);
};
}, [listOpen]); }, [listOpen]);
// Listen for AudioContext resume events so dependent effects re-trigger // Listen for AudioContext resume events so dependent effects re-trigger
@@ -4,7 +4,37 @@ import { AudioManager } from '../../../audio/AudioManager';
import { useAudioDevices } from '../../../hooks/useAudioDevices'; import { useAudioDevices } from '../../../hooks/useAudioDevices';
import { SectionShell, DropdownItem } from './_shared/SettingsPickerPrimitives'; import { SectionShell, DropdownItem } from './_shared/SettingsPickerPrimitives';
/**
* Feature-detect per-element output routing support. iOS Safari has zero
* support for `HTMLMediaElement.setSinkId` (and likewise no AudioContext
* variant); when both are absent, the OS routes audio (Bluetooth menu, etc.)
* and an in-app picker would be non-functional.
*
* Cached at module level since support cannot change within a page lifetime.
*/
let cachedPlatformSupportsSinkSelection: boolean | null = null;
function platformSupportsSinkSelection(): boolean {
if (cachedPlatformSupportsSinkSelection !== null) return cachedPlatformSupportsSinkSelection;
if (typeof window === 'undefined' || typeof HTMLMediaElement === 'undefined') {
cachedPlatformSupportsSinkSelection = false;
return false;
}
const supported = 'setSinkId' in HTMLMediaElement.prototype;
cachedPlatformSupportsSinkSelection = supported;
return supported;
}
export function AudioOutputSection() { export function AudioOutputSection() {
// Hard gate: if the browser cannot route audio per-element at all (iOS
// Safari has zero support for HTMLMediaElement.setSinkId), hide the entire
// section. Users adjust output via OS controls (Bluetooth menu, etc.).
// This check is module-level cached; for a given page lifetime the function
// always returns the same value, so hook order downstream is stable.
if (!platformSupportsSinkSelection()) return null;
return <AudioOutputSectionInner />;
}
function AudioOutputSectionInner() {
const outputDeviceId = useVoiceStore((s) => s.outputDeviceId); const outputDeviceId = useVoiceStore((s) => s.outputDeviceId);
const setOutputDevice = useVoiceStore((s) => s.setOutputDevice); const setOutputDevice = useVoiceStore((s) => s.setOutputDevice);
const outputVolume = useVoiceStore((s) => s.outputVolume); const outputVolume = useVoiceStore((s) => s.outputVolume);
@@ -26,16 +56,22 @@ export function AudioOutputSection() {
const [audioCtxGen, setAudioCtxGen] = useState(0); const [audioCtxGen, setAudioCtxGen] = useState(0);
const dropdownRef = useRef<HTMLDivElement>(null); const dropdownRef = useRef<HTMLDivElement>(null);
// Click-outside-to-close. // Click-outside-to-close. Listens for both mousedown and touchstart so the
// popover dismisses with a single tap on touch devices (iOS Safari does not
// synthesize mousedown reliably from touch).
useEffect(() => { useEffect(() => {
if (!listOpen) return; if (!listOpen) return;
const onMouseDown = (e: MouseEvent) => { const onPointerDown = (e: MouseEvent | TouchEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) { if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) {
setListOpen(false); setListOpen(false);
} }
}; };
document.addEventListener('mousedown', onMouseDown); document.addEventListener('mousedown', onPointerDown);
return () => document.removeEventListener('mousedown', onMouseDown); document.addEventListener('touchstart', onPointerDown, { passive: true });
return () => {
document.removeEventListener('mousedown', onPointerDown);
document.removeEventListener('touchstart', onPointerDown);
};
}, [listOpen]); }, [listOpen]);
// Listen for AudioContext resume events so the supportsSinkId check // Listen for AudioContext resume events so the supportsSinkId check
@@ -120,16 +120,23 @@ export function VideoSection() {
if (videoEl) videoEl.srcObject = null; if (videoEl) videoEl.srcObject = null;
}; };
// Close the dropdown when the user clicks outside it. // Close the dropdown when the user clicks outside it. Listens for both
// mousedown and touchstart so the popover dismisses with a single tap on
// touch devices (iOS Safari does not synthesize mousedown reliably from
// touch).
useEffect(() => { useEffect(() => {
if (!listOpen) return; if (!listOpen) return;
const onMouseDown = (e: MouseEvent) => { const onPointerDown = (e: MouseEvent | TouchEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) { if (dropdownRef.current && !dropdownRef.current.contains(e.target as Node)) {
setListOpen(false); setListOpen(false);
} }
}; };
document.addEventListener('mousedown', onMouseDown); document.addEventListener('mousedown', onPointerDown);
return () => document.removeEventListener('mousedown', onMouseDown); document.addEventListener('touchstart', onPointerDown, { passive: true });
return () => {
document.removeEventListener('mousedown', onPointerDown);
document.removeEventListener('touchstart', onPointerDown);
};
}, [listOpen]); }, [listOpen]);
// Tab-visibility cleanup: release the camera light when the tab is hidden. // Tab-visibility cleanup: release the camera light when the tab is hidden.
@@ -508,6 +515,7 @@ export function VideoSection() {
ref={previewVideoRef} ref={previewVideoRef}
muted muted
playsInline playsInline
autoPlay
className={`w-full h-full object-cover ${isDormant ? 'invisible' : ''}`} className={`w-full h-full object-cover ${isDormant ? 'invisible' : ''}`}
style={{ transform: 'scaleX(-1)' }} style={{ transform: 'scaleX(-1)' }}
/> />
@@ -528,7 +536,11 @@ export function VideoSection() {
<button <button
type="button" type="button"
onClick={stopPreviewFromUser} onClick={stopPreviewFromUser}
className="absolute top-2 right-2 text-[11px] px-2 py-1 rounded-md bg-black/60 hover:bg-black/75 text-white/90 transition-colors opacity-0 group-hover:opacity-100 focus-visible:opacity-100" // Mobile: always visible (no hover state) and 44 px tap target
// per iOS HIG. Desktop: original hover-reveal compact pill.
className="absolute top-2 right-2 rounded-md bg-black/60 hover:bg-black/75 text-white/90 transition-colors focus-visible:opacity-100
min-h-[44px] min-w-[44px] px-3 py-2 text-xs flex items-center justify-center
md:min-h-0 md:min-w-0 md:text-[11px] md:px-2 md:py-1 md:opacity-0 md:group-hover:opacity-100"
> >
Stop preview Stop preview
</button> </button>