Commit Graph
171 Commits
Author SHA1 Message Date
Jannis Braun aeb38a714a rename project from Opencord to Backspace in deployment config
- docker-compose: use external volume/network with backspace naming
- deploy.sh: add --remote/-r flag for off-network deploys, update remote path
- Dockerfile: update DB_PATH to backspace.db
2026-03-01 01:22:03 +01:00
Jannis Braun 773a03b1aa feat: instance-level streaming limits with admin settings panel
Add a server-side instance_settings table (single-row, CHECK(id=1))
that stores admin-configurable streaming bounds: bitrate min/max/step,
allowed resolutions, and allowed framerates.

Backend:
- New instance_settings schema + migrations (is_admin on users, default
  settings row, first-registered-user promoted to admin)
- GET/PATCH /api/settings/streaming endpoints with admin-only writes
  and full input validation including cross-field checks

Frontend:
- settingsStore fetches limits on WebSocket ready, exposes isAdmin flag
- ScreenShareSettingsPopover reads bounds from store instead of
  hardcoded constants, auto-clamps stale localStorage values
- buildScreenShareOptions() clamps bitrate to server limits at build
  time as enforcement backstop
- ServerSettings modal gains a "Streaming" tab (admin-only) with
  bitrate range sliders, resolution/framerate toggles, and save/reset
2026-02-26 03:33:29 +01:00
Jannis Braun 2184ded2c1 fix: use autoSubscribe:false to prevent LiveKit renegotiation storm on screen share
When a viewer joined while a screen share was active, autoSubscribe:true
caused a subscribe-then-unsubscribe dance for screen share tracks,
triggering cascading renegotiations, MaxListeners warnings, and
negotiation timeouts. Now the SFU starts with no subscriptions and we
explicitly subscribe only to non-screen-share tracks (audio, camera).
Screen shares remain controlled by the watch/unwatch UI flow.
2026-02-26 03:02:27 +01:00
Jannis Braun 61e0bf6fe8 fix: lower screen share default bitrates for VP9 codec efficiency 2026-02-26 02:43:17 +01:00
Jannis Braun 4bb69851d3 fix: use LiveKit track.attach() for adaptive stream and switch screen share to VP9 single-layer
Camera was stuck at 180p because our custom <video> rendering bypassed
LiveKit's adaptive stream observer. Replaced manual srcObject binding
with track.attach()/detach() in VoiceUser, StreamTile, and PictureInPicture
so the SFU receives viewport dimensions and forwards the correct
H.264 simulcast layer.

Screen share VP9 SVC with L3T3 spatial layers failed because hardware
VP9 encoders (NVENC, QSV, VCE) don't support spatial scalability —
Chrome silently degrades to L1T1. Reverted to VP9 single-layer
(simulcast: false, no scalabilityMode). Also targets encodings[length-1]
in applyOverdrive() for correct simulcast layer targeting.
2026-02-26 02:33:18 +01:00
Jannis Braun 33e9198de6 perf: enable adaptiveStream and camera simulcast for SFU bandwidth optimization
- adaptiveStream: true — SFU adjusts quality per subscriber viewport size
- Camera simulcast: true — publisher encodes 3 quality layers, SFU picks per viewer
- Screen share keeps simulcast: false (text readability)
- Server-side: livekit.yaml updated with 40 Mbps bandwidth cap, room cleanup,
  playout delay, and A/V sync (applied directly to Pi, not in this commit)
2026-02-26 01:09:05 +01:00
Jannis Braun 2186235d65 feat: manual bitrate override slider for screen share settings
- Add customBitrateKbps to ScreenShareConfig (null = auto matrix lookup)
- Slider in Stream Settings popover: 500 kbps–20 Mbps, step 500 kbps
- "Reset to Auto" clears override back to preset-derived bitrate
- Live updates via existing applyOverdrive() pipeline on active streams
- Persist version 5 → 6 with migration
2026-02-26 00:28:20 +01:00
Jannis Braun 3ceab3d73e fix: hydrate replyTo on optimistic messages so reply preview renders instantly 2026-02-26 00:17:00 +01:00
Jannis Braun c9f9787b99 feat: rich markdown renderer with @mention badges and autocomplete
- Add MarkdownRenderer with syntax-highlighted code blocks (prism-react-renderer),
  GFM tables, and Discord-style theming
- Add MentionBadge that resolves user IDs to display names with role colors
- Add MentionPopover autocomplete triggered by @ in MessageInput
- Fix mention:// URL sanitization — allowlist the scheme in urlTransform so
  react-markdown v9 passes it through to the component override
- Highlight messages that mention the current user (amber border)
- Bump rate limit from 60 to 200 req/min
2026-02-25 23:49:41 +01:00
Jannis Braun 0a157de162 refactor: purge legacy server_members.role column, single source of truth via member_roles
Remove the legacy TEXT role column ('owner'/'admin'/'member') from
server_members and make the bitwise RBAC member_roles junction table
the sole authority for role assignments. Owner detection now uses
servers.ownerId exclusively.

- Remove MemberRole type and role field from shared types
- Remove role from Drizzle schema, raw SQL CREATE TABLE, and seed data
- Rewrite PATCH /members/:uid to accept { roleIds: string[] }
- Fix GET /members to populate roles array (was TODO)
- Replace member.role === 'owner' guard with isServerOwner()
- Remove getMemberRole() helper and legacy bridge code
- MemberSidebar groups by highest-positioned role instead of legacy string
- ServerSettings replaces admin/member dropdown with role checkboxes
- Message.tsx derives color from roles[] with owner fallback via ownerId
- Existing DBs keep vestigial column (Drizzle ignores it); new DBs omit it
2026-02-25 22:46:50 +01:00
Jannis Braun 76b8a43be2 fix: enforce channel-level RBAC across WS broadcasts, REST endpoints, and frontend reactivity
Wire the bitwise permission engine end-to-end:

- Add sendToChannel() to ConnectionManager, filtering WS recipients by VIEW_CHANNEL
- Convert 6 channel-scoped events (messages, typing, reactions) from sendToServer to sendToChannel
- Add broadcastOverrideChange() to push channel_updated/channel_deleted per-user on override mutations
- Bridge legacy server_members.role TEXT to member_roles junction table on PATCH
- Add pushReadyPayload() to force re-sync frontend store after role changes
- Filter channels by VIEW_CHANNEL in GET /api/servers/:id to prevent initial load data leak
- Pre-compute viewers before CASCADE delete on channel_deleted
- Fix frontend channel event handlers to upsert/cleanup channelToServerMap and channelPermissions
- Add ChannelSettingsModal with Private Channel toggle and gear icon in ChannelSidebar
2026-02-24 06:10:11 +01:00
Jannis Braun 8030c89c6c feat: bitwise RBAC engine with channel-level permission overrides
Replace string-based role checks (role === 'admin') with a bitwise BigInt
permission system. Adds computePermissions() resolution engine following
Discord's model: @everyone base → role union → admin shortcut → channel
overrides (role deny/allow → member deny/allow). Ready payload now filters
channels by VIEW_CHANNEL and attaches per-user myPermissions to each
server and channel. Includes channel_overrides table, @everyone role
auto-creation, migration for existing servers, and override CRUD API.
2026-02-24 05:08:59 +01:00
Jannis Braun 024833c470 fix: security hardening and Safari stability
- Remove hardcoded JWT_SECRET fallback (crash on boot if unset)
- Make LiveKit config optional with 503 guard on token endpoint
- Add REST rate limiting via @fastify/rate-limit (auth 10/15m, messages 5/5s, uploads 10/1m, global 60/1m)
- Add WebSocket token bucket rate limiter (30 burst, 2/sec refill)
- Add DM channel ownership (ownerId) with migration, enforce on add-member
- Require friendship to add users to group DMs
- Add silent 20Hz oscillator to prevent Safari AudioContext suspension
- Move WebSocket heartbeat to Web Worker to bypass Safari background throttling
2026-02-24 04:34:36 +01:00
Jannis Braun 36e27121da fix: harden data integrity, connection stability, and memory management
Wrap all multi-write DB operations in atomic transactions (server/channel
creation, message+attachment linking, DM creation, friend acceptance,
cascading deletes) to prevent partial-write corruption.

Batch N+1 queries in WS ready payload into O(1) bulk fetches with
chunked inArray() to respect SQLite's variable limit.

Fix chat history regression where background WS messages bypassed
channel load by switching the guard from messages.has() to hasMore.has().

Add LRU channel eviction (20 cached, evict to 15) and per-channel
message cap (200) to bound client memory growth.

Shorten WS heartbeat from 30s to 15s for aggressive proxy/NAT
environments. Clear all user-scoped stores on logout to prevent
cross-session data leaks.

Extract LiveKit internal accessors into shared livekitInternals utility.
2026-02-24 03:52:22 +01:00
Jannis Braun 2342396fce fix: disable browser DSP on screen share audio and harden ICE stats resolution
Screen share audio was muffled/gated because getDisplayMedia used plain
`audio: true`, letting the browser apply voice-optimized DSP (NS, AEC, AGC)
to desktop audio. Now passes explicit constraints disabling all processing
and requesting stereo capture.

Also improves WebRTC stats: three-tier ICE candidate-pair discovery
(transport → active-bytes heuristic → legacy fallback), height-based
simulcast layer inference, and documents mDNS obfuscation limitation.
2026-02-24 02:31:24 +01:00
Jannis Braun f808a204e7 refactor: dynamic screen share engine with independent resolution/fps/mode axes
Replace rigid SCREEN_QUALITY_MAP (6 hardcoded VideoPreset strings) with a
builder function that computes bitrate, degradation preference, and content
hint from three independent axes (height, fps, content mode). Camera is
decoupled onto a fixed 720p30 preset so screen share changes no longer
affect camera quality. New ScreenShareSettingsPopover replaces the old
VideoQualityPopover with pill-style selectors. Store migrated to v5 with
backwards-compatible migration from videoQuality string.
2026-02-24 02:16:15 +01:00
Jannis Braun 9b0d319ab2 refactor: per-track WebRTC diagnostic engine replacing flat stats model
Extract stats polling into useTrackStats hook with per-sender/receiver
stats via RTCRtpSender.getStats(), delta-based FPS (fixes 25fps overwrite
bug with camera+screenshare), and track-to-source matching via LiveKit
TrackPublication identity. ConnectionInfoPopover becomes a pure display
component with Network/Audio/Video sections. Remove soft-launch diagnostic
loop from useLiveKit.
2026-02-24 00:36:31 +01:00
Jannis Braun b6adf310fc fix: wire DM file attachments through the full send/fetch/broadcast chain
DM uploads silently failed because the 5-point chain (types, frontend,
POST, GET, WS broadcast) was never wired for attachments. Added
buildDmMessageWithUser/getDmMessageWithUser helpers mirroring the server
channel pattern, and plumbed attachmentIds + replyToId through all DM
code paths.
2026-02-24 00:08:45 +01:00
Jannis Braun 900eb67869 feat: add group DM support (3-10 members)
Unlock group DMs by removing the 2-member assumption across the stack.
No schema migration needed — dm_members junction table already supports
N members. Includes dedup bug fix, add/leave member endpoints, late-join
call support, group-aware sidebar rendering, and AddDmMemberModal.
2026-02-23 22:39:06 +01:00
Jannis Braun 653e59bfb2 refactor: unify backend voice signaling with VoiceRoom abstraction
Replace dual voiceStates + activeCalls maps with a single VoiceRoom
system that tracks both server channels and DM calls uniformly.

Fixes four bugs:
- voice_status silently dropped for DM call participants
- DM calls not cleaned up on WebSocket disconnect
- DM call state missing from ready payload on reconnect
- No spatial tracking of DM call participants
2026-02-23 22:15:41 +01:00
Jannis Braun 628d417723 fix: restore VoiceControls and chat split-view in DM calls
VoiceControls was gated on server-only `currentVoiceChannelId`, now also
triggers on `activeDmCall`. Added VoiceChatPanel to DM call view matching
the server voice channel pattern.
2026-02-23 21:49:06 +01:00
Jannis Braun e9ec61db43 fix: revert removeAllListeners() from destroyRoom() to stop SDK teardown errors
Closure guards on all 15 handlers already isolate React state from stale
rooms — the sledgehammer removeAllListeners() was stripping the SDK's own
internal listeners (PCManager, SignalClient), causing "Tried to add a track
for a participant that's not present" errors on rapid channel switches.
2026-02-23 21:30:07 +01:00
Jannis Braun 61e607204f fix: prevent ghost Room memory leak by stripping listeners before disconnect
LiveKit's Room.disconnect() tears down WebRTC but leaves .on() handlers
attached. Every connect() registers ~15 event handlers, which accumulate
on orphaned Room instances during rapid channel switches or HMR, causing
MaxListenersExceededWarning. Added destroyRoom() helper that calls
removeAllListeners() before disconnect() at all four teardown sites.
2026-02-23 21:13:35 +01:00
Jannis Braun bc51fc6f7e refactor: unify DM calls with server voice architecture
Merge connectDm() into connect() with isDm flag, eliminating ~145 lines
of duplicated LiveKit room setup. DM calls now inherit all event handlers
(SpeakingDetector cleanup, deafen broadcasts, metadata changes). Replace
monolithic DmCallView with shared VoiceGrid + VoiceControlBar components,
making DM calls group-DM-ready with full feature parity.
2026-02-23 20:52:58 +01:00
Jannis Braun 5e34b39b78 fix: broadcast camera & screen share status via WebSocket for sidebar visibility
Camera and LIVE badges in the channel sidebar were only visible to users
who had joined the same LiveKit room. Widen the voice_status WS event
from {isMuted, isDeafened} to {isMuted, isDeafened, isCameraOn, isScreenSharing}
so all server members see camera/screenshare indicators without joining voice.
2026-02-23 20:07:42 +01:00
Jannis Braun 77c5bda1fd fix: WebSocket heartbeat to prevent idle drops + debounce reconnect sound
30s ping/pong keepalive prevents proxy/NAT from killing idle connections.
Reconnect sound now only plays if downtime exceeds 3s, suppressing phantom
audio from brief network blips.
2026-02-23 19:45:32 +01:00
Jannis Braun edcdf8b207 fix: defer AEC toggle until after screen picker resolves to prevent mic dropout
AudioManager.setScreenShareActive(true) was firing before the browser's screen
picker, killing the mic stream. The picker suspends getUserMedia while its secure
overlay is open, trapping the mic in a dead state for 5-30s. Now the AEC rebuild
fires after the track is acquired — mic stays alive during the picker.
2026-02-23 19:27:49 +01:00
Jannis Braun fe97212c64 fix: screen share gaming stutter by switching to maintain-framerate + contentHint motion
Root cause: degradationPreference 'maintain-resolution' forced FPS drops under CPU
pressure during fast-motion gaming. Changed to 'maintain-framerate' so the encoder
drops resolution temporarily instead of stuttering. Added contentHint='motion' to
optimize for temporal prediction (more P-frames, fewer I-frames).
2026-02-23 03:46:06 +01:00
Jannis Braun 7e7de41718 fix: centralize screen share pipeline to fix 270p resolution ramp-up failure
Screen sharing was publishing at h360 (640x360) and never ramping to target
resolution due to stale closure in setTimeout, wrong initial quality anchor,
and 5 competing code paths with inconsistent bitrates.

- Create utils/screenShare.ts as single source of truth for all screen share ops
- Publish at target resolution from the start (not h360 → ramp)
- Read store at call time in timers (eliminates stale closure bug)
- Use maintain-resolution for screen content, maintain-framerate for camera
- Fix OS-level "Stop sharing" not resetting store or restoring AEC
- Enable dynacast for SFU quality signaling
- Reconcile QUALITY_MAP to canonical bitrates across all 7 files
2026-02-23 03:25:52 +01:00
Jannis Braun 82eee04946 fix: client-side speaking detection replacing LiveKit's conservative server VAD
LiveKit's server-side VAD has a high, non-configurable threshold that misses
conversational speech. Replace it with a SpeakingDetector singleton that uses
Web Audio AnalyserNodes to read actual RMS levels per participant (50ms poll,
0.008 threshold, 250ms hysteresis hold).
2026-02-23 02:53:40 +01:00
Jannis Braun 9091f08ced fix: reliable speaking indicator via direct store writes + polling safety net
Eliminate double-state architecture (useState → useEffect bridge → store)
that lost speaking events due to React 18 batching. ActiveSpeakersChanged
now writes speakingParticipantIds directly to voiceStore; 200ms poll
catches missed SDK events. Each VoiceUser subscribes to its own identity
via fine-grained selector for minimal re-renders. Also adds connection
quality indicator and ConnectionInfoPopover.
2026-02-23 02:36:39 +01:00
Jannis Braun 176f4db27e chore: remove 62 tsc emit artifacts from src/, add noEmit to tsconfig
tsc was emitting compiled .js files directly into packages/web/src/
alongside the .tsx source files because noEmit was not set. These
artifacts were never used — Vite compiles from .tsx source directly.

- Add noEmit: true to packages/web/tsconfig.json (tsc = type-check only)
- Delete all 62 orphaned .js files from src/ (-6,129 lines)
- Add packages/web/src/**/*.js to .gitignore as safeguard
2026-02-23 00:54:51 +01:00
Jannis Braun 5747267a6b refactor: remove browser NS toggle, default RNNoise on, rework voice settings
- RNNoise (AI Noise Suppression) now enabled by default for all users
- Remove redundant browser Noise Suppression toggle from VoiceControls
  and UserSettings — AudioManager handles it automatically as fallback
- Add AI Noise Suppression toggle to UserSettings panel
- Rename toggleRnnoise → setRnnoiseEnabled for clearer API
- Store migration v3→v4: enable RNNoise for existing users
- Keep Echo Cancellation and Auto Gain Control (orthogonal features)
2026-02-23 00:49:42 +01:00
Jannis Braun dc57856506 fix: RNNoise mono-left-only audio via explicit ChannelMerger stereo upmix
RNNoise worklet outputs mono (1ch) which played only in the left ear.
Replace unreliable automatic up-mixing (channelCount/channelCountMode on
inputGain) with a ChannelMergerNode that duplicates the mono signal to
both L and R channels — guaranteed stereo by the Web Audio spec.
2026-02-23 00:37:50 +01:00
Jannis Braun ccf2001047 feat: RNNoise WASM AudioWorklet for ML-based noise suppression (Phase 3)
Inject @sapphi-red/web-noise-suppressor into AudioManager's input pipeline
as a toggleable AudioWorkletNode. The worklet is loaded lazily on first
enable, then kept alive — toggling bypasses by rewiring the graph without
destroying the WASM instance. Browser NS is forced off when RNNoise is
active to avoid double-processing. InputGain forced to stereo up-mix to
prevent mono-left-only output from the worklet. Also wires Phase 2 output
device routing (setSinkId) into AppLayout/ChannelSidebar.
2026-02-23 00:28:29 +01:00
Jannis Braun dbebd38576 refactor: replace unpublishTrack mute with setMicrophoneEnabled (Phase 1 RNNoise prep)
- syncMic now uses setMicrophoneEnabled(false/true) to mute/unmute the mic
  track in-place instead of tearing down and re-publishing via unpublishTrack().
  This eliminates WebRTC renegotiation on every mute cycle and preserves the
  Web Audio pipeline for future AudioWorklet injection (RNNoise WASM).
- Unify DmCallView mute/deafen with the primary syncMic path — removed direct
  setMicrophoneEnabled calls, added missing auto-mute/unmute on deafen toggle.
- Strip redundant applyConstraints from VoiceControls noise suppression toggle
  that was hardcoding echoCancellation:true/autoGainControl:true and fighting
  AudioManager's constraint pipeline.
2026-02-22 23:17:16 +01:00
Jannis Braun 5fe43bc0c8 fix: DM close visibility flag, wire Remove Friend button, harden deploy script
- Redesign DM close as a visibility flag (closed column) instead of row deletion,
  so message broadcasts still reach users who closed a DM conversation
- Add broadcastDmMessage() helper that auto-resurfaces closed DMs when new messages arrive
- Wire Remove Friend button in DM welcome header with onClick handler and friend check
- Fix deploy.sh to cd to its own directory so rsync always runs from project root
2026-02-22 22:29:40 +01:00
Jannis Braun 6b8fd44a3a feat: real-time sync, notification & social system overhaul
- Add WS events: dm_channel_created, dm_channel_closed, friend_removed,
  channel_created/updated/deleted, server_updated
- Fix first-ever DM: broadcast dm_channel_created to recipient
- Add DELETE /api/dm/:id for closing DMs with re-open support
- Wire Close DM button in sidebar
- Fix dm_message_created for unknown channels (safety net)
- Broadcast friend_removed on friend deletion
- Sound system: track realtimeMessageEvents separately from API loads,
  play notification for messages in all channels, not just current
- Optimistic updates: send/edit/delete messages appear instantly with
  rollback on failure, temp message deduplication on WS echo
- Channel CRUD broadcasts to all server members
- Server update broadcast on PATCH
- Server join registers user in connectionManager immediately
- Reconnect: only reload current channel, preserve other channel caches
- Activity panel, right panel, member list toggle components
2026-02-22 22:15:11 +01:00
Jannis Braun 708bdf5468 fix: auto-disable AEC during screen share to prevent Chrome voice ducking
Chrome's AEC uses getDisplayMedia audio as a reference signal and
aggressively ducks the microphone even when headphones are used.
This adds a screenShareActive flag to AudioManager that forces
echoCancellation off during screen share, plus Chromium-specific
goog* constraints as belt-and-suspenders.
2026-02-20 15:22:20 +01:00
Jannis Braun 25f72aef3b fix: address sender-side voice ducking with voice processing controls
Disable AGC by default to prevent Chrome from crushing mic sensitivity
when stream audio is playing. Add user-facing toggles for echo
cancellation, noise suppression, and auto gain control. Decouple voice
and stream audio by routing through ctx.destination instead of shared
compressor. Track mic stream generation to re-publish when settings change.
2026-02-20 15:00:08 +01:00
Jannis Braun 5c38f076d3 fix: eliminate voice ducking caused by rogue LiveKit audio elements
Add MutationObserver to neutralize LiveKit's re-attached <audio> elements,
mark our keep-alive elements with data-opencord, detach tracks on unsubscribe,
remove dangerous blanket .play(), and soften compressor to transparent limiter.
2026-02-20 07:03:41 +01:00
Jannis Braun 6f777f97ce fix: local stream video visibility and audio double-playback
Show local user's own screen share in their stream tile by bypassing
the isSubscribed check for local participants and auto-watching/unwatching
local streams on publish/unpublish. Eliminate audio double-playback by
detaching LiveKit's auto-attached audio elements so GlobalAudioRenderer
is the sole playback path, making attenuation and volume controls effective.
Fix GainNode leak in useAudioTrackPlayer cleanup paths.
2026-02-20 04:47:50 +01:00
Jannis Braun 44018aa588 fix: remove stream auto-focus and persist audio across navigation
Streams no longer auto-focus into large view when they start — tiles
stay in the equal-size grid until manually clicked. Audio playback is
moved out of VoiceUser/StreamTile into a new GlobalAudioRenderer
component rendered in AppLayout so it survives channel navigation.
2026-02-20 04:27:18 +01:00
Jannis Braun 1aa40cca15 feat: implement Discord-like stream widget system with separate tiles
Streams now appear as separate tiles in the voice grid alongside the
user's camera/avatar tile, matching Discord's model. Each stream tile
has independent volume, mute, watch/unwatch controls, quality badges,
and stream attenuation that ducks audio when someone speaks.
2026-02-20 03:59:28 +01:00
Jannis Braun a8656e6a3b feat: add screen share audio support
Pass audio: true to setScreenShareEnabled so the browser offers the
"Share audio" checkbox. Track ScreenShareAudio from remote participants
and play it through a dedicated audio element with the same volume
pipeline (including boost >100%).
2026-02-20 03:16:34 +01:00
Jannis Braun a8be2dd5da fix: suppress redundant reconnect sounds and broadcasts during active voice 2026-02-20 03:07:53 +01:00
Jannis Braun 9fed8214c7 feat: replace placeholder sounds with actual MP3 recordings and refine audio logic 2026-02-20 02:39:32 +01:00
Jannis Braun 9a79ddf7eb feat: implement global audio effects and presence grace period 2026-02-19 23:46:46 +01:00
Jannis Braun 6784dbcbaa Fix screen share ghosting and improve grid view interactions 2026-02-19 23:25:36 +01:00
Jannis Braun 140c7c38ab Milestone: First fully usable version with robust audio and screen sharing 2026-02-19 23:09:03 +01:00