backspace/docs/systems/message-list.md

Message List & Auto-Scroll

The chat message list (packages/web/src/components/chat/MessageList.tsx) is responsible for rendering the message stream, auto-scrolling to follow new content, restoring per-channel scroll positions during a session, dispatching jump-to-message scrolls from search, and triggering load-more pagination at the top.

Files

File	Responsibility
packages/web/src/components/chat/MessageList.tsx	Render the message stream and own all scroll behavior.
packages/web/src/stores/chatStore.ts	messages, hasMore, scrollPositions (in-memory, session-scoped).
packages/web/src/components/chat/embeds/*.tsx	Embed renderers — must obey the dimension reservation contract below.
packages/web/src/components/chat/AttachmentRenderer.tsx	Reference for the dimension reservation pattern (AttachmentRenderer.tsx:81-100).

Bottom clearance — --composer-clearance

The MessageList content's paddingBottom is dynamic — it reads the CSS variable var(--composer-clearance, 80px) written to the chat region's wrapper element by MessageInput via a ResizeObserver. The variable's value is composer.height + composer.bottom-offset + 12 px so the last message always lands 12 px above the composer's top edge regardless of (a) composer height (reply banner, staged-attachment tile row, multi-line autosize), (b) the composer's own bottom style (desktop 12 px / mobile-keyboard-closed env(safe-area-inset-bottom) + 6 / mobile-keyboard-open 0).

The 80 px fallback is sized for the desktop case during the brief mount window before the first ResizeObserver tick — it matches the previous static pb-20. See docs/systems/mobile-ui.md "Floating Composer" for the full rationale; the short version is that any static value is wrong on iPhone (composer height + safe-area + 6 already exceeds 80 px) and wrong when the composer grows (replies, attachments).

ContainerRef invariant

containerRef.current (and contentRef.current) MUST be non-null for the entire lifetime of any chat view where messages may arrive or scroll-affecting effects can run. Every effect in this file (initial snap A, ResizeObserver B, capture-phase load handler C, scrollend final-pin D, jump-to-message) reads these refs and bails on a null guard. Each of those effects is keyed on [messages.length, channelId] or [hasMessages, channelId], and the inbound transition (0 → N / false → true) is the only re-fire signal during a channel-load lifecycle. If a ref is null at the moment that signal fires, the effect bails — and no later dep change will retry it, leaving the channel permanently broken (initial scroll never lands at bottom, ResizeObserver never observes, scrollend never registers, saved-anchor restore never happens).

Implication for loading UI: the initial-load skeleton is rendered as an absolutely-positioned overlay on top of the scroll container, NOT as an early return that replaces it. See MessageList.tsx (the showInitialSkeleton overlay block) for the canonical pattern. Any future loading/empty/error UI added to this component must follow the same rule — overlay, never replace — or the scroll model breaks under slow loads where the skeleton is visible at the moment messages arrive.

Top-of-list reservation slot

When hasMore === true for the current channel, a constant-height div is rendered as the first child of the scroll container, above the messages container that holds contentRef. This slot exists for the entire hasMore === true lifetime of the channel view and its height NEVER changes. The pagination skeleton's grey-bar contents toggle inside the slot based on showPaginationSkeleton; the slot itself does not mount or unmount with the skeleton. This is the entire mechanism that eliminates the layout-shift bug where the skeleton's appearance pushed message content down and its disappearance snapped it back up — by keeping slot height invariant, no DOM height ever changes from the skeleton's lifecycle.

The constant PAGINATION_SLOT_HEIGHT_PX (defined at module scope in MessageList.tsx) is the single source of truth for three values: the slot's inline height style, the skeleton's structural total height (the markup is tuned so it fits comfortably inside the constant — the last row's mb-5 is stripped to give a buffer), and the load-more trigger threshold (scrollTop < PAGINATION_SLOT_HEIGHT_PX + 50). It is a TS constant rather than a CSS variable to avoid a getComputedStyle round-trip when the threshold computation reads it back. If the skeleton markup changes (different row count, different padding, different bar height), re-measure getBoundingClientRect().height in DevTools and update the constant — round UP to the nearest 4-pixel step.

Slot ↔ WelcomeHeader transition. The slot and WelcomeHeader are two separate components and are mutually exclusive: slot when hasMore, WelcomeHeader when !hasMore. On the final page load that flips hasMore to false, the slot unmounts and WelcomeHeader mounts in its tree position. The (WelcomeHeader_height − slot_height) delta is automatically absorbed by the prepend math, which sees the total change in container.scrollHeight and compensates scrollTop accordingly — no special transition handling required. Keeping the two as separate components is a deliberate clarity choice: each component is unaware of the other and the formula handles the join.

Why not overflow-anchor. CSS scroll-anchoring (overflow-anchor: auto, the default in modern browsers) is not the mechanism in use here. It cannot help: the browser's anchor-pick algorithm requires a stable element ABOVE the changing content in the viewport, and at scrollTop ≈ 0 the slot is the topmost in-viewport element AND the one whose contents change — no stable anchor to pick. Additionally, iOS Safari (until 17) had no support, and historical edge cases interact with rubber-band overscroll at the top. Manual scroll-position compensation in the load-completion rAF is the reliable path.

Auto-scroll model

Three effects cooperate. Their ordering is established by the 2026-03-25 race-fix and the 2026-04-25 sentinel addendum.

Effect A — initial snap / restore. Runs once when messages.length transitions from 0 to N for a channel. Reads chatStore.scrollPositions.get(channelId). If a saved anchor exists, scrolls that message into view and computes the resulting isAtBottomRef from actual distance. Otherwise, sets container.scrollTop = container.scrollHeight, captures the post-clamp value into lastProgrammaticBottomScrollRef, and sets isAtBottomRef.current = true. On subsequent message arrivals (messages.length > prev), if isAtBottomRef.current, sets the typed smooth-scroll intent ('bottom') and smooth-scrolls via bottomRef.scrollIntoView({ behavior: 'smooth' }). The smooth animation lands asynchronously across many frames; the intent ref keeps the at-bottom gate open during that window so late-loading media can re-pin (see "Smooth-scroll intent" below), and Effect D delivers a final defensive instant pin when the animation completes.

Effect B — ResizeObserver. Observes the message-list content container. When height grows and isAtBottomRef.current === true, re-pins to bottom and updates the sentinel. Gated on isAtBottomRef.current so it cannot interfere when the user has scrolled away.

Effect C — capture-phase load listener. Catches image/iframe load completions that ResizeObserver suppresses due to its layout-loop limit. Same gate, same re-pin, same sentinel update.

Effect D — scrollend listener (final defensive pin). Native scrollend event (Chrome 114+, Safari 18+) fires once when a smooth scroll's animation completes. When smoothScrollIntentRef.current === 'bottom' at that moment, performs an instant container.scrollTop = container.scrollHeight, refreshes the sentinel, sets isAtBottomRef = true, and clears the intent. This is the catch-all for layout that grew during the smooth animation but after the animation's terminal target was computed. For browsers without scrollend, beginSmoothScrollIntent arms a setTimeout(800ms) fallback instead — exactly one of the two paths fires per intent. If the user has wheeled away mid-animation past the 5000px threshold (SMOOTH_SCROLL_USER_INTENT_THRESHOLD), the final pin is skipped (we honor the user's gesture).

handleScroll. Runs on every scroll event. First check: if container.scrollTop === lastProgrammaticBottomScrollRef.current, the event was queued by our own command — re-affirm the at-bottom flags, re-pin defensively (layout may have grown since the command), update the sentinel, and return early. Otherwise, invalidate the sentinel immediately (a non-matching event means the user has moved away from the position we last commanded; leaving the stale value live would let a coincidental future scroll-through of the same scrollTop falsely match and yank the user to bottom). Then check the smooth-scroll intent: if intent === 'bottom', the deadline hasn't elapsed, AND the user hasn't wheeled away past the 5000px threshold, suppress the at-bottom flip — keep isAtBottomRef = true so Effects B/C stay open. Otherwise (no intent, expired intent, intent === 'message', or user wheeled away), recompute distanceFromBottom, update isAtBottomRef and isNearBottomRef honestly, track visibleMsgIdRef (for position memory), and trigger loadMoreMessages when scrolled near the top. isNearBottomRef is always updated honestly even during suppression — only the at-bottom gate is held open, never the Jump-to-Present visibility.

Invariant: isAtBottomRef flips from true to false only when (a) the user genuinely scrolls away outside any active smooth-scroll-to-bottom intent, OR (b) a smooth scroll with intent === 'message' legitimately moves the user away from bottom. Layout growth, our own programmatic scrolls, queued scroll events from those programmatic scrolls, and intermediate frames of a smooth-scroll-to-bottom animation do not flip it.

Smooth-scroll intent

Bottom-bound smooth scrolls (new-message arrival in Effect A, Jump-to-Present click) and jump-to-message smooth scrolls (search result click — animates to a non-bottom target) both run scrollIntoView({behavior:'smooth'}), which animates scrollTop over many frames. Each intermediate frame fires handleScroll with a measured distanceFromBottom that does not match the smooth animation's terminal frame. Without intent tracking, those intermediate measurements would flip isAtBottomRef to false, closing the Effect B/C gates so any media (avatars, embeds, attachment images, Spotify thumbs) that finishes loading mid-animation grows scrollHeight while the gate is closed — the smooth scroll then lands at the originally computed (now stale) target, leaving the user above the true bottom.

The fix is a typed intent ref:

Field	Type	Set by
smoothScrollIntentRef	'bottom' | 'message' | null	beginSmoothScrollIntent(intent, label)
smoothScrollDeadlineRef	number (performance.now() ms)	beginSmoothScrollIntent (now + 800)

Behavior by intent:

• 'bottom': handleScroll suppresses the at-bottom flip while the deadline hasn't elapsed and the user hasn't wheeled away past 5000px (SMOOTH_SCROLL_USER_INTENT_THRESHOLD). Effect D fires the final defensive pin via scrollend (or its timeout fallback). Set by: new-message smooth scroll in Effect A, Jump-to-Present onClick.
• 'message': NO suppression — the jump-to-message animation legitimately moves the user away from bottom and isAtBottomRef should flip honestly. Effect D clears the intent at scrollend (no defensive pin). Set by: scrollToMessage in the jump-to-message effect.

The 5000px user-intent threshold matches the nearBottom band: distances larger than that signal a deliberate user gesture (mouse-wheel away mid-animation), and we let the gate flip honestly so the smooth scroll's terminal frames don't fight the user.

Position memory

• Storage: chatStore.scrollPositions: Map<channelId, messageId> — in-memory Zustand state.
• Tracking: handleScroll updates visibleMsgIdRef.current to the topmost visible message in real time on every scroll event (when not near bottom).
• Commit: the channel-change effect commits visibleMsgIdRef.current to chatStore.scrollPositions for the outgoing channel before resetting state for the incoming one. If the user was at the bottom (visibleMsgIdRef.current === null), the entry is removed so the next visit snaps to bottom.
• Lifetime: session-only. Lost on reload, app restart, tab close. This is a deliberate design choice — restoring a scroll position from a previous session would be disorienting and would silently swallow new messages the user hasn't seen.
• Eviction: scroll positions are evicted alongside messages when MAX_CACHED_CHANNELS (chatStore.ts:10) is exceeded.

Embed renderer contract

Every renderer that contains an image, iframe, or video MUST reserve dimensions when they are known. The reference pattern is AttachmentRenderer.tsx:81-100. See docs/systems/embeds.md for the bidirectional server-and-client contract.

When dimensions are not known (probe failed, no OG tags, non-image type), the renderer must rely on a structurally fixed layout (iframe with hardcoded height, fixed-size thumbnail) rather than a dimension-fallback wrapper. Fallback wrappers using a default aspect-ratio (e.g. 4/3) cause visible letterbox bars on content whose true ratio differs — that exact failure caused the revert in commit 0c84029. The sentinel and Effects B/C absorb residual layout shift from un-reserved content.

Renderers known to satisfy the contract today:

• AttachmentRenderer.tsx — reserves from attachment.width/height.
• embeds/ImageEmbed.tsx — reserves from embed.width/height when populated.
• embeds/VideoEmbed.tsx — fixed 16:9 reservation for both branches: aspectRatio: '16/9' for the direct-video container, paddingBottom: 56.25% for the provider-iframe container.
• embeds/RichEmbed.tsx — explicit height from getIframeHeight(); fixed 80×80 thumbnail in collapsed state.
• embeds/GenericEmbed.tsx — fixed 80×80 thumbnail.

Renderers that do not reserve (residual shift, sentinel-covered):

• Bare GIF URLs in Message.tsx — Tenor/Klipy URLs are not embed records, no dims.
• Markdown inline images in MarkdownRenderer.tsx — ![](url) syntax carries no dims.

Known limitations

• Bare GIFs and markdown images shift on load. The sentinel keeps the auto-scroll system from being disabled by their shifts; ResizeObserver/load handlers re-pin to bottom while the user is at the bottom.
• The 150px at-bottom tolerance is generous — sending a new message while the user is reading the last few messages 100px up from the bottom yanks them down. This is intentional today; if changed, update this doc and the spec history.
• The smooth-scroll UX is preserved deliberately for both new-message arrival and Jump-to-Present per UX call. The 2026-04-27 fix (smooth-scroll intent + scrollend final pin) closes the residual above-bottom-landing race without removing the animation.

Out of scope (deferred)

These items were considered and rejected for the 2026-04-25 work; they live here so their failure modes are findable when they actually occur:

• Persisting scrollPositions across reloads. Rejected — would silently hide new messages and confuse users returning to old context.
• Explicit { atBottom: true } flag in scrollPositions. The current "absence of entry = at bottom" coupling is structurally fragile (LRU eviction silently turns "scrolled up" state into "at bottom"), but MAX_CACHED_CHANNELS = 20 makes this rare in practice and there is no verified user-visible failure today.
• Save-on-unmount of the visible-message anchor. The current commit-on-channel-switch path doesn't fire if the component unmounts via route change or tab close. No verified user-visible failure today.
• Tightening the 150px at-bottom tolerance. Separate UX decision, not driven by any current bug.

History

• 2026-03-25 — chat-scroll-race-fix spec: removed isAtBottom from Effect A's deps, gated Effects B and C on isAtBottomRef, set the ref after initial snap. Shipped.
• 2026-03-25 — embed-dimension-reservation spec: server probes image embeds for dimensions; client renderers reserve via aspect-ratio. Server side shipped. Client side partly shipped, then reverted in 0c84029 because the 4/3 fallback caused dark letterbox bars.
• 2026-04-25 — message-list-scroll-completion-and-addendum spec: restored the known-dimension-only branch of the client reservation in ImageEmbed.tsx; added the lastProgrammaticBottomScrollRef sentinel to close the residual handleScroll race; created this file.
• 2026-04-27 — smooth-scroll intent + scrollend final pin: typed smoothScrollIntentRef ('bottom' | 'message' | null) with an 800ms deadline. handleScroll suppresses the at-bottom flip while a 'bottom' intent is active and the user hasn't wheeled past the 5000px threshold; the gate stays open so Effects B/C re-pin to bottom as media loads mid-animation. Effect D fires a final defensive instant pin via the native scrollend event (Chrome 114+, Safari 18+) or a setTimeout(800) fallback. 'message' intent (jump-to-message from search) does NOT suppress — the gate flips honestly so the user is left at the targeted message. Closes the "Jump to Present doesn't fully reach the bottom" residue reproducible on nova.ddns.net Orbit → general. UX call: smooth-scroll animation preserved, not replaced with an instant jump.
• 2026-05-05 — skeleton-as-overlay fix: the initial-load skeleton was rendered as an early return that replaced the JSX containing containerRef / contentRef. On slow loads (mobile hotspot, throttled connection), useDelayedLoading's 200ms threshold flipped showInitialSkeleton=true before messages arrived, unmounting the scroll container. When messages then arrived (messages.length 0→N), Effect A's rAF ran against containerRef.current === null and bailed at the null guard at the top of its callback. Effects B/C/D and the scrollend listener — all keyed on [hasMessages, channelId] — re-fired exactly once when hasMessages flipped false→true (which happened while the skeleton was still up due to useDelayedLoading.minDisplay's 300ms enforcement), hit the same null guard, and never re-attached because no dep changed when the skeleton finally cleared. Net effect: chat opened scrolled to the top instead of the bottom; ResizeObserver never observed the content; scrollend never registered; saved-anchor restore was equally broken. Fix: render the skeleton as an absolute inset-0 z-10 bg-surface-chat pointer-events-none overlay alongside the (always-mounted) scroll container, so all refs stay live across the loading transition. Added the "ContainerRef invariant" section above to encode the constraint for future loading-UI additions. Manual repro recipe (regression check): DevTools → Network tab → throttle to "Slow 3G" → click into a channel that hasn't been opened this session → pre-fix: list lands scrolled to the top; post-fix: list lands at the bottom. Saved-anchor variant: scroll up in a channel, switch away, return under throttle — pre-fix: lands at top; post-fix: lands at the saved anchor.
• 2026-04-27 — pagination cross-channel race fix: handleScroll's load-more block (scrollTop < 50 && hasMore) leaked isLoadingMore = true and the prevScrollHeight value across channel switches that raced the async loadMoreMessages await. The useState slot is the same component instance across channel changes, so the new channel inherited the flag (phantom pagination skeleton) and the post-await requestAnimationFrame applied the outgoing channel's prevScrollHeight to the incoming channel's container DOM (wrong-position scroll). Fix: capture channelId into a requestChannelId local at the start of the load block, mirror the live channelId prop into currentChannelIdRef synchronously each render, and compare twice — once before scheduling the rAF and once inside the rAF callback (the ~16ms gap between scheduling and firing is enough time for a click to switch channels). Wrap the await in try/finally so setIsLoadingMore(false) always runs even on throw. Belt-and-suspenders: the channel-switch effect (Effect 3) also calls setIsLoadingMore(false) so a never-resolving await (network hang) cannot strand the flag on the new channel. The store's currentChannelId was rejected as the live source — it lags one render behind a URL-driven channel switch (set in an AppLayout effect that fires after MessageList renders with the new prop), which would let the guard mis-fire during that one-frame window. Request cancellation was deliberately deferred — the channelId guard already silently drops stale results, and AbortController plumbing through chatStore.loadMoreMessages is a larger refactor; network waste on an abandoned page is not a correctness issue.
• 2026-05-05 — sticky pagination skeleton on federated channels (parallel loadMessages/loadMoreMessages dedup): the 2026-05-05 hook + clamp-scroll fix above did not cover the federated-LAN reproduction. Playwright trace against the live deployment showed every channel mount firing TWO parallel GET /channels/:id/messages?limit=50 requests, and every fast-scroll burst firing N parallel loadMoreMessages requests. Two cooperating root causes: (a) AppLayout.tsx:289 AND MessageList.tsx:267 both call loadMessages(channelId) in their channel-mount effects — MessageList's call is required for surfaces that don't have the AppLayout chrome (VoiceChatPanel), so neither can be removed unilaterally; (b) handleScroll's load-more block sees !isLoadingMore as true in every scroll event of a fast burst because all events share one closure created before the first setIsLoadingMore(true) commits. The chatStore guard if (!force && get().hasMore.has(channelId)) return; only deduplicates calls after the first completes — it does not collapse parallel callers. On a NAT hairpin'd LAN where some federated TCP connections succeed and others hang, having N hung parallel fetches per channel mount keeps chatStore.isLoading and component-level isLoadingMore true while any one of them is still waiting on the 30 s api-client timeout, producing the visually-stuck pagination skeleton across channel switches. Fix: in-flight Promise dedup at the chatStore level. loadMessages and loadMoreMessages each consult a module-level Map<channelId, Promise>; concurrent callers receive the same Promise. The map entry is cleared in a finally block, with a inFlightLoads.get(channelId) === promise self-check so a force-reload (WS reconnect) scheduled mid-flight isn't dropped. force=true callers bypass dedup intentionally — WS reconnect needs a fresh fetch even if a stale one is still pending. Net effect: one federated fetch per channel mount, one per fast-scroll burst, predictable timeout window, skeleton clears within ≤ 30 s + minDisplay even on full hairpin failure.
• 2026-05-05 — sticky pagination skeleton (clamp-scroll trigger + minDisplay deadline drift): two cooperating bugs produced a "ghost" pagination skeleton that stuck at the top of the chat across channel switches, sometimes until reload. (1) When the user switched from a tall channel to a shorter one, the browser clamped scrollTop and dispatched a synthetic scroll event after Effect 3 ran. handleScroll saw scrollTop < 50 && hasMore && !isLoadingMore — for any channel where hasMore defaulted to true (unvisited or LRU-evicted; see MessageList.tsx hasMore selector with the ?? true fallback) the load-more block fired even though the user never scrolled. (2) useDelayedLoading's threshold-timer callback unconditionally wrote displayStartRef.current = Date.now(), including when show was already true. With isLoading cycling true→false→true at periods near threshold (200 ms), each re-fire refreshed the deadline and the next false rescheduled minDisplay from a fresh start — extending the visible skeleton by up to one full minDisplay window per cycle. Together: the clamp event re-armed isLoadingMore against an empty-bail loadMoreMessages call, the hook over-extended the visible window, and rapid channel switches kept feeding the cycle. Fix: (a) useDelayedLoading.ts — only write displayStartRef.current on the false→true show transition, using setShow's functional form to avoid stale closures. (b) MessageList.tsx — suppressNextLoadMoreRef is armed in Effect 3 on every channel change and consumed by the load-more block on the first scroll event after the switch. A 250 ms setTimeout fallback disarms the flag if no clamp event fires (new channel's content fit without clamping), so a real user scroll-to-top isn't permanently suppressed. Suppression is scoped to the load-more block only — isAtBottomRef/isNearBottomRef updates and visible-message tracking still run because the clamped scroll genuinely changes scroll position. Regression test: useDelayedLoading.test.ts (REGRESSION: minDisplay deadline measured from FIRST show, not refreshed by re-fired threshold).
• 2026-05-15 — pagination skeleton layout-shift elimination: the pagination loading skeleton was rendered as a transient inline child at the top of the scroll content (MessageList.tsx:652-664). Each mount pushed message content down by the skeleton's height; each unmount snapped it back up. The existing c.scrollTop = c.scrollHeight - prevScrollHeight compensation at line 618 only ran at load completion and only against the message prepend — it never saw the skeleton's own mount/unmount as an event to compensate. Fix: render a constant-height slot above the messages whenever hasMore === true (defined by the PAGINATION_SLOT_HEIGHT_PX constant; see "Top-of-list reservation slot" above). The skeleton's grey-bar contents toggle inside the slot; slot height never changes from the skeleton's lifecycle. Companion changes: (a) raise the load-more trigger from scrollTop < 50 to scrollTop < PAGINATION_SLOT_HEIGHT_PX + 50 so the load fires while the slot is still off-screen — the skeleton's appearance overlaps with the user still off-slot — with a scrollTop >= 0 guard added to neutralize iOS Safari rubber-band overscroll producing briefly-negative scrollTop values that would otherwise satisfy the upper bound and fire a spurious load during a rubber-band gesture; (b) correct the prepend math from c.scrollTop = c.scrollHeight - prevScrollHeight (an "anchor from bottom" formula that only holds when prevScrollTop ≈ 0) to c.scrollTop = prevScrollTop + (c.scrollHeight - prevScrollHeight), which is correct for any prior scroll position. Without the math correction the raised threshold would cause a jump of up to ~PAGINATION_SLOT_HEIGHT_PX + 50 on every load. The corrected formula also automatically absorbs the (WelcomeHeader_height − slot_height) delta on the final page (slot unmounts, WelcomeHeader mounts), since both pieces are visible in c.scrollHeight - prevScrollHeight. (c) Lower useDelayedLoading's threshold to 50ms for the pagination case (initial-load skeleton keeps the default 200ms) — with no layout shift to flicker, the original purpose of the 200ms delay (avoid showing skeleton on near-instant cache hits) survives but the value was over-tuned for the new mechanic. Manual repro recipe (regression check): DevTools → Network → Slow 3G → open a channel with deep history → scroll to top. Pre-fix: content visibly pushed down when skeleton appears, snaps back up when skeleton leaves. Post-fix: skeleton fades into pre-reserved slot, no motion of content below; new messages appear above previously-visible content without shifting it.