backspace/docs/systems/search.md

Search System

Source files:

• packages/server/src/routes/search.ts — Server-side search endpoints (channel search, DM search, messages-around)
• packages/web/src/components/chat/SearchPopover.tsx — Client-side search UI (filter bar, result rendering, jump-to-message)
• packages/web/src/api/client.ts — API client search namespace and messagesAround methods
• packages/web/src/stores/chatStore.ts — loadMessagesAround() store action
• packages/web/src/components/chat/MessageList.tsx — Jump-to-message scroll + highlight logic
• packages/web/src/components/layout/MainContent.tsx — Search button + popover wiring
• packages/web/src/styles/globals.css — .search-highlight animation

---

Endpoints

Four endpoints, all requiring JWT authentication (preHandler: authenticate).

Endpoint	Auth Check	Response Shape
GET /api/channels/:id/search	VIEW_CHANNEL + READ_MESSAGE_HISTORY via hasPermission()	{ results: MessageWithUser[], totalCount: number }
GET /api/dm/:id/search	isDmMember()	{ results: DmMessageWithUser[], totalCount: number }
GET /api/channels/:id/messages/around	VIEW_CHANNEL + READ_MESSAGE_HISTORY via hasPermission()	MessageWithUser[] (flat array)
GET /api/dm/:id/messages/around	isDmMember()	DmMessageWithUser[] (flat array)

For full endpoint signatures, see api.md under "Search".

---

Filter Syntax

All filters are query string parameters on the search endpoints.

Parameter	Type	Description	SQL Behavior
q	string	Text query (trimmed)	LIKE '%{q}%' on content column — case-insensitive in SQLite by default for ASCII
from	string	Username filter	Exact LIKE match on users.username (not partial — no wildcards added). If no user found, returns empty results immediately (not 404).
has	file | image | link	Attachment/content filter	See "has: filter" section below
before	string	ISO 8601 date	createdAt < new Date(before).getTime() — parsed via new Date(), invalid dates silently ignored
after	string	ISO 8601 date	createdAt > new Date(after).getTime() — parsed via new Date(), invalid dates silently ignored
offset	number	Pagination offset	Math.max(Number(offset) || 0, 0) — floored to 0
limit	number	Page size	Clamped: Math.min(Math.max(Number(limit) || 25, 1), 50) — default 25, max 50

has: Filter Implementation

The has filter uses two different mechanisms depending on the value:

Value	Mechanism	SQL
file	EXISTS subquery on attachments table	EXISTS (SELECT 1 FROM attachments WHERE attachments.message_id = messages.id)
image	EXISTS subquery with mimetype filter	EXISTS (SELECT 1 FROM attachments WHERE attachments.message_id = messages.id AND attachments.mimetype LIKE 'image/%')
link	LIKE on content column	content LIKE '%http%' — appended to the WHERE conditions (not a subquery)

For DM search, the subquery joins on attachments.dm_message_id = dm_messages.id instead.

Important: The has: file/has: image filter uses a raw SQL EXISTS subquery (hasFilter) that is combined with the main whereClause using and(). This filter is applied separately from the main conditions array because Drizzle ORM conditions and raw SQL fragments are combined at query time.

---

Pagination

• Offset-based: Uses offset + limit query params
• Default page size: 25
• Max page size: 50
• Total count: Returned as totalCount in every search response (separate COUNT query)
• Sort order: Results ordered by createdAt DESC (newest first)

---

Result Hydration

Both search endpoints follow the same hydration pipeline after fetching raw message rows:

1. Fetch message rows (filtered, paginated)
2. Batch-fetch users         → userMap (userId → user)
3. Batch-fetch attachments   → attachmentMap (messageId → attachments[])
4. Batch-fetch reactions     → fetchReactionsForMessages() / fetchDmReactionsForMessages()
5. Batch-fetch embeds        → fetchEmbedsForMessages() / fetchDmEmbedsForMessages()
6. Batch-fetch reply parents → fetchReplyToMessages() / inline DM reply fetch
7. Assemble via buildMessageWithUser() / buildDmMessageWithUser()
8. Filter out messages with missing users (null check)

Channel Search Hydration

• Users: Batch SELECT from users with inArray(users.id, userIds)
• Attachments: Batch SELECT from attachments with inArray(attachments.messageId, messageIds)
• Reactions: fetchReactionsForMessages(messageIds) — from routes/messages.ts
• Embeds: fetchEmbedsForMessages(messageIds) — from utils/embedResolver.ts
• Replies: fetchReplyToMessages(messageRows) — from routes/messages.ts
• Assembly: buildMessageWithUser() — from routes/messages.ts

DM Search Hydration

• Users: Same batch pattern
• Attachments: Batch SELECT from attachments with inArray(attachments.dmMessageId, messageIds)
• Reactions: fetchDmReactionsForMessages(messageIds) — from routes/dm.ts
• Embeds: fetchDmEmbedsForMessages(messageIds) — from utils/embedResolver.ts
• Replies: Inline implementation — fetches dmMessages by replyToId, builds minimal DmMessageWithUser (with empty attachments, embeds, reactions arrays)
• Assembly: buildDmMessageWithUser() — from routes/dm.ts

Response Types

Both types are defined in packages/shared/src/types.ts. See database.md for underlying table schemas.

MessageWithUser extends Message with: user: User, attachments: Attachment[], embeds: Embed[], reactions: Reaction[], replyTo?: MessageWithUser | null

DmMessageWithUser extends DmMessage with: user: User, attachments: Attachment[], embeds: Embed[], reactions: Reaction[], replyTo?: DmMessageWithUser | null

---

Messages-Around Endpoint

Used for jump-to-message navigation (from search results and deep links). Loads a window of messages centered on a target message.

Parameters

Parameter	Type	Required	Description
messageId	string	Yes	Target message ID (returns 400 if missing)
limit	number	No	Window size, default 50, max 100, clamped to [1, 100]

Algorithm

1. Validate target message exists in the channel (404 if not found)
2. half = floor(limit / 2)
3. Fetch "before" rows: messages with id <= messageId, ordered DESC, limit half+1 (includes target)
4. Fetch "after" rows:  messages with id > messageId, ordered ASC, limit half
5. Reverse beforeRows to chronological order
6. Concatenate: [...beforeRows, ...afterRows]
7. Deduplicate by id (target may appear in both sets)
8. Hydrate with same pipeline as search results
9. Return flat MessageWithUser[] / DmMessageWithUser[] array

Note: The "before" query uses id <= messageId (not timestamp-based), and the "after" query uses id > messageId. This means the pivot is on Snowflake ID ordering, not createdAt. The ORDER BY clause still uses createdAt, which works because Snowflake IDs are monotonically increasing and correlate with creation time.

Channel vs DM Differences

• Channel: Checks getChannelSpaceId() + hasPermission() with VIEW_CHANNEL | READ_MESSAGE_HISTORY
• DM: Checks isDmMember()
• Channel: Queries messages table, uses fetchReactionsForMessages, fetchEmbedsForMessages, fetchReplyToMessages
• DM: Queries dm_messages table, uses fetchDmReactionsForMessages, fetchDmEmbedsForMessages, inline reply fetch

---

Client-Side: SearchPopover

packages/web/src/components/chat/SearchPopover.tsx

Component Props

interface SearchPopoverProps {
  open: boolean;
  onClose: () => void;
  anchorRef: React.RefObject<HTMLElement | null>;
  channelId: string;
  isDm: boolean;
  onJumpToMessage: (messageId: string) => void;
}

State

State Variable	Type	Purpose
query	string	Text search input
fromFilter	string	Username filter input
hasFilter	string	'' | 'file' | 'image' | 'link' — select dropdown
beforeFilter	string	Date string from <input type="date">
afterFilter	string	Date string from <input type="date">
showFilters	boolean	Filter panel visibility toggle
results	AnyMessage[]	Accumulated search results (MessageWithUser | DmMessageWithUser)
totalCount	number	Total matching results (from server)
isSearching	boolean	Loading state
offset	number	Current pagination offset

Behavior

1. Reset on open/channel change: All state resets when open or channelId changes
2. Debounced search: 300ms debounce on any query/filter change, calls doSearch(0)
3. Empty guard: Search requires at least one of: query, fromFilter, hasFilter, beforeFilter, afterFilter — otherwise clears results
4. Pagination: "Load more" button appends next page. Offset tracked as searchOffset + data.results.length
5. Federation-aware: Uses getChannelOrigin(channelId) and getApiForOrigin(origin) to route API calls to the correct instance
6. Dismiss: Click-outside (mousedown listener) or Escape key closes the popover
7. Auto-focus: Input focused 50ms after popover opens

API Call Flow

SearchPopover.doSearch(offset)
  → getChannelOrigin(channelId)       // resolve federation origin
  → getApiForOrigin(origin)           // get API client for that origin
  → isDm ? client.search.dm(channelId, params)
         : client.search.channel(channelId, params)
  → params: { q, from, has, before, after, offset, limit: 25 }

The client always sends limit: 25 (hardcoded in SearchPopover).

UI Layout

• Container: 420px wide, max 500px tall, glass material, positioned via useFloatingPosition (bottom placement, 8px offset)
• Search input: input-embedded tier with search icon and clear button
• Filter toggle: Collapsed by default, shows active-filter indicator dot when any filter is set
• Filter panel: 2-column grid — From (text), Has (select), Before (date), After (date) — all input-search tier
• Results list: Scrollable area with result count header
• Result items: Avatar + display name + timestamp + content snippet (2-line clamp) + attachment count
• Query highlighting: highlightMatch() wraps matches in <mark> tags with bg-accent-primary/30 styling
• Load more: Shows remaining count, disabled while loading

Result Rendering

Each result shows:

• User avatar (via <Avatar> component)
• Display name (falls back to username, then "Unknown")
• Timestamp via formatTime(): "Today at HH:MM", "Yesterday at HH:MM", or "MM/DD/YYYY HH:MM"
• Content with query term highlighting (case-insensitive regex split)
• Attachment count badge (paperclip icon) if msg.attachments.length > 0

---

Client-Side: Jump-to-Message

The jump-to-message flow spans three components.

Flow

1. User clicks search result in SearchPopover
     → onJumpToMessage(messageId) callback fires
     → MainContent: setJumpToMessageId(id), setSearchOpen(false)

2. MessageList receives jumpToMessageId prop
     → Check if message element exists in DOM: document.getElementById(`msg-${jumpToMessageId}`)
     → If found: scroll + highlight immediately
     → If not found: call loadMessagesAround(channelId, messageId)
         → chatStore.loadMessagesAround() replaces the channel's message cache entirely
         → After React render (double requestAnimationFrame), scroll + highlight

3. Scroll + Highlight:
     → el.scrollIntoView({ behavior: 'smooth', block: 'center' })
     → el.classList.add('search-highlight')
     → setTimeout 2000ms → el.classList.remove('search-highlight')
     → onJumpComplete() → resets jumpToMessageId to null

loadMessagesAround (chatStore)

search.ts:chatStore.loadMessagesAround(channelId, messageId):

• Routes to client.channels.messagesAround() or client.dm.messagesAround() based on isDmChannel()
• Normalizes remote asset URLs for federated channels
• Replaces the entire message cache for that channel (not append/prepend)
• Sets hasMore to true (enables upward scroll loading from the new position)
• Updates channelAccessTimes

search-highlight CSS

Defined in globals.css:

@keyframes search-flash {
  0% { background-color: rgba(124, 108, 246, 0.2); }
  100% { background-color: transparent; }
}
.search-highlight { animation: search-flash 2s ease-out; }

Purple flash (accent color at 20% opacity) that fades to transparent over 2 seconds.

---

Known Limitations

1. SQL LIKE for text search: Uses LIKE '%query%' — no full-text indexing (FTS5), no relevance ranking, no word boundary matching. Performance degrades linearly with message count.
2. from: filter is exact match: Uses LIKE without wildcards on username, but SQLite LIKE is case-insensitive for ASCII by default. Does not search display names.
3. has:link is content-based: Searches for '%http%' in message content — does not check the embeds table. May miss non-HTTP links or match false positives (e.g., a message containing the word "http" in prose).
4. No cross-channel search: Each search is scoped to a single channel or DM. There is no global/space-wide search endpoint.
5. DM reply hydration is minimal: Reply-to messages in DM search results have empty attachments, embeds, and reactions arrays (unlike channel search which uses fetchReplyToMessages with full attachment hydration).
6. Offset pagination: Uses offset/limit (not cursor-based). Large offsets may have performance implications on big result sets since SQLite must scan and skip rows.