feat(admin): manual cleanup of stale tus upload sessions + visibility

Adds an admin-driven sweep on top of the existing 24h auto-expire so
operators can see and reap abandoned `.tus/` sessions without waiting.

- storageJanitor: extract `walkTusDir(predicate)` helper, add
  `getStaleTusInfo` + `cleanupStaleTusSessions(thresholdMs, dryRun)`;
  refactor `cleanupTusStragglers` to delegate while preserving its
  janitor-tick `{ removed }` contract.
- StorageStats gains `staleTusSessions` + `staleTusSize` (fixed 1h
  display threshold).
- New `POST /api/admin/storage/cleanup-tus` route with
  `maxAgeHours` validation (positive finite number, default 1) and
  `dryRun` support; admin-gated.
- StoragePanel: 6th overview card "Stale Uploads" + new cleanup
  subsection mirroring the media-cleanup pattern (preview-then-clean
  with shared result panel styling).
- Tests: 8 new janitor tests covering empty dir, threshold filtering,
  dry-run vs live, oldest-mtime tracking, subdir skipping, and the
  override path on the existing straggler sweep. New
  `routes/admin.test.ts` covers auth/admin gates, validation (zero,
  negative, NaN), default `maxAgeHours`, dry-run vs live unlink.
- Docs: `uploads.md` §Janitor expanded to the full lifecycle (cancel
  DELETE, discard DELETE, auto-expire, straggler sweep, admin route);
  `admin.md` Storage Management updated with the new endpoint and
  StorageStats fields.
This commit is contained in:
Jannis Braun
2026-05-02 18:44:19 +02:00
parent 4e5a440176
commit 2f0940c30b
9 changed files with 635 additions and 27 deletions
+8 -1
View File
@@ -204,8 +204,11 @@ GET /api/admin/storage/stats → StorageStats
GET /api/admin/storage/orphans → { orphans: OrphanedFile[] }
POST /api/admin/storage/cleanup { dryRun?: boolean } → CleanupResult
POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean } → CleanupResult
POST /api/admin/storage/cleanup-tus { maxAgeHours?: number = 1, dryRun?: boolean = false } → CleanupResult
```
The `cleanup-tus` route walks `.tus/`, deleting (or counting, if `dryRun`) any entry whose mtime is older than `maxAgeHours`. No DB rows are touched — `.tus/` is filesystem-only — so `deletedAttachmentRecords` in the response is always `0`. See `docs/systems/uploads.md` §Janitor for the full lifecycle (immediate-DELETE on cancel/discard, automatic 24 h `cleanupTusUploads`, 48 h defensive `cleanupTusStragglers`, and this admin-driven sweep).
**StorageStats shape:**
```typescript
{
@@ -219,10 +222,14 @@ POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean }
unlinkedSize: number;
danglingAttachments: number; // Attachment records pointing to missing files
danglingSize: number;
staleTusSessions: number; // .tus/ payload + sidecar files with mtime > 1 h old
staleTusSize: number; // Total bytes of those stale tus entries
breakdown: { type: string; count: number; size: number }[];
}
```
`staleTusSessions` / `staleTusSize` use a **fixed 1 h display threshold** (active uploads write chunks frequently; a 1 h+ gap means the user walked away). This is distinct from the `maxAgeHours` body parameter on `cleanup-tus`, which is configurable per request.
**CleanupResult shape:**
```typescript
{
@@ -234,7 +241,7 @@ POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean }
}
```
Storage functions (`getStorageStats`, `getOrphanedFiles`, `cleanupStorage`, `cleanupOldMedia`) are implemented in `utils/storageJanitor.ts`. Out of scope here -- if an uploads.md spec is created, document there.
Storage functions (`getStorageStats`, `getOrphanedFiles`, `cleanupStorage`, `cleanupOldMedia`, `cleanupStaleTusSessions`, `getStaleTusInfo`) are implemented in `utils/storageJanitor.ts`. Tus-specific lifecycle details live in `docs/systems/uploads.md` §Janitor.
**Cleanup flow (UI):**
1. Admin clicks "Preview Cleanup" -- calls `cleanupStorage(dryRun=true)` or `cleanupOldMedia(days, dryRun=true)`
+11 -5
View File
@@ -65,11 +65,16 @@ When the final PATCH completes, the hook:
### Janitor
| Function | Sweeps |
|----------|--------|
| `cleanupTusUploads()` | Invokes `@tus/file-store.deleteExpired()` (24 h `Upload-Expires`). |
| `cleanupTusStragglers()` | Defensive sweep of `.tus/` for files older than 48 h that the tus library failed to clean up. |
| `getUnlinkedAttachments()` | Existing 1 h grace still applies to abandoned post-finish attachments. |
| Trigger | Function / Path | Sweeps |
|---------|-----------------|--------|
| User cancels mid-upload | Client `tus.abort(true)` → tus DELETE | Immediate cleanup of the `.tus/` payload + sidecar. |
| User discards a paused/failed bubble | `transferStore.abortUpload` → manual `fetch DELETE` (when no live tus instance) | Immediate cleanup of the `.tus/` payload + sidecar. |
| Janitor tick (every ~30 s) | `cleanupTusUploads()` | Invokes `@tus/file-store.deleteExpired()` (24 h `Upload-Expires` default, configurable via `tusExpirationMs`). |
| Janitor tick (every ~30 s) | `cleanupTusStragglers()` | Defensive unlink of any `.tus/` entry whose mtime is older than `tusStragglerSweepMs` (48 h default) — catches orphans the tus library missed (payload without sidecar, sidecar without payload). |
| Admin-triggered | `POST /api/admin/storage/cleanup-tus``cleanupStaleTusSessions(thresholdMs, dryRun)` | Manual sweep with configurable `maxAgeHours` (default 1 h). Supports preview (`dryRun=true`) before live deletion. |
| Janitor tick (post-finalize) | `getUnlinkedAttachments()` | 1 h grace for finalized attachment rows that were never linked to a message. |
Stats: `getStorageStats()` exposes `staleTusSessions` + `staleTusSize` for the admin Storage Overview, computed via `getStaleTusInfo(60 * 60 * 1000)` — entries with mtime older than 1 h. The display threshold is fixed (matches the cleanup default); the admin route's `maxAgeHours` is what's actually configurable.
### Security
@@ -449,6 +454,7 @@ The admin routes (`routes/admin.ts`) expose the janitor functions via REST:
| `GET /api/admin/storage/orphans` | GET | `getOrphanedFiles()` |
| `POST /api/admin/storage/cleanup` | POST | `cleanupStorage(dryRun)` |
| `POST /api/admin/storage/cleanup-media` | POST | `cleanupOldMedia(maxAgeDays, dryRun)` |
| `POST /api/admin/storage/cleanup-tus` | POST | `cleanupStaleTusSessions(maxAgeHours * 3600 * 1000, dryRun)` |
All require JWT + admin role. See `docs/systems/api.md` for request/response formats.