feat(admin): manual cleanup of stale tus upload sessions + visibility
Adds an admin-driven sweep on top of the existing 24h auto-expire so
operators can see and reap abandoned `.tus/` sessions without waiting.
- storageJanitor: extract `walkTusDir(predicate)` helper, add
`getStaleTusInfo` + `cleanupStaleTusSessions(thresholdMs, dryRun)`;
refactor `cleanupTusStragglers` to delegate while preserving its
janitor-tick `{ removed }` contract.
- StorageStats gains `staleTusSessions` + `staleTusSize` (fixed 1h
display threshold).
- New `POST /api/admin/storage/cleanup-tus` route with
`maxAgeHours` validation (positive finite number, default 1) and
`dryRun` support; admin-gated.
- StoragePanel: 6th overview card "Stale Uploads" + new cleanup
subsection mirroring the media-cleanup pattern (preview-then-clean
with shared result panel styling).
- Tests: 8 new janitor tests covering empty dir, threshold filtering,
dry-run vs live, oldest-mtime tracking, subdir skipping, and the
override path on the existing straggler sweep. New
`routes/admin.test.ts` covers auth/admin gates, validation (zero,
negative, NaN), default `maxAgeHours`, dry-run vs live unlink.
- Docs: `uploads.md` §Janitor expanded to the full lifecycle (cancel
DELETE, discard DELETE, auto-expire, straggler sweep, admin route);
`admin.md` Storage Management updated with the new endpoint and
StorageStats fields.
This commit is contained in:
@@ -204,8 +204,11 @@ GET /api/admin/storage/stats → StorageStats
|
||||
GET /api/admin/storage/orphans → { orphans: OrphanedFile[] }
|
||||
POST /api/admin/storage/cleanup { dryRun?: boolean } → CleanupResult
|
||||
POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean } → CleanupResult
|
||||
POST /api/admin/storage/cleanup-tus { maxAgeHours?: number = 1, dryRun?: boolean = false } → CleanupResult
|
||||
```
|
||||
|
||||
The `cleanup-tus` route walks `.tus/`, deleting (or counting, if `dryRun`) any entry whose mtime is older than `maxAgeHours`. No DB rows are touched — `.tus/` is filesystem-only — so `deletedAttachmentRecords` in the response is always `0`. See `docs/systems/uploads.md` §Janitor for the full lifecycle (immediate-DELETE on cancel/discard, automatic 24 h `cleanupTusUploads`, 48 h defensive `cleanupTusStragglers`, and this admin-driven sweep).
|
||||
|
||||
**StorageStats shape:**
|
||||
```typescript
|
||||
{
|
||||
@@ -219,10 +222,14 @@ POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean }
|
||||
unlinkedSize: number;
|
||||
danglingAttachments: number; // Attachment records pointing to missing files
|
||||
danglingSize: number;
|
||||
staleTusSessions: number; // .tus/ payload + sidecar files with mtime > 1 h old
|
||||
staleTusSize: number; // Total bytes of those stale tus entries
|
||||
breakdown: { type: string; count: number; size: number }[];
|
||||
}
|
||||
```
|
||||
|
||||
`staleTusSessions` / `staleTusSize` use a **fixed 1 h display threshold** (active uploads write chunks frequently; a 1 h+ gap means the user walked away). This is distinct from the `maxAgeHours` body parameter on `cleanup-tus`, which is configurable per request.
|
||||
|
||||
**CleanupResult shape:**
|
||||
```typescript
|
||||
{
|
||||
@@ -234,7 +241,7 @@ POST /api/admin/storage/cleanup-media { maxAgeDays: number, dryRun?: boolean }
|
||||
}
|
||||
```
|
||||
|
||||
Storage functions (`getStorageStats`, `getOrphanedFiles`, `cleanupStorage`, `cleanupOldMedia`) are implemented in `utils/storageJanitor.ts`. Out of scope here -- if an uploads.md spec is created, document there.
|
||||
Storage functions (`getStorageStats`, `getOrphanedFiles`, `cleanupStorage`, `cleanupOldMedia`, `cleanupStaleTusSessions`, `getStaleTusInfo`) are implemented in `utils/storageJanitor.ts`. Tus-specific lifecycle details live in `docs/systems/uploads.md` §Janitor.
|
||||
|
||||
**Cleanup flow (UI):**
|
||||
1. Admin clicks "Preview Cleanup" -- calls `cleanupStorage(dryRun=true)` or `cleanupOldMedia(days, dryRun=true)`
|
||||
|
||||
+11
-5
@@ -65,11 +65,16 @@ When the final PATCH completes, the hook:
|
||||
|
||||
### Janitor
|
||||
|
||||
| Function | Sweeps |
|
||||
|----------|--------|
|
||||
| `cleanupTusUploads()` | Invokes `@tus/file-store.deleteExpired()` (24 h `Upload-Expires`). |
|
||||
| `cleanupTusStragglers()` | Defensive sweep of `.tus/` for files older than 48 h that the tus library failed to clean up. |
|
||||
| `getUnlinkedAttachments()` | Existing 1 h grace still applies to abandoned post-finish attachments. |
|
||||
| Trigger | Function / Path | Sweeps |
|
||||
|---------|-----------------|--------|
|
||||
| User cancels mid-upload | Client `tus.abort(true)` → tus DELETE | Immediate cleanup of the `.tus/` payload + sidecar. |
|
||||
| User discards a paused/failed bubble | `transferStore.abortUpload` → manual `fetch DELETE` (when no live tus instance) | Immediate cleanup of the `.tus/` payload + sidecar. |
|
||||
| Janitor tick (every ~30 s) | `cleanupTusUploads()` | Invokes `@tus/file-store.deleteExpired()` (24 h `Upload-Expires` default, configurable via `tusExpirationMs`). |
|
||||
| Janitor tick (every ~30 s) | `cleanupTusStragglers()` | Defensive unlink of any `.tus/` entry whose mtime is older than `tusStragglerSweepMs` (48 h default) — catches orphans the tus library missed (payload without sidecar, sidecar without payload). |
|
||||
| Admin-triggered | `POST /api/admin/storage/cleanup-tus` → `cleanupStaleTusSessions(thresholdMs, dryRun)` | Manual sweep with configurable `maxAgeHours` (default 1 h). Supports preview (`dryRun=true`) before live deletion. |
|
||||
| Janitor tick (post-finalize) | `getUnlinkedAttachments()` | 1 h grace for finalized attachment rows that were never linked to a message. |
|
||||
|
||||
Stats: `getStorageStats()` exposes `staleTusSessions` + `staleTusSize` for the admin Storage Overview, computed via `getStaleTusInfo(60 * 60 * 1000)` — entries with mtime older than 1 h. The display threshold is fixed (matches the cleanup default); the admin route's `maxAgeHours` is what's actually configurable.
|
||||
|
||||
### Security
|
||||
|
||||
@@ -449,6 +454,7 @@ The admin routes (`routes/admin.ts`) expose the janitor functions via REST:
|
||||
| `GET /api/admin/storage/orphans` | GET | `getOrphanedFiles()` |
|
||||
| `POST /api/admin/storage/cleanup` | POST | `cleanupStorage(dryRun)` |
|
||||
| `POST /api/admin/storage/cleanup-media` | POST | `cleanupOldMedia(maxAgeDays, dryRun)` |
|
||||
| `POST /api/admin/storage/cleanup-tus` | POST | `cleanupStaleTusSessions(maxAgeHours * 3600 * 1000, dryRun)` |
|
||||
|
||||
All require JWT + admin role. See `docs/systems/api.md` for request/response formats.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user