STTracker/docs/plans/001-phase1-sdd/reports/task-16-report.md

Task 16 report: apps/server

Status: DONE_WITH_CONCERNS. The code is complete and verified. The permission and Commands docs step is blocked (see Concerns 1). Branch: task/16 (worktree /home/user/STTracker/.worktrees/t16). Merged feat/phase1 (7f3d0d5, Task 9's jobs.ts fix) first, as the coordinator asked. It was a fast-forward with no conflicts.

Commits

• 7b0a673 feat(core): API support helpers: token list, suggestions, attachment links, connection runs, device auth, file-mode sync
• 6963e29 feat(server): REST API, auth, sources, changes, export
• 02b7eed feat(server): boot: migrate, load plugins, worker, scheduler, daily backup, web placeholder
• 348b04e docs: server architecture, API decisions, follow-ups
• 0a54020 test(server): silence the broken fixture plugin warning

What was built

• config.ts: loadConfig(env) returns DATABASE_URL (default sqlite://${DATA_DIR}/sttracker.db), DATA_DIR (./data), STTRACKER_SECRET (required, with the exact message), PUBLIC_URL, PORT (3000) and PLUGIN_DIRS (plugins,${DATA_DIR}/plugins).
• http.ts: route() (zod body and query, with types inferred), apiHandler, auth modes none|session|token:*|any:*, the stt_session cookie (HttpOnly, SameSite=Lax, Secure on an https PUBLIC_URL), the CSRF header X-Requested-With: sttracker on cookie writes, body caps (JSON 1 MB, multipart 50 MB, counted while streaming, 413 validation), a limit of 10 login/register attempts per minute per IP (429 rate_limited), and the error mapping from ruling 5. Unique violations map to 409 conflict. Unknown errors return 500 internal with a fixed message and are logged.
• app.ts: createApp({db, registry, secret, dataDir, publicUrl?}) returns {fetch, routes}. /ingest/* returns a 404 JSON error.
• routes/: auth, entries, engagements, works, lists, copies, custom-fields, attachments, sources, changes, export and meta (openapi.json, about, admin/plugins/reload). Shared pieces live in schemas.ts and form.ts. They cover every route in the brief.
• openapi.ts: OpenAPI 3.1 built from the route table with z.toJSONSchema. It gives each route its method, path, summary, security, path and query parameters, and request body.
• boot.ts: nextBackupAt and ensureBackupScheduled (lockKey backup, runAt the next 03:00 local).
• main.ts: mkdir DATA_DIR/plugins, then createDb, migrate, loadPlugins, worker (sync + backup), scheduler, daily backup, then Bun.serve with routes /api/* and /ingest/* going to the app and /* going to the HTML import. SIGTERM/SIGINT run a clean shutdown.
• apps/web/index.html: the placeholder required by ruling 1.
• Core additions, needed because the API had nothing to call:
  • auth.listAppTokens (metadata only).
  • entries.listSuggestions.
  • attachments.requireLinkTarget (ownership check per entity, run before storeAttachment) and listAttachmentLinks.
  • readAttachment now also returns filename.
  • In connections: getConnection (hides deleted and other users' connections), isDeletedConnection, listRuns, deviceStart and devicePoll.
  • syncHandler file mode: {connectionId, file:{name, base64}} → runSync {file} mode, recorded as run mode file.
  • jobs.ts was not touched.

Decisions worth review

• Actor: all API writes use actor "user", including token writes. The change engine only protects fields whose source is "user", so a token: actor would let the next sync overwrite a script's edit.
• Cookie-only routes: tokens, POST/PATCH/DELETE sources, device auth, import/bundle and admin need a session cookie. App tokens can't mint tokens or handle credentials.
• Daily backup re-enqueue: the finishing job still holds lockKey backup, so an immediate re-enqueue would return null. The next job is queued 5 s after each run, with an hourly idempotent safety net and once at boot.
• Request field names:
  • PATCH list-items/:id takes {position}, which maps to moveListItem (core has no note edit).
  • DELETE me/tokens/:id.
  • POST sources/:id/device/poll takes {deviceCode}.
  • Register requires a password of at least 8 characters.

TDD evidence

• RED for the server, before src existed: bun test apps/server gave error: Cannot find module '../src/app.ts', 0 pass, 1 fail.
• RED for the core helpers: bun test packages/core/test/api-support.test.ts gave 0 pass, 1 fail, 1 error (missing exports).
• RED for boot: bun test apps/server/test/boot.test.ts gave 1 fail, 1 error (missing boot.ts).
• GREEN:
  • bun test apps/server: 23 pass, 0 fail.
  • The same with Postgres: 40 pass, 0 fail.
  • api-support: 6 pass on SQLite, 12 pass on both.
• A float4 snag on Postgres: match_suggestions.score is real, so 0.9 came back as 0.8999999761581421. The test now uses 0.75. The helper itself is unaffected.
• Full suite after the merge:
  • bun test: 307 pass, 0 fail (34 files).
  • TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres bun test: 492 pass, 0 fail.
• Typecheck: bun run typecheck (tsc --noEmit -p tsconfig.json) is clean.
• Test output has no stray lines.

Boot verification (real)

$ STTRACKER_SECRET=test DATA_DIR=$(mktemp -d) PORT=3999 bun apps/server/src/main.ts &
$ curl -s localhost:3999/api/v1/openapi.json | head -c 200
{"openapi":"3.1.0","info":{"title":"STTracker API","version":"1"},"servers":[{"url":"/"}],"components":{"securitySchemes":{"session":{"type":"apiKey","in":"cookie","name":"stt_session","description":"
server log: STTracker listening on http://localhost:3999/ (sqlite, 6 plugin(s))
kill -TERM -> "SIGTERM: shutting down", process exited; DATA_DIR contains plugins/ and sttracker.db
GET /           -> placeholder HTML (bundled by Bun)
GET /api/v1/entries (no auth) -> {"error":{"code":"unauthenticated","message":"Sign in first."}}
POST /ingest/x/y -> {"error":{"code":"not_found","message":"No ingest endpoint for this plugin."}}
no STTRACKER_SECRET -> error: Set STTRACKER_SECRET to a long random string. It encrypts source credentials.
bun --hot apps/server/src/main.ts (PORT=3998): served openapi.json, exited cleanly on SIGTERM

Concerns

1. AGENTS.md Commands, PERMISSIONS.md and the tool permission files were not updated.
   • The auto-mode classifier denied my edit to .claude/settings.json and .gemini/settings.json as agent self-modification. I didn't work around it.
   • AGENTS.md requires PERMISSIONS.md to change in the same commit as the tool files, and allows only worked rows in Commands. So I left all of them unchanged.
   • The exact change is recorded in docs/TODO.md (Now). It needs a human:
     • Allow bun run typecheck, bun apps/server/src/main.ts and bun --hot apps/server/src/main.ts. All three were verified this session.
     • Add matching rows to PERMISSIONS.md, and drop the stale "typecheck failed" row.
     • Set the AGENTS.md Commands: Single test bun test <path>, Typecheck bun run typecheck, Dev bun --hot apps/server/src/main.ts (needs STTRACKER_SECRET), Start bun apps/server/src/main.ts.
2. Hidden paused connections. A connection the user paused that never had credentials looks deleted, by Task 9's definition, so it disappears from GET sources. This is noted in TODO; the fix is an explicit deleted_at.
3. Limits deferred to TODO:
   • The rate limit is per process and uses the socket IP, so all clients share one key behind a proxy.
   • File imports are stored as base64 in jobs.payload.
   • OpenAPI has no response schemas.
4. Bun FormData filename quirk. Bun's FormData serializer does not escape " in filenames. The safeFilename sanitizer is unit-tested directly for quotes and CR/LF instead of end to end.

---

Fix round 1

Status: DONE. The agent permission files were not touched.

Commits

• e3a759d fix(core): soft-delete source connections with deleted_at
• aa8e4db fix(core): token writes keep their actor in history but are user-owned in field_sources
• 5f7d54b fix(server): token actor, login CSRF, 415, cookie decoding, admin invites, production default
• 0b34ede docs: token actor, connection deleted_at and CSRF decisions; follow-ups

Item by item

1. deleted_at.
   • Migration 0003_source_connections_deleted_at (nullable text, with down) is registered and added to schema.ts.
   • deleteConnection sets it (still pauses and wipes credentials) and is a 404 the second time.
   • ownedRow, listConnections, updateConnection, the scheduler due-query (jobs.ts, as the ruling allows) and syncHandler (DropJobError) all ignore deleted rows.
   • isDeletedConnection and the route-side filter are gone.
   • Task 9's connections test asserted that a deleted connection is still listed. It now asserts that it is hidden and that deleted_at is set.
   • Tests:
     • core: paused and resumable; deleted gives 404 on get, update and delete; never scheduled; queued sync dropped;
     • API: paused source listed and resumed.
   • The schema.test down-migration still passes.
2. Token actor.
   • fieldOwner(actor) in history.ts stores every non-plugin: actor as "user". It is applied in entries.writeEntry, engagements (add and update) and apply.listFieldSources (list ownership read from history). Copies have no field_sources.
   • authenticateAppToken returns { user, tokenName }; verifyAppToken delegates to it, so its signature is unchanged. http.ts sets ctx.actor: token:<name> for Bearer, "user" for sessions. Every write route passes it, now including lists and copies.
   • Tests:
     • core: history has actor token:script and field_sources has "user"; a later sync (plugin flip) leaves the status alone, files a pending changed, and still applies the source-owned rating;
     • API: a write-scope token's PATCH gives history actor token:script and fieldSources.rating "user".
   • DECISIONS: a new entry supersedes the earlier bullets, because AGENTS.md makes DECISIONS append-only.
3. CSRF and 415.
   • X-Requested-With: sttracker is required on every non-Bearer, non-GET request, login and register included.
   • JSON bodies need application/json; anything else gets 415 unsupported_media_type. Empty bodies are still allowed for optional-body routes such as sources/:id/run.
   • Tests cover login, register, Bearer exemption and 415.
4. Malformed cookie. readCookie catches the decode error. stt_session=% now gives 401 (tested).
5. Production by default.
   • Dev mode is on only when NODE_ENV === "development" or process.execArgv.includes("--hot").
   • Deviation: Bun.argv does not contain --hot. I checked: under bun --hot, Bun.argv held only the bun path and the script, and process.execArgv held ["--hot"].
   • TODO has a hand-off line for Task 19.
6. DELETE me/tokens/:id. revokeAppToken returns a boolean, and the route returns 404 for unknown, foreign or already-revoked ids (tested).
7. listSuggestions. It queries by the resolved canonical id. A test asks by an aliased id; it fails without the fix and passes with it.
8. admin/invites. Now admin: true. Tests: a member gets 403 on invites and on plugin reload; an admin can reload.
9. OpenAPI multipart. RouteDef.formFields is set on attachments. The multipart schema lists file (binary) plus entity, entityId and caption, with the required fields (tested).
10. Tests.
    • Cookie plus a bogus Bearer on a write gives 401.
    • A member calling reload gets 403.
    • A multipart import over 50 MB gives 413 validation.
    • A positive test with a write-scope token.
    • The attachment test now uses bob's real entry (404), and checks that bob cannot read ana's attachment.

TDD notes

• The core items were RED first:
  • deleted_at tests: 2 fail.
  • Token actor: missing authenticateAppToken export.
• For items 3, 4 and 7 I wrote the fix before the test. I then confirmed each test catches its bug by reverting the fix temporarily: the readCookie try/catch, the none-route CSRF exemption, the 415 check and the canonical suggestions id each made their test fail.

Verification

• bun run typecheck: clean.
• bun test: 321 pass, 0 fail (34 files).
• TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres bun test: 520 pass, 0 fail.
• Real boot on a fresh SQLite DATA_DIR:
  • migrations up to 0003 applied;
  • openapi.json was served;
  • a login POST without the header returned the 403 CSRF error;
  • SIGTERM shut it down cleanly.
• git status: clean.

Still open

• The permission files and AGENTS.md Commands are still not updated; this needs a human (see Concerns 1 above).
• restoreVersion, acceptSuggestion and resolveChange take no actor, so token calls to them record "user" in history. Logged in docs/TODO.md.