STTracker/docs/plans/001-phase1-sdd/reports/task-16-report.md

# Task 16 report: `apps/server`

**Status:** DONE_WITH_CONCERNS. The code is complete and verified. The permission and Commands docs step is blocked (see Concerns 1).
**Branch:** `task/16` (worktree `/home/user/STTracker/.worktrees/t16`). Merged `feat/phase1` (7f3d0d5, Task 9's jobs.ts fix) first, as the coordinator asked. It was a fast-forward with no conflicts.

## Commits
- 7b0a673 feat(core): API support helpers: token list, suggestions, attachment links, connection runs, device auth, file-mode sync
- 6963e29 feat(server): REST API, auth, sources, changes, export
- 02b7eed feat(server): boot: migrate, load plugins, worker, scheduler, daily backup, web placeholder
- 348b04e docs: server architecture, API decisions, follow-ups
- 0a54020 test(server): silence the broken fixture plugin warning

## What was built
- `config.ts`: `loadConfig(env)` returns DATABASE_URL (default `sqlite://${DATA_DIR}/sttracker.db`), DATA_DIR (`./data`), STTRACKER_SECRET (required, with the exact message), PUBLIC_URL, PORT (3000) and PLUGIN_DIRS (`plugins,${DATA_DIR}/plugins`).
- `http.ts`: `route()` (zod body and query, with types inferred), `apiHandler`, auth modes `none|session|token:*|any:*`, the `stt_session` cookie (HttpOnly, SameSite=Lax, Secure on an https PUBLIC_URL), the CSRF header `X-Requested-With: sttracker` on cookie writes, body caps (JSON 1 MB, multipart 50 MB, counted while streaming, 413 `validation`), a limit of 10 login/register attempts per minute per IP (429 `rate_limited`), and the error mapping from ruling 5. Unique violations map to 409 `conflict`. Unknown errors return 500 `internal` with a fixed message and are logged.
- `app.ts`: `createApp({db, registry, secret, dataDir, publicUrl?})` returns `{fetch, routes}`. `/ingest/*` returns a 404 JSON error.
- `routes/`: auth, entries, engagements, works, lists, copies, custom-fields, attachments, sources, changes, export and meta (openapi.json, about, admin/plugins/reload). Shared pieces live in `schemas.ts` and `form.ts`. They cover every route in the brief.
- `openapi.ts`: OpenAPI 3.1 built from the route table with `z.toJSONSchema`. It gives each route its method, path, summary, security, path and query parameters, and request body.
- `boot.ts`: `nextBackupAt` and `ensureBackupScheduled` (lockKey `backup`, runAt the next 03:00 local).
- `main.ts`: mkdir DATA_DIR/plugins, then createDb, migrate, loadPlugins, worker (sync + backup), scheduler, daily backup, then `Bun.serve` with routes `/api/*` and `/ingest/*` going to the app and `/*` going to the HTML import. SIGTERM/SIGINT run a clean shutdown.
- `apps/web/index.html`: the placeholder required by ruling 1.
- Core additions, needed because the API had nothing to call:
  - `auth.listAppTokens` (metadata only).
  - `entries.listSuggestions`.
  - `attachments.requireLinkTarget` (ownership check per entity, run before `storeAttachment`) and `listAttachmentLinks`.
  - `readAttachment` now also returns `filename`.
  - In `connections`: `getConnection` (hides deleted and other users' connections), `isDeletedConnection`, `listRuns`, `deviceStart` and `devicePoll`.
  - `syncHandler` file mode: `{connectionId, file:{name, base64}}` → `runSync` `{file}` mode, recorded as run mode `file`.
  - `jobs.ts` was not touched.

## Decisions worth review
- **Actor:** all API writes use actor `"user"`, including token writes. The change engine only protects fields whose source is `"user"`, so a `token:` actor would let the next sync overwrite a script's edit.
- **Cookie-only routes:** tokens, `POST/PATCH/DELETE sources`, device auth, `import/bundle` and admin need a session cookie. App tokens can't mint tokens or handle credentials.
- **Daily backup re-enqueue:** the finishing job still holds lockKey `backup`, so an immediate re-enqueue would return null. The next job is queued 5 s after each run, with an hourly idempotent safety net and once at boot.
- **Request field names:**
  - `PATCH list-items/:id` takes `{position}`, which maps to `moveListItem` (core has no note edit).
  - `DELETE me/tokens/:id`.
  - `POST sources/:id/device/poll` takes `{deviceCode}`.
  - Register requires a password of at least 8 characters.

## TDD evidence
- RED for the server, before src existed: `bun test apps/server` gave `error: Cannot find module '../src/app.ts'`, 0 pass, 1 fail.
- RED for the core helpers: `bun test packages/core/test/api-support.test.ts` gave 0 pass, 1 fail, 1 error (missing exports).
- RED for boot: `bun test apps/server/test/boot.test.ts` gave 1 fail, 1 error (missing `boot.ts`).
- GREEN:
  - `bun test apps/server`: 23 pass, 0 fail.
  - The same with Postgres: 40 pass, 0 fail.
  - `api-support`: 6 pass on SQLite, 12 pass on both.
- A float4 snag on Postgres: `match_suggestions.score` is `real`, so 0.9 came back as 0.8999999761581421. The test now uses 0.75. The helper itself is unaffected.
- Full suite after the merge:
  - `bun test`: 307 pass, 0 fail (34 files).
  - `TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres bun test`: 492 pass, 0 fail.
- Typecheck: `bun run typecheck` (`tsc --noEmit -p tsconfig.json`) is clean.
- Test output has no stray lines.

## Boot verification (real)
```
$ STTRACKER_SECRET=test DATA_DIR=$(mktemp -d) PORT=3999 bun apps/server/src/main.ts &
$ curl -s localhost:3999/api/v1/openapi.json | head -c 200
{"openapi":"3.1.0","info":{"title":"STTracker API","version":"1"},"servers":[{"url":"/"}],"components":{"securitySchemes":{"session":{"type":"apiKey","in":"cookie","name":"stt_session","description":"
server log: STTracker listening on http://localhost:3999/ (sqlite, 6 plugin(s))
kill -TERM -> "SIGTERM: shutting down", process exited; DATA_DIR contains plugins/ and sttracker.db
GET /           -> placeholder HTML (bundled by Bun)
GET /api/v1/entries (no auth) -> {"error":{"code":"unauthenticated","message":"Sign in first."}}
POST /ingest/x/y -> {"error":{"code":"not_found","message":"No ingest endpoint for this plugin."}}
no STTRACKER_SECRET -> error: Set STTRACKER_SECRET to a long random string. It encrypts source credentials.
bun --hot apps/server/src/main.ts (PORT=3998): served openapi.json, exited cleanly on SIGTERM
```

## Concerns
1. **AGENTS.md Commands, PERMISSIONS.md and the tool permission files were not updated.**
   - The auto-mode classifier denied my edit to `.claude/settings.json` and `.gemini/settings.json` as agent self-modification. I didn't work around it.
   - AGENTS.md requires PERMISSIONS.md to change in the same commit as the tool files, and allows only `worked` rows in Commands. So I left all of them unchanged.
   - The exact change is recorded in `docs/TODO.md` (Now). It needs a human:
     - Allow `bun run typecheck`, `bun apps/server/src/main.ts` and `bun --hot apps/server/src/main.ts`. All three were verified this session.
     - Add matching rows to PERMISSIONS.md, and drop the stale "typecheck failed" row.
     - Set the AGENTS.md Commands: Single test `bun test <path>`, Typecheck `bun run typecheck`, Dev `bun --hot apps/server/src/main.ts` (needs STTRACKER_SECRET), Start `bun apps/server/src/main.ts`.
2. **Hidden paused connections.** A connection the user paused that never had credentials looks deleted, by Task 9's definition, so it disappears from `GET sources`. This is noted in TODO; the fix is an explicit `deleted_at`.
3. **Limits deferred to TODO:**
   - The rate limit is per process and uses the socket IP, so all clients share one key behind a proxy.
   - File imports are stored as base64 in `jobs.payload`.
   - OpenAPI has no response schemas.
4. **Bun FormData filename quirk.** Bun's FormData serializer does not escape `"` in filenames. The `safeFilename` sanitizer is unit-tested directly for quotes and CR/LF instead of end to end.

---

# Fix round 1

**Status:** DONE. The agent permission files were not touched.

## Commits
- e3a759d fix(core): soft-delete source connections with deleted_at
- aa8e4db fix(core): token writes keep their actor in history but are user-owned in field_sources
- 5f7d54b fix(server): token actor, login CSRF, 415, cookie decoding, admin invites, production default
- 0b34ede docs: token actor, connection deleted_at and CSRF decisions; follow-ups

## Item by item
1. **`deleted_at`.**
   - Migration `0003_source_connections_deleted_at` (nullable text, with `down`) is registered and added to `schema.ts`.
   - `deleteConnection` sets it (still pauses and wipes credentials) and is a 404 the second time.
   - `ownedRow`, `listConnections`, `updateConnection`, the scheduler due-query (`jobs.ts`, as the ruling allows) and `syncHandler` (`DropJobError`) all ignore deleted rows.
   - `isDeletedConnection` and the route-side filter are gone.
   - Task 9's connections test asserted that a deleted connection is still listed. It now asserts that it is hidden and that `deleted_at` is set.
   - Tests:
     - core: paused and resumable; deleted gives 404 on get, update and delete; never scheduled; queued sync dropped;
     - API: paused source listed and resumed.
   - The `schema.test` down-migration still passes.
2. **Token actor.**
   - `fieldOwner(actor)` in `history.ts` stores every non-`plugin:` actor as `"user"`. It is applied in `entries.writeEntry`, `engagements` (add and update) and `apply.listFieldSources` (list ownership read from history). Copies have no `field_sources`.
   - `authenticateAppToken` returns `{ user, tokenName }`; `verifyAppToken` delegates to it, so its signature is unchanged. `http.ts` sets `ctx.actor`: `token:<name>` for Bearer, `"user"` for sessions. Every write route passes it, now including lists and copies.
   - Tests:
     - core: history has actor `token:script` and `field_sources` has `"user"`; a later sync (plugin `flip`) leaves the status alone, files a pending `changed`, and still applies the source-owned rating;
     - API: a write-scope token's PATCH gives history actor `token:script` and `fieldSources.rating` `"user"`.
   - DECISIONS: a new entry supersedes the earlier bullets, because AGENTS.md makes DECISIONS append-only.
3. **CSRF and 415.**
   - `X-Requested-With: sttracker` is required on every non-Bearer, non-GET request, login and register included.
   - JSON bodies need `application/json`; anything else gets 415 `unsupported_media_type`. Empty bodies are still allowed for optional-body routes such as `sources/:id/run`.
   - Tests cover login, register, Bearer exemption and 415.
4. **Malformed cookie.** `readCookie` catches the decode error. `stt_session=%` now gives 401 (tested).
5. **Production by default.**
   - Dev mode is on only when `NODE_ENV === "development"` or `process.execArgv.includes("--hot")`.
   - Deviation: `Bun.argv` does not contain `--hot`. I checked: under `bun --hot`, `Bun.argv` held only the bun path and the script, and `process.execArgv` held `["--hot"]`.
   - TODO has a hand-off line for Task 19.
6. **`DELETE me/tokens/:id`.** `revokeAppToken` returns a boolean, and the route returns 404 for unknown, foreign or already-revoked ids (tested).
7. **`listSuggestions`.** It queries by the resolved canonical id. A test asks by an aliased id; it fails without the fix and passes with it.
8. **`admin/invites`.** Now `admin: true`. Tests: a member gets 403 on invites and on plugin reload; an admin can reload.
11. **OpenAPI multipart.** `RouteDef.formFields` is set on attachments. The multipart schema lists `file` (binary) plus `entity`, `entityId` and `caption`, with the required fields (tested).
12. **Tests.**
    - Cookie plus a bogus Bearer on a write gives 401.
    - A member calling reload gets 403.
    - A multipart import over 50 MB gives 413 `validation`.
    - A positive test with a write-scope token.
    - The attachment test now uses bob's real entry (404), and checks that bob cannot read ana's attachment.

## TDD notes
- The core items were RED first:
  - `deleted_at` tests: 2 fail.
  - Token actor: missing `authenticateAppToken` export.
- For items 3, 4 and 7 I wrote the fix before the test. I then confirmed each test catches its bug by reverting the fix temporarily: the readCookie try/catch, the `none`-route CSRF exemption, the 415 check and the canonical suggestions id each made their test fail.

## Verification
- `bun run typecheck`: clean.
- `bun test`: 321 pass, 0 fail (34 files).
- `TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres bun test`: 520 pass, 0 fail.
- Real boot on a fresh SQLite DATA_DIR:
  - migrations up to 0003 applied;
  - `openapi.json` was served;
  - a login POST without the header returned the 403 CSRF error;
  - SIGTERM shut it down cleanly.
- `git status`: clean.

## Still open
- The permission files and AGENTS.md Commands are still not updated; this needs a human (see Concerns 1 above).
- `restoreVersion`, `acceptSuggestion` and `resolveChange` take no actor, so token calls to them record `"user"` in history. Logged in `docs/TODO.md`.