diff --git a/docs/superpowers/plans/2026-10-30-ai.md b/docs/superpowers/plans/2026-10-30-ai.md new file mode 100644 index 0000000..6578ce5 --- /dev/null +++ b/docs/superpowers/plans/2026-10-30-ai.md @@ -0,0 +1,178 @@ +# WP-D AI Implementation Plan + +> **For agentic workers:** executed inline by the WP-D agent with TDD (failing test first), one commit per task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Opt-in AI for STCourses: a course-aware learner assistant (retrieval, course tools, MCP tools, mentor mode) and a studio generator that writes house-style drafts. AI is off until an admin configures a server provider or a user saves their own key (BYOK). + +**Architecture:** +- `packages/feature-ai` exports `aiFeature(ctx, opts?): Feature`, `aiWeb: WebFeature` and `migrationsDirs`. +- Providers implement the frozen `AiProvider` contract with plain `fetch` (no SDK): Anthropic Messages API (streaming SSE, tool use, default model `claude-opus-5-5`) and OpenAI-compatible chat completions (covers Ollama). +- Settings live in the existing `ai_settings` and `user_api_keys` tables (no new tables). Secrets are AES-GCM encrypted with a key derived by HKDF-SHA-256 from `ctx.config.secret`; they never leave the server. +- Retrieval is an in-process BM25 index per loaded course (course Markdown, transcripts, code, deck/transcript text under `/cache//`), plus the requesting user's notes at query time. Solution files are never indexed. +- The assistant runs a small tool loop over `search_course`, `read_lesson`, `list_notes` and admin-configured MCP servers (stdio and streamable HTTP), and streams NDJSON events to the browser. +- The generator builds house-style prompts (Pure Snake module shape, Systemic Design beats), parses `=== FILE: … ===` blocks from the model output, and hands them to a `DraftSink`. The default sink writes to `/drafts///`; Phase 2 swaps in the studio's `createDraftWriter(ctx)`. + +**Tech Stack:** Bun 1.3, TypeScript strict, WebCrypto, `Bun.SQL`, `Bun.spawn`, React 19 + React Router 6, `@stc/ui` primitives. + +**Spec:** sections 9.2, 7, 2 (anti-goals). Brief: `.superpowers/sdd/2026-10-30-stcourses/phase-1-brief.md` (WP-D). + +## Global Constraints + +Inherited from `docs/superpowers/plans/2026-10-30-stcourses.md` (Bun only, plain CSS with tokens, both databases, AGPL, never write into the reference course repos, never commit licensed content, keyboard rules, commit trailer). Package-specific: + +- **AI is off by default.** No provider configured and no BYOK key means every AI route answers `enabled: false` / 409, and the UI says how to turn it on. +- **Keys never go to the client.** API responses carry `hasApiKey: boolean`, never ciphertext or plaintext. +- **No real network in tests.** Providers take an injectable `fetch`; MCP tests use a local fake server (stdio child process and `Bun.serve` on port 0). +- **Mentor mode is on by default** and its rules go into the system prompt; when off, the UI shows "Mentor mode off: I'll give full answers". +- **Generated material is never published.** The generator only writes through a `DraftSink`. + +## Review Focus + +1. **Secrets:** BYOK and server keys are encrypted at rest (AES-GCM, random IV, HKDF key from `STC_SECRET`), a wrong secret fails to decrypt, and no route returns key material. Tests: Tasks 1, 2, 9. +2. **Mentor anti-goal:** mentor rules are in the system prompt unless the learner switched them off for that course; solution files (`browser.solution`, `solutions/`, `reference/`, `expansion/solution/`) are never retrieved or readable by tools. Tests: Tasks 5, 7. +3. **Generation never publishes:** drafts land only under the sink location, paths are sanitised (no `..`, no absolute paths). Test: Task 8. +4. **Access control:** assistant routes require that the user can view the course (`canView`); generator routes require owner, the `author` role or a `course_authors` row; admin routes require owner. Test: Task 9. +5. **Thinking blocks:** the Anthropic provider replays an assistant turn's raw content blocks (thinking, text, tool_use) unchanged inside one tool loop. Test: Task 3. + +--- + +### Task 1: Package scaffold and secret box + +**Files:** `packages/feature-ai/{package.json,src/index.ts,src/crypto.ts}`, test `packages/feature-ai/test/crypto.test.ts` + +**Interfaces (Produces):** +- `createSecretBox(secret: string): { encrypt(plain: string): Promise; decrypt(token: string): Promise }`. HKDF-SHA-256 (salt `"stcourses/byok"`, info `"aes-gcm-256/v1"`) → AES-GCM-256; token `v1..`. +- `migrationsDirs: string[]` (empty: the package adds no tables). + +- [ ] Tests: round trip; two encryptions differ (random IV); a box with another secret rejects; a tampered token rejects. +- [ ] Implement; commit `feat(ai): package scaffold and AES-GCM secret box`. + +### Task 2: Settings repository + +**Files:** `src/settings.ts`, test `test/settings.test.ts` (uses `forEachDb` from `packages/db/test/helpers.ts`) + +**Interfaces:** +- `ProviderKind = "anthropic" | "openai"`; `ProviderConfig = { kind, model?, baseUrl?, effort? }`. +- `McpServerConfig = { id, name, transport: "stdio" | "http", command?, args?, env?, url?, headers?, courseIds?: string[] }` (no `courseIds` = global). +- `aiSettingsRepo(db, box)`: + - `getServer(): Promise<{ provider: (ProviderConfig & { hasApiKey }) | null; mcpServers }>`, `saveServer({ provider: (ProviderConfig & { apiKey?: string; clearApiKey?: boolean }) | null, mcpServers })`, `serverApiKey(): Promise`. + - `getUser(userId): Promise<{ byok: (ProviderConfig & { hasApiKey }) | null; mentorOff: Record }>`, `saveUserKey(userId, cfg & { apiKey })`, `removeUserKey(userId)`, `userApiKey(userId, kind)`, `setMentorOff(userId, courseId, off)`, `isMentorOff(userId, courseId)`. +- Storage: `ai_settings` row `('server','')` holds `{ provider, apiKeyCipher, mcpServers }`; row `('user', userId)` holds `{ byok, mentorOff }`. The per-course flag lives in `config_json` because the frozen primary key is `(scope, owner_id)`; the `mentor_off` column mirrors "off for every course" (always 0 for now). Contract request filed. Keys go in `user_api_keys` encrypted. Upserts are select-then-insert-or-update in a transaction. + +- [ ] Tests (both DBs): empty server config is `null` provider; saving a key stores ciphertext only (raw row does not contain the key) and `serverApiKey()` returns it; omitting `apiKey` on save keeps the old key; BYOK save/remove; mentor toggle per course is independent. +- [ ] Implement; commit `feat(ai): settings repository for server config, BYOK keys and mentor toggles`. + +### Task 3: SSE parsing and the Anthropic provider + +**Files:** `src/providers/sse.ts`, `src/providers/anthropic.ts`, test `test/anthropic.test.ts` + +**Interfaces:** +- `readSse(body: ReadableStream): AsyncIterable<{ event: string; data: string }>`. +- `createAnthropicProvider({ apiKey, model = "claude-opus-5-5", baseUrl = "https://api.anthropic.com", effort?, maxTokens = 64000, fetch? }): AiProvider`. + - POST `/v1/messages` with `stream: true`, headers `x-api-key`, `anthropic-version: 2023-06-01`. No `thinking`, `temperature` or forced `tool_choice` (Opus 5.5 rejects them). On the first-party endpoint, `fallbacks: "default"` with beta header `server-side-fallback-2026-07-01`. + - Message mapping: `system` messages join into `system`; an assistant `ChatMessage` with `toolCallId` and `name` is a tool call (content = JSON input); `tool` messages become `tool_result` blocks; consecutive same-role blocks merge. + - The provider remembers each response's full content blocks (thinking with signature, text, tool_use) by tool_use id and replays them unchanged for the matching assistant tool-call messages. + - Streams `{ text }` per `text_delta`, `{ toolCall }` per finished `tool_use` block; HTTP errors and `error` events throw with the API message; `stop_reason: "refusal"` yields a plain sentence. + +- [ ] Tests with a fake `fetch` that returns canned SSE: text streams in order; a tool_use with split `input_json_delta` yields one parsed toolCall; request body has model, system, tools as `input_schema`, no `thinking`/`temperature`; a second call after a tool round replays the thinking block with its signature unchanged before `tool_result`; non-2xx throws with the error message. +- [ ] Implement; commit `feat(ai): Anthropic provider over fetch with streaming tool use`. + +### Task 4: OpenAI-compatible provider + +**Files:** `src/providers/openai.ts`, `src/providers/index.ts`, test `test/openai.test.ts` + +**Interfaces:** +- `createOpenAiProvider({ baseUrl, apiKey?, model, fetch? }): AiProvider`. POST `/chat/completions`, `stream: true`, tools as `{ type: "function", function: { name, description, parameters } }`. Assistant tool-call messages map to `tool_calls`; `tool` messages to `{ role: "tool", tool_call_id }`. Accumulates `delta.tool_calls` by index and emits on finish or `[DONE]`. +- `createProvider(cfg: ProviderConfig, apiKey: string | null, fetchImpl?): AiProvider`. + +- [ ] Tests: text deltas; fragmented tool-call arguments; no `Authorization` header without a key (Ollama); request mapping of a tool round. +- [ ] Implement; commit `feat(ai): OpenAI-compatible provider (covers Ollama)`. + +### Task 5: Retrieval + +**Files:** `src/retrieval/{bm25.ts,course-index.ts}`, test `test/retrieval.test.ts` + +**Interfaces:** +- `Bm25` with `add(text, item)`, `search(query, k): { item, score }[]`. +- `Chunk = { path, kind: "lesson" | "transcript" | "markdown" | "code" | "deck" | "note", text, heading?, lessonId?, lessonTitle? }`. +- `buildCourseIndex(course: LoadedCourse, dataDir: string): Promise`; `CourseIndex.search(query, k, extra?: Chunk[]): Chunk[]`; `isSolutionPath(course, rel): boolean`. +- `getCourseIndex(course, dataDir)` caches per `LoadedCourse` object (a repository reload yields new objects) with a 5-minute TTL. +- Sources: every `.md` under the course root (depth-limited, skipping `.git`, `node_modules`), manifest code dirs and common code extensions (size cap 200 KB per file, 2 000 files per course), `.md/.txt/.vtt/.srt/.json` text under `/cache//`. Markdown splits at headings; long chunks split at ~1 500 chars. Paths are mapped to lessons through `file`, `transcript` and `hints.file` (anchor heading when it matches). +- Never indexed: `browser.solution` files and any path segment `solutions`, `solution`, `reference` under `expansion/`, or `reference/` in Systemic-Design style modules (see `isSolutionPath`). + +- [ ] Tests on the `sample` fixture: "sum of its arguments" hits `lessons/04-exercise.md` with lessonId `m2-text` or `m2-ex-add`; `exercises/add/solution.js` never appears; a deck text file in a temp `cache/sample/decks/d1/page-1.txt` is found with kind `deck`; note chunks passed as `extra` rank. +- [ ] Implement; commit `feat(ai): BM25 course retrieval over lessons, transcripts, code, deck text and notes`. + +### Task 6: MCP client + +**Files:** `src/mcp/client.ts`, test `test/mcp.test.ts`, fixture `test/fixtures/fake-mcp.ts` + +**Interfaces:** +- `connectMcp(cfg: McpServerConfig, opts?: { fetch?, timeoutMs? }): Promise`; `McpClient = { tools(): Promise<{ name, description, inputSchema }[]>; call(name, args): Promise; close(): Promise }`. +- JSON-RPC 2.0: `initialize` (protocolVersion `2025-06-18`), `notifications/initialized`, `tools/list`, `tools/call`. stdio = newline-delimited JSON over `Bun.spawn`; streamable HTTP = POST with `Accept: application/json, text/event-stream`, JSON or SSE replies, `Mcp-Session-Id` echoed. +- `mcpServersFor(courseId, servers)` selects global servers plus ones listing the course. + +- [ ] Tests: the fake server over stdio lists `echo` and `tools/call` returns its text; the same over HTTP (fake via `Bun.serve` on port 0, replying with SSE for `tools/call`) with the session id echoed; a server that exits gives a rejected promise, not a hang. +- [ ] Implement; commit `feat(ai): MCP client over stdio and streamable HTTP`. + +### Task 7: Assistant core + +**Files:** `src/assistant/{prompt.ts,tools.ts,loop.ts}`, test `test/assistant.test.ts` + +**Interfaces:** +- `buildSystemPrompt({ courseTitle, mentorText: string | null, mentorOn, sources: Chunk[], lessonTitle? }): string`. With mentor on and text present: a `## Course mentor rules` section with the file's text. With mentor off: a line saying full answers are allowed. Sources are numbered `[n] path (lesson title)` with their text; the prompt asks for citations by path. +- `courseTools({ ctx, course, userId, index })`: `ToolSpec[]` plus `run(name, input)` for `search_course`, `read_lesson` (refuses solution files) and `list_notes`. +- `runAssistant({ provider, messages, tools, signal, maxSteps = 6 }): AsyncIterable` where `AssistantEvent = { type: "text", text } | { type: "tool", name } | { type: "error", message } | { type: "done", text }`. Tool calls become assistant messages with `toolCallId`/`name` and `tool` result messages; unknown tools and tool errors are fed back as error text. +- Citations: `citationsFor(chunks): { path, lessonId?, lessonTitle?, kind }[]` deduplicated. + +- [ ] Tests: mentor on puts the sample `MENTOR.md` text in the system prompt; mentor off omits it and states full answers; a fake provider that calls `search_course` then answers gets the tool result and the loop ends with text; `read_lesson` on the exercise returns markdown; solution paths are refused. +- [ ] Implement; commit `feat(ai): assistant prompt, course tools and tool loop`. + +### Task 8: Generator and draft sink + +**Files:** `src/generator/{templates.ts,parse.ts,generate.ts,draft-sink.ts}`, test `test/generator.test.ts` + +**Interfaces:** +- `DraftSink = { writeDraftFiles(courseId, authorId, files: { path, content }[], message): Promise<{ location: string; paths: string[] }> }`; `createFsDraftSink(dataDir)` writes to `/drafts///` plus `DRAFT.json` (author, message, time). Same signature as the studio's `createDraftWriter(ctx).writeDraftFiles`. +- `GenerateRequest = { courseId, kind: "document" | "exercise" | "questions", style: "pure-snake" | "systemic-design", topic, sources: ({ kind: "lesson", lessonId } | { kind: "file", path } | { kind: "text", label, text })[], instructions? }`. +- `buildGeneratorPrompt(req, material, course)`: house-style system prompt (module shape: why, objectives, concepts, puzzle with "Think about it first" fold, everyday-programming bridge, exercises, hints as nudge / stronger hint / near-solution folds, check your understanding, in production, further reading; or Observe → Predict (+Discussion fold) → Spec → Build (Hint 1 — nudge / Hint 2 — concept / Hint 3 — pseudo-code) → Name it → Expansion Drop (+what this drop measures) → Further reading). Exercises: `README.md`, `starter.js`, `tests.js` (`export const tests` per the `ExerciseTest` contract), `solution.js`, shared fixture JSON across tracks when the course has tracks. Questions: check-your-understanding list with at least one predict and one "what would break if". +- `parseFiles(text): { files, warnings }` reads `=== FILE: ===` … `=== END FILE ===`; rejects absolute and `..` paths. +- `checkHouseStyle(req, files): string[]` warnings for missing beats/tiers. +- `generateDraft({ provider, req, course, ctx, sink, authorId }): Promise<{ location, paths, warnings }>`; every Markdown file starts with a draft marker comment. + +- [ ] Tests: prompt for each style contains its beats; parse handles two files and rejects `../x`; fake provider output written by the fs sink lands under `/drafts/sample/…` and nothing changes in the course folder; missing tiers give warnings. +- [ ] Implement; commit `feat(ai): house-style generator writing drafts through a DraftSink`. + +### Task 9: Feature routes and jobs + +**Files:** `src/feature.ts`, test `test/feature.test.ts` + +**Interfaces:** `aiFeature(ctx, opts?: { fetch?; draftSink?; providerFactory?(cfg, key): AiProvider }): Feature` with: +- `GET /api/ai/status?courseId=` → `{ enabled, source: "byok" | "server" | null, mentor?: { available, on } }`. +- `GET|PUT /api/ai/admin/config` (owner), `POST /api/ai/admin/mcp/test` (owner; lists a server's tools). +- `GET /api/ai/account`, `PUT /api/ai/account/key`, `DELETE /api/ai/account/key` (signed in). +- `PUT /api/ai/courses/:courseId/mentor` `{ off }`. +- `POST /api/ai/courses/:courseId/chat` `{ message, conversationId? }` → `application/x-ndjson` stream: `{type:"meta", conversationId, mentorOn, citations}`, `text`, `tool`, `done`, `error`. Stores user and assistant turns in `ai_conversations`/`ai_messages`. +- `GET /api/ai/courses/:courseId/conversations`, `GET|DELETE /api/ai/conversations/:id`. +- `GET /api/ai/studio/sources?courseId=`, `POST /api/ai/studio/generate` (enqueues job `ai.generate`), `GET /api/ai/studio/jobs/:id`. +- Job `ai.generate` runs `generateDraft` with the requesting user's provider. + +- [ ] Tests through `createApp` with a fake `AuthService`: status disabled by default; admin config hides the key; non-owner gets 403; BYOK enables the assistant for that user only; chat with a fake provider streams text and `meta.citations` include `lessons/04-exercise.md`; system prompt holds mentor rules, and not after `PUT mentor {off:true}`; anonymous on a restricted course gets 404; generate as a learner gets 403; the job handler writes drafts under `/drafts/sample`. +- [ ] Implement; commit `feat(ai): HTTP routes, chat streaming and generation job`. + +### Task 10: Web UI + +**Files:** `src/web/{index.ts,api.ts,AssistantPage.tsx,AdminAiPage.tsx,AccountAiPage.tsx,GeneratePage.tsx,ai.css}`, `apps/web/src/features.ts` (one line), test `test/web.test.tsx` + +- `aiWeb: WebFeature` routes `/c/:courseId/assistant`, `/admin/ai` (nav admin "AI"), `/account/ai`, `/studio/generate`. +- Assistant: course name, mentor segmented toggle, the plain banner "Mentor mode off: I'll give full answers" when off, conversation rows, sources as rows with paths, composer with a primary Send pill. Disabled state explains how to enable AI. +- Admin: provider kind, model, base URL, write-only key, MCP server rows. Account: BYOK form and remove. Generate: course, kind, style, sources checklist, topic, instructions, job progress, list of draft files with location. +- `readNdjson(stream)` client helper. + +- [ ] Tests: `readNdjson` splits across chunk boundaries; `renderToString` of the banner component shows the exact sentence when off and nothing when on. +- [ ] Implement, check in headless Chrome via `packages/feature-ai/scripts/dev.ts` on port 3464; commit `feat(ai): assistant panel, AI settings and studio generator UI`. + +### Task 11: Contract requests and report + +- [ ] `docs/contract-requests/ai.md`: feature factories in `startServer`; tool-call fields on `ChatMessage`; per-course `ai_settings` key; studio `createDraftWriter` wiring; cache layout for deck text and transcripts; learner lesson URL. +- [ ] Run `bun test`, `bun run typecheck`, the Postgres run; write `wp-ai-report.md`.