STCourses/docs/superpowers/plans/2026-10-30-ai.md

WP-D AI Implementation Plan

For agentic workers: executed inline by the WP-D agent with TDD (failing test first), one commit per task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Opt-in AI for STCourses: a course-aware learner assistant (retrieval, course tools, MCP tools, mentor mode) and a studio generator that writes house-style drafts. AI is off until an admin configures a server provider or a user saves their own key (BYOK).

Architecture:

• packages/feature-ai exports aiFeature(ctx, opts?): Feature, aiWeb: WebFeature and migrationsDirs.
• Providers implement the frozen AiProvider contract with plain fetch (no SDK): Anthropic Messages API (streaming SSE, tool use, default model claude-opus-5-5) and OpenAI-compatible chat completions (covers Ollama).
• Settings live in the existing ai_settings and user_api_keys tables (no new tables). Secrets are AES-GCM encrypted with a key derived by HKDF-SHA-256 from ctx.config.secret; they never leave the server.
• Retrieval is an in-process BM25 index per loaded course (course Markdown, transcripts, code, deck/transcript text under <dataDir>/cache/<courseId>/), plus the requesting user's notes at query time. Solution files are never indexed.
• The assistant runs a small tool loop over search_course, read_lesson, list_notes and admin-configured MCP servers (stdio and streamable HTTP), and streams NDJSON events to the browser.
• The generator builds house-style prompts (Pure Snake module shape, Systemic Design beats), parses === FILE: … === blocks from the model output, and hands them to a DraftSink. The default sink writes to <dataDir>/drafts/<courseId>/<runId>/; Phase 2 swaps in the studio's createDraftWriter(ctx).

Tech Stack: Bun 1.3, TypeScript strict, WebCrypto, Bun.SQL, Bun.spawn, React 19 + React Router 6, @stc/ui primitives.

Spec: sections 9.2, 7, 2 (anti-goals). Brief: .superpowers/sdd/2026-10-30-stcourses/phase-1-brief.md (WP-D).

Global Constraints

Inherited from docs/superpowers/plans/2026-10-30-stcourses.md (Bun only, plain CSS with tokens, both databases, AGPL, never write into the reference course repos, never commit licensed content, keyboard rules, commit trailer). Package-specific:

• AI is off by default. No provider configured and no BYOK key means every AI route answers enabled: false / 409, and the UI says how to turn it on.
• Keys never go to the client. API responses carry hasApiKey: boolean, never ciphertext or plaintext.
• No real network in tests. Providers take an injectable fetch; MCP tests use a local fake server (stdio child process and Bun.serve on port 0).
• Mentor mode is on by default and its rules go into the system prompt; when off, the UI shows "Mentor mode off: I'll give full answers".
• Generated material is never published. The generator only writes through a DraftSink.

Review Focus

1. Secrets: BYOK and server keys are encrypted at rest (AES-GCM, random IV, HKDF key from STC_SECRET), a wrong secret fails to decrypt, and no route returns key material. Tests: Tasks 1, 2, 9.
2. Mentor anti-goal: mentor rules are in the system prompt unless the learner switched them off for that course; solution files (browser.solution, solutions/, reference/, expansion/solution/) are never retrieved or readable by tools. Tests: Tasks 5, 7.
3. Generation never publishes: drafts land only under the sink location, paths are sanitised (no .., no absolute paths). Test: Task 8.
4. Access control: assistant routes require that the user can view the course (canView); generator routes require owner, the author role or a course_authors row; admin routes require owner. Test: Task 9.
5. Thinking blocks: the Anthropic provider replays an assistant turn's raw content blocks (thinking, text, tool_use) unchanged inside one tool loop. Test: Task 3.

---

Task 1: Package scaffold and secret box

Files: packages/feature-ai/{package.json,src/index.ts,src/crypto.ts}, test packages/feature-ai/test/crypto.test.ts

Interfaces (Produces):

• createSecretBox(secret: string): { encrypt(plain: string): Promise<string>; decrypt(token: string): Promise<string> }. HKDF-SHA-256 (salt "stcourses/byok", info "aes-gcm-256/v1") → AES-GCM-256; token v1.<b64 iv>.<b64 ciphertext>.

• migrationsDirs: string[] (empty: the package adds no tables).

• Tests: round trip; two encryptions differ (random IV); a box with another secret rejects; a tampered token rejects.

• Implement; commit feat(ai): package scaffold and AES-GCM secret box.

Task 2: Settings repository

Files: src/settings.ts, test test/settings.test.ts (uses forEachDb from packages/db/test/helpers.ts)

Interfaces:

• ProviderKind = "anthropic" | "openai"; ProviderConfig = { kind, model?, baseUrl?, effort? }.

• McpServerConfig = { id, name, transport: "stdio" | "http", command?, args?, env?, url?, headers?, courseIds?: string[] } (no courseIds = global).

• aiSettingsRepo(db, box):

  • getServer(): Promise<{ provider: (ProviderConfig & { hasApiKey }) | null; mcpServers }>, saveServer({ provider: (ProviderConfig & { apiKey?: string; clearApiKey?: boolean }) | null, mcpServers }), serverApiKey(): Promise<string | null>.
  • getUser(userId): Promise<{ byok: (ProviderConfig & { hasApiKey }) | null; mentorOff: Record<courseId, boolean> }>, saveUserKey(userId, cfg & { apiKey }), removeUserKey(userId), userApiKey(userId, kind), setMentorOff(userId, courseId, off), isMentorOff(userId, courseId).

• Storage: ai_settings row ('server','') holds { provider, apiKeyCipher, mcpServers }; row ('user', userId) holds { byok, mentorOff }. The per-course flag lives in config_json because the frozen primary key is (scope, owner_id); the mentor_off column mirrors "off for every course" (always 0 for now). Contract request filed. Keys go in user_api_keys encrypted. Upserts are select-then-insert-or-update in a transaction.

• Tests (both DBs): empty server config is null provider; saving a key stores ciphertext only (raw row does not contain the key) and serverApiKey() returns it; omitting apiKey on save keeps the old key; BYOK save/remove; mentor toggle per course is independent.

• Implement; commit feat(ai): settings repository for server config, BYOK keys and mentor toggles.

Task 3: SSE parsing and the Anthropic provider

Files: src/providers/sse.ts, src/providers/anthropic.ts, test test/anthropic.test.ts

Interfaces:

• readSse(body: ReadableStream<Uint8Array>): AsyncIterable<{ event: string; data: string }>.

• createAnthropicProvider({ apiKey, model = "claude-opus-5-5", baseUrl = "https://api.anthropic.com", effort?, maxTokens = 64000, fetch? }): AiProvider.

  • POST /v1/messages with stream: true, headers x-api-key, anthropic-version: 2023-06-01. No thinking, temperature or forced tool_choice (Opus 5.5 rejects them). On the first-party endpoint, fallbacks: "default" with beta header server-side-fallback-2026-07-01.
  • Message mapping: system messages join into system; an assistant ChatMessage with toolCallId and name is a tool call (content = JSON input); tool messages become tool_result blocks; consecutive same-role blocks merge.
  • The provider remembers each response's full content blocks (thinking with signature, text, tool_use) by tool_use id and replays them unchanged for the matching assistant tool-call messages.
  • Streams { text } per text_delta, { toolCall } per finished tool_use block; HTTP errors and error events throw with the API message; stop_reason: "refusal" yields a plain sentence.

• Tests with a fake fetch that returns canned SSE: text streams in order; a tool_use with split input_json_delta yields one parsed toolCall; request body has model, system, tools as input_schema, no thinking/temperature; a second call after a tool round replays the thinking block with its signature unchanged before tool_result; non-2xx throws with the error message.

• Implement; commit feat(ai): Anthropic provider over fetch with streaming tool use.

Task 4: OpenAI-compatible provider

Files: src/providers/openai.ts, src/providers/index.ts, test test/openai.test.ts

Interfaces:

• createOpenAiProvider({ baseUrl, apiKey?, model, fetch? }): AiProvider. POST <baseUrl>/chat/completions, stream: true, tools as { type: "function", function: { name, description, parameters } }. Assistant tool-call messages map to tool_calls; tool messages to { role: "tool", tool_call_id }. Accumulates delta.tool_calls by index and emits on finish or [DONE].

• createProvider(cfg: ProviderConfig, apiKey: string | null, fetchImpl?): AiProvider.

• Tests: text deltas; fragmented tool-call arguments; no Authorization header without a key (Ollama); request mapping of a tool round.

• Implement; commit feat(ai): OpenAI-compatible provider (covers Ollama).

Task 5: Retrieval

Files: src/retrieval/{bm25.ts,course-index.ts}, test test/retrieval.test.ts

Interfaces:

• Bm25<T> with add(text, item), search(query, k): { item, score }[].

• Chunk = { path, kind: "lesson" | "transcript" | "markdown" | "code" | "deck" | "note", text, heading?, lessonId?, lessonTitle? }.

• buildCourseIndex(course: LoadedCourse, dataDir: string): Promise<CourseIndex>; CourseIndex.search(query, k, extra?: Chunk[]): Chunk[]; isSolutionPath(course, rel): boolean.

• getCourseIndex(course, dataDir) caches per LoadedCourse object (a repository reload yields new objects) with a 5-minute TTL.

• Sources: every .md under the course root (depth-limited, skipping .git, node_modules), manifest code dirs and common code extensions (size cap 200 KB per file, 2 000 files per course), .md/.txt/.vtt/.srt/.json text under <dataDir>/cache/<courseId>/. Markdown splits at headings; long chunks split at ~1 500 chars. Paths are mapped to lessons through file, transcript and hints.file (anchor heading when it matches).

• Never indexed: browser.solution files and any path segment solutions, solution, reference under expansion/, or reference/ in Systemic-Design style modules (see isSolutionPath).

• Tests on the sample fixture: "sum of its arguments" hits lessons/04-exercise.md with lessonId m2-text or m2-ex-add; exercises/add/solution.js never appears; a deck text file in a temp cache/sample/decks/d1/page-1.txt is found with kind deck; note chunks passed as extra rank.

• Implement; commit feat(ai): BM25 course retrieval over lessons, transcripts, code, deck text and notes.

Task 6: MCP client

Files: src/mcp/client.ts, test test/mcp.test.ts, fixture test/fixtures/fake-mcp.ts

Interfaces:

• connectMcp(cfg: McpServerConfig, opts?: { fetch?, timeoutMs? }): Promise<McpClient>; McpClient = { tools(): Promise<{ name, description, inputSchema }[]>; call(name, args): Promise<string>; close(): Promise<void> }.

• JSON-RPC 2.0: initialize (protocolVersion 2025-06-18), notifications/initialized, tools/list, tools/call. stdio = newline-delimited JSON over Bun.spawn; streamable HTTP = POST with Accept: application/json, text/event-stream, JSON or SSE replies, Mcp-Session-Id echoed.

• mcpServersFor(courseId, servers) selects global servers plus ones listing the course.

• Tests: the fake server over stdio lists echo and tools/call returns its text; the same over HTTP (fake via Bun.serve on port 0, replying with SSE for tools/call) with the session id echoed; a server that exits gives a rejected promise, not a hang.

• Implement; commit feat(ai): MCP client over stdio and streamable HTTP.

Task 7: Assistant core

Files: src/assistant/{prompt.ts,tools.ts,loop.ts}, test test/assistant.test.ts

Interfaces:

• buildSystemPrompt({ courseTitle, mentorText: string | null, mentorOn, sources: Chunk[], lessonTitle? }): string. With mentor on and text present: a ## Course mentor rules section with the file's text. With mentor off: a line saying full answers are allowed. Sources are numbered [n] path (lesson title) with their text; the prompt asks for citations by path.

• courseTools({ ctx, course, userId, index }): ToolSpec[] plus run(name, input) for search_course, read_lesson (refuses solution files) and list_notes.

• runAssistant({ provider, messages, tools, signal, maxSteps = 6 }): AsyncIterable<AssistantEvent> where AssistantEvent = { type: "text", text } | { type: "tool", name } | { type: "error", message } | { type: "done", text }. Tool calls become assistant messages with toolCallId/name and tool result messages; unknown tools and tool errors are fed back as error text.

• Citations: citationsFor(chunks): { path, lessonId?, lessonTitle?, kind }[] deduplicated.

• Tests: mentor on puts the sample MENTOR.md text in the system prompt; mentor off omits it and states full answers; a fake provider that calls search_course then answers gets the tool result and the loop ends with text; read_lesson on the exercise returns markdown; solution paths are refused.

• Implement; commit feat(ai): assistant prompt, course tools and tool loop.

Task 8: Generator and draft sink

Files: src/generator/{templates.ts,parse.ts,generate.ts,draft-sink.ts}, test test/generator.test.ts

Interfaces:

• DraftSink = { writeDraftFiles(courseId, authorId, files: { path, content }[], message): Promise<{ location: string; paths: string[] }> }; createFsDraftSink(dataDir) writes to <dataDir>/drafts/<courseId>/<runId>/ plus DRAFT.json (author, message, time). Same signature as the studio's createDraftWriter(ctx).writeDraftFiles.

• GenerateRequest = { courseId, kind: "document" | "exercise" | "questions", style: "pure-snake" | "systemic-design", topic, sources: ({ kind: "lesson", lessonId } | { kind: "file", path } | { kind: "text", label, text })[], instructions? }.

• buildGeneratorPrompt(req, material, course): house-style system prompt (module shape: why, objectives, concepts, puzzle with "Think about it first" fold, everyday-programming bridge, exercises, hints as nudge / stronger hint / near-solution folds, check your understanding, in production, further reading; or Observe → Predict (+Discussion fold) → Spec → Build (Hint 1 — nudge / Hint 2 — concept / Hint 3 — pseudo-code) → Name it → Expansion Drop (+what this drop measures) → Further reading). Exercises: README.md, starter.js, tests.js (export const tests per the ExerciseTest contract), solution.js, shared fixture JSON across tracks when the course has tracks. Questions: check-your-understanding list with at least one predict and one "what would break if".

• parseFiles(text): { files, warnings } reads === FILE: <path> === … === END FILE ===; rejects absolute and .. paths.

• checkHouseStyle(req, files): string[] warnings for missing beats/tiers.

• generateDraft({ provider, req, course, ctx, sink, authorId }): Promise<{ location, paths, warnings }>; every Markdown file starts with a draft marker comment.

• Tests: prompt for each style contains its beats; parse handles two files and rejects ../x; fake provider output written by the fs sink lands under <dataDir>/drafts/sample/… and nothing changes in the course folder; missing tiers give warnings.

• Implement; commit feat(ai): house-style generator writing drafts through a DraftSink.

Task 9: Feature routes and jobs

Files: src/feature.ts, test test/feature.test.ts

Interfaces: aiFeature(ctx, opts?: { fetch?; draftSink?; providerFactory?(cfg, key): AiProvider }): Feature with:

• GET /api/ai/status?courseId= → { enabled, source: "byok" | "server" | null, mentor?: { available, on } }.

• GET|PUT /api/ai/admin/config (owner), POST /api/ai/admin/mcp/test (owner; lists a server's tools).

• GET /api/ai/account, PUT /api/ai/account/key, DELETE /api/ai/account/key (signed in).

• PUT /api/ai/courses/:courseId/mentor { off }.

• POST /api/ai/courses/:courseId/chat { message, conversationId? } → application/x-ndjson stream: {type:"meta", conversationId, mentorOn, citations}, text, tool, done, error. Stores user and assistant turns in ai_conversations/ai_messages.

• GET /api/ai/courses/:courseId/conversations, GET|DELETE /api/ai/conversations/:id.

• GET /api/ai/studio/sources?courseId=, POST /api/ai/studio/generate (enqueues job ai.generate), GET /api/ai/studio/jobs/:id.

• Job ai.generate runs generateDraft with the requesting user's provider.

• Tests through createApp with a fake AuthService: status disabled by default; admin config hides the key; non-owner gets 403; BYOK enables the assistant for that user only; chat with a fake provider streams text and meta.citations include lessons/04-exercise.md; system prompt holds mentor rules, and not after PUT mentor {off:true}; anonymous on a restricted course gets 404; generate as a learner gets 403; the job handler writes drafts under <dataDir>/drafts/sample.

• Implement; commit feat(ai): HTTP routes, chat streaming and generation job.

Task 10: Web UI

Files: src/web/{index.ts,api.ts,AssistantPage.tsx,AdminAiPage.tsx,AccountAiPage.tsx,GeneratePage.tsx,ai.css}, apps/web/src/features.ts (one line), test test/web.test.tsx

• aiWeb: WebFeature routes /c/:courseId/assistant, /admin/ai (nav admin "AI"), /account/ai, /studio/generate.

• Assistant: course name, mentor segmented toggle, the plain banner "Mentor mode off: I'll give full answers" when off, conversation rows, sources as rows with paths, composer with a primary Send pill. Disabled state explains how to enable AI.

• Admin: provider kind, model, base URL, write-only key, MCP server rows. Account: BYOK form and remove. Generate: course, kind, style, sources checklist, topic, instructions, job progress, list of draft files with location.

• readNdjson(stream) client helper.

• Tests: readNdjson splits across chunk boundaries; renderToString of the banner component shows the exact sentence when off and nothing when on.

• Implement, check in headless Chrome via packages/feature-ai/scripts/dev.ts on port 3464; commit feat(ai): assistant panel, AI settings and studio generator UI.

Task 11: Contract requests and report

• docs/contract-requests/ai.md: feature factories in startServer; tool-call fields on ChatMessage; per-course ai_settings key; studio createDraftWriter wiring; cache layout for deck text and transcripts; learner lesson URL.
• Run bun test, bun run typecheck, the Postgres run; write wp-ai-report.md.