diff --git a/docs/contract-requests/ai.md b/docs/contract-requests/ai.md index c29e6af..7afe0a5 100644 --- a/docs/contract-requests/ai.md +++ b/docs/contract-requests/ai.md @@ -35,17 +35,30 @@ provider keeps each response's raw content blocks (thinking with signature, text and replays them unchanged. Proposal: `ChatMessage.toolCalls?: { id, name, input }[]` and an opaque `providerData?: unknown` the provider can round-trip. +**Settled in v1.1 (`feat/v11-ai`):** `ChatMessage.toolCalls?: ToolCall[]` and `providerData?: unknown` are in +`@stc/contracts`, and providers may yield `{ providerData }` once per turn. The assistant loop appends one assistant +message per model turn. Anthropic returns `{ anthropic: }` and replays it verbatim (no in-memory map); +OpenAI maps `toolCalls` to `tool_calls`. + ## 4. Per-course mentor toggle in `ai_settings` The frozen key is `(scope, owner_id)` with one `mentor_off` column, so a per-course toggle cannot be a row. The flag lives in the user row's `config_json.mentorOff[courseId]`; `mentor_off` stays 0 (reserved for "off everywhere"). Proposal: add `course_id TEXT NOT NULL DEFAULT ''` to the key, or a small `ai_mentor_prefs(user_id, course_id, off)` table. +**Settled in v1.1:** table `ai_mentor_prefs(user_id, course_id, off, updated_at)` from +`packages/feature-ai/migrations/0002_ai_mentor_prefs.sql`. `upgradeAiData` (src/upgrade.ts) copies v1 +`config_json.mentorOff` toggles once (marker `ai_data_v11` in `schema_migrations`). + ## 5. Citations on `ai_messages` `ai_messages` has no column for an assistant message's sources. They are stored as an extra row with role `sources` (JSON) right after the assistant row and folded back when reading. Proposal: `ai_messages.meta_json TEXT`. +**Settled in v1.1:** column `ai_messages.citations_json` from `0001_ai_message_citations.sql`, which also folds v1 +`sources` rows into the assistant row before them. (MCP `env`/`headers` are now encrypted at rest too, masked in the +admin API.) + ## 6. Derived text cache layout (shared with WP-B) Retrieval reads the importer's layout: diff --git a/docs/superpowers/plans/2026-10-30-v11-ai.md b/docs/superpowers/plans/2026-10-30-v11-ai.md new file mode 100644 index 0000000..845f6d4 --- /dev/null +++ b/docs/superpowers/plans/2026-10-30-v11-ai.md @@ -0,0 +1,94 @@ +# STCourses v1.1 — Work package A: deferred AI contract requests 3–5 + +> **For agentic workers:** executed inline with superpowers:executing-plans and TDD. Steps use checkbox (`- [ ]`) syntax. One commit per task. + +**Goal:** settle AI contract requests 3, 4 and 5 (`docs/contract-requests/ai.md`) and the v1 deferred minor "MCP `env`/`headers` in plaintext": +- (3) structured tool calls on `ChatMessage`, with the Anthropic and OpenAI providers mapping them natively; +- (4) per-course mentor toggles in their own table, migrated from `ai_settings.config_json`; +- (5) a citations column on `ai_messages`, migrated from the v1 `sources` rows; +- MCP `env` and `headers` encrypted at rest (same AES-GCM/HKDF box as BYOK keys) and only ever returned masked. + +**Spec:** section 9.2. **Parent plan:** `docs/superpowers/plans/2026-10-30-v1-1.md` (Global Constraints, package A). + +## Global Constraints + +- Everything in `CLAUDE.md` "Project rules" and the v1.1 Global Constraints applies. +- Every DB change is a new portable migration in `packages/feature-ai/migrations/` that runs unchanged on SQLite and Postgres, and upgrades a database already on the v1 schema. File names carry the `ai` prefix (core `migrate()` requires unique names across packages). +- An upgrade test seeds v1 rows (core `0001_init.sql` only), runs the AI migrations and the data upgrade, and asserts the result, on SQLite and on Postgres (`STC_TEST_PG_URL`). +- v1 safeguards that must keep holding (each already has tests; they stay green and unchanged in meaning): + - BYOK keys only reach the official endpoints or allowlisted origins, without redirects; + - error bodies from BYOK calls are never echoed to the user; + - stdio MCP processes get PATH, HOME, LANG plus exactly the configured variables; + - the solution guard works on canonical paths; + - the banner text is exactly "Mentor mode off: I'll give full answers"; + - tests use fakes only, no network. +- Gates per task: `bun test`, `bun run typecheck`, and the Postgres run with `STC_TEST_PG_URL=postgres://postgres:stc@127.0.0.1:55433/postgres`. +- Manual check: dev server on port 3493 with `STC_DATA_DIR=/tmp/stc-v11-ai`. Never port 3480 or `/home/user/STCourses/data`. + +## Design decisions + +1. **Schema vs data migrations.** Pure-SQL migrations cover what portable SQL can do: the new table, the new column, and folding v1 `sources` rows into `citations_json` (a correlated subquery that runs on both engines). Two upgrades cannot be portable SQL: reading `config_json.mentorOff` needs engine-specific JSON functions, and encrypting MCP secrets needs `STC_SECRET`. These run in code, in `upgradeAiData(db, box)`: one transaction, idempotent, recorded in `schema_migrations` under the name `ai_data_v11` (no `.sql` suffix, so it cannot clash with a file). The settings repo awaits it once (memoised; retried after a failure) before its first read or write, and `aiFeature` starts it at construction so plaintext does not linger until the first request. +2. **Citations column name.** `ai_messages.citations_json TEXT NULL` (the parent plan asks for "a citations column"; a typed name beats a generic `meta_json`). v1 `sources` rows are folded into the assistant row immediately before them (by id order, the order v1 reads in) and then deleted. +3. **Mentor table.** `ai_mentor_prefs (user_id, course_id, off INTEGER, updated_at, PK (user_id, course_id))`. A row exists only while the mentor is off (turning it back on deletes the row). `ai_settings.mentor_off` stays reserved for "off everywhere" and is not read. The upgrade copies every `mentorOff[courseId] === true` and removes the `mentorOff` key from the user's `config_json`. +4. **Tool calls on `ChatMessage`.** Contract change in `@stc/contracts`: + - `ToolCall = { id, name, input }`; + - `ChatMessage.toolCalls?: ToolCall[]` (assistant turn) and `ChatMessage.providerData?: unknown` (opaque, round-tripped by the provider that produced it); + - `AiProvider.chat` may also yield `{ providerData }` once per turn. + The assistant loop appends one assistant message per model turn (`content` = the turn's text, `toolCalls`, `providerData`), then one `tool` message per call (`toolCallId`, `name`). The v1 one-message-per-call encoding (`assistant` + `toolCallId`) is removed. + - Anthropic: `providerData = { anthropic: }` (thinking with signature, text, tool_use). An assistant message carrying it is replayed verbatim; otherwise it maps to text + `tool_use` blocks. The in-memory `rawByToolId` map goes away, so a provider instance holds no conversation state. + - OpenAI: assistant `toolCalls` map to `tool_calls` (`function.arguments` = JSON of input) with `content` = text or null. No `providerData`. +5. **MCP secrets at rest.** In `config_json` each MCP server keeps its non-secret fields; `env` and `headers` move into `secretCipher` = `box.encrypt(JSON.stringify({ env, headers }))`. The admin GET returns `env` and `headers` with the keys and masked values: `••••` plus the last 4 characters when the value has at least 12 characters, otherwise `••••••••`. On save, a posted value equal to the mask of the stored value for the same server id and key keeps the stored value; any other value starting with `••••` is refused with 400 ("re-enter the value"), so a mask is never stored as a secret. The MCP test endpoint resolves masks the same way. Only server code gets full values (`mcpServersWithSecrets()`). If the secret cannot be decrypted (changed `STC_SECRET`), the server is listed without `env`/`headers` and the client starts it without them. + +## Tasks + +### Task 1: AI migrations folder, test helper and citations column + +**Files:** +- Create: `packages/feature-ai/migrations/0001_ai_message_citations.sql` +- Modify: `packages/feature-ai/src/index.ts` (`migrationsDirs`), `packages/feature-ai/scripts/dev.ts` (migrate with the AI folder), `packages/feature-ai/src/feature.ts` (write/read `citations_json`, no more `sources` rows) +- Test: `packages/feature-ai/test/upgrade.test.ts` (new), `test/helpers.ts` (`forEachAiDb`), `test/feature.test.ts` + +- [ ] Write `upgrade.test.ts`: on a v1 database (core migrations only) seed a conversation with user, assistant, sources, user, assistant (no sources), and an orphan sources row; run `migrate(db, { "feature-ai": migrationsDirs })`; assert the first assistant row has the citations JSON, the second has NULL, no `sources` rows remain, and a second migrate applies nothing. Add a feature test that a stored assistant message has `citations_json` set and no `sources` row is written. +- [ ] Run: fails (no migrations folder). +- [ ] Implement the SQL (`ALTER TABLE ai_messages ADD COLUMN citations_json TEXT`; correlated `UPDATE` picking the next message by id when it is a `sources` row; `DELETE ... WHERE role = 'sources'`), export the folder, switch `feature.ts` to the column, switch AI tests that touch the DB to `forEachAiDb`. +- [ ] Run tests on SQLite and Postgres; typecheck; commit `feat(ai): citations column on ai_messages (contract request 5)`. + +### Task 2: Mentor preferences table and the data upgrade runner + +**Files:** +- Create: `packages/feature-ai/migrations/0002_ai_mentor_prefs.sql`, `packages/feature-ai/src/upgrade.ts` +- Modify: `packages/feature-ai/src/settings.ts`, `src/feature.ts`, `src/index.ts` +- Test: `test/upgrade.test.ts`, `test/settings.test.ts` + +- [ ] Tests: v1 user rows with `mentorOff: { sample: true, other: false }` plus a BYOK provider become one `ai_mentor_prefs` row (`u1`, `sample`) and a `config_json` without `mentorOff` but with `byok` intact; `isMentorOff` reads the table; running the upgrade twice changes nothing; a row with invalid JSON is left alone. Settings test: toggling writes and deletes table rows and never touches `config_json.mentorOff`. +- [ ] Run: fails. +- [ ] Implement table SQL, `upgradeAiData(db, box)` (mentor part plus the `ai_data_v11` marker), repo `ready()` memo, mentor methods on the table, `getUser().mentorOff` from the table. +- [ ] Gates; commit `feat(ai): per-course mentor preferences table (contract request 4)`. + +### Task 3: Encrypt MCP env and headers at rest, masked in the UI + +**Files:** +- Modify: `src/settings.ts`, `src/upgrade.ts`, `src/feature.ts` (admin PUT 400 on stale masks, chat uses `mcpServersWithSecrets`, MCP test route resolves masks), `src/web/AdminAiPage.tsx` (one-line hint) +- Test: `test/upgrade.test.ts`, `test/settings.test.ts`, `test/feature.test.ts` + +- [ ] Tests: v1 server row with plaintext `env`/`headers` is encrypted by the upgrade (raw `config_json` no longer contains the values; `mcpServersWithSecrets()` returns them); `getServer()` returns masked values only; saving the masked values back keeps the stored secrets; saving a new value replaces it; a stray `••••` value is refused; the admin config API never contains a secret value; the MCP test route with masked values reaches the server with the real header (fake fetch). +- [ ] Run: fails. +- [ ] Implement. +- [ ] Gates; commit `feat(ai): encrypt MCP env and headers at rest and mask them in the admin API`. + +### Task 4: Structured tool calls on ChatMessage + +**Files:** +- Modify: `packages/contracts/src/ai.ts`, `packages/contracts/test/types.test.ts`, `packages/feature-ai/src/assistant/loop.ts`, `src/providers/anthropic.ts`, `src/providers/openai.ts` +- Test: `test/anthropic.test.ts`, `test/openai.test.ts`, `test/assistant.test.ts`, `test/helpers.ts` + +- [ ] Tests: the loop appends one assistant message with `toolCalls` (two calls in one turn) and the provider's `providerData`, then a tool message per call; Anthropic replays `providerData` blocks verbatim (thinking signature intact) and maps a message without it to text + `tool_use` blocks; a fresh provider instance replays history it did not produce (no in-memory state); OpenAI maps `toolCalls` to one assistant message with `tool_calls` and text content. +- [ ] Run: fails. +- [ ] Implement contract fields, provider mapping, loop change; remove `rawByToolId` and the legacy encoding. +- [ ] Gates; commit `feat(ai): structured tool calls on ChatMessage (contract request 3)`. + +### Task 5: Docs and manual check + +- [ ] Mark requests 3–5 settled in `docs/contract-requests/ai.md`; update `settings.ts`/`index.ts` comments. +- [ ] Dev server on 3493 with `STC_DATA_DIR=/tmp/stc-v11-ai`: migrations apply on a fresh DB, admin config shows masked MCP values. +- [ ] Full gates; commit `docs(ai): contract requests 3-5 settled in v1.1`. diff --git a/packages/contracts/src/ai.ts b/packages/contracts/src/ai.ts index e0815c1..44c591c 100644 --- a/packages/contracts/src/ai.ts +++ b/packages/contracts/src/ai.ts @@ -1,3 +1,19 @@ -export interface ChatMessage { role: "system" | "user" | "assistant" | "tool"; content: string; toolCallId?: string; name?: string } +/** One tool call the model asked for in an assistant turn. */ +export interface ToolCall { id: string; name: string; input: unknown } +/** + * A chat message. An assistant turn that calls tools carries them in `toolCalls` (v1.1, contract request 3), with + * its text in `content`; each result follows as a `tool` message with `toolCallId` (and `name`). + * `providerData` is opaque: the provider that produced the turn yields it and gets it back unchanged (for example + * Anthropic's raw content blocks with thinking signatures). Other providers ignore it. + */ +export interface ChatMessage { + role: "system" | "user" | "assistant" | "tool"; + content: string; + toolCallId?: string; + name?: string; + toolCalls?: ToolCall[]; + providerData?: unknown; +} export interface ToolSpec { name: string; description: string; inputSchema: Record } -export interface AiProvider { id: string; chat(messages: ChatMessage[], opts: { tools?: ToolSpec[]; signal?: AbortSignal }): AsyncIterable<{ text?: string; toolCall?: { id: string; name: string; input: unknown } }> } +/** `chat` streams text and tool calls, and may yield `providerData` once per turn for the assistant message. */ +export interface AiProvider { id: string; chat(messages: ChatMessage[], opts: { tools?: ToolSpec[]; signal?: AbortSignal }): AsyncIterable<{ text?: string; toolCall?: ToolCall; providerData?: unknown }> } diff --git a/packages/contracts/test/types.test.ts b/packages/contracts/test/types.test.ts index a49a1fa..88fc793 100644 --- a/packages/contracts/test/types.test.ts +++ b/packages/contracts/test/types.test.ts @@ -1,6 +1,6 @@ import { expect, test } from "bun:test"; import type { - AiProvider, AppConfig, AuthService, AuthUser, Assert, ChatMessage, ExerciseRunner, ExerciseTest, + AiProvider, AppConfig, AuthService, AuthUser, Assert, ChatMessage, ToolCall, ExerciseRunner, ExerciseTest, Feature, ImportResult, ImportSource, JobContext, JobHandler, JobQueue, JobRecord, RequestContext, RouteHandler, ServerContext, TestResult, ToolSpec, WebFeature, WebRoute, } from "../src/index"; @@ -38,10 +38,12 @@ test("contract interfaces accept well-formed literals", async () => { run: async () => [{ name: "t", passed: true } satisfies TestResult], }; const msg: ChatMessage = { role: "user", content: "hi" }; + const call: ToolCall = { id: "1", name: "x", input: {} }; + const turn: ChatMessage = { role: "assistant", content: "", toolCalls: [call], providerData: { any: "thing" } }; const tool: ToolSpec = { name: "x", description: "d", inputSchema: {} }; const provider: AiProvider = { id: "p", - async *chat() { yield { text: "hi" }; yield { toolCall: { id: "1", name: "x", input: {} } }; }, + async *chat() { yield { text: "hi" }; yield { toolCall: { id: "1", name: "x", input: {} } }; yield { providerData: [] }; }, }; const source: ImportSource = { kind: "pdf-deck", path: "/tmp/a.pdf", options: {} }; const result: ImportResult = { courseId: "c", manifestPath: "/tmp/stcourse.yaml", warnings: [] }; @@ -53,5 +55,5 @@ test("contract interfaces accept well-formed literals", async () => { expect(typeof feature.routes).toBe("object"); expect(await queue.enqueue("noop", {})).toBe("j1"); expect((await runner.run("", "", 1000))[0]!.passed).toBe(true); - void [rc, jc, ctx, assert, t, msg, tool, provider, source, result, web]; + void [rc, jc, ctx, assert, t, msg, turn, tool, provider, source, result, web]; }); diff --git a/packages/feature-ai/migrations/0001_ai_message_citations.sql b/packages/feature-ai/migrations/0001_ai_message_citations.sql new file mode 100644 index 0000000..4e04b03 --- /dev/null +++ b/packages/feature-ai/migrations/0001_ai_message_citations.sql @@ -0,0 +1,10 @@ +-- v1.1 (contract request 5): an assistant message's citations live in its own column. +-- v1 stored them as an extra row with role 'sources' right after the assistant row (ids sort in write order). +-- Fold each such row into the assistant row before it, then drop every 'sources' row. +ALTER TABLE ai_messages ADD COLUMN citations_json TEXT; +UPDATE ai_messages SET citations_json = ( + SELECT s.content FROM ai_messages s + WHERE s.conversation_id = ai_messages.conversation_id AND s.role = 'sources' + AND s.id = (SELECT MIN(n.id) FROM ai_messages n WHERE n.conversation_id = ai_messages.conversation_id AND n.id > ai_messages.id) +) WHERE role = 'assistant'; +DELETE FROM ai_messages WHERE role = 'sources'; diff --git a/packages/feature-ai/migrations/0002_ai_mentor_prefs.sql b/packages/feature-ai/migrations/0002_ai_mentor_prefs.sql new file mode 100644 index 0000000..13da201 --- /dev/null +++ b/packages/feature-ai/migrations/0002_ai_mentor_prefs.sql @@ -0,0 +1,7 @@ +-- v1.1 (contract request 4): per-course mentor toggles get their own table. A row exists only while the learner +-- has switched the course's mentor rules off. v1 kept them in ai_settings.config_json.mentorOff; reading that JSON +-- is not portable SQL, so upgradeAiData (src/upgrade.ts) copies it on first start. +CREATE TABLE ai_mentor_prefs ( + user_id TEXT NOT NULL, course_id TEXT NOT NULL, off INTEGER NOT NULL, updated_at TEXT NOT NULL, + PRIMARY KEY (user_id, course_id) +); diff --git a/packages/feature-ai/scripts/dev.ts b/packages/feature-ai/scripts/dev.ts index a31fe02..55e0af2 100644 --- a/packages/feature-ai/scripts/dev.ts +++ b/packages/feature-ai/scripts/dev.ts @@ -7,6 +7,7 @@ import { createCourseRepository } from "@stc/course-format"; import { connect, migrate } from "@stc/db"; import indexHtml from "../../../apps/web/index.html"; import { aiFeature } from "../src/feature"; +import { migrationsDirs } from "../src/migrations"; const config = loadConfig(); const who = process.env.STC_AI_DEV_USER ?? "owner"; @@ -32,7 +33,7 @@ const fake: AiProvider = { }; const db = await connect(config.databaseUrl); -await migrate(db); +await migrate(db, { "feature-ai": migrationsDirs }); const courses = await createCourseRepository(config.courseDirs); const ctx = { config, db, courses, auth } as ServerContext; const feature = aiFeature(ctx, process.env.STC_AI_DEV_FAKE ? { providerFactory: () => fake } : {}); diff --git a/packages/feature-ai/src/assistant/loop.ts b/packages/feature-ai/src/assistant/loop.ts index 0a3a0e0..3683d5c 100644 --- a/packages/feature-ai/src/assistant/loop.ts +++ b/packages/feature-ai/src/assistant/loop.ts @@ -1,4 +1,4 @@ -import type { AiProvider, ChatMessage } from "@stc/contracts"; +import type { AiProvider, ChatMessage, ToolCall } from "@stc/contracts"; import type { ToolSet } from "./tools"; export type AssistantEvent = @@ -9,7 +9,8 @@ export type AssistantEvent = /** * Runs the model with tools until it answers without calling a tool, or until maxSteps model calls. - * Mutates `messages` by appending the tool round trips, so callers can inspect them. + * Mutates `messages` by appending the tool round trips (an assistant message with `toolCalls`, then one `tool` + * message per call), so callers can inspect them. */ export async function* runAssistant(o: { provider: AiProvider; @@ -22,11 +23,13 @@ export async function* runAssistant(o: { let answer = ""; for (let step = 0; step < maxSteps; step++) { let text = ""; - const calls: { id: string; name: string; input: unknown }[] = []; + const calls: ToolCall[] = []; + let providerData: unknown; try { for await (const ev of o.provider.chat(o.messages, { tools: o.tools.specs, signal: o.signal })) { if (ev.text) { text += ev.text; yield { type: "text", text: ev.text }; } if (ev.toolCall) calls.push(ev.toolCall); + if (ev.providerData !== undefined) providerData = ev.providerData; } } catch (e) { yield { type: "error", message: (e as Error).message }; @@ -37,8 +40,10 @@ export async function* runAssistant(o: { yield { type: "done", text: answer }; return; } - if (text) o.messages.push({ role: "assistant", content: text }); - for (const c of calls) o.messages.push({ role: "assistant", content: JSON.stringify(c.input ?? {}), toolCallId: c.id, name: c.name }); + // One assistant message per model turn: its text, every tool call, and the provider's opaque data. + const turn: ChatMessage = { role: "assistant", content: text, toolCalls: calls }; + if (providerData !== undefined) turn.providerData = providerData; + o.messages.push(turn); for (const c of calls) { yield { type: "tool", name: c.name }; let result: string; diff --git a/packages/feature-ai/src/crypto.ts b/packages/feature-ai/src/crypto.ts index 10f6456..a139844 100644 --- a/packages/feature-ai/src/crypto.ts +++ b/packages/feature-ai/src/crypto.ts @@ -1,41 +1,82 @@ /** - * Secret box for API keys at rest: AES-GCM-256 with a key derived by HKDF-SHA-256 from STC_SECRET. - * Token format: `v1..`. + * Secret box for secrets at rest (API keys, MCP env and headers): AES-GCM-256 with keys derived by HKDF-SHA-256 + * from STC_SECRET. + * + * - `v2..` (v1.1): each purpose has its own HKDF key (`info` = + * `stcourses//v2`), and the AES-GCM additional data binds the token to the row it belongs to (for + * example the user id and provider of a BYOK key, or the MCP server id and field). A token copied to another + * row or purpose does not open, so the server cannot be used to decrypt one secret as another. + * - `v1..` (v1): one key for everything (info `aes-gcm-256/v1`), no additional data. Only the v1.1 data + * upgrade opens these, to re-seal them as v2 (`openLegacy`); `open` refuses them. */ +export type SecretPurpose = "byok" | "server-api-key" | "mcp"; + export interface SecretBox { - encrypt(plain: string): Promise; - decrypt(token: string): Promise; + seal(plain: string, purpose: SecretPurpose, aad: string): Promise; + open(token: string, purpose: SecretPurpose, aad: string): Promise; + /** v1 tokens only; for the data upgrade. */ + openLegacy(token: string): Promise; } const enc = new TextEncoder(); const dec = new TextDecoder(); const b64 = (u: Uint8Array) => Buffer.from(u).toString("base64"); -const unb64 = (s: string) => new Uint8Array(Buffer.from(s, "base64")); +const unb64 = (s: string): Uint8Array => new Uint8Array(Buffer.from(s, "base64")); +const SALT = enc.encode("stcourses/byok"); + +export const isLegacyToken = (token: string): boolean => token.startsWith("v1."); + +function deriveKey(secret: string, info: string): Promise { + return (async () => { + const ikm = await crypto.subtle.importKey("raw", enc.encode(secret), "HKDF", false, ["deriveKey"]); + return crypto.subtle.deriveKey({ name: "HKDF", hash: "SHA-256", salt: SALT, info: enc.encode(info) }, ikm, { name: "AES-GCM", length: 256 }, false, ["encrypt", "decrypt"]); + })(); +} + +function parseToken(token: string, version: "v1" | "v2"): { iv: Uint8Array; ct: Uint8Array } { + const [v, iv, ct, extra] = token.split("."); + if (v !== version || !iv || !ct || extra !== undefined) throw new Error("Unrecognised secret format"); + return { iv: unb64(iv), ct: unb64(ct) }; +} + +/** Additional data: purpose and row identity, length-prefixed so no two (purpose, aad) pairs encode alike. */ +const additionalData = (purpose: SecretPurpose, aad: string) => enc.encode(`stc/${purpose.length}:${purpose}/${aad.length}:${aad}`); export function createSecretBox(secret: string): SecretBox { - let keyP: Promise | null = null; - const key = () => - (keyP ??= (async () => { - const ikm = await crypto.subtle.importKey("raw", enc.encode(secret), "HKDF", false, ["deriveKey"]); - return crypto.subtle.deriveKey( - { name: "HKDF", hash: "SHA-256", salt: enc.encode("stcourses/byok"), info: enc.encode("aes-gcm-256/v1") }, - ikm, - { name: "AES-GCM", length: 256 }, - false, - ["encrypt", "decrypt"], - ); - })()); + const keys = new Map>(); + const key = (info: string) => { + let k = keys.get(info); + if (!k) keys.set(info, (k = deriveKey(secret, info))); + return k; + }; return { - async encrypt(plain) { + async seal(plain, purpose, aad) { const iv = crypto.getRandomValues(new Uint8Array(12)); - const ct = new Uint8Array(await crypto.subtle.encrypt({ name: "AES-GCM", iv }, await key(), enc.encode(plain))); - return `v1.${b64(iv)}.${b64(ct)}`; + const ct = new Uint8Array(await crypto.subtle.encrypt({ name: "AES-GCM", iv, additionalData: additionalData(purpose, aad) }, await key(`stcourses/${purpose}/v2`), enc.encode(plain))); + return `v2.${b64(iv)}.${b64(ct)}`; }, - async decrypt(token) { - const [v, iv, ct] = token.split("."); - if (v !== "v1" || !iv || !ct) throw new Error("Unrecognised secret format"); - const plain = await crypto.subtle.decrypt({ name: "AES-GCM", iv: unb64(iv) }, await key(), unb64(ct)); + async open(token, purpose, aad) { + const { iv, ct } = parseToken(token, "v2"); + const plain = await crypto.subtle.decrypt({ name: "AES-GCM", iv, additionalData: additionalData(purpose, aad) }, await key(`stcourses/${purpose}/v2`), ct); return dec.decode(plain); }, + async openLegacy(token) { + const { iv, ct } = parseToken(token, "v1"); + return dec.decode(await crypto.subtle.decrypt({ name: "AES-GCM", iv }, await key("aes-gcm-256/v1"), ct)); + }, }; } + +/** The v1 token format, exactly as v1 wrote it. Only upgrade tests use it, to build v1 data. */ +export async function legacyV1Encrypt(secret: string, plain: string): Promise { + const iv = crypto.getRandomValues(new Uint8Array(12)); + const ct = new Uint8Array(await crypto.subtle.encrypt({ name: "AES-GCM", iv }, await deriveKey(secret, "aes-gcm-256/v1"), enc.encode(plain))); + return `v1.${b64(iv)}.${b64(ct)}`; +} + +/** AAD for a user's own key: user id and provider kind (the user_api_keys row). */ +export const byokAad = (userId: string, provider: string) => `${userId}\n${provider}`; +/** AAD for the admin's provider key (one per server). */ +export const SERVER_KEY_AAD = "server"; +/** AAD for an MCP server's env or headers: server id and field. */ +export const mcpAad = (serverId: string, field: "env" | "headers") => `${serverId}\n${field}`; diff --git a/packages/feature-ai/src/feature.ts b/packages/feature-ai/src/feature.ts index 13b5180..2169bcb 100644 --- a/packages/feature-ai/src/feature.ts +++ b/packages/feature-ai/src/feature.ts @@ -14,6 +14,7 @@ import { connectMcp, mcpServersFor } from "./mcp/client"; import { createProvider } from "./providers"; import { PRESET_BASE_URLS, byokBaseUrl, noRedirectFetch, parseAllowedOrigins, publicProviderError, sanitizeProviderErrors } from "./endpoints"; import { getCourseIndex, noteChunks } from "./retrieval/course-index"; +import { MaskedSecretError } from "./mcp/secrets"; import { aiSettingsRepo, type McpServerConfig, type ProviderConfig } from "./settings"; export interface AiFeatureOptions { @@ -90,6 +91,10 @@ function parseMcp(v: unknown): McpServerConfig[] | "invalid" { if (s.headers && typeof s.headers === "object") cfg.headers = Object.fromEntries(Object.entries(s.headers).map(([k, val]) => [k, String(val)])); } if (Array.isArray(s.courseIds)) cfg.courseIds = s.courseIds.map(String).filter(Boolean); + if (s.clearSecrets !== undefined) { + if (!Array.isArray(s.clearSecrets) || !s.clearSecrets.every((f) => f === "env" || f === "headers")) return "invalid"; + if (s.clearSecrets.length) cfg.clearSecrets = [...new Set(s.clearSecrets as ("env" | "headers")[])]; + } out.push(cfg); } return out; @@ -145,9 +150,17 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat const progress = progressRepo(ctx.db); const sink = opts.draftSink ?? createFsDraftSink(ctx.config.dataDir); const log = opts.log ?? ((m: string) => console.error(`[ai] ${m}`)); + // The v1 data upgrade (mentor toggles, MCP secrets) starts now; every settings call also waits for it. + // Unreadable secrets are kept, never dropped; say so once at startup (the admin page shows the details). + settings.ready() + .then(() => settings.getServer()) + .then((s) => { + if (s.unreadableSecrets.length) log(`${s.unreadableSecrets.length} stored AI secret(s) cannot be decrypted with the current STC_SECRET; they are kept until re-entered or the previous secret is restored`); + }) + .catch((e) => log(`v1.1 data upgrade failed (retried on next use): ${(e as Error).message}`)); // AppConfig.aiAllowedBaseUrls (STC_AI_ALLOWED_BASE_URLS) plus the admin's list in the server AI settings. const configOrigins = parseAllowedOrigins((ctx.config.aiAllowedBaseUrls ?? []).join(",")); - const allowedOrigins = async () => [...new Set([...configOrigins, ...(await settings.getServer()).allowedBaseUrls])]; + const allowedOrigins = async () => [...new Set([...configOrigins, ...(await settings.allowedBaseUrls())])]; /** * Providers for a user's own key never follow redirects, and their errors reach the user only as the HTTP * status plus a fixed sentence (the full text is logged here). Server-scope providers are the admin's own. @@ -169,12 +182,12 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat if (key && base) return { source: "byok", cfg: { ...u.byok, baseUrl: base }, key }; } } - const s = await settings.getServer(); - if (!s.provider) return null; - if (s.provider.kind === "anthropic" && !s.provider.hasApiKey) return null; - const key = s.provider.hasApiKey ? await settings.serverApiKey().catch(() => null) : null; - if (s.provider.kind === "anthropic" && !key) return null; - return { source: "server", cfg: s.provider, key }; + const provider = await settings.serverProvider(); + if (!provider) return null; + if (provider.kind === "anthropic" && !provider.hasApiKey) return null; + const key = provider.hasApiKey ? await settings.serverApiKey().catch(() => null) : null; + if (provider.kind === "anthropic" && !key) return null; + return { source: "server", cfg: provider, key }; }; const viewable = async (user: AuthUser | null, courseId: string): Promise => { @@ -198,22 +211,23 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat const rows = await ctx.db.sql`SELECT id, user_id, course_id, title, created_at FROM ai_conversations WHERE id = ${id} AND user_id = ${userId}`; return rows[0] as { id: string; user_id: string; course_id: string; title: string | null; created_at: string } | undefined; }; - // Roles: "user", "assistant", and "sources" (JSON citations for the assistant message before it). - const addMessage = async (conversationId: string, role: "user" | "assistant" | "sources", content: string) => { - await ctx.db.sql`INSERT INTO ai_messages (id, conversation_id, role, content, created_at) VALUES (${Bun.randomUUIDv7()}, ${conversationId}, ${role}, ${content}, ${new Date().toISOString()})`; + /** Roles "user" and "assistant"; an assistant message's citations go in `citations_json` (contract request 5). */ + const addMessage = async (conversationId: string, role: "user" | "assistant", content: string, citations?: unknown[]) => { + const cites = citations ? JSON.stringify(citations) : null; + await ctx.db.sql`INSERT INTO ai_messages (id, conversation_id, role, content, citations_json, created_at) VALUES (${Bun.randomUUIDv7()}, ${conversationId}, ${role}, ${content}, ${cites}, ${new Date().toISOString()})`; }; - const messagesOf = async (conversationId: string) => - (await ctx.db.sql`SELECT id, role, content, created_at FROM ai_messages WHERE conversation_id = ${conversationId}`) - .map((r: { id: string; role: string; content: string; created_at: string }) => ({ id: r.id, role: r.role, content: r.content, createdAt: r.created_at })) - .sort((a: { id: string }, b: { id: string }) => (a.id < b.id ? -1 : 1)) as { id: string; role: "user" | "assistant" | "sources"; content: string; createdAt: string }[]; - /** Chat turns with each assistant message's stored citations attached. */ + /** Chat turns in write order (UUIDv7 ids), each assistant message with its stored citations. */ const turnsOf = async (conversationId: string) => { + const rows = (await ctx.db.sql`SELECT id, role, content, citations_json, created_at FROM ai_messages WHERE conversation_id = ${conversationId}`) as + { id: string; role: string; content: string; citations_json: string | null; created_at: string }[]; const out: { id: string; role: "user" | "assistant"; content: string; createdAt: string; citations?: unknown[] }[] = []; - for (const m of await messagesOf(conversationId)) { - if (m.role === "sources") { - const last = out[out.length - 1]; - if (last?.role === "assistant") { try { last.citations = JSON.parse(m.content); } catch {} } - } else out.push({ ...m, role: m.role }); + for (const r of rows.sort((a, b) => (a.id < b.id ? -1 : 1))) { + if (r.role !== "user" && r.role !== "assistant") continue; + const turn: (typeof out)[number] = { id: r.id, role: r.role, content: r.content, createdAt: r.created_at }; + if (r.role === "assistant" && r.citations_json) { + try { const c = JSON.parse(r.citations_json); if (Array.isArray(c)) turn.citations = c; } catch {} + } + out.push(turn); } return out; }; @@ -264,7 +278,7 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat let tools: ToolSet | null = null; try { const course_ = courseTools({ repo: ctx.courses, course, index, listNotes: (lid) => notes.list(user.id, course.id, lid), getProgress: () => progress.get(user.id, course.id) }); - const servers = mcpServersFor(course.id, (await settings.getServer()).mcpServers); + const servers = mcpServersFor(course.id, await settings.mcpServersWithSecrets()); const mcp = servers.length ? await mcpTools(servers, { fetch: opts.fetch, timeoutMs: opts.mcpTimeoutMs }) : null; for (const w of mcp?.warnings ?? []) send({ type: "notice", message: w }); tools = combineTools(mcp ? [course_, mcp] : [course_]); @@ -282,10 +296,7 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat send({ type: "done", text: ev.text, citations: cited }); } else send(ev); } - if (answer) { - await addMessage(convId, "assistant", answer); - await addMessage(convId, "sources", JSON.stringify(cited)); - } + if (answer) await addMessage(convId, "assistant", answer, cited); } catch (e) { const m = (e as Error).message; if (resolved.source === "byok") log(`chat error for user ${user.id} (own key): ${m}`); @@ -344,7 +355,12 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat allowedBaseUrls = parseAllowedOrigins(list); if (allowedBaseUrls.length !== new Set(list.map((x) => x.replace(/\/+$/, ""))).size) return err(400, "Each allowed base URL must be an origin such as http://ollama.internal:11434."); } - await settings.saveServer({ provider, mcpServers, allowedBaseUrls }); + try { + await settings.saveServer({ provider, mcpServers, allowedBaseUrls }); + } catch (e) { + if (e instanceof MaskedSecretError) return err(400, e.message); + throw e; + } return json(await settings.getServer()); }, }, @@ -355,8 +371,13 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat const b = await body(req); const parsed = parseMcp(b ? [b] : null); if (parsed === "invalid" || !parsed.length) return err(400, "Invalid MCP server."); + let cfg: McpServerConfig; + try { cfg = await settings.resolveMcpMasks(parsed[0]!); } catch (e) { + if (e instanceof MaskedSecretError) return err(400, e.message); + throw e; + } try { - const c = await connectMcp(parsed[0]!, { fetch: opts.fetch, timeoutMs: opts.mcpTimeoutMs ?? 15_000 }); + const c = await connectMcp(cfg, { fetch: opts.fetch, timeoutMs: opts.mcpTimeoutMs ?? 15_000 }); try { return json({ ok: true, tools: (await c.tools()).map((t) => ({ name: t.name, description: t.description })) }); } finally { await c.close(); } } catch (e) { return json({ ok: false, error: (e as Error).message }); @@ -367,8 +388,8 @@ export function aiFeature(ctx: ServerContext, opts: AiFeatureOptions = {}): Feat GET: async (_req, rc) => { if (!rc.user) return err(401, "Sign in first."); const u = await settings.getUser(rc.user.id); - const s = await settings.getServer(); - return json({ byok: u.byok, serverProvider: s.provider ? { kind: s.provider.kind, model: s.provider.model ?? null } : null }); + const p = await settings.serverProvider(); + return json({ byok: u.byok, serverProvider: p ? { kind: p.kind, model: p.model ?? null } : null }); }, }, "/api/ai/account/key": { diff --git a/packages/feature-ai/src/index.ts b/packages/feature-ai/src/index.ts index 405c90e..d0cae5b 100644 --- a/packages/feature-ai/src/index.ts +++ b/packages/feature-ai/src/index.ts @@ -1,8 +1,8 @@ export { createSecretBox, type SecretBox } from "./crypto"; -/** Migration folders for this package. Empty: the AI package uses the ai_* tables from 0001_init.sql. */ -export const migrationsDirs: string[] = []; +export { migrationsDirs } from "./migrations"; export * from "./settings"; +export { upgradeAiData, AI_DATA_UPGRADE, AI_SECRETS_UPGRADE } from "./upgrade"; export * from "./providers"; export * from "./retrieval/course-index"; export { Bm25, tokenize } from "./retrieval/bm25"; diff --git a/packages/feature-ai/src/mcp/secrets.ts b/packages/feature-ai/src/mcp/secrets.ts new file mode 100644 index 0000000..f0ed5d9 --- /dev/null +++ b/packages/feature-ai/src/mcp/secrets.ts @@ -0,0 +1,154 @@ +import { mcpAad, type SecretBox } from "../crypto"; +import type { McpServerConfig } from "../settings"; + +/** + * MCP `env` and `headers` at rest: each field is sealed on its own (`envCipher`, `headersCipher`) with the "mcp" + * purpose key and the server id plus field name as additional data, so a cipher moved to another server, field or + * table does not open. The admin API only ever sees masked values; server code opens them to start or call a server. + * + * Older shapes, re-sealed by the data upgrade (src/upgrade.ts): plaintext `env`/`headers` (v1) and one v1-token + * `secretCipher` holding `{ env, headers }` (the first v1.1 draft). + */ +export type SecretField = "env" | "headers"; +export const SECRET_FIELDS: SecretField[] = ["env", "headers"]; +const CIPHER: Record = { env: "envCipher", headers: "headersCipher" }; + +export type StoredMcpServer = Omit & { + envCipher?: string; + headersCipher?: string; + /** First v1.1 draft: v1 token of `{ env, headers }`. */ + secretCipher?: string; + /** v1: plaintext. */ + env?: Record; + headers?: Record; +}; + +export interface OpenedMcpServer { + cfg: McpServerConfig; + /** Fields with a stored cipher that cannot be opened (STC_SECRET changed, or a cipher moved from elsewhere). */ + unreadable: SecretField[]; +} + +export const MASK_PREFIX = "••••"; + +/** `••••` plus the last 4 characters for values of 12 characters or more, otherwise `••••••••`. */ +export function maskSecret(value: string): string { + return value.length >= 12 ? `${MASK_PREFIX}${value.slice(-4)}` : `${MASK_PREFIX}${MASK_PREFIX}`; +} + +/** A posted value looks like a mask but matches no stored value for that server and key. */ +export class MaskedSecretError extends Error { + constructor(serverId: string, key: string) { + super(`Re-enter the value of ${key} for MCP server "${serverId}": a masked value cannot be saved.`); + this.name = "MaskedSecretError"; + } +} + +const nonEmpty = (r: Record | undefined) => + (r && typeof r === "object" && !Array.isArray(r) && Object.keys(r).length ? r : undefined); + +const isRecordOfStrings = (v: unknown): v is Record => + !!v && typeof v === "object" && !Array.isArray(v) && Object.values(v).every((x) => typeof x === "string"); + +/** The non-secret part of a server config (also drops `clearSecrets`, which is a save instruction only). */ +function baseOf(s: McpServerConfig | StoredMcpServer): Omit { + const { env: _e, headers: _h, envCipher: _ec, headersCipher: _hc, secretCipher: _sc, clearSecrets: _cs, ...rest } = s as StoredMcpServer & McpServerConfig; + return rest; +} + +/** Opens a stored server for server code. Unreadable ciphers are reported, never thrown and never discarded. */ +export async function openMcpServer(box: SecretBox, s: StoredMcpServer): Promise { + const cfg: McpServerConfig = { ...baseOf(s) }; + const unreadable: SecretField[] = []; + for (const f of SECRET_FIELDS) { + const token = s[CIPHER[f]]; + if (token) { + try { + const v = JSON.parse(await box.open(token, "mcp", mcpAad(s.id, f))); + if (!isRecordOfStrings(v)) throw new Error("not a record"); + if (nonEmpty(v)) cfg[f] = v; + } catch { unreadable.push(f); } + } else if (nonEmpty(s[f])) cfg[f] = s[f]; // v1 plaintext not yet upgraded + } + // A draft-format cipher the upgrade could not open: report the field this transport uses. + if (s.secretCipher) { + const f: SecretField = s.transport === "stdio" ? "env" : "headers"; + if (!s[CIPHER[f]] && !unreadable.includes(f)) unreadable.push(f); + } + return { cfg, unreadable }; +} + +/** + * Seals a posted (already unmasked) config. For each field: `clearSecrets` drops it; posted values replace it; with + * nothing posted, a stored cipher that could not be opened is kept as it is, so a temporary STC_SECRET mismatch + * never turns into data loss on the next save. + */ +export async function sealMcpServer(box: SecretBox, cfg: McpServerConfig, previous?: { stored: StoredMcpServer; unreadable: SecretField[] }): Promise { + const out: StoredMcpServer = { ...baseOf(cfg) }; + for (const f of SECRET_FIELDS) { + if (cfg.clearSecrets?.includes(f)) continue; + const values = nonEmpty(cfg[f]); + if (values) out[CIPHER[f]] = await box.seal(JSON.stringify(values), "mcp", mcpAad(cfg.id, f)); + else if (previous?.unreadable.includes(f)) { + if (previous.stored[CIPHER[f]]) out[CIPHER[f]] = previous.stored[CIPHER[f]]; + else if (previous.stored.secretCipher) out.secretCipher = previous.stored.secretCipher; + } + } + return out; +} + +/** Re-seals v1 plaintext and a readable draft `secretCipher` into per-field v2 ciphers. `pending`: something legacy stays. */ +export async function upgradeStoredMcpServer(box: SecretBox, s: StoredMcpServer): Promise<{ server: StoredMcpServer; changed: boolean; pending: boolean }> { + let legacy: { env?: Record; headers?: Record } = {}; + let pending = false; + let changed = false; + if (s.secretCipher) { + try { + const v = JSON.parse(await box.openLegacy(s.secretCipher)); + legacy = v && typeof v === "object" ? v : {}; + } catch { pending = true; } + } + const out: StoredMcpServer = { ...baseOf(s) }; + for (const f of SECRET_FIELDS) { + if (s[CIPHER[f]]) { out[CIPHER[f]] = s[CIPHER[f]]; continue; } + const values = nonEmpty(isRecordOfStrings(s[f]) ? s[f] : undefined) ?? nonEmpty(isRecordOfStrings(legacy[f]) ? legacy[f] : undefined); + if (values) out[CIPHER[f]] = await box.seal(JSON.stringify(values), "mcp", mcpAad(s.id, f)); + } + if (pending) out.secretCipher = s.secretCipher; + if (s.env !== undefined || s.headers !== undefined || (s.secretCipher && !pending)) changed = true; + return { server: out, changed, pending }; +} + +const maskRecord = (r: Record | undefined) => (r ? Object.fromEntries(Object.entries(r).map(([k, v]) => [k, maskSecret(v)])) : undefined); + +/** What the admin API returns: keys visible, values masked. */ +export function maskMcpServer(s: McpServerConfig): McpServerConfig { + const out: McpServerConfig = { ...s }; + if (s.env) out.env = maskRecord(s.env); + if (s.headers) out.headers = maskRecord(s.headers); + return out; +} + +/** + * Replaces each posted value equal to the mask of the stored value (same server id, same key) with the stored + * value. Any other value starting with the mask prefix throws MaskedSecretError, so a mask is never saved or sent. + */ +export function unmaskMcpServer(posted: McpServerConfig, previous: McpServerConfig[]): McpServerConfig { + const prev = previous.find((p) => p.id === posted.id); + const fill = (field: SecretField) => { + const r = posted[field]; + if (!r) return undefined; + return Object.fromEntries(Object.entries(r).map(([k, v]) => { + if (!v.startsWith(MASK_PREFIX)) return [k, v]; + const old = prev?.[field]?.[k]; + if (old !== undefined && maskSecret(old) === v) return [k, old]; + throw new MaskedSecretError(posted.id, k); + })); + }; + const out: McpServerConfig = { ...posted }; + const env = fill("env"); + const headers = fill("headers"); + if (env) out.env = env; + if (headers) out.headers = headers; + return out; +} diff --git a/packages/feature-ai/src/migrations.ts b/packages/feature-ai/src/migrations.ts new file mode 100644 index 0000000..4c24074 --- /dev/null +++ b/packages/feature-ai/src/migrations.ts @@ -0,0 +1,7 @@ +import { join } from "node:path"; + +/** + * Migration folders for this package; the server passes them to core `migrate()`. The files upgrade the v1 ai_* + * tables of core's 0001_init.sql. Data that portable SQL cannot move runs in `upgradeAiData` (src/upgrade.ts). + */ +export const migrationsDirs: string[] = [join(import.meta.dir, "..", "migrations")]; diff --git a/packages/feature-ai/src/providers/anthropic.ts b/packages/feature-ai/src/providers/anthropic.ts index 1576bf3..724f583 100644 --- a/packages/feature-ai/src/providers/anthropic.ts +++ b/packages/feature-ai/src/providers/anthropic.ts @@ -17,19 +17,24 @@ export interface AnthropicOptions { type Block = Record & { type: string }; type ApiMessage = { role: "user" | "assistant"; content: Block[] }; +/** This provider's own `providerData` (`{ anthropic: Block[] }`); anything else is ignored. */ +const rawBlocks = (data: unknown): Block[] | null => { + const blocks = data && typeof data === "object" ? (data as { anthropic?: unknown }).anthropic : undefined; + return Array.isArray(blocks) && blocks.length && blocks.every((b) => b && typeof b === "object" && typeof (b as Block).type === "string") ? (blocks as Block[]) : null; +}; + /** * Anthropic Messages API over plain fetch with streaming and tool use. * - Thinking is left to the model default (adaptive on current models); no temperature or forced tool_choice. - * - The frozen ChatMessage contract has no slot for thinking blocks, so the provider remembers every - * response's raw content blocks by tool_use id and replays them unchanged when the caller sends back - * the matching assistant tool-call message (required for thinking + tool use). + * - Each turn ends with `providerData = { anthropic: }` (thinking with signature, text, + * tool_use). An assistant message carrying it is replayed verbatim, as thinking + tool use requires; one + * without it maps to text plus tool_use blocks from `toolCalls`. The provider keeps no conversation state. * - On the first-party endpoint, server-side refusal fallbacks are on ("default" routing). */ export function createAnthropicProvider(o: AnthropicOptions): AiProvider { const doFetch = o.fetch ?? fetch; const base = (o.baseUrl || FIRST_PARTY).replace(/\/+$/, ""); const firstParty = base === FIRST_PARTY; - const rawByToolId = new Map(); const toApi = (messages: ChatMessage[]): { system: string; messages: ApiMessage[] } => { const system = messages.filter((m) => m.role === "system").map((m) => m.content).join("\n\n"); @@ -42,18 +47,12 @@ export function createAnthropicProvider(o: AnthropicOptions): AiProvider { for (const m of messages) { if (m.role === "system") continue; if (m.role === "tool") push("user", { type: "tool_result", tool_use_id: m.toolCallId ?? "", content: m.content }); - else if (m.role === "assistant" && m.toolCallId) { - let input: unknown = {}; - try { input = JSON.parse(m.content || "{}"); } catch { input = { value: m.content }; } - push("assistant", { type: "tool_use", id: m.toolCallId, name: m.name ?? "", input }); - } else if (m.content) push(m.role === "assistant" ? "assistant" : "user", { type: "text", text: m.content }); - } - // Replace assistant turns we produced with their exact original blocks (thinking signatures intact). - for (const msg of out) { - if (msg.role !== "assistant") continue; - const id = msg.content.find((b) => b.type === "tool_use")?.id as string | undefined; - const raw = id ? rawByToolId.get(id) : undefined; - if (raw) msg.content = structuredClone(raw); + else if (m.role === "assistant") { + const raw = rawBlocks(m.providerData); + if (raw) { for (const b of structuredClone(raw)) push("assistant", b); continue; } + if (m.content) push("assistant", { type: "text", text: m.content }); + for (const c of m.toolCalls ?? []) push("assistant", { type: "tool_use", id: c.id, name: c.name, input: c.input ?? {} }); + } else if (m.content) push("user", { type: "text", text: m.content }); } return { system, messages: out }; }; @@ -85,7 +84,6 @@ export function createAnthropicProvider(o: AnthropicOptions): AiProvider { const blocks: Block[] = []; const partialJson = new Map(); - const toolIds: string[] = []; let stopReason: string | null = null; for await (const { data } of readSse(res.body)) { let ev: any; @@ -117,7 +115,6 @@ export function createAnthropicProvider(o: AnthropicOptions): AiProvider { let input: unknown = {}; try { input = json ? JSON.parse(json) : (b.input ?? {}); } catch { input = { _invalidJson: json }; } b.input = input; - toolIds.push(b.id as string); yield { toolCall: { id: b.id as string, name: b.name as string, input } }; } break; @@ -130,7 +127,7 @@ export function createAnthropicProvider(o: AnthropicOptions): AiProvider { } } const content = blocks.filter(Boolean); - for (const id of toolIds) rawByToolId.set(id, content); + if (content.length) yield { providerData: { anthropic: content } }; if (stopReason === "refusal" && !content.some((b) => b.type === "text" && b.text)) { yield { text: "The model declined to answer this request." }; } diff --git a/packages/feature-ai/src/providers/openai.ts b/packages/feature-ai/src/providers/openai.ts index f1c477d..8af5dbd 100644 --- a/packages/feature-ai/src/providers/openai.ts +++ b/packages/feature-ai/src/providers/openai.ts @@ -11,11 +11,9 @@ type OaMessage = function toOpenAi(messages: ChatMessage[]): OaMessage[] { const out: OaMessage[] = []; for (const m of messages) { - if (m.role === "assistant" && m.toolCallId) { - const call = { id: m.toolCallId, type: "function" as const, function: { name: m.name ?? "", arguments: m.content || "{}" } }; - const last = out[out.length - 1]; - if (last && last.role === "assistant") (last.tool_calls ??= []).push(call); - else out.push({ role: "assistant", content: null, tool_calls: [call] }); + if (m.role === "assistant" && m.toolCalls?.length) { + const tool_calls = m.toolCalls.map((c) => ({ id: c.id, type: "function" as const, function: { name: c.name, arguments: JSON.stringify(c.input ?? {}) } })); + out.push({ role: "assistant", content: m.content || null, tool_calls }); } else if (m.role === "tool") out.push({ role: "tool", tool_call_id: m.toolCallId ?? "", content: m.content }); else out.push({ role: m.role, content: m.content } as OaMessage); } diff --git a/packages/feature-ai/src/settings.ts b/packages/feature-ai/src/settings.ts index ecc9776..1e0438a 100644 --- a/packages/feature-ai/src/settings.ts +++ b/packages/feature-ai/src/settings.ts @@ -1,5 +1,9 @@ import type { Db } from "@stc/db"; -import type { SecretBox } from "./crypto"; +import { byokAad, SERVER_KEY_AAD, type SecretBox } from "./crypto"; +import { maskMcpServer, openMcpServer, sealMcpServer, unmaskMcpServer, type OpenedMcpServer, type SecretField, type StoredMcpServer } from "./mcp/secrets"; +import { upgradeAiData } from "./upgrade"; + +export { maskSecret, MaskedSecretError } from "./mcp/secrets"; export type ProviderKind = "anthropic" | "openai"; export type Effort = "low" | "medium" | "high" | "xhigh" | "max"; @@ -17,20 +21,31 @@ export interface McpServerConfig { headers?: Record; /** Courses this server is offered in. Absent or empty: every course (global). */ courseIds?: string[]; + /** Save only: drop these stored fields, even if they cannot be decrypted. Never stored or returned. */ + clearSecrets?: SecretField[]; } -/** `allowedBaseUrls`: origins learners' own keys (BYOK) may be sent to, besides the official endpoints. */ -export interface ServerSettings { provider: PublicProviderConfig | null; mcpServers: McpServerConfig[]; allowedBaseUrls: string[] } +/** A stored secret the current STC_SECRET cannot open. It is kept until the admin replaces or clears it. */ +export type UnreadableSecret = + | { kind: "api-key" } + | { kind: "mcp"; serverId: string; name: string; field: SecretField }; + +/** + * `allowedBaseUrls`: origins learners' own keys (BYOK) may be sent to, besides the official endpoints. + * `mcpServers` here carry masked `env`/`headers` values (see mcp/secrets.ts); full values: mcpServersWithSecrets(). + */ +export interface ServerSettings { provider: PublicProviderConfig | null; mcpServers: McpServerConfig[]; allowedBaseUrls: string[]; unreadableSecrets: UnreadableSecret[] } export interface ServerSettingsInput { provider: (ProviderConfig & { apiKey?: string; clearApiKey?: boolean }) | null; mcpServers: McpServerConfig[]; /** Omitted: keep the saved list. */ allowedBaseUrls?: string[]; } -export interface UserSettings { byok: PublicProviderConfig | null; mentorOff: Record } +/** `keyUnreadable`: the saved key cannot be decrypted with the current STC_SECRET; the user must re-enter it. */ +export interface UserSettings { byok: (PublicProviderConfig & { keyUnreadable?: true }) | null; mentorOff: Record } -type ServerRow = { provider: ProviderConfig | null; apiKeyCipher: string | null; mcpServers: McpServerConfig[]; allowedBaseUrls?: string[] }; -type UserRow = { byok: ProviderConfig | null; mentorOff: Record }; +type ServerRow = { provider: ProviderConfig | null; apiKeyCipher: string | null; mcpServers: StoredMcpServer[]; allowedBaseUrls?: string[] }; +type UserRow = { byok: ProviderConfig | null }; const pickProvider = (p: ProviderConfig): ProviderConfig => { const out: ProviderConfig = { kind: p.kind }; @@ -46,14 +61,19 @@ const parse = (s: string | undefined, fallback: T): T => { }; /** - * AI settings in the ai_settings and user_api_keys tables of 0001_init.sql. - * - ('server', '') holds the admin provider (key encrypted) and MCP servers. - * - ('user', ) holds the BYOK provider choice and per-course mentor toggles. The frozen primary key - * is (scope, owner_id), so the per-course flag lives in config_json; the mentor_off column means - * "off for every course" and stays 0 (see docs/contract-requests/ai.md). - * Keys never leave this module except through serverApiKey/userApiKey, which only server code calls. + * AI settings in ai_settings and user_api_keys (core 0001_init.sql) and ai_mentor_prefs (this package's 0002). + * - ('server', '') holds the admin provider (key encrypted) and MCP servers (env and headers encrypted). + * - ('user', ) holds the BYOK provider choice. The mentor_off column ("off for every course") stays 0. + * - ai_mentor_prefs has a row per (user, course) whose mentor rules the learner switched off. + * Keys and MCP secrets leave this module only through serverApiKey, userApiKey and mcpServersWithSecrets, which + * only server code calls. + * Every method first awaits the one-time v1 data upgrade (upgradeAiData); a failed attempt is retried next call. */ export function aiSettingsRepo(db: Db, box: SecretBox) { + let upgraded: Promise | null = null; + const ready = () => + (upgraded ??= upgradeAiData(db, box).catch((e) => { upgraded = null; throw e; })); + const read = async (scope: "server" | "user", owner: string): Promise => { const rows = await db.sql`SELECT config_json FROM ai_settings WHERE scope = ${scope} AND owner_id = ${owner}`; return rows[0]?.config_json; @@ -68,40 +88,107 @@ export function aiSettingsRepo(db: Db, box: SecretBox) { }); }; const serverRow = async (): Promise => parse(await read("server", ""), { provider: null, apiKeyCipher: null, mcpServers: [] }); - const userRow = async (userId: string): Promise => parse(await read("user", userId), { byok: null, mentorOff: {} }); + const userRow = async (userId: string): Promise => { + const { mentorOff: _legacy, ...row } = parse(await read("user", userId), { byok: null } as UserRow & { mentorOff?: unknown }); + return row; + }; + const storedServers = (r: ServerRow): StoredMcpServer[] => + (Array.isArray(r.mcpServers) ? r.mcpServers : []).filter((m) => m && typeof m === "object" && typeof m.id === "string"); + const openServers = async (r: ServerRow): Promise => Promise.all(storedServers(r).map((m) => openMcpServer(box, m))); + const openServerKey = async (r: ServerRow): Promise => + r.apiKeyCipher ? box.open(r.apiKeyCipher, "server-api-key", SERVER_KEY_AAD).catch(() => null) : null; + const openUserKey = async (userId: string, k: { provider: string; ciphertext: string }): Promise => + box.open(k.ciphertext, "byok", byokAad(userId, k.provider)).catch(() => null); const keyRow = async (userId: string) => (await db.sql`SELECT provider, ciphertext FROM user_api_keys WHERE user_id = ${userId}`)[0] as { provider: string; ciphertext: string } | undefined; return { + /** Resolves once the v1 data upgrade is done (aiFeature starts it at construction). */ + ready: async (): Promise => { await ready(); }, async getServer(): Promise { + await ready(); const r = await serverRow(); - return { provider: r.provider ? { ...r.provider, hasApiKey: !!r.apiKeyCipher } : null, mcpServers: r.mcpServers ?? [], allowedBaseUrls: r.allowedBaseUrls ?? [] }; + const opened = await openServers(r); + const unreadableSecrets: UnreadableSecret[] = []; + if (r.provider && r.apiKeyCipher && (await openServerKey(r)) === null) unreadableSecrets.push({ kind: "api-key" }); + for (const o of opened) for (const field of o.unreadable) unreadableSecrets.push({ kind: "mcp", serverId: o.cfg.id, name: o.cfg.name, field }); + return { + provider: r.provider ? { ...r.provider, hasApiKey: !!r.apiKeyCipher } : null, + mcpServers: opened.map((o) => maskMcpServer(o.cfg)), + allowedBaseUrls: r.allowedBaseUrls ?? [], + unreadableSecrets, + }; }, + /** The admin provider without opening any secret (for resolving which provider a request uses). */ + async serverProvider(): Promise { + await ready(); + const r = await serverRow(); + return r.provider ? { ...r.provider, hasApiKey: !!r.apiKeyCipher } : null; + }, + /** The admin's BYOK allowlist, without opening any secret. */ + async allowedBaseUrls(): Promise { + await ready(); + return (await serverRow()).allowedBaseUrls ?? []; + }, + /** Server code only: MCP servers with their real env and headers (unreadable fields left out). */ + async mcpServersWithSecrets(): Promise { + await ready(); + return (await openServers(await serverRow())).map((o) => o.cfg); + }, + /** A posted (masked) MCP config with stored values filled in; throws MaskedSecretError for a stray mask. */ + async resolveMcpMasks(cfg: McpServerConfig): Promise { + await ready(); + const { clearSecrets: _c, ...rest } = unmaskMcpServer(cfg, (await openServers(await serverRow())).map((o) => o.cfg)); + return rest; + }, + /** + * Saves the admin settings. Secrets that cannot be decrypted are never discarded implicitly: the API key stays + * unless replaced or cleared (clearApiKey), and an MCP field stays unless new values are posted or the field is + * listed in that server's clearSecrets. + */ async saveServer(input: ServerSettingsInput): Promise { + await ready(); const prev = await serverRow(); let apiKeyCipher: string | null = null; if (input.provider) { - if (input.provider.apiKey) apiKeyCipher = await box.encrypt(input.provider.apiKey); + if (input.provider.apiKey) apiKeyCipher = await box.seal(input.provider.apiKey, "server-api-key", SERVER_KEY_AAD); else if (!input.provider.clearApiKey) apiKeyCipher = prev.apiKeyCipher; } + const stored = storedServers(prev); + const opened = await openServers(prev); + const previous = opened.map((o) => o.cfg); + const mcpServers: StoredMcpServer[] = []; + for (const m of input.mcpServers ?? []) { + const i = stored.findIndex((x) => x.id === m.id); + mcpServers.push(await sealMcpServer(box, unmaskMcpServer(m, previous), i >= 0 ? { stored: stored[i]!, unreadable: opened[i]!.unreadable } : undefined)); + } await write("server", "", { provider: input.provider ? pickProvider(input.provider) : null, apiKeyCipher, - mcpServers: input.mcpServers ?? [], + mcpServers, allowedBaseUrls: input.allowedBaseUrls ?? prev.allowedBaseUrls ?? [], }); }, async serverApiKey(): Promise { + await ready(); const r = await serverRow(); - return r.provider && r.apiKeyCipher ? box.decrypt(r.apiKeyCipher) : null; + return r.provider ? openServerKey(r) : null; }, async getUser(userId: string): Promise { + await ready(); const r = await userRow(userId); const k = await keyRow(userId); - return { byok: r.byok && k ? { ...r.byok, hasApiKey: true } : null, mentorOff: r.mentorOff ?? {} }; + const prefs = (await db.sql`SELECT course_id FROM ai_mentor_prefs WHERE user_id = ${userId} AND off = 1`) as { course_id: string }[]; + let byok: UserSettings["byok"] = null; + if (r.byok && k) { + byok = { ...r.byok, hasApiKey: true }; + if ((await openUserKey(userId, k)) === null) byok.keyUnreadable = true; + } + return { byok, mentorOff: Object.fromEntries(prefs.map((p) => [p.course_id, true])) }; }, async saveUserKey(userId: string, cfg: ProviderConfig & { apiKey: string }): Promise { - const ciphertext = await box.encrypt(cfg.apiKey); + await ready(); + const ciphertext = await box.seal(cfg.apiKey, "byok", byokAad(userId, cfg.kind)); const now = new Date().toISOString(); await db.sql.begin(async (tx) => { await tx`DELETE FROM user_api_keys WHERE user_id = ${userId}`; @@ -111,23 +198,26 @@ export function aiSettingsRepo(db: Db, box: SecretBox) { await write("user", userId, { ...r, byok: pickProvider(cfg) }); }, async removeUserKey(userId: string): Promise { + await ready(); await db.sql`DELETE FROM user_api_keys WHERE user_id = ${userId}`; const r = await userRow(userId); await write("user", userId, { ...r, byok: null }); }, async userApiKey(userId: string): Promise { + await ready(); const k = await keyRow(userId); - return k ? box.decrypt(k.ciphertext) : null; + return k ? openUserKey(userId, k) : null; }, async setMentorOff(userId: string, courseId: string, off: boolean): Promise { - const r = await userRow(userId); - const mentorOff = { ...r.mentorOff }; - if (off) mentorOff[courseId] = true; - else delete mentorOff[courseId]; - await write("user", userId, { ...r, mentorOff }); + await ready(); + await db.sql.begin(async (tx) => { + await tx`DELETE FROM ai_mentor_prefs WHERE user_id = ${userId} AND course_id = ${courseId}`; + if (off) await tx`INSERT INTO ai_mentor_prefs (user_id, course_id, off, updated_at) VALUES (${userId}, ${courseId}, 1, ${new Date().toISOString()})`; + }); }, async isMentorOff(userId: string, courseId: string): Promise { - return (await userRow(userId)).mentorOff?.[courseId] === true; + await ready(); + return (await db.sql`SELECT 1 AS x FROM ai_mentor_prefs WHERE user_id = ${userId} AND course_id = ${courseId} AND off = 1`).length > 0; }, }; } diff --git a/packages/feature-ai/src/upgrade.ts b/packages/feature-ai/src/upgrade.ts new file mode 100644 index 0000000..a5ad34e --- /dev/null +++ b/packages/feature-ai/src/upgrade.ts @@ -0,0 +1,111 @@ +import type { Db } from "@stc/db"; +import { byokAad, isLegacyToken, SERVER_KEY_AAD, type SecretBox } from "./crypto"; +import { upgradeStoredMcpServer, type StoredMcpServer } from "./mcp/secrets"; + +/** Recorded in schema_migrations once the data upgrade has run (no `.sql` suffix, so no file can share it). */ +export const AI_DATA_UPGRADE = "ai_data_v11"; +/** Recorded once no secret is left in a pre-v2 format. */ +export const AI_SECRETS_UPGRADE = "ai_secrets_v2"; + +type Tx = Bun.TransactionSQL; + +const parseObject = (s: string): Record | null => { + try { + const v = JSON.parse(s); + return v && typeof v === "object" && !Array.isArray(v) ? v : null; + } catch { return null; } +}; + +/** v1 kept per-course toggles in the user row's config_json.mentorOff; move them to ai_mentor_prefs. */ +async function moveMentorToggles(tx: Tx, now: string) { + const rows = (await tx`SELECT owner_id, config_json FROM ai_settings WHERE scope = 'user'`) as { owner_id: string; config_json: string }[]; + for (const r of rows) { + const cfg = parseObject(r.config_json); + if (!cfg || !("mentorOff" in cfg)) continue; + const off = cfg.mentorOff && typeof cfg.mentorOff === "object" ? (cfg.mentorOff as Record) : {}; + for (const [courseId, v] of Object.entries(off)) { + if (v !== true || !courseId) continue; + const exists = await tx`SELECT 1 AS x FROM ai_mentor_prefs WHERE user_id = ${r.owner_id} AND course_id = ${courseId}`; + if (!exists.length) await tx`INSERT INTO ai_mentor_prefs (user_id, course_id, off, updated_at) VALUES (${r.owner_id}, ${courseId}, 1, ${now})`; + } + const { mentorOff: _drop, ...rest } = cfg; + await tx`UPDATE ai_settings SET config_json = ${JSON.stringify(rest)}, updated_at = ${now} WHERE scope = 'user' AND owner_id = ${r.owner_id}`; + } +} + +/** + * Re-seals every secret still in an older format as a v2 token (own HKDF key per purpose, AAD bound to the row): + * BYOK keys, the server API key, and MCP env/headers (v1 plaintext or the draft `secretCipher`). A v1 token that + * does not open (STC_SECRET changed) is left untouched and the step stays pending, so it is retried on every start + * and completes once the previous secret is restored. Returns { changed, pending }. + */ +async function resealSecrets(tx: Tx, box: SecretBox, now: string): Promise<{ changed: boolean; pending: boolean }> { + let changed = false; + let pending = false; + const keys = (await tx`SELECT user_id, provider, ciphertext FROM user_api_keys`) as { user_id: string; provider: string; ciphertext: string }[]; + for (const k of keys) { + if (!isLegacyToken(k.ciphertext)) continue; + let plain: string; + try { plain = await box.openLegacy(k.ciphertext); } catch { pending = true; continue; } + const sealed = await box.seal(plain, "byok", byokAad(k.user_id, k.provider)); + await tx`UPDATE user_api_keys SET ciphertext = ${sealed} WHERE user_id = ${k.user_id} AND provider = ${k.provider}`; + changed = true; + } + + const rows = (await tx`SELECT config_json FROM ai_settings WHERE scope = 'server' AND owner_id = ''`) as { config_json: string }[]; + const cfg = rows[0] ? parseObject(rows[0].config_json) : null; + if (!cfg) return { changed, pending }; + let rowChanged = false; + if (typeof cfg.apiKeyCipher === "string" && isLegacyToken(cfg.apiKeyCipher)) { + try { + cfg.apiKeyCipher = await box.seal(await box.openLegacy(cfg.apiKeyCipher), "server-api-key", SERVER_KEY_AAD); + rowChanged = true; + } catch { pending = true; } + } + if (Array.isArray(cfg.mcpServers)) { + const servers: unknown[] = []; + for (const s of cfg.mcpServers) { + // Malformed entries are kept as they are (the settings code skips them); they must not stop the upgrade. + if (!s || typeof s !== "object" || typeof (s as StoredMcpServer).id !== "string") { servers.push(s); continue; } + const r = await upgradeStoredMcpServer(box, s as StoredMcpServer); + servers.push(r.server); + if (r.changed) rowChanged = true; + if (r.pending) pending = true; + } + cfg.mcpServers = servers; + } + if (rowChanged) { + await tx`UPDATE ai_settings SET config_json = ${JSON.stringify(cfg)}, updated_at = ${now} WHERE scope = 'server' AND owner_id = ''`; + changed = true; + } + return { changed, pending }; +} + +/** + * Data the v1.1 SQL migrations cannot move portably, in one transaction. Each step is recorded in schema_migrations + * when complete (names without `.sql`, so no file can share them): + * - `ai_data_v11`: per-course mentor toggles from config_json to ai_mentor_prefs; + * - `ai_secrets_v2`: every secret re-sealed in the v2 format (marked only when nothing legacy is left). + * Returns true if it changed anything now. The settings repo awaits it before first use. + */ +export async function upgradeAiData(db: Db, box: SecretBox): Promise { + return db.sql.begin(async (tx) => { + const done = new Set(((await tx`SELECT name FROM schema_migrations WHERE name IN (${AI_DATA_UPGRADE}, ${AI_SECRETS_UPGRADE})`) as { name: string }[]).map((r) => r.name)); + const now = new Date().toISOString(); + let changed = false; + if (!done.has(AI_DATA_UPGRADE)) { + await moveMentorToggles(tx, now); + await tx`INSERT INTO schema_migrations (name, applied_at) VALUES (${AI_DATA_UPGRADE}, ${now})`; + changed = true; + } + if (!done.has(AI_SECRETS_UPGRADE)) { + const r = await resealSecrets(tx, box, now); + if (r.changed) changed = true; + if (!r.pending) { + await tx`INSERT INTO schema_migrations (name, applied_at) VALUES (${AI_SECRETS_UPGRADE}, ${now})`; + changed = true; + } + } + return changed; + }); +} diff --git a/packages/feature-ai/src/web/AccountAiPage.tsx b/packages/feature-ai/src/web/AccountAiPage.tsx index 137cfe8..79fd713 100644 --- a/packages/feature-ai/src/web/AccountAiPage.tsx +++ b/packages/feature-ai/src/web/AccountAiPage.tsx @@ -1,10 +1,10 @@ import { useEffect, useState } from "react"; import { Pill } from "@stc/ui"; import { api, ApiError } from "./api"; -import { Field, PageHead } from "./components"; +import { Field, PageHead, UNREADABLE_KEY_TEXT } from "./components"; interface Account { - byok: { kind: "anthropic" | "openai"; model?: string; baseUrl?: string; hasApiKey: boolean } | null; + byok: { kind: "anthropic" | "openai"; model?: string; baseUrl?: string; hasApiKey: boolean; keyUnreadable?: boolean } | null; serverProvider: { kind: string; model: string | null } | null; } @@ -51,7 +51,8 @@ export default function AccountAiPage() {

- {acct.byok ?

A {acct.byok.kind === "anthropic" ? "Anthropic" : "OpenAI-compatible"} key is saved{acct.byok.model ? ` (model ${acct.byok.model})` : ""}.

: null} + {acct.byok?.keyUnreadable ?

{UNREADABLE_KEY_TEXT}

: null} + {acct.byok && !acct.byok.keyUnreadable ?

A {acct.byok.kind === "anthropic" ? "Anthropic" : "OpenAI-compatible"} key is saved{acct.byok.model ? ` (model ${acct.byok.model})` : ""}.

: null}
{ e.preventDefault(); save(); }}>
diff --git a/packages/feature-ai/src/web/AdminAiPage.tsx b/packages/feature-ai/src/web/AdminAiPage.tsx index 21c4ad5..220527d 100644 --- a/packages/feature-ai/src/web/AdminAiPage.tsx +++ b/packages/feature-ai/src/web/AdminAiPage.tsx @@ -1,14 +1,14 @@ import { useEffect, useState } from "react"; import { Pill } from "@stc/ui"; import { api, ApiError } from "./api"; -import { Field, PageHead } from "./components"; +import { Field, PageHead, UnreadableSecretsWarning, type UnreadableSecretItem } from "./components"; type Kind = "" | "anthropic" | "openai"; interface ProviderOut { kind: "anthropic" | "openai"; model?: string; baseUrl?: string; effort?: string; hasApiKey: boolean } -interface McpServer { id: string; name: string; transport: "stdio" | "http"; command?: string; args?: string[]; env?: Record; url?: string; headers?: Record; courseIds?: string[] } -interface Config { provider: ProviderOut | null; mcpServers: McpServer[]; allowedBaseUrls: string[] } +interface McpServer { id: string; name: string; transport: "stdio" | "http"; command?: string; args?: string[]; env?: Record; url?: string; headers?: Record; courseIds?: string[]; clearSecrets?: ("env" | "headers")[] } +interface Config { provider: ProviderOut | null; mcpServers: McpServer[]; allowedBaseUrls: string[]; unreadableSecrets?: UnreadableSecretItem[] } -type ServerDraft = { id: string; name: string; transport: "stdio" | "http"; command: string; args: string; env: string; url: string; headers?: Record; courses: string; test?: string }; +type ServerDraft = { id: string; name: string; transport: "stdio" | "http"; command: string; args: string; env: string; url: string; headers?: Record; courses: string; test?: string; clearSecrets?: ("env" | "headers")[] }; const toDraft = (s: McpServer): ServerDraft => ({ id: s.id, name: s.name, transport: s.transport, command: s.command ?? "", args: (s.args ?? []).join(" "), env: Object.entries(s.env ?? {}).map(([k, v]) => `${k}=${v}`).join("\n"), url: s.url ?? "", headers: s.headers, courses: (s.courseIds ?? []).join(", "), @@ -19,6 +19,7 @@ const fromDraft = (d: ServerDraft): McpServer => ({ id: d.id, name: d.name || d.id, transport: d.transport, ...(d.transport === "stdio" ? { command: d.command, args: d.args.trim() ? d.args.trim().split(/\s+/) : [], env: parseEnv(d.env) } : { url: d.url, ...(d.headers ? { headers: d.headers } : {}) }), courseIds: d.courses.split(",").map((s) => s.trim()).filter(Boolean), + ...(d.clearSecrets?.length ? { clearSecrets: d.clearSecrets } : {}), }); export default function AdminAiPage() { @@ -80,6 +81,19 @@ export default function AdminAiPage() {

AI is off until you choose a provider here. Learners and authors can also add their own key, which is used only for their own requests. Keys are encrypted on the server and never shown again.

+ { + if (item.kind !== "mcp") return Paste the key again below, or tick "Remove the saved key".; + const i = servers.findIndex((x) => x.id === item.serverId); + const marked = i >= 0 && !!servers[i]!.clearSecrets?.includes(item.field); + if (i < 0) return null; + return marked ? Will be discarded when you save. : ( + setServer(i, { clearSecrets: [...(servers[i]!.clearSecrets ?? []), item.field] })}>Discard stored value + ); + }} + /> + { e.preventDefault(); save(); }}>

Server provider

@@ -157,7 +171,7 @@ export default function AdminAiPage() {
)} {s.transport === "stdio" ? ( - +