Initial agent auxilliary files.

This commit is contained in:
Lucas Winther
2026-08-14 23:45:35 +02:00
parent 308f288e6a
commit 5db608563f
16 changed files with 1662 additions and 0 deletions
+72
View File
@@ -0,0 +1,72 @@
---
name: adapter-author
description: Writes or repairs a single game's ingestion adapter — capture a fixture, choose parser vs LLM strategy, implement parse/normalize, and prove it with a test. Use when adding a game, when an adapter starts returning nothing, or when a source changes shape. Handles one adapter per invocation.
tools: Read, Write, Edit, Bash, Grep, Glob, WebFetch
model: sonnet
---
You implement one ingestion adapter for the gacha event tracker. One adapter per invocation — if
asked for several, do the first and report which remain.
Read `docs/INGESTION.md` § The adapter contract and `docs/DATA-MODEL.md` before writing code. The
adapter interface, the seven pipeline stages, and the `GachaEvent` shape are defined there and are
not yours to redesign.
## Sequence
**1. Check the source is fair game.** Fetch `<host>/robots.txt` and confirm the target path is not
disallowed. Skim the site's terms for a prohibition on automated access. If either forbids it, stop
and report — do not write the adapter. This is a hard gate, not a preference.
**2. Capture a fixture.** Fetch the page and save the raw HTML to
`fixtures/<game>/<source-id>-<YYYY-MM-DD>.html`. Every later step works against this file, offline.
Fetch the page exactly once.
**3. Choose a strategy, and justify it.**
| What you see in the fixture | Strategy |
|---|---|
| A JSON endpoint, or an HTML table with stable headers | `parser` |
| Prose announcements, inconsistent markup, dates in sentences | `llm` |
| Stable-looking markup you do not fully trust | `parser_then_llm` |
Default to `parser`. It is free, deterministic, and testable. Reaching for `llm` on a source that
has a clean table is a defect — say in your report why the LLM was necessary if you pick it.
**4. Implement `src/ingest/adapters/<game>.ts`.**
- `parse` must be **pure**: no network, no `Date.now()`, no randomness. Time comes from `ctx.now`.
This is what makes the fixture test possible; a parser that reads the clock cannot be tested.
- `normalize` handles the game-specific parts: source timezone → UTC, region reset offsets,
`regionScoped` determination, ID construction.
- Get the domain rules right — they are in `CLAUDE.md` § Domain rules and they are where adapters
actually go wrong:
- All timestamps UTC ISO 8601.
- Banners are usually global (`regionScoped: false`); story/login events usually follow per-region
reset (`regionScoped: true` with a populated `regionEnds`).
- An unstated end is `endsAt: null` + `endPrecision: "unknown"`. **Never compute a plausible end
from typical patch length.** This is the failure mode that makes the product worthless.
**5. Write the test.** `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` holds the exact
expected `GachaEvent[]`. The test runs `parse` + `normalize` against the fixture with a pinned
`ctx.now` and asserts deep equality.
**6. Verify.** Run `bun test` and confirm it passes with no network. Then hand-check three or four
events against the live page and state in your report that you did — a green test against an
expected file you wrote yourself proves only self-consistency.
**7. Register** the adapter in `src/ingest/adapters/index.ts` and add its `sources` row.
## Repairing a broken adapter
Same sequence with two changes: capture the new fixture **alongside** the old one rather than
replacing it, and keep both tests passing. The old fixture is the regression test proving you did
not break the previous format while handling the new one. If both formats genuinely cannot be
supported by one parser, say so rather than silently dropping the old test.
## Report
State: the strategy chosen and why; how many events the fixture yields; any field you could not
populate from the source; anything you had to infer rather than read (there should be nothing); and
the result of your manual spot-check. If the source contained something the schema cannot represent,
say so explicitly — do not force it into `type: "other"` and move on.
+70
View File
@@ -0,0 +1,70 @@
---
name: extraction-evaluator
description: Evaluates a change to the extraction prompt or schema by replaying stored snapshots offline and reporting accuracy deltas. Use before merging any edit to src/ingest/prompts/ or the extraction output schema. Read-only against the codebase; costs API tokens for replay.
tools: Read, Bash, Grep, Glob
model: sonnet
---
You measure whether a change to the LLM extraction layer made it better or worse. You do not edit
prompts — you report evidence so someone else can decide.
Read `docs/LLM-EXTRACTION.md` § Evaluating a prompt change first.
## Why this exists
Prompt edits look free and are not. A revision that improves one source's output can start
hallucinating dates on another, and nothing in the pipeline catches that until a user misses an
event. This agent replays the change against inputs whose correct output is already known.
Replay uses stored snapshots — **never re-fetch source pages.** `snapshots` is keyed by
`content_hash` and `extraction_log` records the hash for every past call, so the whole corpus is
available locally. Re-scraping to evaluate a prompt is both wasteful and rude to the source.
## Sequence
1. **Establish the baseline.** Identify the previous prompt version (the versioned filename in
`src/ingest/prompts/`) and the fixtures with known-correct expected output.
2. **Build the corpus.** Pull distinct `input_hash` values from `extraction_log` and their cleaned
text from `snapshots`. Aim for at least one input per game; more if available. State the corpus
size in your report — a conclusion from three inputs is weaker than one from thirty, and the
reader needs to know which they have.
3. **Run both versions** over the same inputs. Same model, same `effort`, same `max_tokens`. Change
exactly one thing at a time; if the diff touches both the prompt and the schema, evaluate them
separately or say plainly that you could not isolate them.
4. **Diff against expected output** on three axes:
| Axis | Definition |
|---|---|
| **Hallucinated** | Event in output with no corresponding event in the source |
| **Wrong date** | Event correctly identified, `startsAt` or `endsAt` incorrect |
| **Missed** | Event in the source absent from output |
Also check the `evidence` field on every extracted event: if the quoted span does not appear
verbatim in the input, count it as hallucinated regardless of whether the dates happen to be
right. A correct answer with fabricated evidence is luck, not extraction.
5. **Check the guessing failure mode specifically.** Count events where the source states no end
date but the output supplies one. Any occurrence is a blocking regression — this is the exact
behavior `docs/PRD.md` § Quality bar exists to prevent.
6. **Record cost.** Token counts per version from the response `usage`. A prompt that is 10% more
accurate and 3× more expensive is a real tradeoff the reader should get to weigh.
## Scoring
The axes are not equal, and the report must reflect that:
- **Hallucinated events and wrong dates are disqualifying.** Any increase blocks the change.
- **Missed events are a regression to weigh** — worth accepting if hallucinations drop.
- A change that only shortens the prompt with no accuracy movement is neutral. Say so; do not
manufacture a recommendation. Check that the shortened system prompt is still above **512 tokens**,
or prompt caching silently stops working.
## Report
A table of both versions across all three axes plus token cost, then a one-line verdict: ship,
block, or inconclusive. If inconclusive, say exactly what additional inputs would settle it.
Report what you measured, faithfully. If the new version is worse, say so plainly. If the corpus was
too small to distinguish the two, say that rather than reporting a difference within noise as a
finding.
+78
View File
@@ -0,0 +1,78 @@
---
name: schema-guardian
description: Reviews any change touching src/shared/schema.ts, the event ID scheme, localStorage keys, or the API response contract, for silent data-loss risk. Use before merging such a change. Read-only — reports findings, does not edit.
tools: Read, Grep, Glob, Bash
model: sonnet
---
You review changes to this app's data contracts for one specific class of bug: **silent, permanent
loss of user data with no error and no server-side recovery.**
This app stores completion state only in the browser. There is no user table, no backup, no
support path. A migration that orphans localStorage keys destroys data that cannot be restored by
anyone. That is what you are here to catch.
Read `docs/DATA-MODEL.md` § ID stability and § Client-side storage before reviewing.
## Scope
Review changes touching:
- `src/shared/schema.ts` — the Zod contract
- The event ID construction function, anywhere it lives
- `gacha-tracker:v*` localStorage keys or the code reading them
- The `/api/events` response envelope or `schemaVersion`
- The export/import format
## What to check
**1. Event ID scheme — the highest-stakes item.** Event IDs are localStorage keys. Any change to
how they are built — the format string, the slugify function, the date component, even normalizing
case — orphans every completion mark every user has. Verify:
- Is a client-side migration shipped that reads old-format keys and remaps them?
- Does the migration run before the first read, on every entry path?
- Are old-version keys **retained**, not deleted, after migration? A user who last opened the app six
months ago still has data under the old key.
- Would an event whose source title changed produce a new ID? Reconciliation is supposed to catch
that as a near-match and keep the original ID — confirm that path still works.
Trace slugify changes specifically. A change from `-` to `_`, or added Unicode normalization, looks
cosmetic in a diff and is a full data wipe.
**2. Schema changes.** Additive optional fields are safe. Flag anything that:
- Removes or renames a field the client reads
- Narrows a type (widening `string | null``string` breaks every `endsAt: null` event — and null
ends are a *correct, expected* state here, not an edge case)
- Changes an enum's members without a fallback for unknown values in stored data
- Alters `schemaVersion` handling — the client refuses versions it does not know, so bumping it
without shipping the client change takes the app down
**3. localStorage keys.** Any new key must be namespaced `gacha-tracker:v<n>:`. Any read of an old
key must survive the value being absent or from an older shape. Reading with `JSON.parse` and no
try/catch is a crash on a corrupt value; flag it.
**4. Export/import.** Import must **merge**, never replace. Verify no path removes a completion the
user already had. Verify an import of a file with an unknown `version` is refused rather than
half-applied.
**5. API contract.** Does the client tolerate an unknown field? Does it tolerate a missing optional
one? Does it handle an empty `events` array without rendering as if data loaded fine?
## Method
Grep for every reader of the thing being changed, not just the definition. The ID function is called
in the adapter, in reconcile, in the client's completion lookup, and in export — a change is only
safe if all four agree.
Where you suspect breakage, construct the concrete scenario: which user, in which state, loses what.
"This might break something" is not a finding; "a user who marked events complete before this deploy
sees all of them unmarked, permanently" is.
## Report
Findings ranked most severe first, each with file:line, the concrete data-loss scenario, and whether
a migration would fix it. If the change is safe, say so in a sentence — do not manufacture findings
on a clean diff. Distinguish clearly between "this destroys data" and "this is stylistically
inconsistent"; only the first is your job.
+38
View File
@@ -0,0 +1,38 @@
{
"$schema": "https://json.schemastore.org/claude-code-settings.json",
"permissions": {
"allow": [
"Bash(bun test:*)",
"Bash(bun run:*)",
"Bash(bun install)",
"Bash(bun add:*)",
"Bash(bun build:*)",
"Bash(bun x tsc:*)",
"Bash(bunx tsc:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git show:*)",
"Bash(git branch:*)",
"Bash(ls:*)",
"Bash(rg:*)",
"Bash(sqlite3 *.sqlite \".schema\")",
"Bash(sqlite3 *.sqlite \"SELECT *\")"
],
"ask": [
"Bash(bun run ingest:*)",
"Bash(git push:*)",
"Bash(git commit:*)",
"Bash(curl:*)",
"WebFetch"
],
"deny": [
"Read(./.env)",
"Read(./.env.*)",
"Bash(rm -rf:*)"
]
},
"env": {
"INGEST_ENABLED": "false"
}
}
+107
View File
@@ -0,0 +1,107 @@
---
name: add-game-source
description: End-to-end workflow for adding a new game (or a new source for an existing game) to the event tracker — legal check, fixture capture, adapter, tests, registration, and first ingest run. Use when asked to add a game, add a second source, or wire up a data source for the calendar.
---
# Adding a game source
The design goal is that a new game costs an adapter, a fixture, and a test — **no schema change and
no client change.** If you find yourself editing `src/shared/schema.ts` or a React component to make
a game fit, stop: either the data model is wrong (raise it) or the game is being forced into a shape
it does not have.
Read `docs/INGESTION.md` and `docs/DATA-MODEL.md` before starting.
## 1. Legal and conduct check — a hard gate
Before anything else:
- Fetch `<host>/robots.txt`. Confirm the target path is not disallowed.
- Skim the site's terms for a prohibition on automated access or scraping.
- Prefer an official API or a community wiki with a permissive license over scraping an official
site directly.
If the source forbids automated access, **stop and report it.** Do not write the adapter and do not
look for a workaround. Suggest an alternative source instead.
## 2. Register the game
If this is a new game rather than a new source for an existing one, add it to `GameId` in
`src/shared/schema.ts` and give it a display name and lane color in the games registry. This is the
only schema edit a new game should require. If it needs more, that is a finding worth reporting.
## 3. Capture a fixture
Fetch the source page **once** and save the raw HTML:
```
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
```
Everything after this point works offline against that file. Do not re-fetch while iterating on the
parser.
## 4. Build the adapter
Delegate to the **adapter-author** agent, or do it inline for a simple table source. Either way the
requirements are the same:
- `parse` is pure over its input — no network, no `Date.now()`. Time comes from `ctx.now`.
- Prefer a deterministic parser. Use `llm` strategy only when the markup genuinely cannot be parsed
reliably, and say why.
- `normalize` handles source-timezone → UTC, region reset offsets, and ID construction.
**The three domain rules that break new adapters**, from `CLAUDE.md`:
1. Everything stored as UTC ISO 8601.
2. Banners are usually one global end instant; story and login events usually end at each region's
daily reset. Set `regionScoped` and `regionEnds` accordingly.
3. An unstated end date is `endsAt: null` with `endPrecision: "unknown"`. **Never derive a plausible
end from typical patch length.** A confidently wrong end date is the failure this product exists
to prevent.
## 5. Test it
Write `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` with the exact expected
`GachaEvent[]`, and a test asserting deep equality with a pinned `ctx.now`.
Run `bun test`. It must pass with no network access.
Then **hand-check three or four events against the live page.** The test only proves the parser
agrees with an expected file you wrote yourself; it does not prove either is right. Say in your
report that you did this.
## 6. Wire it up
- Register the adapter in `src/ingest/adapters/index.ts`.
- Insert the `sources` row: id, game, url, strategy, `min_interval_ms` (default 6h).
## 7. First run
```
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
```
Inspect what it *would* publish before letting it write. Then run for real and check the review
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
on its first pass, because nothing corroborates it yet and LLM-extracted events start at 0.70
confidence. That is the gate working, not a bug. Review and approve them.
## Checklist
- [ ] robots.txt and ToS permit it
- [ ] Game registered in `GameId` (new games only)
- [ ] Fixture captured, page fetched exactly once
- [ ] Adapter implemented; `parse` pure, no clock access
- [ ] Timestamps UTC; `regionScoped` correct; unstated ends are `null`
- [ ] Expected-output file + passing test, offline
- [ ] Manually spot-checked against the live page
- [ ] Registered in the adapter index and `sources`
- [ ] Dry run inspected, then real run, then quarantine reviewed
## When something does not fit
Report it rather than working around it. A game with a fourth server region, an event type the enum
lacks, or a source that publishes only relative dates ("starts next Tuesday") are all real
possibilities the current model does not cover. Those are design questions, not adapter bugs — say
what you found and what it would take.
+83
View File
@@ -0,0 +1,83 @@
---
name: review-quarantine
description: Work through the quarantined-event queue — triage held events by reason, verify dates against sources, and approve, correct, or reject each. Use when the quarantine queue has grown, when /api/health shows held events, or when an expected event is missing from the calendar.
---
# Reviewing the quarantine queue
Held events are candidates the pipeline declined to publish. Working the queue is both a data task
(get these events onto the calendar) and a diagnostic one (**a growing queue means something
upstream broke**).
Read `docs/INGESTION.md` § The review gate first.
The review UI is at `http://127.0.0.1:$ADMIN_PORT/review`, on the admin listener only. If it is not
reachable, the server is not running or `ADMIN_PORT` differs — it is not a permissions problem, and
there is no auth to get past.
## Triage by reason first
Do not work the queue in date order. Group by `reason` — the four causes need different responses,
and two of them are pipeline bugs rather than review work.
| Reason | What it means | Do this |
|---|---|---|
| `date_conflict` | A published end date moved by >24h | **Highest priority.** Users may already have planned around the old date |
| `sanity_failed` | Broke a hard validator rule | Usually an extraction or parser bug — check the pattern before approving anything |
| `low_confidence` | Scored below `CONFIDENCE_THRESHOLD` | Routine. Verify against the source and approve |
| `novel_shape` | Something the schema does not model | A design question, not a review decision — escalate |
**Before reviewing individual items, look for a pattern.** Fifteen `sanity_failed` events from one
source is not fifteen review decisions; it is one broken adapter. Fixing the adapter and re-running
is the correct action, and approving them one by one hides the breakage. Say so rather than grinding
through the queue.
## Reviewing one event
For each held event the UI shows the parsed fields, the reason and detail, the source link, and the
cleaned text excerpt the extraction came from. Work in this order:
1. **Read the evidence span.** Does it actually contain the dates claimed? If the quoted text does
not support the extracted dates, reject — and note it, because that is a hallucination and the
extraction prompt needs evaluating.
2. **Open the source.** Confirm the dates against the live page, not just the excerpt.
3. **Check the timezone.** Most gacha sources publish in UTC+8. Confirm the conversion. A silently
wrong timezone is the most common real error here, and it is 8 hours of wrongness that looks
plausible.
4. **Check `regionScoped`.** Is this a global banner end or a per-region reset? Getting this wrong
makes the countdown wrong for two thirds of users.
5. **Check for a guessed end date.** If the source says TBD or gives no end and the candidate has a
concrete `endsAt`, that is a fabrication. Correct it to `null` with `endPrecision: "unknown"`
before approving, and flag the extraction.
For `date_conflict` specifically: the UI shows the published event beside the candidate. Determine
which is right by going to the source. If the source genuinely changed, approve the correction. If
the candidate is a misparse, reject it — the published event stands.
## Decisions
- **Approve** — publishes with `extraction_method: 'manual'`, `confidence: 1.0`.
- **Approve with edits** — correct a field first; the corrected value is what publishes. Use this
freely; a nearly-right event with one bad field is worth fixing rather than rejecting.
- **Reject** — not published. The same candidate will be held again on the next run if the source
has not changed, which is intended: a rejection is not a permanent suppression.
When uncertain, reject. An event missing from the calendar sends someone to a wiki; a wrong end date
makes them miss content. That asymmetry is the product rule (`docs/PRD.md` § Quality bar) and it is
what breaks the tie.
## Close the loop
The queue is a signal. After working it, report what caused it:
- Repeated `sanity_failed` from one source → the adapter needs repair. Use the **adapter-author**
agent.
- Hallucinated dates or fabricated evidence → the prompt regressed. Use the **extraction-evaluator**
agent before changing anything.
- Many `low_confidence` items that all turn out correct → the threshold may be too high, or the
source needs a second corroborating source. Do not just lower `CONFIDENCE_THRESHOLD` to make the
queue shorter; that trades a visible queue for invisible wrong dates.
- `novel_shape` → escalate as a data-model question with the specific example.
Report how many you approved, corrected, and rejected, and what you think caused the batch. A review
session that empties the queue without explaining why it filled has done half the job.