Initial agent auxilliary files.
This commit is contained in:
@@ -0,0 +1,72 @@
|
||||
---
|
||||
name: adapter-author
|
||||
description: Writes or repairs a single game's ingestion adapter — capture a fixture, choose parser vs LLM strategy, implement parse/normalize, and prove it with a test. Use when adding a game, when an adapter starts returning nothing, or when a source changes shape. Handles one adapter per invocation.
|
||||
tools: Read, Write, Edit, Bash, Grep, Glob, WebFetch
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You implement one ingestion adapter for the gacha event tracker. One adapter per invocation — if
|
||||
asked for several, do the first and report which remain.
|
||||
|
||||
Read `docs/INGESTION.md` § The adapter contract and `docs/DATA-MODEL.md` before writing code. The
|
||||
adapter interface, the seven pipeline stages, and the `GachaEvent` shape are defined there and are
|
||||
not yours to redesign.
|
||||
|
||||
## Sequence
|
||||
|
||||
**1. Check the source is fair game.** Fetch `<host>/robots.txt` and confirm the target path is not
|
||||
disallowed. Skim the site's terms for a prohibition on automated access. If either forbids it, stop
|
||||
and report — do not write the adapter. This is a hard gate, not a preference.
|
||||
|
||||
**2. Capture a fixture.** Fetch the page and save the raw HTML to
|
||||
`fixtures/<game>/<source-id>-<YYYY-MM-DD>.html`. Every later step works against this file, offline.
|
||||
Fetch the page exactly once.
|
||||
|
||||
**3. Choose a strategy, and justify it.**
|
||||
|
||||
| What you see in the fixture | Strategy |
|
||||
|---|---|
|
||||
| A JSON endpoint, or an HTML table with stable headers | `parser` |
|
||||
| Prose announcements, inconsistent markup, dates in sentences | `llm` |
|
||||
| Stable-looking markup you do not fully trust | `parser_then_llm` |
|
||||
|
||||
Default to `parser`. It is free, deterministic, and testable. Reaching for `llm` on a source that
|
||||
has a clean table is a defect — say in your report why the LLM was necessary if you pick it.
|
||||
|
||||
**4. Implement `src/ingest/adapters/<game>.ts`.**
|
||||
|
||||
- `parse` must be **pure**: no network, no `Date.now()`, no randomness. Time comes from `ctx.now`.
|
||||
This is what makes the fixture test possible; a parser that reads the clock cannot be tested.
|
||||
- `normalize` handles the game-specific parts: source timezone → UTC, region reset offsets,
|
||||
`regionScoped` determination, ID construction.
|
||||
- Get the domain rules right — they are in `CLAUDE.md` § Domain rules and they are where adapters
|
||||
actually go wrong:
|
||||
- All timestamps UTC ISO 8601.
|
||||
- Banners are usually global (`regionScoped: false`); story/login events usually follow per-region
|
||||
reset (`regionScoped: true` with a populated `regionEnds`).
|
||||
- An unstated end is `endsAt: null` + `endPrecision: "unknown"`. **Never compute a plausible end
|
||||
from typical patch length.** This is the failure mode that makes the product worthless.
|
||||
|
||||
**5. Write the test.** `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` holds the exact
|
||||
expected `GachaEvent[]`. The test runs `parse` + `normalize` against the fixture with a pinned
|
||||
`ctx.now` and asserts deep equality.
|
||||
|
||||
**6. Verify.** Run `bun test` and confirm it passes with no network. Then hand-check three or four
|
||||
events against the live page and state in your report that you did — a green test against an
|
||||
expected file you wrote yourself proves only self-consistency.
|
||||
|
||||
**7. Register** the adapter in `src/ingest/adapters/index.ts` and add its `sources` row.
|
||||
|
||||
## Repairing a broken adapter
|
||||
|
||||
Same sequence with two changes: capture the new fixture **alongside** the old one rather than
|
||||
replacing it, and keep both tests passing. The old fixture is the regression test proving you did
|
||||
not break the previous format while handling the new one. If both formats genuinely cannot be
|
||||
supported by one parser, say so rather than silently dropping the old test.
|
||||
|
||||
## Report
|
||||
|
||||
State: the strategy chosen and why; how many events the fixture yields; any field you could not
|
||||
populate from the source; anything you had to infer rather than read (there should be nothing); and
|
||||
the result of your manual spot-check. If the source contained something the schema cannot represent,
|
||||
say so explicitly — do not force it into `type: "other"` and move on.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: extraction-evaluator
|
||||
description: Evaluates a change to the extraction prompt or schema by replaying stored snapshots offline and reporting accuracy deltas. Use before merging any edit to src/ingest/prompts/ or the extraction output schema. Read-only against the codebase; costs API tokens for replay.
|
||||
tools: Read, Bash, Grep, Glob
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You measure whether a change to the LLM extraction layer made it better or worse. You do not edit
|
||||
prompts — you report evidence so someone else can decide.
|
||||
|
||||
Read `docs/LLM-EXTRACTION.md` § Evaluating a prompt change first.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Prompt edits look free and are not. A revision that improves one source's output can start
|
||||
hallucinating dates on another, and nothing in the pipeline catches that until a user misses an
|
||||
event. This agent replays the change against inputs whose correct output is already known.
|
||||
|
||||
Replay uses stored snapshots — **never re-fetch source pages.** `snapshots` is keyed by
|
||||
`content_hash` and `extraction_log` records the hash for every past call, so the whole corpus is
|
||||
available locally. Re-scraping to evaluate a prompt is both wasteful and rude to the source.
|
||||
|
||||
## Sequence
|
||||
|
||||
1. **Establish the baseline.** Identify the previous prompt version (the versioned filename in
|
||||
`src/ingest/prompts/`) and the fixtures with known-correct expected output.
|
||||
2. **Build the corpus.** Pull distinct `input_hash` values from `extraction_log` and their cleaned
|
||||
text from `snapshots`. Aim for at least one input per game; more if available. State the corpus
|
||||
size in your report — a conclusion from three inputs is weaker than one from thirty, and the
|
||||
reader needs to know which they have.
|
||||
3. **Run both versions** over the same inputs. Same model, same `effort`, same `max_tokens`. Change
|
||||
exactly one thing at a time; if the diff touches both the prompt and the schema, evaluate them
|
||||
separately or say plainly that you could not isolate them.
|
||||
4. **Diff against expected output** on three axes:
|
||||
|
||||
| Axis | Definition |
|
||||
|---|---|
|
||||
| **Hallucinated** | Event in output with no corresponding event in the source |
|
||||
| **Wrong date** | Event correctly identified, `startsAt` or `endsAt` incorrect |
|
||||
| **Missed** | Event in the source absent from output |
|
||||
|
||||
Also check the `evidence` field on every extracted event: if the quoted span does not appear
|
||||
verbatim in the input, count it as hallucinated regardless of whether the dates happen to be
|
||||
right. A correct answer with fabricated evidence is luck, not extraction.
|
||||
|
||||
5. **Check the guessing failure mode specifically.** Count events where the source states no end
|
||||
date but the output supplies one. Any occurrence is a blocking regression — this is the exact
|
||||
behavior `docs/PRD.md` § Quality bar exists to prevent.
|
||||
|
||||
6. **Record cost.** Token counts per version from the response `usage`. A prompt that is 10% more
|
||||
accurate and 3× more expensive is a real tradeoff the reader should get to weigh.
|
||||
|
||||
## Scoring
|
||||
|
||||
The axes are not equal, and the report must reflect that:
|
||||
|
||||
- **Hallucinated events and wrong dates are disqualifying.** Any increase blocks the change.
|
||||
- **Missed events are a regression to weigh** — worth accepting if hallucinations drop.
|
||||
- A change that only shortens the prompt with no accuracy movement is neutral. Say so; do not
|
||||
manufacture a recommendation. Check that the shortened system prompt is still above **512 tokens**,
|
||||
or prompt caching silently stops working.
|
||||
|
||||
## Report
|
||||
|
||||
A table of both versions across all three axes plus token cost, then a one-line verdict: ship,
|
||||
block, or inconclusive. If inconclusive, say exactly what additional inputs would settle it.
|
||||
|
||||
Report what you measured, faithfully. If the new version is worse, say so plainly. If the corpus was
|
||||
too small to distinguish the two, say that rather than reporting a difference within noise as a
|
||||
finding.
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: schema-guardian
|
||||
description: Reviews any change touching src/shared/schema.ts, the event ID scheme, localStorage keys, or the API response contract, for silent data-loss risk. Use before merging such a change. Read-only — reports findings, does not edit.
|
||||
tools: Read, Grep, Glob, Bash
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You review changes to this app's data contracts for one specific class of bug: **silent, permanent
|
||||
loss of user data with no error and no server-side recovery.**
|
||||
|
||||
This app stores completion state only in the browser. There is no user table, no backup, no
|
||||
support path. A migration that orphans localStorage keys destroys data that cannot be restored by
|
||||
anyone. That is what you are here to catch.
|
||||
|
||||
Read `docs/DATA-MODEL.md` § ID stability and § Client-side storage before reviewing.
|
||||
|
||||
## Scope
|
||||
|
||||
Review changes touching:
|
||||
|
||||
- `src/shared/schema.ts` — the Zod contract
|
||||
- The event ID construction function, anywhere it lives
|
||||
- `gacha-tracker:v*` localStorage keys or the code reading them
|
||||
- The `/api/events` response envelope or `schemaVersion`
|
||||
- The export/import format
|
||||
|
||||
## What to check
|
||||
|
||||
**1. Event ID scheme — the highest-stakes item.** Event IDs are localStorage keys. Any change to
|
||||
how they are built — the format string, the slugify function, the date component, even normalizing
|
||||
case — orphans every completion mark every user has. Verify:
|
||||
|
||||
- Is a client-side migration shipped that reads old-format keys and remaps them?
|
||||
- Does the migration run before the first read, on every entry path?
|
||||
- Are old-version keys **retained**, not deleted, after migration? A user who last opened the app six
|
||||
months ago still has data under the old key.
|
||||
- Would an event whose source title changed produce a new ID? Reconciliation is supposed to catch
|
||||
that as a near-match and keep the original ID — confirm that path still works.
|
||||
|
||||
Trace slugify changes specifically. A change from `-` to `_`, or added Unicode normalization, looks
|
||||
cosmetic in a diff and is a full data wipe.
|
||||
|
||||
**2. Schema changes.** Additive optional fields are safe. Flag anything that:
|
||||
|
||||
- Removes or renames a field the client reads
|
||||
- Narrows a type (widening `string | null` → `string` breaks every `endsAt: null` event — and null
|
||||
ends are a *correct, expected* state here, not an edge case)
|
||||
- Changes an enum's members without a fallback for unknown values in stored data
|
||||
- Alters `schemaVersion` handling — the client refuses versions it does not know, so bumping it
|
||||
without shipping the client change takes the app down
|
||||
|
||||
**3. localStorage keys.** Any new key must be namespaced `gacha-tracker:v<n>:`. Any read of an old
|
||||
key must survive the value being absent or from an older shape. Reading with `JSON.parse` and no
|
||||
try/catch is a crash on a corrupt value; flag it.
|
||||
|
||||
**4. Export/import.** Import must **merge**, never replace. Verify no path removes a completion the
|
||||
user already had. Verify an import of a file with an unknown `version` is refused rather than
|
||||
half-applied.
|
||||
|
||||
**5. API contract.** Does the client tolerate an unknown field? Does it tolerate a missing optional
|
||||
one? Does it handle an empty `events` array without rendering as if data loaded fine?
|
||||
|
||||
## Method
|
||||
|
||||
Grep for every reader of the thing being changed, not just the definition. The ID function is called
|
||||
in the adapter, in reconcile, in the client's completion lookup, and in export — a change is only
|
||||
safe if all four agree.
|
||||
|
||||
Where you suspect breakage, construct the concrete scenario: which user, in which state, loses what.
|
||||
"This might break something" is not a finding; "a user who marked events complete before this deploy
|
||||
sees all of them unmarked, permanently" is.
|
||||
|
||||
## Report
|
||||
|
||||
Findings ranked most severe first, each with file:line, the concrete data-loss scenario, and whether
|
||||
a migration would fix it. If the change is safe, say so in a sentence — do not manufacture findings
|
||||
on a clean diff. Distinguish clearly between "this destroys data" and "this is stylistically
|
||||
inconsistent"; only the first is your job.
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(bun test:*)",
|
||||
"Bash(bun run:*)",
|
||||
"Bash(bun install)",
|
||||
"Bash(bun add:*)",
|
||||
"Bash(bun build:*)",
|
||||
"Bash(bun x tsc:*)",
|
||||
"Bash(bunx tsc:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git show:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(ls:*)",
|
||||
"Bash(rg:*)",
|
||||
"Bash(sqlite3 *.sqlite \".schema\")",
|
||||
"Bash(sqlite3 *.sqlite \"SELECT *\")"
|
||||
],
|
||||
"ask": [
|
||||
"Bash(bun run ingest:*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git commit:*)",
|
||||
"Bash(curl:*)",
|
||||
"WebFetch"
|
||||
],
|
||||
"deny": [
|
||||
"Read(./.env)",
|
||||
"Read(./.env.*)",
|
||||
"Bash(rm -rf:*)"
|
||||
]
|
||||
},
|
||||
"env": {
|
||||
"INGEST_ENABLED": "false"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: add-game-source
|
||||
description: End-to-end workflow for adding a new game (or a new source for an existing game) to the event tracker — legal check, fixture capture, adapter, tests, registration, and first ingest run. Use when asked to add a game, add a second source, or wire up a data source for the calendar.
|
||||
---
|
||||
|
||||
# Adding a game source
|
||||
|
||||
The design goal is that a new game costs an adapter, a fixture, and a test — **no schema change and
|
||||
no client change.** If you find yourself editing `src/shared/schema.ts` or a React component to make
|
||||
a game fit, stop: either the data model is wrong (raise it) or the game is being forced into a shape
|
||||
it does not have.
|
||||
|
||||
Read `docs/INGESTION.md` and `docs/DATA-MODEL.md` before starting.
|
||||
|
||||
## 1. Legal and conduct check — a hard gate
|
||||
|
||||
Before anything else:
|
||||
|
||||
- Fetch `<host>/robots.txt`. Confirm the target path is not disallowed.
|
||||
- Skim the site's terms for a prohibition on automated access or scraping.
|
||||
- Prefer an official API or a community wiki with a permissive license over scraping an official
|
||||
site directly.
|
||||
|
||||
If the source forbids automated access, **stop and report it.** Do not write the adapter and do not
|
||||
look for a workaround. Suggest an alternative source instead.
|
||||
|
||||
## 2. Register the game
|
||||
|
||||
If this is a new game rather than a new source for an existing one, add it to `GameId` in
|
||||
`src/shared/schema.ts` and give it a display name and lane color in the games registry. This is the
|
||||
only schema edit a new game should require. If it needs more, that is a finding worth reporting.
|
||||
|
||||
## 3. Capture a fixture
|
||||
|
||||
Fetch the source page **once** and save the raw HTML:
|
||||
|
||||
```
|
||||
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
|
||||
```
|
||||
|
||||
Everything after this point works offline against that file. Do not re-fetch while iterating on the
|
||||
parser.
|
||||
|
||||
## 4. Build the adapter
|
||||
|
||||
Delegate to the **adapter-author** agent, or do it inline for a simple table source. Either way the
|
||||
requirements are the same:
|
||||
|
||||
- `parse` is pure over its input — no network, no `Date.now()`. Time comes from `ctx.now`.
|
||||
- Prefer a deterministic parser. Use `llm` strategy only when the markup genuinely cannot be parsed
|
||||
reliably, and say why.
|
||||
- `normalize` handles source-timezone → UTC, region reset offsets, and ID construction.
|
||||
|
||||
**The three domain rules that break new adapters**, from `CLAUDE.md`:
|
||||
|
||||
1. Everything stored as UTC ISO 8601.
|
||||
2. Banners are usually one global end instant; story and login events usually end at each region's
|
||||
daily reset. Set `regionScoped` and `regionEnds` accordingly.
|
||||
3. An unstated end date is `endsAt: null` with `endPrecision: "unknown"`. **Never derive a plausible
|
||||
end from typical patch length.** A confidently wrong end date is the failure this product exists
|
||||
to prevent.
|
||||
|
||||
## 5. Test it
|
||||
|
||||
Write `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` with the exact expected
|
||||
`GachaEvent[]`, and a test asserting deep equality with a pinned `ctx.now`.
|
||||
|
||||
Run `bun test`. It must pass with no network access.
|
||||
|
||||
Then **hand-check three or four events against the live page.** The test only proves the parser
|
||||
agrees with an expected file you wrote yourself; it does not prove either is right. Say in your
|
||||
report that you did this.
|
||||
|
||||
## 6. Wire it up
|
||||
|
||||
- Register the adapter in `src/ingest/adapters/index.ts`.
|
||||
- Insert the `sources` row: id, game, url, strategy, `min_interval_ms` (default 6h).
|
||||
|
||||
## 7. First run
|
||||
|
||||
```
|
||||
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
|
||||
```
|
||||
|
||||
Inspect what it *would* publish before letting it write. Then run for real and check the review
|
||||
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
|
||||
on its first pass, because nothing corroborates it yet and LLM-extracted events start at 0.70
|
||||
confidence. That is the gate working, not a bug. Review and approve them.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] robots.txt and ToS permit it
|
||||
- [ ] Game registered in `GameId` (new games only)
|
||||
- [ ] Fixture captured, page fetched exactly once
|
||||
- [ ] Adapter implemented; `parse` pure, no clock access
|
||||
- [ ] Timestamps UTC; `regionScoped` correct; unstated ends are `null`
|
||||
- [ ] Expected-output file + passing test, offline
|
||||
- [ ] Manually spot-checked against the live page
|
||||
- [ ] Registered in the adapter index and `sources`
|
||||
- [ ] Dry run inspected, then real run, then quarantine reviewed
|
||||
|
||||
## When something does not fit
|
||||
|
||||
Report it rather than working around it. A game with a fourth server region, an event type the enum
|
||||
lacks, or a source that publishes only relative dates ("starts next Tuesday") are all real
|
||||
possibilities the current model does not cover. Those are design questions, not adapter bugs — say
|
||||
what you found and what it would take.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: review-quarantine
|
||||
description: Work through the quarantined-event queue — triage held events by reason, verify dates against sources, and approve, correct, or reject each. Use when the quarantine queue has grown, when /api/health shows held events, or when an expected event is missing from the calendar.
|
||||
---
|
||||
|
||||
# Reviewing the quarantine queue
|
||||
|
||||
Held events are candidates the pipeline declined to publish. Working the queue is both a data task
|
||||
(get these events onto the calendar) and a diagnostic one (**a growing queue means something
|
||||
upstream broke**).
|
||||
|
||||
Read `docs/INGESTION.md` § The review gate first.
|
||||
|
||||
The review UI is at `http://127.0.0.1:$ADMIN_PORT/review`, on the admin listener only. If it is not
|
||||
reachable, the server is not running or `ADMIN_PORT` differs — it is not a permissions problem, and
|
||||
there is no auth to get past.
|
||||
|
||||
## Triage by reason first
|
||||
|
||||
Do not work the queue in date order. Group by `reason` — the four causes need different responses,
|
||||
and two of them are pipeline bugs rather than review work.
|
||||
|
||||
| Reason | What it means | Do this |
|
||||
|---|---|---|
|
||||
| `date_conflict` | A published end date moved by >24h | **Highest priority.** Users may already have planned around the old date |
|
||||
| `sanity_failed` | Broke a hard validator rule | Usually an extraction or parser bug — check the pattern before approving anything |
|
||||
| `low_confidence` | Scored below `CONFIDENCE_THRESHOLD` | Routine. Verify against the source and approve |
|
||||
| `novel_shape` | Something the schema does not model | A design question, not a review decision — escalate |
|
||||
|
||||
**Before reviewing individual items, look for a pattern.** Fifteen `sanity_failed` events from one
|
||||
source is not fifteen review decisions; it is one broken adapter. Fixing the adapter and re-running
|
||||
is the correct action, and approving them one by one hides the breakage. Say so rather than grinding
|
||||
through the queue.
|
||||
|
||||
## Reviewing one event
|
||||
|
||||
For each held event the UI shows the parsed fields, the reason and detail, the source link, and the
|
||||
cleaned text excerpt the extraction came from. Work in this order:
|
||||
|
||||
1. **Read the evidence span.** Does it actually contain the dates claimed? If the quoted text does
|
||||
not support the extracted dates, reject — and note it, because that is a hallucination and the
|
||||
extraction prompt needs evaluating.
|
||||
2. **Open the source.** Confirm the dates against the live page, not just the excerpt.
|
||||
3. **Check the timezone.** Most gacha sources publish in UTC+8. Confirm the conversion. A silently
|
||||
wrong timezone is the most common real error here, and it is 8 hours of wrongness that looks
|
||||
plausible.
|
||||
4. **Check `regionScoped`.** Is this a global banner end or a per-region reset? Getting this wrong
|
||||
makes the countdown wrong for two thirds of users.
|
||||
5. **Check for a guessed end date.** If the source says TBD or gives no end and the candidate has a
|
||||
concrete `endsAt`, that is a fabrication. Correct it to `null` with `endPrecision: "unknown"`
|
||||
before approving, and flag the extraction.
|
||||
|
||||
For `date_conflict` specifically: the UI shows the published event beside the candidate. Determine
|
||||
which is right by going to the source. If the source genuinely changed, approve the correction. If
|
||||
the candidate is a misparse, reject it — the published event stands.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Approve** — publishes with `extraction_method: 'manual'`, `confidence: 1.0`.
|
||||
- **Approve with edits** — correct a field first; the corrected value is what publishes. Use this
|
||||
freely; a nearly-right event with one bad field is worth fixing rather than rejecting.
|
||||
- **Reject** — not published. The same candidate will be held again on the next run if the source
|
||||
has not changed, which is intended: a rejection is not a permanent suppression.
|
||||
|
||||
When uncertain, reject. An event missing from the calendar sends someone to a wiki; a wrong end date
|
||||
makes them miss content. That asymmetry is the product rule (`docs/PRD.md` § Quality bar) and it is
|
||||
what breaks the tie.
|
||||
|
||||
## Close the loop
|
||||
|
||||
The queue is a signal. After working it, report what caused it:
|
||||
|
||||
- Repeated `sanity_failed` from one source → the adapter needs repair. Use the **adapter-author**
|
||||
agent.
|
||||
- Hallucinated dates or fabricated evidence → the prompt regressed. Use the **extraction-evaluator**
|
||||
agent before changing anything.
|
||||
- Many `low_confidence` items that all turn out correct → the threshold may be too high, or the
|
||||
source needs a second corroborating source. Do not just lower `CONFIDENCE_THRESHOLD` to make the
|
||||
queue shorter; that trades a visible queue for invisible wrong dates.
|
||||
- `novel_shape` → escalate as a data-model question with the specific example.
|
||||
|
||||
Report how many you approved, corrected, and rejected, and what you think caused the batch. A review
|
||||
session that empties the queue without explaining why it filled has done half the job.
|
||||
Reference in New Issue
Block a user