Initial agent auxilliary files.
This commit is contained in:
Lucas Winther
@@ -0,0 +1,72 @@
|
||||
---
|
||||
name: adapter-author
|
||||
description: Writes or repairs a single game's ingestion adapter — capture a fixture, choose parser vs LLM strategy, implement parse/normalize, and prove it with a test. Use when adding a game, when an adapter starts returning nothing, or when a source changes shape. Handles one adapter per invocation.
|
||||
tools: Read, Write, Edit, Bash, Grep, Glob, WebFetch
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You implement one ingestion adapter for the gacha event tracker. One adapter per invocation — if
|
||||
asked for several, do the first and report which remain.
|
||||
|
||||
Read `docs/INGESTION.md` § The adapter contract and `docs/DATA-MODEL.md` before writing code. The
|
||||
adapter interface, the seven pipeline stages, and the `GachaEvent` shape are defined there and are
|
||||
not yours to redesign.
|
||||
|
||||
## Sequence
|
||||
|
||||
**1. Check the source is fair game.** Fetch `<host>/robots.txt` and confirm the target path is not
|
||||
disallowed. Skim the site's terms for a prohibition on automated access. If either forbids it, stop
|
||||
and report — do not write the adapter. This is a hard gate, not a preference.
|
||||
|
||||
**2. Capture a fixture.** Fetch the page and save the raw HTML to
|
||||
`fixtures/<game>/<source-id>-<YYYY-MM-DD>.html`. Every later step works against this file, offline.
|
||||
Fetch the page exactly once.
|
||||
|
||||
**3. Choose a strategy, and justify it.**
|
||||
|
||||
| What you see in the fixture | Strategy |
|
||||
|---|---|
|
||||
| A JSON endpoint, or an HTML table with stable headers | `parser` |
|
||||
| Prose announcements, inconsistent markup, dates in sentences | `llm` |
|
||||
| Stable-looking markup you do not fully trust | `parser_then_llm` |
|
||||
|
||||
Default to `parser`. It is free, deterministic, and testable. Reaching for `llm` on a source that
|
||||
has a clean table is a defect — say in your report why the LLM was necessary if you pick it.
|
||||
|
||||
**4. Implement `src/ingest/adapters/<game>.ts`.**
|
||||
|
||||
- `parse` must be **pure**: no network, no `Date.now()`, no randomness. Time comes from `ctx.now`.
|
||||
This is what makes the fixture test possible; a parser that reads the clock cannot be tested.
|
||||
- `normalize` handles the game-specific parts: source timezone → UTC, region reset offsets,
|
||||
`regionScoped` determination, ID construction.
|
||||
- Get the domain rules right — they are in `CLAUDE.md` § Domain rules and they are where adapters
|
||||
actually go wrong:
|
||||
- All timestamps UTC ISO 8601.
|
||||
- Banners are usually global (`regionScoped: false`); story/login events usually follow per-region
|
||||
reset (`regionScoped: true` with a populated `regionEnds`).
|
||||
- An unstated end is `endsAt: null` + `endPrecision: "unknown"`. **Never compute a plausible end
|
||||
from typical patch length.** This is the failure mode that makes the product worthless.
|
||||
|
||||
**5. Write the test.** `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` holds the exact
|
||||
expected `GachaEvent[]`. The test runs `parse` + `normalize` against the fixture with a pinned
|
||||
`ctx.now` and asserts deep equality.
|
||||
|
||||
**6. Verify.** Run `bun test` and confirm it passes with no network. Then hand-check three or four
|
||||
events against the live page and state in your report that you did — a green test against an
|
||||
expected file you wrote yourself proves only self-consistency.
|
||||
|
||||
**7. Register** the adapter in `src/ingest/adapters/index.ts` and add its `sources` row.
|
||||
|
||||
## Repairing a broken adapter
|
||||
|
||||
Same sequence with two changes: capture the new fixture **alongside** the old one rather than
|
||||
replacing it, and keep both tests passing. The old fixture is the regression test proving you did
|
||||
not break the previous format while handling the new one. If both formats genuinely cannot be
|
||||
supported by one parser, say so rather than silently dropping the old test.
|
||||
|
||||
## Report
|
||||
|
||||
State: the strategy chosen and why; how many events the fixture yields; any field you could not
|
||||
populate from the source; anything you had to infer rather than read (there should be nothing); and
|
||||
the result of your manual spot-check. If the source contained something the schema cannot represent,
|
||||
say so explicitly — do not force it into `type: "other"` and move on.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: extraction-evaluator
|
||||
description: Evaluates a change to the extraction prompt or schema by replaying stored snapshots offline and reporting accuracy deltas. Use before merging any edit to src/ingest/prompts/ or the extraction output schema. Read-only against the codebase; costs API tokens for replay.
|
||||
tools: Read, Bash, Grep, Glob
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You measure whether a change to the LLM extraction layer made it better or worse. You do not edit
|
||||
prompts — you report evidence so someone else can decide.
|
||||
|
||||
Read `docs/LLM-EXTRACTION.md` § Evaluating a prompt change first.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Prompt edits look free and are not. A revision that improves one source's output can start
|
||||
hallucinating dates on another, and nothing in the pipeline catches that until a user misses an
|
||||
event. This agent replays the change against inputs whose correct output is already known.
|
||||
|
||||
Replay uses stored snapshots — **never re-fetch source pages.** `snapshots` is keyed by
|
||||
`content_hash` and `extraction_log` records the hash for every past call, so the whole corpus is
|
||||
available locally. Re-scraping to evaluate a prompt is both wasteful and rude to the source.
|
||||
|
||||
## Sequence
|
||||
|
||||
1. **Establish the baseline.** Identify the previous prompt version (the versioned filename in
|
||||
`src/ingest/prompts/`) and the fixtures with known-correct expected output.
|
||||
2. **Build the corpus.** Pull distinct `input_hash` values from `extraction_log` and their cleaned
|
||||
text from `snapshots`. Aim for at least one input per game; more if available. State the corpus
|
||||
size in your report — a conclusion from three inputs is weaker than one from thirty, and the
|
||||
reader needs to know which they have.
|
||||
3. **Run both versions** over the same inputs. Same model, same `effort`, same `max_tokens`. Change
|
||||
exactly one thing at a time; if the diff touches both the prompt and the schema, evaluate them
|
||||
separately or say plainly that you could not isolate them.
|
||||
4. **Diff against expected output** on three axes:
|
||||
|
||||
| Axis | Definition |
|
||||
|---|---|
|
||||
| **Hallucinated** | Event in output with no corresponding event in the source |
|
||||
| **Wrong date** | Event correctly identified, `startsAt` or `endsAt` incorrect |
|
||||
| **Missed** | Event in the source absent from output |
|
||||
|
||||
Also check the `evidence` field on every extracted event: if the quoted span does not appear
|
||||
verbatim in the input, count it as hallucinated regardless of whether the dates happen to be
|
||||
right. A correct answer with fabricated evidence is luck, not extraction.
|
||||
|
||||
5. **Check the guessing failure mode specifically.** Count events where the source states no end
|
||||
date but the output supplies one. Any occurrence is a blocking regression — this is the exact
|
||||
behavior `docs/PRD.md` § Quality bar exists to prevent.
|
||||
|
||||
6. **Record cost.** Token counts per version from the response `usage`. A prompt that is 10% more
|
||||
accurate and 3× more expensive is a real tradeoff the reader should get to weigh.
|
||||
|
||||
## Scoring
|
||||
|
||||
The axes are not equal, and the report must reflect that:
|
||||
|
||||
- **Hallucinated events and wrong dates are disqualifying.** Any increase blocks the change.
|
||||
- **Missed events are a regression to weigh** — worth accepting if hallucinations drop.
|
||||
- A change that only shortens the prompt with no accuracy movement is neutral. Say so; do not
|
||||
manufacture a recommendation. Check that the shortened system prompt is still above **512 tokens**,
|
||||
or prompt caching silently stops working.
|
||||
|
||||
## Report
|
||||
|
||||
A table of both versions across all three axes plus token cost, then a one-line verdict: ship,
|
||||
block, or inconclusive. If inconclusive, say exactly what additional inputs would settle it.
|
||||
|
||||
Report what you measured, faithfully. If the new version is worse, say so plainly. If the corpus was
|
||||
too small to distinguish the two, say that rather than reporting a difference within noise as a
|
||||
finding.
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: schema-guardian
|
||||
description: Reviews any change touching src/shared/schema.ts, the event ID scheme, localStorage keys, or the API response contract, for silent data-loss risk. Use before merging such a change. Read-only — reports findings, does not edit.
|
||||
tools: Read, Grep, Glob, Bash
|
||||
model: sonnet
|
||||
---
|
||||
|
||||
You review changes to this app's data contracts for one specific class of bug: **silent, permanent
|
||||
loss of user data with no error and no server-side recovery.**
|
||||
|
||||
This app stores completion state only in the browser. There is no user table, no backup, no
|
||||
support path. A migration that orphans localStorage keys destroys data that cannot be restored by
|
||||
anyone. That is what you are here to catch.
|
||||
|
||||
Read `docs/DATA-MODEL.md` § ID stability and § Client-side storage before reviewing.
|
||||
|
||||
## Scope
|
||||
|
||||
Review changes touching:
|
||||
|
||||
- `src/shared/schema.ts` — the Zod contract
|
||||
- The event ID construction function, anywhere it lives
|
||||
- `gacha-tracker:v*` localStorage keys or the code reading them
|
||||
- The `/api/events` response envelope or `schemaVersion`
|
||||
- The export/import format
|
||||
|
||||
## What to check
|
||||
|
||||
**1. Event ID scheme — the highest-stakes item.** Event IDs are localStorage keys. Any change to
|
||||
how they are built — the format string, the slugify function, the date component, even normalizing
|
||||
case — orphans every completion mark every user has. Verify:
|
||||
|
||||
- Is a client-side migration shipped that reads old-format keys and remaps them?
|
||||
- Does the migration run before the first read, on every entry path?
|
||||
- Are old-version keys **retained**, not deleted, after migration? A user who last opened the app six
|
||||
months ago still has data under the old key.
|
||||
- Would an event whose source title changed produce a new ID? Reconciliation is supposed to catch
|
||||
that as a near-match and keep the original ID — confirm that path still works.
|
||||
|
||||
Trace slugify changes specifically. A change from `-` to `_`, or added Unicode normalization, looks
|
||||
cosmetic in a diff and is a full data wipe.
|
||||
|
||||
**2. Schema changes.** Additive optional fields are safe. Flag anything that:
|
||||
|
||||
- Removes or renames a field the client reads
|
||||
- Narrows a type (widening `string | null` → `string` breaks every `endsAt: null` event — and null
|
||||
ends are a *correct, expected* state here, not an edge case)
|
||||
- Changes an enum's members without a fallback for unknown values in stored data
|
||||
- Alters `schemaVersion` handling — the client refuses versions it does not know, so bumping it
|
||||
without shipping the client change takes the app down
|
||||
|
||||
**3. localStorage keys.** Any new key must be namespaced `gacha-tracker:v<n>:`. Any read of an old
|
||||
key must survive the value being absent or from an older shape. Reading with `JSON.parse` and no
|
||||
try/catch is a crash on a corrupt value; flag it.
|
||||
|
||||
**4. Export/import.** Import must **merge**, never replace. Verify no path removes a completion the
|
||||
user already had. Verify an import of a file with an unknown `version` is refused rather than
|
||||
half-applied.
|
||||
|
||||
**5. API contract.** Does the client tolerate an unknown field? Does it tolerate a missing optional
|
||||
one? Does it handle an empty `events` array without rendering as if data loaded fine?
|
||||
|
||||
## Method
|
||||
|
||||
Grep for every reader of the thing being changed, not just the definition. The ID function is called
|
||||
in the adapter, in reconcile, in the client's completion lookup, and in export — a change is only
|
||||
safe if all four agree.
|
||||
|
||||
Where you suspect breakage, construct the concrete scenario: which user, in which state, loses what.
|
||||
"This might break something" is not a finding; "a user who marked events complete before this deploy
|
||||
sees all of them unmarked, permanently" is.
|
||||
|
||||
## Report
|
||||
|
||||
Findings ranked most severe first, each with file:line, the concrete data-loss scenario, and whether
|
||||
a migration would fix it. If the change is safe, say so in a sentence — do not manufacture findings
|
||||
on a clean diff. Distinguish clearly between "this destroys data" and "this is stylistically
|
||||
inconsistent"; only the first is your job.
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(bun test:*)",
|
||||
"Bash(bun run:*)",
|
||||
"Bash(bun install)",
|
||||
"Bash(bun add:*)",
|
||||
"Bash(bun build:*)",
|
||||
"Bash(bun x tsc:*)",
|
||||
"Bash(bunx tsc:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git show:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(ls:*)",
|
||||
"Bash(rg:*)",
|
||||
"Bash(sqlite3 *.sqlite \".schema\")",
|
||||
"Bash(sqlite3 *.sqlite \"SELECT *\")"
|
||||
],
|
||||
"ask": [
|
||||
"Bash(bun run ingest:*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git commit:*)",
|
||||
"Bash(curl:*)",
|
||||
"WebFetch"
|
||||
],
|
||||
"deny": [
|
||||
"Read(./.env)",
|
||||
"Read(./.env.*)",
|
||||
"Bash(rm -rf:*)"
|
||||
]
|
||||
},
|
||||
"env": {
|
||||
"INGEST_ENABLED": "false"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: add-game-source
|
||||
description: End-to-end workflow for adding a new game (or a new source for an existing game) to the event tracker — legal check, fixture capture, adapter, tests, registration, and first ingest run. Use when asked to add a game, add a second source, or wire up a data source for the calendar.
|
||||
---
|
||||
|
||||
# Adding a game source
|
||||
|
||||
The design goal is that a new game costs an adapter, a fixture, and a test — **no schema change and
|
||||
no client change.** If you find yourself editing `src/shared/schema.ts` or a React component to make
|
||||
a game fit, stop: either the data model is wrong (raise it) or the game is being forced into a shape
|
||||
it does not have.
|
||||
|
||||
Read `docs/INGESTION.md` and `docs/DATA-MODEL.md` before starting.
|
||||
|
||||
## 1. Legal and conduct check — a hard gate
|
||||
|
||||
Before anything else:
|
||||
|
||||
- Fetch `<host>/robots.txt`. Confirm the target path is not disallowed.
|
||||
- Skim the site's terms for a prohibition on automated access or scraping.
|
||||
- Prefer an official API or a community wiki with a permissive license over scraping an official
|
||||
site directly.
|
||||
|
||||
If the source forbids automated access, **stop and report it.** Do not write the adapter and do not
|
||||
look for a workaround. Suggest an alternative source instead.
|
||||
|
||||
## 2. Register the game
|
||||
|
||||
If this is a new game rather than a new source for an existing one, add it to `GameId` in
|
||||
`src/shared/schema.ts` and give it a display name and lane color in the games registry. This is the
|
||||
only schema edit a new game should require. If it needs more, that is a finding worth reporting.
|
||||
|
||||
## 3. Capture a fixture
|
||||
|
||||
Fetch the source page **once** and save the raw HTML:
|
||||
|
||||
```
|
||||
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
|
||||
```
|
||||
|
||||
Everything after this point works offline against that file. Do not re-fetch while iterating on the
|
||||
parser.
|
||||
|
||||
## 4. Build the adapter
|
||||
|
||||
Delegate to the **adapter-author** agent, or do it inline for a simple table source. Either way the
|
||||
requirements are the same:
|
||||
|
||||
- `parse` is pure over its input — no network, no `Date.now()`. Time comes from `ctx.now`.
|
||||
- Prefer a deterministic parser. Use `llm` strategy only when the markup genuinely cannot be parsed
|
||||
reliably, and say why.
|
||||
- `normalize` handles source-timezone → UTC, region reset offsets, and ID construction.
|
||||
|
||||
**The three domain rules that break new adapters**, from `CLAUDE.md`:
|
||||
|
||||
1. Everything stored as UTC ISO 8601.
|
||||
2. Banners are usually one global end instant; story and login events usually end at each region's
|
||||
daily reset. Set `regionScoped` and `regionEnds` accordingly.
|
||||
3. An unstated end date is `endsAt: null` with `endPrecision: "unknown"`. **Never derive a plausible
|
||||
end from typical patch length.** A confidently wrong end date is the failure this product exists
|
||||
to prevent.
|
||||
|
||||
## 5. Test it
|
||||
|
||||
Write `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` with the exact expected
|
||||
`GachaEvent[]`, and a test asserting deep equality with a pinned `ctx.now`.
|
||||
|
||||
Run `bun test`. It must pass with no network access.
|
||||
|
||||
Then **hand-check three or four events against the live page.** The test only proves the parser
|
||||
agrees with an expected file you wrote yourself; it does not prove either is right. Say in your
|
||||
report that you did this.
|
||||
|
||||
## 6. Wire it up
|
||||
|
||||
- Register the adapter in `src/ingest/adapters/index.ts`.
|
||||
- Insert the `sources` row: id, game, url, strategy, `min_interval_ms` (default 6h).
|
||||
|
||||
## 7. First run
|
||||
|
||||
```
|
||||
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
|
||||
```
|
||||
|
||||
Inspect what it *would* publish before letting it write. Then run for real and check the review
|
||||
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
|
||||
on its first pass, because nothing corroborates it yet and LLM-extracted events start at 0.70
|
||||
confidence. That is the gate working, not a bug. Review and approve them.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] robots.txt and ToS permit it
|
||||
- [ ] Game registered in `GameId` (new games only)
|
||||
- [ ] Fixture captured, page fetched exactly once
|
||||
- [ ] Adapter implemented; `parse` pure, no clock access
|
||||
- [ ] Timestamps UTC; `regionScoped` correct; unstated ends are `null`
|
||||
- [ ] Expected-output file + passing test, offline
|
||||
- [ ] Manually spot-checked against the live page
|
||||
- [ ] Registered in the adapter index and `sources`
|
||||
- [ ] Dry run inspected, then real run, then quarantine reviewed
|
||||
|
||||
## When something does not fit
|
||||
|
||||
Report it rather than working around it. A game with a fourth server region, an event type the enum
|
||||
lacks, or a source that publishes only relative dates ("starts next Tuesday") are all real
|
||||
possibilities the current model does not cover. Those are design questions, not adapter bugs — say
|
||||
what you found and what it would take.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: review-quarantine
|
||||
description: Work through the quarantined-event queue — triage held events by reason, verify dates against sources, and approve, correct, or reject each. Use when the quarantine queue has grown, when /api/health shows held events, or when an expected event is missing from the calendar.
|
||||
---
|
||||
|
||||
# Reviewing the quarantine queue
|
||||
|
||||
Held events are candidates the pipeline declined to publish. Working the queue is both a data task
|
||||
(get these events onto the calendar) and a diagnostic one (**a growing queue means something
|
||||
upstream broke**).
|
||||
|
||||
Read `docs/INGESTION.md` § The review gate first.
|
||||
|
||||
The review UI is at `http://127.0.0.1:$ADMIN_PORT/review`, on the admin listener only. If it is not
|
||||
reachable, the server is not running or `ADMIN_PORT` differs — it is not a permissions problem, and
|
||||
there is no auth to get past.
|
||||
|
||||
## Triage by reason first
|
||||
|
||||
Do not work the queue in date order. Group by `reason` — the four causes need different responses,
|
||||
and two of them are pipeline bugs rather than review work.
|
||||
|
||||
| Reason | What it means | Do this |
|
||||
|---|---|---|
|
||||
| `date_conflict` | A published end date moved by >24h | **Highest priority.** Users may already have planned around the old date |
|
||||
| `sanity_failed` | Broke a hard validator rule | Usually an extraction or parser bug — check the pattern before approving anything |
|
||||
| `low_confidence` | Scored below `CONFIDENCE_THRESHOLD` | Routine. Verify against the source and approve |
|
||||
| `novel_shape` | Something the schema does not model | A design question, not a review decision — escalate |
|
||||
|
||||
**Before reviewing individual items, look for a pattern.** Fifteen `sanity_failed` events from one
|
||||
source is not fifteen review decisions; it is one broken adapter. Fixing the adapter and re-running
|
||||
is the correct action, and approving them one by one hides the breakage. Say so rather than grinding
|
||||
through the queue.
|
||||
|
||||
## Reviewing one event
|
||||
|
||||
For each held event the UI shows the parsed fields, the reason and detail, the source link, and the
|
||||
cleaned text excerpt the extraction came from. Work in this order:
|
||||
|
||||
1. **Read the evidence span.** Does it actually contain the dates claimed? If the quoted text does
|
||||
not support the extracted dates, reject — and note it, because that is a hallucination and the
|
||||
extraction prompt needs evaluating.
|
||||
2. **Open the source.** Confirm the dates against the live page, not just the excerpt.
|
||||
3. **Check the timezone.** Most gacha sources publish in UTC+8. Confirm the conversion. A silently
|
||||
wrong timezone is the most common real error here, and it is 8 hours of wrongness that looks
|
||||
plausible.
|
||||
4. **Check `regionScoped`.** Is this a global banner end or a per-region reset? Getting this wrong
|
||||
makes the countdown wrong for two thirds of users.
|
||||
5. **Check for a guessed end date.** If the source says TBD or gives no end and the candidate has a
|
||||
concrete `endsAt`, that is a fabrication. Correct it to `null` with `endPrecision: "unknown"`
|
||||
before approving, and flag the extraction.
|
||||
|
||||
For `date_conflict` specifically: the UI shows the published event beside the candidate. Determine
|
||||
which is right by going to the source. If the source genuinely changed, approve the correction. If
|
||||
the candidate is a misparse, reject it — the published event stands.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Approve** — publishes with `extraction_method: 'manual'`, `confidence: 1.0`.
|
||||
- **Approve with edits** — correct a field first; the corrected value is what publishes. Use this
|
||||
freely; a nearly-right event with one bad field is worth fixing rather than rejecting.
|
||||
- **Reject** — not published. The same candidate will be held again on the next run if the source
|
||||
has not changed, which is intended: a rejection is not a permanent suppression.
|
||||
|
||||
When uncertain, reject. An event missing from the calendar sends someone to a wiki; a wrong end date
|
||||
makes them miss content. That asymmetry is the product rule (`docs/PRD.md` § Quality bar) and it is
|
||||
what breaks the tie.
|
||||
|
||||
## Close the loop
|
||||
|
||||
The queue is a signal. After working it, report what caused it:
|
||||
|
||||
- Repeated `sanity_failed` from one source → the adapter needs repair. Use the **adapter-author**
|
||||
agent.
|
||||
- Hallucinated dates or fabricated evidence → the prompt regressed. Use the **extraction-evaluator**
|
||||
agent before changing anything.
|
||||
- Many `low_confidence` items that all turn out correct → the threshold may be too high, or the
|
||||
source needs a second corroborating source. Do not just lower `CONFIDENCE_THRESHOLD` to make the
|
||||
queue shorter; that trades a visible queue for invisible wrong dates.
|
||||
- `novel_shape` → escalate as a data-model question with the specific example.
|
||||
|
||||
Report how many you approved, corrected, and rejected, and what you think caused the batch. A review
|
||||
session that empties the queue without explaining why it filled has done half the job.
|
||||
Generated
+10
@@ -0,0 +1,10 @@
|
||||
# Default ignored files
|
||||
/shelf/
|
||||
/workspace.xml
|
||||
# Editor-based HTTP Client requests
|
||||
/httpRequests/
|
||||
# Ignored default folder with query files
|
||||
/queries/
|
||||
# Datasource local storage ignored files
|
||||
/dataSources/
|
||||
/dataSources.local.xml
|
||||
Generated
+8
@@ -0,0 +1,8 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<module type="WEB_MODULE" version="4">
|
||||
<component name="NewModuleRootManager">
|
||||
<content url="file://$MODULE_DIR$" />
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
</component>
|
||||
</module>
|
||||
Generated
+8
@@ -0,0 +1,8 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="ProjectModuleManager">
|
||||
<modules>
|
||||
<module fileurl="file://$PROJECT_DIR$/.idea/gacha-event-tracker.iml" filepath="$PROJECT_DIR$/.idea/gacha-event-tracker.iml" />
|
||||
</modules>
|
||||
</component>
|
||||
</project>
|
||||
Generated
+6
@@ -0,0 +1,6 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="VcsDirectoryMappings">
|
||||
<mapping directory="" vcs="Git" />
|
||||
</component>
|
||||
</project>
|
||||
@@ -0,0 +1,138 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
A web app that aggregates live and upcoming events across popular gacha games (Genshin Impact,
|
||||
Honkai: Star Rail, Zenless Zone Zero, Wuthering Waves, Arknights, Arknights: Endfield), plots them
|
||||
on a calendar, sorts them by end date, and lets a user mark events completed.
|
||||
|
||||
**Status: specification only.** No application code exists yet. `docs/` is the source of truth for
|
||||
what to build; everything below describes the intended system, not an existing one. When you write
|
||||
the first code, follow `docs/ARCHITECTURE.md` and update this file's Commands section with the real
|
||||
commands.
|
||||
|
||||
## Two constraints that shape everything
|
||||
|
||||
1. **No accounts, no logins, no user records.** Completion state lives in the browser's
|
||||
`localStorage`, keyed by event ID. There is no user table and no session. Any feature request
|
||||
that implies "sync across devices" must be solved with export/import JSON, not a server-side
|
||||
user.
|
||||
2. **A server is allowed, and is where all secrets live.** The Bun server owns scraping, LLM
|
||||
extraction, and the SQLite database. `ANTHROPIC_API_KEY` never reaches the browser. The client
|
||||
only ever calls this app's own `/api/*`.
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Choice |
|
||||
|---|---|
|
||||
| Runtime / server / bundler / test runner | Bun (single dependency — `Bun.serve`, `bun:sqlite`, `bun test`, `bun build`) |
|
||||
| UI | React 19 + TypeScript (strict) + Tailwind |
|
||||
| Storage | SQLite via `bun:sqlite` (file is gitignored — `*.sqlite`) |
|
||||
| Validation | Zod — one schema module shared by server and client |
|
||||
| LLM | Anthropic TypeScript SDK (`@anthropic-ai/sdk`), model `claude-opus-5` |
|
||||
|
||||
TypeScript runs `strict: true` **and** `noUncheckedIndexedAccess`. Do not add a bundler, test
|
||||
runner, or process manager — Bun covers all three.
|
||||
|
||||
## Architecture in one paragraph
|
||||
|
||||
A scheduled job inside the Bun process runs one **adapter** per game. Each adapter fetches a source
|
||||
page, cleans it, and hands it to a **deterministic parser** when the source has a stable shape, or
|
||||
to **Claude structured extraction** when it doesn't. Results are validated with Zod plus calendar
|
||||
sanity rules, then either published to the `events` table or held in `events_quarantine` for human
|
||||
review at an unauthenticated `/review` route bound to `127.0.0.1`. The React client fetches
|
||||
`/api/events`, renders a calendar and an ends-soonest list, and stores completion ticks in
|
||||
`localStorage`. Full detail: `docs/ARCHITECTURE.md`.
|
||||
|
||||
The important consequence: **the LLM runs at ingestion time, never in a request path.** A page load
|
||||
must never trigger an API call to Anthropic. If you find yourself adding one, the design is wrong.
|
||||
|
||||
## Reading order for a new task
|
||||
|
||||
| Task | Read |
|
||||
|---|---|
|
||||
| Anything at all | `docs/ARCHITECTURE.md` |
|
||||
| Adding/changing an event field | `docs/DATA-MODEL.md` — the schema is versioned and the client depends on it |
|
||||
| Adding a game, fixing a broken adapter | `docs/INGESTION.md`, then invoke the `add-game-source` skill |
|
||||
| Touching prompts, extraction, or cost | `docs/LLM-EXTRACTION.md` |
|
||||
| Product questions (what does the calendar show?) | `docs/PRD.md` |
|
||||
|
||||
## Domain rules that are not obvious from the code
|
||||
|
||||
These come from how gacha games actually schedule things, and they are the source of most bugs in
|
||||
this kind of app:
|
||||
|
||||
- **Store every timestamp as UTC ISO 8601. Never store a local wall-clock time.** Sources publish
|
||||
in a mix of UTC+8, server-local, and "after maintenance".
|
||||
- **Banner ends are usually global and simultaneous; event ends are usually per-region.** Genshin
|
||||
and HSR character banners end at the same instant worldwide, while story/login events end at each
|
||||
region's daily reset (Asia / America / Europe are offset by hours). The `regionScoped` flag and
|
||||
the optional `regionEnds` map exist for exactly this — do not collapse them into one timestamp.
|
||||
- **"Ends after maintenance" and "TBD" are real values.** An event whose end is genuinely unknown
|
||||
gets `endsAt: null` and `endPrecision: "unknown"`. Never invent a plausible date to satisfy a
|
||||
non-null type — that is the single worst failure mode for this app, because the user's whole
|
||||
reason for visiting is trusting the end date.
|
||||
- **Version 1.x patch cycles are ~6 weeks (42 days), split into two banner phases.** Any extracted
|
||||
event with a duration over 180 days is almost certainly a parse error, not a long event. The
|
||||
validator rejects it.
|
||||
|
||||
## Working with the LLM extraction layer
|
||||
|
||||
Read `docs/LLM-EXTRACTION.md` before editing any prompt or request. The rules that will actually
|
||||
bite you:
|
||||
|
||||
- **Model is `claude-opus-5`.** That is the exact, complete ID — never append a date suffix.
|
||||
- **Use structured outputs, not prompt-and-parse.** `client.messages.parse()` with
|
||||
`zodOutputFormat(EventExtractionSchema)` from `@anthropic-ai/sdk/helpers/zod`. Read
|
||||
`response.parsed_output`. Do not write a JSON-repair or regex-extraction fallback — if the schema
|
||||
is right, there is nothing to repair.
|
||||
- **Never set `temperature`, `top_p`, or `top_k`.** They are removed on `claude-opus-5` and return
|
||||
a 400. Steer with the prompt.
|
||||
- **Never set `thinking: {type: "enabled", budget_tokens: N}`.** Removed — returns 400. Thinking is
|
||||
on by default; control depth with `output_config.effort`.
|
||||
- **Deterministic parsers come first.** The LLM is for sources whose markup is unstable. A source
|
||||
with a clean JSON API or a stable table must not go through the model.
|
||||
- **Skip unchanged sources by content hash.** This is the main cost lever — most refresh cycles
|
||||
should make zero API calls.
|
||||
|
||||
## Cost discipline
|
||||
|
||||
Every ingestion run should be able to answer "why did this cost anything?" Extraction is billed at
|
||||
`claude-opus-5` rates ($5/MTok input, $25/MTok output). Three levers, in order of impact:
|
||||
|
||||
1. **Content-hash skip** — unchanged source, no call at all.
|
||||
2. **HTML pre-cleaning** — strip `<script>`, `<style>`, nav, and footer before sending. Cuts a
|
||||
typical wiki page from ~15k to ~5k tokens.
|
||||
3. **Batch API for scheduled runs** — 50% off, results within an hour, which is fine for a 6-hour
|
||||
cadence. Use the synchronous API only for the manual `refresh --now` path.
|
||||
|
||||
Prompt caching applies to the shared extraction system prompt (`cache_control: {type: "ephemeral"}`,
|
||||
1-hour TTL). The minimum cacheable prefix on `claude-opus-5` is **512 tokens** — the system prompt
|
||||
is deliberately above it. If you shorten the system prompt below 512 tokens, caching silently stops
|
||||
working and nothing errors; check `usage.cache_read_input_tokens` after any prompt edit.
|
||||
|
||||
## Scraping conduct
|
||||
|
||||
Sources are community wikis and official news pages. Treat them as a guest would:
|
||||
|
||||
- Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
|
||||
- One request per source per refresh cycle, minimum 6 hours apart. No parallel hammering.
|
||||
- Send `If-None-Match` / `If-Modified-Since` and treat `304` as "skip, unchanged".
|
||||
- Cache raw snapshots locally so re-running extraction never re-fetches.
|
||||
- Record `sourceUrl` on every event and surface attribution in the UI.
|
||||
|
||||
A new source that forbids automated access in its ToS does not get an adapter. Flag it and ask.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Zod schemas are the single source of truth for types.** Derive TypeScript types with
|
||||
`z.infer<>`; never hand-write an interface that duplicates a schema.
|
||||
- **Event IDs are stable and deterministic**: `${gameId}:${slugify(title)}:${startsAt.slice(0,10)}`.
|
||||
They are the localStorage keys, so **changing the ID scheme silently wipes every user's completion
|
||||
state.** If you must change it, ship a migration in the client that remaps old keys.
|
||||
- **Adapters are pure over their input.** `fetch` is separate from `parse` so parsers can be tested
|
||||
against checked-in HTML fixtures with no network.
|
||||
- Every adapter ships a fixture in `fixtures/<game>/` and a test asserting the parsed output. This
|
||||
is how a source silently changing shape gets caught.
|
||||
@@ -0,0 +1,161 @@
|
||||
# Architecture
|
||||
|
||||
## Shape
|
||||
|
||||
One Bun process serves the static React build, exposes a read-only JSON API, and runs the
|
||||
ingestion scheduler on a timer. SQLite is the only datastore. There is no auth layer because there
|
||||
are no users.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
game wikis ───► │ Bun process │
|
||||
news pages │ │
|
||||
│ scheduler (every 6h) │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ingest pipeline │
|
||||
│ fetch → clean → parse|extract → validate │
|
||||
│ → reconcile → gate → publish │
|
||||
│ │ │ │
|
||||
│ │ └──► quarantine │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────┐ │ │
|
||||
│ │ SQLite │◄────────────────┘ │
|
||||
│ └─────────────┘ /review (127.0.0.1) │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ GET /api/events static React build │
|
||||
└────────┬─────────────────────────┬───────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
browser fetch browser render
|
||||
│
|
||||
▼
|
||||
localStorage: completions,
|
||||
filters, region ← never leaves the device
|
||||
```
|
||||
|
||||
Anthropic API calls happen only inside the ingest pipeline. **No request path — not `/api/events`,
|
||||
not a page load — ever calls the model.** If a feature seems to need live inference, it needs a
|
||||
precomputed field instead.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
server/
|
||||
index.ts Bun.serve entry: static + API + scheduler bootstrap
|
||||
routes/
|
||||
events.ts GET /api/events, /api/events.json
|
||||
games.ts GET /api/games
|
||||
review.ts /review UI + approve/reject (localhost-bound)
|
||||
health.ts GET /api/health
|
||||
db/
|
||||
client.ts bun:sqlite handle, WAL, pragmas
|
||||
migrations/ NNN-name.sql, applied in order at boot
|
||||
queries.ts all SQL lives here — no SQL in route handlers
|
||||
ingest/
|
||||
scheduler.ts timer + jitter + per-source lock
|
||||
pipeline.ts the 7 stages, orchestration only
|
||||
clean.ts HTML → text reduction before extraction
|
||||
extract.ts Anthropic client, prompts, batch submission
|
||||
validate.ts zod parse + calendar sanity rules
|
||||
reconcile.ts diff vs published, confidence, conflict detection
|
||||
adapters/
|
||||
index.ts registry: GameId → Adapter
|
||||
genshin.ts
|
||||
hsr.ts
|
||||
zzz.ts
|
||||
wuwa.ts
|
||||
arknights.ts
|
||||
endfield.ts
|
||||
shared/
|
||||
schema.ts zod schemas — the contract, imported by both sides
|
||||
types.ts z.infer types only
|
||||
time.ts region reset math, duration formatting
|
||||
client/
|
||||
main.tsx
|
||||
App.tsx
|
||||
views/
|
||||
Timeline.tsx F1
|
||||
EndingSoon.tsx F2
|
||||
EventDetail.tsx
|
||||
state/
|
||||
completions.ts localStorage read/write + export/import
|
||||
prefs.ts region, filters
|
||||
api.ts typed fetch of /api/events
|
||||
fixtures/<game>/ checked-in raw HTML + expected parse output
|
||||
docs/
|
||||
```
|
||||
|
||||
## Request paths
|
||||
|
||||
| Route | Purpose | Notes |
|
||||
|---|---|---|
|
||||
| `GET /` + assets | React SPA | Served from the `bun build` output |
|
||||
| `GET /api/events?from&to&game` | Filtered feed | `ETag` + `Cache-Control: public, max-age=300` |
|
||||
| `GET /api/events.json` | Whole published feed | Cheap; the client mostly uses this and filters locally |
|
||||
| `GET /api/games` | Game metadata: id, name, color, lastUpdatedAt | Drives the freshness badges (F7) |
|
||||
| `GET /api/health` | Per-source last-success, quarantine depth | For an operator, not the UI |
|
||||
| `GET /review` | Quarantine review UI | **Bound to `127.0.0.1` only** |
|
||||
| `POST /api/review/:id/approve` \| `/reject` | Promote or discard a quarantined event | Same binding |
|
||||
|
||||
### Why `/review` needs no auth
|
||||
|
||||
`Bun.serve` runs two listeners: the public one on `0.0.0.0:PORT` with the SPA and `/api/*`, and a
|
||||
second on `127.0.0.1:ADMIN_PORT` with `/review` and `/api/review/*`. The review routes are not
|
||||
registered on the public listener at all — they are unreachable from off-box, so there is nothing
|
||||
to authenticate. This is the mechanism that satisfies "no logins" without leaving an open admin
|
||||
endpoint on the internet.
|
||||
|
||||
**This is load-bearing.** If someone later puts a reverse proxy in front of the admin port, or
|
||||
merges the two listeners "to simplify", the review UI becomes a public write endpoint. Any change
|
||||
in that area needs an explicit auth story first.
|
||||
|
||||
## Data flow, concretely
|
||||
|
||||
1. **Scheduler** wakes every 6h (± jitter). For each source not fetched within its `minIntervalMs`,
|
||||
it acquires a per-source lock row and enqueues a run.
|
||||
2. **Pipeline** executes the seven stages in `docs/INGESTION.md`. Every stage writes to
|
||||
`ingest_runs` so a failure is diagnosable after the fact without re-running.
|
||||
3. **Publish** upserts into `events` by stable ID, bumping `version` and `updatedAt` when any field
|
||||
changed. Events that vanish from a source are *not* deleted — they are marked
|
||||
`status = 'delisted'` so a source outage cannot silently empty the calendar.
|
||||
4. **Client** fetches the feed, merges the completion set from `localStorage` by event ID, and
|
||||
renders. Merge is a client-side join; the server never learns what the user completed.
|
||||
|
||||
## Concurrency and failure
|
||||
|
||||
- One in-flight run per source, enforced by a lock row with a stale-lock timeout of 15 minutes.
|
||||
- A source that fails keeps its previously published events. A failed run never deletes or blanks
|
||||
data — worst case, the game's lane goes stale and gets a warning badge (F7).
|
||||
- Three consecutive failures for one source raises its `health` to `failing` in `/api/health`. It
|
||||
does not stop the schedule; a wiki being down for a day is normal.
|
||||
- Extraction results are written to `extraction_log` with the input hash, so a prompt change can be
|
||||
evaluated against previously-seen inputs without re-fetching or re-paying.
|
||||
|
||||
## Deployment
|
||||
|
||||
Single process, single SQLite file, no external services beyond the Anthropic API.
|
||||
|
||||
```
|
||||
PORT=3000
|
||||
ADMIN_PORT=3001 # bound to 127.0.0.1
|
||||
DATABASE_PATH=./data/events.sqlite
|
||||
ANTHROPIC_API_KEY=sk-ant-...
|
||||
INGEST_INTERVAL_MS=21600000
|
||||
INGEST_ENABLED=true # false for local UI work — never hits the network or the API
|
||||
EXTRACTION_MODE=batch # batch | sync
|
||||
CONFIDENCE_THRESHOLD=0.8
|
||||
```
|
||||
|
||||
`INGEST_ENABLED=false` is the default for local development. Frontend work should run against a
|
||||
seeded SQLite file and cost nothing.
|
||||
|
||||
## Deliberate non-choices
|
||||
|
||||
- **No ORM.** `bun:sqlite` plus hand-written SQL in `queries.ts`. The schema is six tables.
|
||||
- **No Redis / job queue.** The scheduler is a timer and a lock row. Restarting the process resumes
|
||||
cleanly because state is in SQLite.
|
||||
- **No server-side rendering.** The feed is small and cacheable; a static SPA is enough.
|
||||
- **No websockets.** Events change on a scale of hours; a 5-minute cache is more than adequate.
|
||||
@@ -0,0 +1,248 @@
|
||||
# Data Model
|
||||
|
||||
`src/shared/schema.ts` is the single source of truth. TypeScript types are derived with
|
||||
`z.infer<>` — never hand-write an interface that duplicates a schema.
|
||||
|
||||
## The Event
|
||||
|
||||
```ts
|
||||
import { z } from "zod";
|
||||
|
||||
export const GameId = z.enum([
|
||||
"genshin", "hsr", "zzz", "wuwa", "arknights", "endfield",
|
||||
]);
|
||||
|
||||
export const EventType = z.enum([
|
||||
"banner", // limited character/weapon rate-up
|
||||
"story", // main or side story chapter, limited-time
|
||||
"rerun", // returning event
|
||||
"challenge", // combat/endgame cycle (Abyss, Memory of Chaos, ...)
|
||||
"login", // login rewards / check-in
|
||||
"shop", // limited shop or exchange window
|
||||
"maintenance", // server downtime
|
||||
"other",
|
||||
]);
|
||||
|
||||
export const Region = z.enum(["asia", "america", "europe"]);
|
||||
|
||||
/** How much we actually know about a boundary timestamp. */
|
||||
export const Precision = z.enum([
|
||||
"exact", // sourced to the minute
|
||||
"day", // date known, time-of-day inferred from the game's reset
|
||||
"unknown", // genuinely not announced — endsAt is null
|
||||
]);
|
||||
|
||||
export const GachaEvent = z.object({
|
||||
id: z.string(), // `${game}:${slug}:${YYYY-MM-DD}` — see Stability below
|
||||
game: GameId,
|
||||
title: z.string().min(1).max(200),
|
||||
type: EventType,
|
||||
summary: z.string().max(500).nullable(),
|
||||
|
||||
startsAt: z.string().datetime(), // UTC ISO 8601, always
|
||||
startPrecision: Precision,
|
||||
endsAt: z.string().datetime().nullable(),
|
||||
endPrecision: Precision,
|
||||
|
||||
/** True when the end time follows each region's daily reset rather than a global instant. */
|
||||
regionScoped: z.boolean(),
|
||||
/** Populated only when regionScoped; per-region resolved UTC instants. */
|
||||
regionEnds: z.record(Region, z.string().datetime()).nullable(),
|
||||
|
||||
sourceUrl: z.string().url(),
|
||||
sourceId: z.string(), // which adapter/source produced this
|
||||
|
||||
status: z.enum(["published", "delisted"]),
|
||||
confidence: z.number().min(0).max(1),
|
||||
extractionMethod: z.enum(["parser", "llm", "manual"]),
|
||||
|
||||
version: z.number().int().positive(),
|
||||
firstSeenAt: z.string().datetime(),
|
||||
updatedAt: z.string().datetime(),
|
||||
});
|
||||
|
||||
export type GachaEvent = z.infer<typeof GachaEvent>;
|
||||
```
|
||||
|
||||
### Field notes that matter
|
||||
|
||||
**`endsAt: null` is a first-class state, not an error.** Many events are announced with "duration
|
||||
TBD" or "until the next version update". The correct representation is `endsAt: null` with
|
||||
`endPrecision: "unknown"`. The extractor is instructed to produce this and the UI renders it
|
||||
distinctly (PRD F1). Filling in a plausible date instead is the single worst bug this codebase can
|
||||
ship.
|
||||
|
||||
**`regionScoped` + `regionEnds`.** Character banners end at one global instant — `regionScoped:
|
||||
false`, `regionEnds: null`. Story and login events end at each region's daily reset — `regionScoped:
|
||||
true`, with `regionEnds` carrying the three resolved UTC instants. The client picks one using the
|
||||
user's stored region (PRD F5). Collapsing these into a single timestamp loses up to 13 hours of
|
||||
accuracy and will make the countdown wrong for two thirds of users.
|
||||
|
||||
**`confidence`** is assigned during reconciliation, not by the model's self-report. See
|
||||
`docs/LLM-EXTRACTION.md` § Scoring — a model asserting "I am 0.95 confident" is not evidence.
|
||||
|
||||
**`status: "delisted"`** means the event stopped appearing at its source. It is never deleted,
|
||||
because a source outage would otherwise silently empty the calendar. Delisted events are excluded
|
||||
from the API feed but retained for debugging and for the case where a source flickers.
|
||||
|
||||
### ID stability — read before changing
|
||||
|
||||
```
|
||||
`${game}:${slugify(title)}:${startsAt.slice(0, 10)}`
|
||||
→ "genshin:windblume-festival:2026-03-14"
|
||||
```
|
||||
|
||||
**Event IDs are the localStorage keys for completion state.** Changing the scheme orphans every
|
||||
completion mark every user has ever made, silently, with no error and no way to recover it
|
||||
server-side (the server never had the data). If the scheme must change, ship a client-side
|
||||
migration that reads the old keys and remaps them, and keep that migration for at least a year.
|
||||
|
||||
The date suffix disambiguates reruns of the same event. Title is slugified from the *source's*
|
||||
title, so a wiki renaming an event creates a new ID — reconciliation detects this as a near-match
|
||||
(same game, overlapping dates, high title similarity) and treats it as an update rather than a new
|
||||
event, preserving the original ID.
|
||||
|
||||
## SQLite schema
|
||||
|
||||
```sql
|
||||
-- Published feed. One row per event.
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
game TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
summary TEXT,
|
||||
starts_at TEXT NOT NULL,
|
||||
start_precision TEXT NOT NULL,
|
||||
ends_at TEXT,
|
||||
end_precision TEXT NOT NULL,
|
||||
region_scoped INTEGER NOT NULL DEFAULT 0,
|
||||
region_ends TEXT, -- JSON object or NULL
|
||||
source_url TEXT NOT NULL,
|
||||
source_id TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'published',
|
||||
confidence REAL NOT NULL,
|
||||
extraction_method TEXT NOT NULL,
|
||||
version INTEGER NOT NULL DEFAULT 1,
|
||||
first_seen_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX idx_events_ends ON events (ends_at) WHERE status = 'published';
|
||||
CREATE INDEX idx_events_game ON events (game, starts_at);
|
||||
CREATE INDEX idx_events_window ON events (starts_at, ends_at);
|
||||
|
||||
-- Candidates held back by the review gate. Same shape plus why.
|
||||
CREATE TABLE events_quarantine (
|
||||
id TEXT PRIMARY KEY,
|
||||
payload TEXT NOT NULL, -- full GachaEvent JSON
|
||||
reason TEXT NOT NULL, -- 'low_confidence' | 'date_conflict' | 'sanity_failed' | 'novel_shape'
|
||||
detail TEXT NOT NULL, -- human-readable explanation for the reviewer
|
||||
conflicts_with TEXT, -- events.id, when reason = 'date_conflict'
|
||||
run_id TEXT NOT NULL REFERENCES ingest_runs(id),
|
||||
created_at TEXT NOT NULL,
|
||||
resolved_at TEXT,
|
||||
resolution TEXT -- 'approved' | 'rejected' | NULL
|
||||
);
|
||||
CREATE INDEX idx_quarantine_open ON events_quarantine (created_at) WHERE resolved_at IS NULL;
|
||||
|
||||
-- One row per configured source.
|
||||
CREATE TABLE sources (
|
||||
id TEXT PRIMARY KEY, -- 'genshin-wiki-events'
|
||||
game TEXT NOT NULL,
|
||||
url TEXT NOT NULL,
|
||||
strategy TEXT NOT NULL, -- 'parser' | 'llm' | 'parser_then_llm'
|
||||
min_interval_ms INTEGER NOT NULL DEFAULT 21600000,
|
||||
etag TEXT,
|
||||
last_modified TEXT,
|
||||
content_hash TEXT, -- sha256 of cleaned content; the skip check
|
||||
last_success_at TEXT,
|
||||
last_attempt_at TEXT,
|
||||
consecutive_failures INTEGER NOT NULL DEFAULT 0,
|
||||
health TEXT NOT NULL DEFAULT 'ok', -- 'ok' | 'degraded' | 'failing'
|
||||
lock_holder TEXT,
|
||||
lock_expires_at TEXT
|
||||
);
|
||||
|
||||
-- One row per pipeline execution. The audit trail.
|
||||
CREATE TABLE ingest_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
source_id TEXT NOT NULL REFERENCES sources(id),
|
||||
started_at TEXT NOT NULL,
|
||||
finished_at TEXT,
|
||||
outcome TEXT, -- 'published' | 'skipped_unchanged' | 'quarantined' | 'failed'
|
||||
stage_failed TEXT,
|
||||
error TEXT,
|
||||
events_seen INTEGER DEFAULT 0,
|
||||
events_changed INTEGER DEFAULT 0,
|
||||
events_held INTEGER DEFAULT 0
|
||||
);
|
||||
|
||||
-- One row per LLM call. Enables replaying prompt changes against past inputs.
|
||||
CREATE TABLE extraction_log (
|
||||
id TEXT PRIMARY KEY,
|
||||
run_id TEXT NOT NULL REFERENCES ingest_runs(id),
|
||||
source_id TEXT NOT NULL,
|
||||
model TEXT NOT NULL,
|
||||
prompt_version TEXT NOT NULL,
|
||||
input_hash TEXT NOT NULL, -- of the cleaned text sent to the model
|
||||
input_tokens INTEGER,
|
||||
output_tokens INTEGER,
|
||||
cache_read_tokens INTEGER,
|
||||
cache_write_tokens INTEGER,
|
||||
stop_reason TEXT,
|
||||
refusal_category TEXT,
|
||||
duration_ms INTEGER,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX idx_extraction_input ON extraction_log (input_hash);
|
||||
|
||||
-- Cached raw + cleaned snapshots so re-extraction never re-fetches.
|
||||
CREATE TABLE snapshots (
|
||||
content_hash TEXT PRIMARY KEY,
|
||||
source_id TEXT NOT NULL,
|
||||
fetched_at TEXT NOT NULL,
|
||||
raw BLOB NOT NULL,
|
||||
cleaned TEXT NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
`region_ends` and `payload` hold JSON as TEXT; parse them through the Zod schema on read so a
|
||||
malformed row surfaces at the boundary rather than deep in the UI.
|
||||
|
||||
## Client-side storage
|
||||
|
||||
Namespaced, versioned, and small. Nothing here ever goes to the server.
|
||||
|
||||
```ts
|
||||
"gacha-tracker:v1:completions" // { [eventId]: { completedAt: string } }
|
||||
"gacha-tracker:v1:prefs" // { region, hiddenGames[], hiddenTypes[], showCompleted }
|
||||
"gacha-tracker:v1:feedCache" // { fetchedAt, events } — offline fallback
|
||||
```
|
||||
|
||||
The `v1` segment is the migration hook. On boot, the client checks for keys at older versions and
|
||||
migrates them forward before reading. **Never delete an old-version key until the migration has
|
||||
shipped and run** — a user who has not opened the app in six months still has their data under the
|
||||
old key.
|
||||
|
||||
### Export format (PRD F6)
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "gacha-tracker-export",
|
||||
"version": 1,
|
||||
"exportedAt": "2026-08-14T12:00:00.000Z",
|
||||
"completions": { "genshin:windblume-festival:2026-03-14": { "completedAt": "..." } },
|
||||
"prefs": { "region": "europe", "hiddenGames": [] }
|
||||
}
|
||||
```
|
||||
|
||||
Import **merges**: a completion present in either the file or the current device stays completed.
|
||||
Import never removes a completion. Losing a user's marks to a bad import is unrecoverable, so the
|
||||
merge is deliberately one-directional.
|
||||
|
||||
## Schema versioning
|
||||
|
||||
`/api/events` responses carry `{ schemaVersion: 1, generatedAt, events: [...] }`. The client
|
||||
refuses to render a `schemaVersion` it does not know and shows a "refresh the page" prompt instead
|
||||
of guessing at unfamiliar fields. Additive fields do not bump the version; removing or retyping a
|
||||
field does.
|
||||
@@ -0,0 +1,221 @@
|
||||
# Ingestion Pipeline
|
||||
|
||||
Seven stages, run per source. Every stage writes its outcome to `ingest_runs` so a failure two days
|
||||
ago can be diagnosed without re-running or re-paying.
|
||||
|
||||
```
|
||||
fetch → clean → parse|extract → validate → reconcile → gate → publish
|
||||
│
|
||||
└──► quarantine
|
||||
```
|
||||
|
||||
## The adapter contract
|
||||
|
||||
An adapter is the only per-game code. Everything downstream of `parse` is shared.
|
||||
|
||||
```ts
|
||||
export interface Adapter {
|
||||
id: string; // 'genshin-wiki-events'
|
||||
game: GameId;
|
||||
url: string;
|
||||
strategy: "parser" | "llm" | "parser_then_llm";
|
||||
minIntervalMs?: number; // default 6h
|
||||
|
||||
/** Narrow the cleaned document to just the region containing event data. */
|
||||
select?(cleaned: string): string;
|
||||
|
||||
/**
|
||||
* Deterministic parse. Return null to fall through to LLM extraction
|
||||
* (only meaningful when strategy is 'parser_then_llm').
|
||||
* Pure over its input — no network, no clock, no randomness. This is what
|
||||
* makes fixture tests possible.
|
||||
*/
|
||||
parse?(cleaned: string, ctx: ParseContext): RawEvent[] | null;
|
||||
|
||||
/** Extra instructions appended to the shared extraction prompt. */
|
||||
extractionHints?: string;
|
||||
|
||||
/** Game-specific normalization: reset times, region offsets, patch cadence. */
|
||||
normalize(raw: RawEvent, ctx: ParseContext): GachaEvent;
|
||||
}
|
||||
|
||||
export interface ParseContext {
|
||||
now: string; // injected, never Date.now() — keeps parse pure and testable
|
||||
sourceUrl: string;
|
||||
sourceId: string;
|
||||
game: GameId;
|
||||
}
|
||||
```
|
||||
|
||||
**`parse` must not read the clock.** It takes `now` from `ctx`. This is what lets a fixture test
|
||||
assert exact output for a page captured last March.
|
||||
|
||||
### Choosing a strategy
|
||||
|
||||
| Source shape | Strategy |
|
||||
|---|---|
|
||||
| JSON API, or a stable HTML table with consistent headers | `parser` |
|
||||
| Free-form patch notes, announcement prose, inconsistent markup | `llm` |
|
||||
| Mostly-stable markup that occasionally changes | `parser_then_llm` |
|
||||
|
||||
Prefer `parser`. It is free, deterministic, and instantly testable. The LLM exists for sources that
|
||||
genuinely cannot be parsed reliably, not as the default. A source with a clean API that goes through
|
||||
the model is a bug.
|
||||
|
||||
## Stage 1 — fetch
|
||||
|
||||
- Send `If-None-Match` / `If-Modified-Since` from `sources.etag` / `last_modified`. A `304` ends
|
||||
the run as `skipped_unchanged` with zero further cost.
|
||||
- `User-Agent: gacha-event-tracker/1.0 (+https://github.com/<owner>/gacha-event-tracker)`.
|
||||
- Honor `robots.txt`. Cache the parsed robots per host for 24h.
|
||||
- 20s timeout; retry twice with exponential backoff on 5xx and network errors; never retry 4xx.
|
||||
- Store the raw bytes in `snapshots`.
|
||||
|
||||
On failure: increment `consecutive_failures`, leave published events untouched, end the run as
|
||||
`failed`. A source being down never mutates the feed.
|
||||
|
||||
## Stage 2 — clean
|
||||
|
||||
Reduce the document before it costs anything. This stage is the second-biggest cost lever after the
|
||||
content-hash skip.
|
||||
|
||||
- Drop `<script>`, `<style>`, `<svg>`, `<noscript>`, comments, nav, header, footer, and known
|
||||
wiki chrome (edit links, category boxes, reference lists).
|
||||
- Collapse whitespace; convert tables to pipe-delimited text; keep headings as `#` markers so
|
||||
section structure survives.
|
||||
- Apply `adapter.select()` if present to isolate the event region.
|
||||
- Hash the result (sha256) → `content_hash`.
|
||||
|
||||
**If `content_hash` matches `sources.content_hash`, end the run as `skipped_unchanged`.** This is
|
||||
the check that keeps a 6-hourly schedule from costing anything on a quiet week — most runs should
|
||||
end here.
|
||||
|
||||
A typical wiki page goes from ~15k tokens raw to ~5k cleaned. Verify with `messages.count_tokens`
|
||||
when tuning, not by guessing.
|
||||
|
||||
## Stage 3 — parse or extract
|
||||
|
||||
Per strategy. `parse` produces `RawEvent[]` directly. `extract` sends the cleaned text to
|
||||
`claude-opus-5` with a structured-output schema — see `docs/LLM-EXTRACTION.md` for the request
|
||||
shape, prompt, and cost rules.
|
||||
|
||||
`parser_then_llm` calls `parse` first and falls through to `extract` only when it returns `null`.
|
||||
When that fallthrough happens, log it loudly: it means the source changed shape and the parser needs
|
||||
updating. A `parser_then_llm` source that is silently always falling through is paying LLM prices
|
||||
for a parser that no longer works.
|
||||
|
||||
## Stage 4 — validate
|
||||
|
||||
Zod parse against `GachaEvent`, then calendar sanity rules. Anything failing a hard rule goes to
|
||||
quarantine with `reason: 'sanity_failed'` — never to the feed.
|
||||
|
||||
**Hard rules (reject):**
|
||||
|
||||
| Rule | Rationale |
|
||||
|---|---|
|
||||
| `endsAt > startsAt` when both present | A backwards interval is always a parse error |
|
||||
| Duration ≤ 180 days | Patch cycles are ~6 weeks; 180d means a year was misread as a range |
|
||||
| `startsAt` within [now − 2y, now + 1y] | Catches century typos and relative-date misreads |
|
||||
| `endsAt` null ⟺ `endPrecision === "unknown"` | The two fields must agree |
|
||||
| `regionEnds` non-null ⟺ `regionScoped` | Same |
|
||||
| All `regionEnds` values within 24h of each other | Region resets differ by hours, not days |
|
||||
| `title` non-empty, ≤ 200 chars, not a placeholder ("TBD", "Event", "Unknown") | Catches header rows scraped as events |
|
||||
|
||||
**Soft rules (reduce confidence, do not reject):**
|
||||
|
||||
- Duration under 1 hour or over 60 days → −0.2
|
||||
- `startPrecision` or `endPrecision` is `"day"` → −0.1
|
||||
- Title very similar to another event in the same batch → −0.15 (likely a duplicate row)
|
||||
|
||||
## Stage 5 — reconcile
|
||||
|
||||
Diff the validated candidates against currently published events for this source.
|
||||
|
||||
1. **Exact ID match** → compare fields. Unchanged: no-op. Changed: candidate update.
|
||||
2. **Near match** — same game, date windows overlap, title similarity ≥ 0.85 — → treat as an update
|
||||
to the existing event, **keeping the existing ID**. This is what survives a wiki renaming an
|
||||
event without orphaning every user's completion mark.
|
||||
3. **No match** → new event.
|
||||
4. **Published event absent from this run's candidates** → mark `status = 'delisted'`. Do not
|
||||
delete.
|
||||
|
||||
**Conflict detection.** A candidate that changes an already-published `endsAt` by more than 24 hours
|
||||
is a `date_conflict`. This is the case worth being paranoid about: the user may have already planned
|
||||
around the old date, and a silent shift is exactly the failure the product exists to prevent. Route
|
||||
it to quarantine regardless of confidence.
|
||||
|
||||
### Scoring
|
||||
|
||||
Confidence is computed here, from evidence — **not** taken from the model's self-report. A model
|
||||
saying "confidence: 0.95" is a token prediction, not a measurement.
|
||||
|
||||
```
|
||||
base parser → 0.95 llm → 0.70
|
||||
+0.15 the same event was extracted identically from a previous run
|
||||
+0.10 both timestamps have precision "exact"
|
||||
+0.10 a second source for the same game corroborates within 1 hour
|
||||
−0.20 any soft rule fired
|
||||
−0.30 this is a date_conflict against a published event
|
||||
```
|
||||
|
||||
Clamp to [0, 1]. `CONFIDENCE_THRESHOLD` (default 0.8) is the gate.
|
||||
|
||||
## Stage 6 — gate
|
||||
|
||||
| Condition | Destination |
|
||||
|---|---|
|
||||
| `confidence >= CONFIDENCE_THRESHOLD` and no conflict | publish |
|
||||
| `confidence < CONFIDENCE_THRESHOLD` | quarantine — `low_confidence` |
|
||||
| `date_conflict` | quarantine — `date_conflict`, with `conflicts_with` set |
|
||||
| failed a hard rule | quarantine — `sanity_failed` |
|
||||
| new event type or field the schema does not recognize | quarantine — `novel_shape` |
|
||||
|
||||
A quarantined event does not block its siblings. If eight events in a run pass and two are held, the
|
||||
eight publish.
|
||||
|
||||
## Stage 7 — publish
|
||||
|
||||
Upsert by ID inside a transaction. Bump `version` and `updatedAt` only when a field actually
|
||||
changed — an unchanged run must not churn `updatedAt`, or the freshness badge (PRD F7) becomes
|
||||
meaningless. Update `sources.content_hash`, `etag`, `last_success_at`, and reset
|
||||
`consecutive_failures`.
|
||||
|
||||
## The review gate
|
||||
|
||||
Quarantined events surface at `GET /review` on the admin listener (`127.0.0.1:ADMIN_PORT`). See
|
||||
`docs/ARCHITECTURE.md` § Why `/review` needs no auth — the routes are simply not registered on the
|
||||
public listener.
|
||||
|
||||
The review UI shows, per held event: the parsed fields, the reason and detail, the conflicting
|
||||
published event side-by-side when applicable, a link to the source, and the exact cleaned text
|
||||
excerpt the extraction came from. A reviewer needs to answer "is this date right?" without leaving
|
||||
the page.
|
||||
|
||||
- `POST /api/review/:id/approve` — writes to `events` with `extraction_method: 'manual'` and
|
||||
`confidence: 1.0`, and stamps `resolved_at` / `resolution`.
|
||||
- `POST /api/review/:id/reject` — stamps resolution only. The event is not published, and the same
|
||||
candidate will be re-held on the next run if the source has not changed.
|
||||
|
||||
Approving with edits is supported: the reviewer can correct a date before approving. That corrected
|
||||
value is the one that publishes.
|
||||
|
||||
**Quarantine depth is the health signal for the whole pipeline.** A growing queue means a source
|
||||
changed shape or the prompt regressed. `/api/health` exposes the count; watch it.
|
||||
|
||||
## Testing
|
||||
|
||||
Every adapter ships:
|
||||
|
||||
1. `fixtures/<game>/<source>-<YYYY-MM-DD>.html` — a real captured page.
|
||||
2. `fixtures/<game>/<source>-<YYYY-MM-DD>.expected.json` — the exact `GachaEvent[]` it should
|
||||
produce.
|
||||
3. A test running `parse` + `normalize` against the fixture with a pinned `ctx.now`, asserting
|
||||
deep equality.
|
||||
|
||||
`bun test` must pass with no network access. When a source changes shape, capture a new fixture
|
||||
alongside the old one and keep both — the old fixture is the regression test proving the parser
|
||||
still handles the previous format.
|
||||
|
||||
Validator rules get their own unit tests with deliberately broken inputs: backwards intervals,
|
||||
1000-year durations, `endsAt` set with `endPrecision: "unknown"`. These rules are the last line of
|
||||
defense before a wrong date reaches a user; test them like it.
|
||||
@@ -0,0 +1,291 @@
|
||||
# LLM Extraction
|
||||
|
||||
Stage 3 of the pipeline, for sources whose markup is too unstable to parse deterministically. Read
|
||||
`docs/INGESTION.md` first for where this sits.
|
||||
|
||||
## Rules
|
||||
|
||||
1. **The model runs at ingestion time only.** No request path calls Anthropic. A page load must
|
||||
never trigger inference.
|
||||
2. **Deterministic parsers win.** If a source has a JSON API or a stable table, it gets a parser,
|
||||
not a prompt.
|
||||
3. **The model's job is transcription, not judgment.** It converts prose and tables into structured
|
||||
dates. It does not decide what is important, does not infer missing dates, and does not resolve
|
||||
contradictions — it reports them.
|
||||
4. **Confidence is computed from evidence in `reconcile`, not asserted by the model.** The output
|
||||
schema has no confidence field. A model claiming 0.95 confidence has predicted a token, not
|
||||
measured anything.
|
||||
|
||||
## Model and parameters
|
||||
|
||||
| Setting | Value | Why |
|
||||
|---|---|---|
|
||||
| `model` | `claude-opus-5` | Exact, complete ID — never append a date suffix |
|
||||
| `max_tokens` | `16000` | Non-streaming; keeps the request under SDK HTTP timeouts |
|
||||
| `output_config.effort` | `"medium"` | Transcription, not reasoning. Sweep low/medium/high against fixtures before settling |
|
||||
| `output_config.format` | `zodOutputFormat(ExtractionResult)` | Schema-conformant output, validated by the SDK |
|
||||
| `thinking` | *omit* | On by default on `claude-opus-5`; the default is correct here |
|
||||
| `temperature` / `top_p` / `top_k` | **never set** | Removed on `claude-opus-5` — sending any of them returns 400 |
|
||||
|
||||
`thinking: {type: "enabled", budget_tokens: N}` is also removed and returns 400. If you want less
|
||||
thinking, lower `effort`.
|
||||
|
||||
Note that `max_tokens` caps thinking *and* output together. If extraction on a large page returns
|
||||
`stop_reason: "max_tokens"`, raise it rather than trimming the schema.
|
||||
|
||||
## The output schema
|
||||
|
||||
The model returns a list of candidate events plus explicit uncertainty. Note what is **absent**:
|
||||
no confidence score, no "importance" ranking, no summary of the page.
|
||||
|
||||
```ts
|
||||
// src/ingest/extract.ts
|
||||
import { z } from "zod";
|
||||
|
||||
const ExtractedEvent = z.object({
|
||||
title: z.string().describe("The event name exactly as written in the source, not paraphrased."),
|
||||
type: z.enum(["banner","story","rerun","challenge","login","shop","maintenance","other"]),
|
||||
summary: z.string().nullable()
|
||||
.describe("One sentence from the source describing the event. Null if the source gives none."),
|
||||
|
||||
startsAt: z.string().nullable()
|
||||
.describe("ISO 8601 UTC. Null only if the source truly does not state a start."),
|
||||
startPrecision: z.enum(["exact","day","unknown"])
|
||||
.describe("'exact' if a time of day is stated; 'day' if only a date; 'unknown' if neither."),
|
||||
|
||||
endsAt: z.string().nullable()
|
||||
.describe("ISO 8601 UTC. Null when the source says TBD, 'until further notice', or gives no end."),
|
||||
endPrecision: z.enum(["exact","day","unknown"]),
|
||||
|
||||
regionScoped: z.boolean()
|
||||
.describe("True if the end follows each server region's daily reset rather than one global instant."),
|
||||
|
||||
sourceTimezone: z.string().nullable()
|
||||
.describe("The timezone the source stated, e.g. 'UTC+8', 'server local'. Null if unstated."),
|
||||
evidence: z.string()
|
||||
.describe("The verbatim span from the input that gave you the dates. Must appear in the input."),
|
||||
});
|
||||
|
||||
const ExtractionResult = z.object({
|
||||
events: z.array(ExtractedEvent),
|
||||
ambiguities: z.array(z.object({
|
||||
title: z.string(),
|
||||
issue: z.string().describe("What is unclear or contradictory in the source."),
|
||||
})).describe("Events you could not confidently transcribe. These are held for human review."),
|
||||
});
|
||||
```
|
||||
|
||||
`evidence` is the load-bearing field. It is checked in `validate`: if the quoted span does not
|
||||
appear in the input text, the event is quarantined as `sanity_failed`. That check is what turns a
|
||||
fabricated date into a caught error instead of a shipped one.
|
||||
|
||||
## The system prompt
|
||||
|
||||
Kept in `src/ingest/prompts/extract-events.v1.md`, versioned in the filename, and logged as
|
||||
`prompt_version` in `extraction_log` so a regression can be traced to a specific revision.
|
||||
|
||||
It must stay **above 512 tokens** — that is the minimum cacheable prefix on `claude-opus-5`. Below
|
||||
it, prompt caching silently stops working with no error. After any prompt edit, check
|
||||
`usage.cache_read_input_tokens` is non-zero on the second call.
|
||||
|
||||
```markdown
|
||||
You extract scheduled in-game events from gacha game source pages into structured data.
|
||||
|
||||
Your output is consumed by a calendar that players rely on to avoid missing limited-time content.
|
||||
A wrong end date is worse than a missing event: a missing event sends someone to a wiki, a wrong
|
||||
one makes them miss content permanently. Transcribe what the source says; never supply what it
|
||||
omits.
|
||||
|
||||
## What counts as an event
|
||||
|
||||
Anything with a start and a bounded or open-ended run: character and weapon banners, story
|
||||
chapters, side events, login campaigns, limited shops, combat cycles, announced maintenance.
|
||||
|
||||
Not events: permanent features, general game descriptions, patch version numbers on their own,
|
||||
speculation or leaks, community posts, and anything phrased as expected, rumored, or datamined.
|
||||
|
||||
## Dates
|
||||
|
||||
- Emit UTC ISO 8601 with an explicit `Z`.
|
||||
- When the source states a timezone (commonly UTC+8 for Chinese-developed titles), convert to UTC
|
||||
and record what it stated in `sourceTimezone`.
|
||||
- When only a date is given, set the timestamp to 00:00:00Z and `precision: "day"`. Do not guess a
|
||||
time of day.
|
||||
- When the source says the end is TBD, "until further notice", "with the next version update", or
|
||||
gives no end at all: `endsAt: null` and `endPrecision: "unknown"`. This is a correct, expected
|
||||
answer. Do not compute a plausible date from a typical patch length.
|
||||
- `regionScoped` is true when the end is tied to each server's daily reset, false when the source
|
||||
gives one simultaneous global instant. Character banners are usually global; story and login
|
||||
events are usually region-scoped. Use what the source says over this heuristic when it says
|
||||
anything.
|
||||
|
||||
## Evidence
|
||||
|
||||
For every event, `evidence` must be a verbatim span copied from the input that contains the dates
|
||||
you reported. It is checked against the input automatically. If you cannot quote a span, the event
|
||||
belongs in `ambiguities` instead.
|
||||
|
||||
## Ambiguities
|
||||
|
||||
Put an entry in `ambiguities` — not in `events` — when the source contradicts itself, gives dates
|
||||
you cannot reconcile, or describes something that may not be a scheduled event. A human reviews
|
||||
these. Reporting uncertainty is a successful outcome, not a failure; guessing to avoid it is the
|
||||
one thing that breaks this system.
|
||||
|
||||
## Scope
|
||||
|
||||
Report every qualifying event on the page and nothing else. Do not rank them, do not summarize the
|
||||
page, do not comment on your process, and do not add fields the schema does not have. If the page
|
||||
contains no events, return empty arrays.
|
||||
```
|
||||
|
||||
### Why the prompt reads the way it does
|
||||
|
||||
`claude-opus-5` follows instructions literally and verifies its own work without being told, so the
|
||||
prompt states scope and boundaries plainly instead of adding emphasis or self-check scaffolding.
|
||||
Specifically: **do not add "double-check your answer" or "verify before responding"** here. On this
|
||||
model that produces over-verification with no accuracy gain. If extraction quality drops, change the
|
||||
schema descriptions or `effort` — not the volume of the prompt.
|
||||
|
||||
## Request shape
|
||||
|
||||
```ts
|
||||
import Anthropic from "@anthropic-ai/sdk";
|
||||
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
|
||||
|
||||
const client = new Anthropic(); // reads ANTHROPIC_API_KEY from env
|
||||
|
||||
const response = await client.messages.parse({
|
||||
model: "claude-opus-5",
|
||||
max_tokens: 16000,
|
||||
system: [
|
||||
{
|
||||
type: "text",
|
||||
text: EXTRACTION_SYSTEM_PROMPT, // stable across every source — cached
|
||||
cache_control: { type: "ephemeral", ttl: "1h" },
|
||||
},
|
||||
],
|
||||
output_config: {
|
||||
effort: "medium",
|
||||
format: zodOutputFormat(ExtractionResult),
|
||||
},
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
`Game: ${adapter.game}`,
|
||||
`Source: ${adapter.url}`,
|
||||
`Today (UTC): ${ctx.now}`,
|
||||
adapter.extractionHints ?? "",
|
||||
"",
|
||||
"--- SOURCE TEXT ---",
|
||||
cleanedText,
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
// parsed_output is null if the model refused or hit max_tokens — check before use.
|
||||
const result = response.parsed_output;
|
||||
```
|
||||
|
||||
**Ordering matters for caching.** The system prompt is byte-identical across every source, so it
|
||||
sits first and stays cached. Everything volatile — game, URL, `now`, the page text — goes in the
|
||||
user turn, after the cache breakpoint. Interpolating `ctx.now` into the system prompt would
|
||||
invalidate the cache on every single call; it is in the user turn for exactly that reason.
|
||||
|
||||
## Batch mode
|
||||
|
||||
`EXTRACTION_MODE=batch` is the default for scheduled runs: 50% cheaper, results typically within an
|
||||
hour, which is irrelevant against a 6-hour cadence. The synchronous path above is for
|
||||
`refresh --now` and for local development.
|
||||
|
||||
```ts
|
||||
const batch = await client.messages.batches.create({
|
||||
requests: sourcesToExtract.map((s) => ({
|
||||
custom_id: s.runId, // key results by this — order is not guaranteed
|
||||
params: { /* same body as above */ },
|
||||
})),
|
||||
});
|
||||
|
||||
// Poll batches.retrieve(batch.id) until processing_status === "ended",
|
||||
// then stream batches.results(batch.id) and key each result by custom_id.
|
||||
```
|
||||
|
||||
Results arrive in **any order**. Key by `custom_id`, never by array position. The batch job's
|
||||
poll loop lives in `scheduler.ts` and persists `batch.id` so a process restart resumes rather than
|
||||
resubmitting.
|
||||
|
||||
## Handling non-success responses
|
||||
|
||||
Check `stop_reason` before touching `parsed_output`:
|
||||
|
||||
| `stop_reason` | Meaning | Action |
|
||||
|---|---|---|
|
||||
| `end_turn` | Normal | Proceed |
|
||||
| `max_tokens` | Output truncated; `parsed_output` unusable | Raise `max_tokens`, or split the page by section. Fail the run — never publish a partial list |
|
||||
| `refusal` | Safety classifier declined | Log `stop_details.category` to `extraction_log.refusal_category`, fail the run, alert. Vanishingly unlikely for game wiki content — if it fires, the input is probably not what you think it is |
|
||||
|
||||
`stop_details` can be `null` even on a refusal, so branch on `stop_reason` and treat `stop_details`
|
||||
as informational.
|
||||
|
||||
**Do not write a JSON-repair or regex-extraction fallback.** Structured outputs guarantee schema
|
||||
conformance; if parsing fails, the response was truncated or refused, and both cases are handled
|
||||
above. A repair path would silently paper over truncation and publish half a calendar.
|
||||
|
||||
## Cost
|
||||
|
||||
At `claude-opus-5` rates — $5/MTok input, $25/MTok output.
|
||||
|
||||
One extraction of a cleaned wiki page:
|
||||
|
||||
```
|
||||
input ~5,000 tok × $5/MTok = $0.025
|
||||
output ~1,500 tok × $25/MTok = $0.0375
|
||||
───────
|
||||
sync $0.063
|
||||
batch $0.031 (50% off)
|
||||
```
|
||||
|
||||
Six sources, four runs a day:
|
||||
|
||||
| Scenario | Extractions/day | Cost/day | Cost/month |
|
||||
|---|---|---|---|
|
||||
| Realistic — ~20% of runs see changed content | ~5 | $0.16 | **~$4.70** |
|
||||
| Worst case — every source changes every run | 24 | $0.74 | ~$22 |
|
||||
| No content-hash skip, no batch | 24 | $1.51 | ~$45 |
|
||||
|
||||
The gap between rows one and three is the whole argument for the skip check and batch mode. Track
|
||||
actuals by summing `extraction_log` token columns — do not rely on these estimates once there is
|
||||
real data.
|
||||
|
||||
Prompt caching contributes modestly (the ~1k-token system prompt at 0.1× on reads) but is free to
|
||||
keep. Its real value is that it makes the prompt cheap to grow if extraction quality needs more
|
||||
instruction.
|
||||
|
||||
## Evaluating a prompt change
|
||||
|
||||
`extraction_log` stores `input_hash` for every call, and `snapshots` stores the cleaned text keyed
|
||||
by the same hash. So a prompt revision is evaluated by replaying past inputs — no re-fetching, no
|
||||
new scraping load:
|
||||
|
||||
1. Pull the last N distinct `input_hash` values with known-correct expected output.
|
||||
2. Run the new prompt against each cleaned snapshot.
|
||||
3. Diff against `fixtures/*/*.expected.json`.
|
||||
4. Compare on three axes: dates correct, events missed, events hallucinated. **Hallucinated events
|
||||
and wrong dates are disqualifying; a missed event is a regression to weigh.** That asymmetry is
|
||||
the product rule from `docs/PRD.md` restated as an eval criterion.
|
||||
|
||||
Bump the prompt filename version and record it as `prompt_version` so the log distinguishes
|
||||
"extraction got worse" from "the source changed".
|
||||
|
||||
## Things not to do
|
||||
|
||||
- Do not ask the model to output a confidence score. Confidence is computed in `reconcile`.
|
||||
- Do not ask the model to resolve a contradiction between two sources. Route it to quarantine.
|
||||
- Do not send raw HTML. Always clean first — it is a 3× cost difference and it improves accuracy.
|
||||
- Do not add a second model call to "verify" the first. That is a scaffolding pattern this model
|
||||
does not need, and it doubles cost for no measured gain. The evidence-span check and the
|
||||
validator rules are the verification layer.
|
||||
- Do not lower `max_tokens` to save money. Output tokens scale with the number of events found;
|
||||
truncation costs a whole run.
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
# Gacha Event Tracker — Product Spec
|
||||
|
||||
## The problem
|
||||
|
||||
A player of three or four gacha games is tracking a dozen concurrent, overlapping, time-boxed
|
||||
events across as many different in-game calendars. The information exists — on wikis, in patch
|
||||
notes, in-game — but never in one place and never sorted by the thing that actually matters:
|
||||
**what expires next.** The failure mode is missing a limited event by a day.
|
||||
|
||||
## What this app is
|
||||
|
||||
A single-page web app that answers three questions:
|
||||
|
||||
1. What is running right now, across all my games?
|
||||
2. What ends soonest?
|
||||
3. Which of these have I already finished?
|
||||
|
||||
## What this app is not
|
||||
|
||||
- Not an account system. There is no login, no profile, no cloud sync.
|
||||
- Not a wiki. It does not explain how to complete an event, only that it exists and when it ends.
|
||||
- Not a notification service. No push, no email, no background alerts. (A browser-local reminder
|
||||
is a plausible v2; it is out of scope for v1.)
|
||||
- Not a damage calculator, build planner, or pull tracker.
|
||||
|
||||
## Users
|
||||
|
||||
One persona: a player of 2–5 gacha games who checks in a few times a week, most often on mobile.
|
||||
They care about accuracy of end dates above everything else — a wrong date is worse than a missing
|
||||
event, because a missing event sends them to a wiki while a wrong one makes them miss content.
|
||||
|
||||
## Scope — v1
|
||||
|
||||
### Games at launch
|
||||
|
||||
| Game | ID |
|
||||
|---|---|
|
||||
| Genshin Impact | `genshin` |
|
||||
| Honkai: Star Rail | `hsr` |
|
||||
| Zenless Zone Zero | `zzz` |
|
||||
| Wuthering Waves | `wuwa` |
|
||||
| Arknights | `arknights` |
|
||||
| Arknights: Endfield | `endfield` |
|
||||
|
||||
Adding a seventh game must require no schema change — only a new adapter. That is the test of
|
||||
whether the data model is right.
|
||||
|
||||
### Features
|
||||
|
||||
**F1 — Calendar view (default).**
|
||||
A horizontal timeline, one lane per game, spanning a scrollable date range with "today" pinned as a
|
||||
vertical marker. Each event is a bar from `startsAt` to `endsAt`. Bars are colored by game, and
|
||||
completed events render at reduced opacity with a check. Clicking a bar opens a detail panel with
|
||||
title, type, exact start/end in the user's local timezone, source link, and a completion toggle.
|
||||
|
||||
An event with `endsAt: null` renders as a bar with a frayed right edge and the label "end date
|
||||
unknown" — it must be visually distinct from an event that ends far in the future.
|
||||
|
||||
**F2 — Ends-soonest list.**
|
||||
A flat list of all *currently running* events sorted ascending by end date, with a relative
|
||||
countdown ("ends in 2 days", "ends in 4 hours"). Under 24 hours, the row is emphasized. This is the
|
||||
view that justifies the app; it should be reachable in one tap from the calendar and is the better
|
||||
default on narrow screens.
|
||||
|
||||
**F3 — Mark completed.**
|
||||
A toggle on every event, in both views. State is written to `localStorage` immediately and
|
||||
optimistically — there is no server round trip and no failure case. Completed events stay visible
|
||||
but de-emphasized; a filter toggles them out entirely.
|
||||
|
||||
**F4 — Filters.**
|
||||
Filter by game (multi-select, persisted) and by event type. Hiding a game hides it from both views.
|
||||
Preferences persist in `localStorage`.
|
||||
|
||||
**F5 — Region selection.**
|
||||
A user picks Asia / America / Europe once. For events where `regionScoped` is true, all displayed
|
||||
end times resolve to that region's server reset. This is stored in `localStorage` and defaults to a
|
||||
guess from the browser timezone, shown as a dismissible "showing America server times — change".
|
||||
|
||||
**F6 — Export / import.**
|
||||
Because there are no accounts, moving between devices is manual: download a JSON file of completed
|
||||
IDs and preferences, upload it elsewhere. Import merges rather than replaces, and never removes a
|
||||
completion the user already has.
|
||||
|
||||
**F7 — Freshness disclosure.**
|
||||
The footer shows when the feed was last updated, per game. If a game's data is more than 48 hours
|
||||
stale, its lane carries a warning badge. Never present stale data as current — the whole value
|
||||
proposition is trust in the dates.
|
||||
|
||||
## Out of scope for v1
|
||||
|
||||
Accounts and sync; push notifications; per-event checklists or progress tracking; in-game resource
|
||||
or pull tracking; user-submitted events; mobile apps; localization beyond English.
|
||||
|
||||
## Success criteria
|
||||
|
||||
- A user can identify their next expiring event within **5 seconds** of load, on mobile.
|
||||
- Published end dates are correct for **99%+** of events. This is a data-quality target, and it is
|
||||
what the review gate in `docs/INGESTION.md` exists to protect. Prefer publishing nothing to
|
||||
publishing a guess.
|
||||
- Adding a new game is an adapter plus a fixture plus a test — no schema migration, no client
|
||||
change.
|
||||
|
||||
## Quality bar for dates — the core product rule
|
||||
|
||||
The app's entire value is that the dates are right. Therefore:
|
||||
|
||||
- An event with an uncertain end date is published with `endsAt: null`, **not** with a plausible
|
||||
guess.
|
||||
- An event whose extraction confidence is below threshold is not published at all until a human
|
||||
approves it.
|
||||
- Every event links to its source so a skeptical user can verify in one click.
|
||||
|
||||
An empty calendar is a recoverable disappointment. A confidently wrong end date is the failure this
|
||||
product exists to prevent.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Does the calendar need a month/grid view, or is the timeline enough? (Assumption: timeline is
|
||||
enough for v1; revisit after use.)
|
||||
- Should events the user has hidden by game filter still count toward "ends soonest"?
|
||||
(Assumption: no — the filter is global.)
|
||||
- Is 6 hours the right refresh cadence? (Assumption: yes; events are announced days ahead, so
|
||||
sub-hourly refresh buys nothing and costs API spend.)
|
||||
Reference in New Issue
Block a user