docs: bring every markdown file up to date with the code

The docs had drifted in ways that would mislead: DATA-MODEL documented a
localStorage shape the code stopped using (completedAt, no ignored store),
INGESTION claimed three Game8 templates when five are known, ARCHITECTURE
still listed the whole client and time.ts as unbuilt, and the review-quarantine
skill described a pipeline that does not exist yet without saying so.

Adds the parser roster and the six date formats as tables, documents the
subpath/base-href and offline behaviour, and records the new product surface
(first-run picker, ignore, offline, credit) as PRD features.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-15 01:27:56 +02:00
co-authored by Claude Opus 5
parent f2cf9ba0cb
commit 692b5a83d5
9 changed files with 211 additions and 67 deletions
+13 -5
View File
@@ -24,8 +24,8 @@ Fetch the page exactly once.
**3. Decide whether an existing parser covers it.**
Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game — one Game8 parser serves
every Game8 page. Check `PARSERS` first:
Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game. Two exist: `game8`
(six sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first:
- **Existing parser handles it** → add one entry to `SOURCES` in `adapters/index.ts`. No new
parsing code. This is the common case and should be the first thing you try.
@@ -34,6 +34,10 @@ every Game8 page. Check `PARSERS` first:
- **Undatable source** (no year, no end date, image-grid schedule) → **stop and report.** There is
no LLM fallback in this pipeline by design. Suggest a different source.
**Check every table before concluding a page is undatable.** Endfield was written off on a pass that
only inspected its `Duration` rows; its real events were in a table further down, and a second
source (wiki.gg) turned out to publish ISO timestamps with per-region timers.
**4. Implement the parser (only if step 3 says you need one).**
- `parse` must be **pure**: no network, no `Date.now()`, no randomness. Time comes from `ctx.now`.
@@ -48,9 +52,13 @@ every Game8 page. Check `PARSERS` first:
- An unstated end is `endsAt: null` + `endPrecision: "unknown"`. **Never compute a plausible end
from typical patch length.** This is the failure mode that makes the product worthless.
**5. Write the test.** `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` holds the exact
expected `GachaEvent[]`. The test runs `parse` + `normalize` against the fixture with a pinned
`ctx.now` and asserts deep equality.
**5. Write the test.** `fixtures/<game>/<site>-events-<YYYY-MM-DD>.expected.json` holds the exact
expected `GachaEvent[]`. Add the source to `CASES` in `test/adapters/game8.test.ts`; the shared
assertions (schema, determinism, no backwards intervals, 180-day cap, unique IDs) then apply for
free. Regenerate the expected file with `bun run parse <id> <fixture> --json`.
**Fixture names matter**: `build-feed` selects by `<site>-*` within the game directory, so a game
with two sources needs distinct site prefixes.
**6. Verify.** Run `bun test` and confirm it passes with no network. Then hand-check three or four
events against the live page and state in your report that you did — a green test against an
+13 -7
View File
@@ -35,9 +35,13 @@ only schema edit a new game should require. If it needs more, that is a finding
Fetch the source page **once** and save the raw HTML:
```
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
fixtures/<game>/<site>-events-<YYYY-MM-DD>.html
```
The `<site>` prefix is load-bearing: `build-feed` picks fixtures by site within the game directory,
so a game with two sources whose files share a prefix will hand one site's page to the other's
parser.
Everything after this point works offline against that file. Do not re-fetch while iterating on the
parser.
@@ -77,16 +81,18 @@ report that you did this.
`parserId`, and optionally `priority` (higher wins when sources disagree).
- Insert the matching `sources` row: id, game, url, parser_id, priority, `min_interval_ms`.
## 7. First run
## 7. Rebuild the feed
```
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
bun run build:feed # regenerates public/data/events.v1.json
bun run dev # build and serve on :3000
```
Inspect what it *would* publish before letting it write. Then run for real and check the review
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
on its first pass, because nothing corroborates it yet. That is the gate working, not a bug.
Review and approve them.
Check the event count and any conflicts the merge reports. A game with two sources will surface
disagreements — those are the gate working, not a bug.
The scheduler, quarantine table and `/review` queue described in `docs/INGESTION.md` are **not built
yet**; today the feed is generated offline from fixtures.
## Checklist
@@ -5,6 +5,11 @@ description: Work through the quarantined-event queue — triage held events by
# Reviewing the quarantine queue
> **Not built yet.** The quarantine table, the `/review` route and the ingest scheduler are specified
> in `docs/INGESTION.md` but do not exist in the tree. Today the closest equivalent is the conflict
> list `bun run build:feed` prints when two sources disagree. Use this skill once the pipeline lands;
> until then, treat it as the spec for what that review flow should do.
Held events are candidates the pipeline declined to publish. Working the queue is both a data task
(get these events onto the calendar) and a diagnostic one (**a growing queue means something
upstream broke**).