docs: correct the counts three docs had drifted past

Each of these was true when written and stopped being true when a source
landed. dates.ts holds ten formats, not nine; INGESTION says Game8 uses
"at least four page templates" directly above a list of seven; its testing
note still credits "both current adapters" when there are eleven; PRD
lists the seven launch games against ten in GameId. AGENTS also missed
fandom from the parsers-are-keyed-by-site bullet and build-static.ts from
the tree, which matters because another section leans on the latter.

The PRD table stays as launch scope rather than becoming a roster to keep
in sync — GameId is named as the live answer instead, which is the same
move that keeps this drift from recurring. Its no-schema-change claim is
sharpened to name GachaEvent, because Reverse: 1999 did add a field:
resetHourLocal, to game metadata, which moves no stored key. The claim as
written read as false against a commit that plainly added one.

Also draws sanitize into the stage diagram as 2.5. It runs on every source
without the pipeline arranging it, which is exactly why it kept being
absent from the picture.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-17 22:55:45 +02:00
co-authored by Claude Opus 5
parent 414dc840fd
commit 823556f2c5
3 changed files with 26 additions and 13 deletions
+6 -5
View File
@@ -105,14 +105,14 @@ re-verify a sample against the live page afterward.
``` ```
src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts
custom.ts — reader-authored games and events, and their key spaces custom.ts — reader-authored games and events, and their key spaces
src/ingest/ html.ts, dates.ts (nine formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts src/ingest/ html.ts, dates.ts (ten formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts
parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts — keyed by SITE, not game parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts — keyed by SITE, not game
adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam
src/client/ React app, service worker, manifest src/client/ React app, service worker, manifest
state/ progress, daily log, ignores, prefs, sort — all localStorage state/ progress, daily log, ignores, prefs, sort — all localStorage
useCustom.ts — the reader's own games and events (PRD F13) useCustom.ts — the reader's own games and events (PRD F13)
lens.ts — who sees which rows (focus, outstanding, next-to-expire); pure lens.ts — who sees which rows (focus, outstanding, next-to-expire); pure
scripts/ build-feed.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches) scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
serve.ts static server + /api/health serve.ts static server + /api/health
test/ 466 tests test/ 466 tests
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
@@ -146,9 +146,10 @@ These come from how gacha games actually schedule things, and they cause most bu
- **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing - **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing
year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a
recoverable disappointment; a confidently wrong date is the failure this product exists to prevent. recoverable disappointment; a confidently wrong date is the failure this product exists to prevent.
- **Parsers are keyed by site, not game.** One `game8` parser serves eight sources; `wikigg` and - **Parsers are keyed by site, not game.** One `game8` parser serves eight sources; `wikigg`,
`akwiki` serve one each — same host family, entirely different templates. Adding a source for a `akwiki` and `fandom` serve one each — the first two share a host family and have entirely
known site is one `SOURCES` entry; a new site is a parser module. different templates. Adding a source for a known site is one `SOURCES` entry; a new site is a
parser module.
- **A source may publish more than one region's schedule.** Arknights' wiki lists CN and Global on - **A source may publish more than one region's schedule.** Arknights' wiki lists CN and Global on
every row, five months apart. Publish the one our readers are on and skip the row that lacks it — every row, five months apart. Publish the one our readers are on and skip the row that lacks it —
a CN date on a Global calendar is a confidently wrong date, not a near miss. a CN date on a Global calendar is a confidently wrong date, not a near miss.
+10 -5
View File
@@ -4,11 +4,14 @@ Six stages, run per source. Every stage writes its outcome to `ingest_runs` so a
ago can be diagnosed without re-running. ago can be diagnosed without re-running.
``` ```
fetch → parse → merge → validate → reconcile → gate publish fetch → parse → merge → validate → reconcile → gate and publish
└──► quarantine (sanitize) └──► quarantine
``` ```
Sanitizing is stage 2.5 rather than a stage of its own: it is wired into the adapter seam, so it
runs on every source without the pipeline arranging it.
## No LLM ## No LLM
Event data is extracted by deterministic code only. There is no model call anywhere in this Event data is extracted by deterministic code only. There is no model call anywhere in this
@@ -156,7 +159,8 @@ produced it.
| Free-form prose with no table structure | Find a different source | | Free-form prose with no table structure | Find a different source |
| A clean table whose newest row is months old | **Not a source, an archive.** Check the *latest* date before writing anything: `bluearchive.fandom.com` parses perfectly and yields zero live events, which shows up as an empty lane and a permanently rejected snapshot rather than as an error | | A clean table whose newest row is months old | **Not a source, an archive.** Check the *latest* date before writing anything: `bluearchive.fandom.com` parses perfectly and yields zero live events, which shows up as an empty lane and a permanently rejected snapshot rather than as an error |
Game8 uses at least four page templates and a game's page may use any of them: Game8 uses at least seven page templates, a game's page may use any of them, and one page may mix
several:
1. **Label/value detail tables**`Event Start` / `Event End` rows under a per-event `h3`, full 1. **Label/value detail tables**`Event Start` / `Event End` rows under a per-event `h3`, full
dates with year. *(Genshin Impact)* dates with year. *(Genshin Impact)*
@@ -393,7 +397,8 @@ Every adapter ships:
**Regenerating `.expected.json` from the parser makes the test self-consistent, not correct.** After **Regenerating `.expected.json` from the parser makes the test self-consistent, not correct.** After
an intentional change, re-verify a sample against the live page — and ideally extract the same data an intentional change, re-verify a sample against the live page — and ideally extract the same data
a second way (a throwaway script over the fixture) to confirm counts and dates independently. That a second way (a throwaway script over the fixture) to confirm counts and dates independently. That
independent check is what caught the exact event counts for both current adapters. independent check is what pinned the exact event counts on every adapter here, and it is what caught
Endfield's real events sitting in a table the first pass never read.
When a source changes shape, capture a new fixture **alongside** the old one and keep both — the old When a source changes shape, capture a new fixture **alongside** the old one and keep both — the old
fixture is the regression test proving the parser still handles the previous format. fixture is the regression test proving the parser still handles the previous format.
+10 -3
View File
@@ -45,9 +45,16 @@ event, because a missing event sends them to a wiki while a wrong one makes them
| Arknights: Endfield | `endfield` | | Arknights: Endfield | `endfield` |
| Neverness to Everness | `nte` | | Neverness to Everness | `nte` |
Adding a game must require no schema change — only a `GameId` entry and a source registration. Added since launch, on the strength of the release thread (`docs/FEEDBACK.md` § P1): Infinity Nikki
That is the test of whether the data model is right. A game may have several sources; see (`nikki`), Persona 5: The Phantom X (`p5x`), Reverse: 1999 (`r1999`). **`GameId` in
`docs/INGESTION.md` § Three layers. `src/shared/schema.ts` is the live answer** and `SOURCES` says which of them actually have a source —
this table is the launch scope, not a roster to keep in sync.
Adding a game must require no change to `GachaEvent` — only a `GameId` entry, its `games.ts`
metadata, and a source registration. That is the test of whether the data model is right, and it has
held: the ten games here have cost the event schema nothing. Per-game *metadata* does occasionally
grow (Reverse: 1999 needed `resetHourLocal` for a 05:00 reset), which is a different file and moves
no stored key. A game may have several sources; see `docs/INGESTION.md` § Three layers.
### Features ### Features