diff --git a/.claude/agents/adapter-author.md b/.claude/agents/adapter-author.md index f79e08f..0918d98 100644 --- a/.claude/agents/adapter-author.md +++ b/.claude/agents/adapter-author.md @@ -25,7 +25,7 @@ Fetch the page exactly once. **3. Decide whether an existing parser covers it.** Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game. Two exist: `game8` -(six sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first: +(eight sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first: - **Existing parser handles it** → add one entry to `SOURCES` in `adapters/index.ts`. No new parsing code. This is the common case and should be the first thing you try. diff --git a/CLAUDE.md b/CLAUDE.md index 707e495..895be3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,8 +8,8 @@ A web app that aggregates live and upcoming events across popular gacha games, p calendar, sorts them by end date or by what the reader is partway through, tracks day-by-day progress on events that repeat daily, and lets a user mark events completed. -**Status: working app, refreshing itself on a schedule.** Schema, two parsers, seven sources across -six games, the full interface, offline support, a static server, a Docker image and CI all exist and +**Status: working app, refreshing itself on a schedule.** Schema, two parsers, nine sources across +eight games, the full interface, offline support, a static server, a Docker image and CI all exist and are tested. The refresh runner (`bun run refresh`) fetches, caches raw snapshots and rebuilds the feed; `.github/workflows/refresh.yml` runs it twice a day and commits only when a page actually changed. The SQLite layer and the review queue are still specified in `docs/` but not built, so the @@ -76,7 +76,7 @@ re-verify a sample against the live page afterward. ``` src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts -src/ingest/ html.ts, dates.ts (six formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts +src/ingest/ html.ts, dates.ts (eight formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts parsers/ game8.ts, wikigg.ts — keyed by SITE, not game adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam src/client/ React app, service worker, manifest @@ -84,7 +84,7 @@ src/client/ React app, service worker, manifest lens.ts — who sees which rows (focus, outstanding, next-to-expire); pure scripts/ build-feed.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches) serve.ts static server + /api/health -test/ 318 tests +test/ 348 tests fixtures// raw HTML + .expected.json per source — pinned, kept forever snapshots/ current page per source, rewritten by refresh — see its README ``` @@ -116,14 +116,19 @@ These come from how gacha games actually schedule things, and they cause most bu - **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a recoverable disappointment; a confidently wrong date is the failure this product exists to prevent. -- **Parsers are keyed by site, not game.** One `game8` parser serves six sources; `wikigg` serves +- **Parsers are keyed by site, not game.** One `game8` parser serves eight sources; `wikigg` serves one. Adding a source for a known site is one `SOURCES` entry; a new site is a parser module. -- **Game8 has no single template.** Five shapes are known and a page may mix them: label/value +- **Game8 has no single template.** Seven shapes are known and a page may mix them: label/value detail tables, column tables, image-grid schedules (unsupportable), combined label+range+blurb - cells, and rowspan Start/End pairs. Full table in `docs/INGESTION.md`. Before assuming a new + cells, rowspan Start/End pairs, labelled `Start: … End: …` cells, and `
`-separated date pairs. + Full table in `docs/INGESTION.md`. Before assuming a new Game8 page will work, dump its structure and check **every** table — Endfield was written off as undatable on a pass that only inspected its `Duration` rows, and its real events were further down the page. +- **Check what fences a section off.** Inclusion is decided by headings, and the level varies: Persona + 5 hides fifty finished events behind nothing but an `

Finished Events

` in a collapsed + accordion, while Genshin uses `h4` for sub-headings *inside* one event. So `h4` gates sections but + never names one — an unrecognised `h4` must leave the current event title alone. - **Prefer a source that states machine-readable times.** wiki.gg emits ISO timestamps with a timer per server region, which is the only reason `regionEnds` carries real data anywhere. - **Silent drops are the dangerous failure.** A date format the parser does not recognise makes diff --git a/docs/DATA-MODEL.md b/docs/DATA-MODEL.md index ba45953..b117e10 100644 --- a/docs/DATA-MODEL.md +++ b/docs/DATA-MODEL.md @@ -9,7 +9,7 @@ import { z } from "zod"; export const GameId = z.enum([ - "genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", + "genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", ]); export const EventType = z.enum([ @@ -254,9 +254,15 @@ reader cannot see. | Game | Reset (server local) | Server offset | Reset (UTC) | Copenhagen, summer / winter | |---|---|---|---|---| | Genshin, Star Rail, ZZZ, Wuwa, NTE | 04:00 | region (EU = UTC+1) | 03:00 | 05:00 / 04:00 | +| Infinity Nikki, P5X | 04:00 | region (assumed) | 03:00 | 05:00 / 04:00 | | Endfield, Europe | 04:00 | UTC-5 (on the Americas server) | 09:00 | 11:00 / 10:00 | | Endfield, Asia / Americas | 04:00 | regional default | 20:00 / 09:00 | — | +Infinity Nikki and P5X carry **no `resetOffsets` entry**, so they take the regional default. That is +an assumption, not a verified server map — neither source states one. Confirm it against the games +before relying on it, and note that adding an override later re-labels the game-day of ticks readers +have already logged, which is the change this table warns about below. + These server offsets are **fixed and do not observe DST**, so the reader's local reset time moves by an hour across the European clock change while the UTC instant stays put. diff --git a/docs/INGESTION.md b/docs/INGESTION.md index 0403a80..9bb1861 100644 --- a/docs/INGESTION.md +++ b/docs/INGESTION.md @@ -45,7 +45,7 @@ Consequences worth internalising: | Parser | Site | Sources using it | |---|---|---| -| `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE | +| `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE, Infinity Nikki, Persona 5: The Phantom X | | `wikigg` | wiki.gg MediaWiki `mp-event` templates | Endfield | `wikigg` is the better shape by a distance: it emits ISO timestamps with one timer per server @@ -63,11 +63,19 @@ All live in `src/ingest/dates.ts`, each returning null rather than inferring any | `parseFullRange` | `Aug. 14, 2026 - Aug. 24, 2026` (a year each side) | Star Rail, Wuthering Waves | | `parseShortSlashRange` | `08/09/26 - 08/30/26` | Endfield | | `parseSlashDateTimeRange` | `2021/01/16 04:00 - 2021/01/31 03:59` | Genshin past events | +| `parseLabelledStartEnd` | `Start: January 24, 2025 End: Permanent` | Infinity Nikki | +| `parseAdjacentFullRange` | `July 30, 2026 August 13, 2026` (halves split by an `
`) | Persona 5: The Phantom X | | `parseOpenRange` | `Jul. 24, 2026 - End of 4.6`, `July 10, 2026 - Permanent` | Star Rail, Wuthering Waves | `parseOpenRange` is tried last because it is the most permissive — it accepts any leading full date and reports no end. +The last two are anchored at both ends and require a year on each half, which is what keeps them from +eating prose. `August 12, 2026 Day 3 rewards are doubled` would otherwise read "Day 3" as an end, and +`June 25, 2026 July 16/30, 2026` names *two* candidate ends — so it takes neither, and the leftover is +not shown as a summary either (a date the parser refused to trust must not reappear dressed as +information). + ### The parser interface ```ts @@ -125,12 +133,25 @@ Game8 uses at least four page templates and a game's page may use any of them: (`Period: 08/09/26 - 08/30/26 During the event...`). *(Arknights: Endfield)* 5. **Rowspan Start/End pairs** — the event name spans two rows, so a flat cell reader sees `[title, "Start", date]` then `["End", date]`. *(Zenless Zone Zero)* +6. **Labelled cells** — one cell holding `Start: ` and `End: ` split by a `
`, where + the end half is often the word `Permanent`. *(Infinity Nikki)* +7. **`
`-separated pairs** — a `Event | Duration` table whose two dates are divided by a rule + rather than a dash, so a tag-stripping reader sees only whitespace between them. The same page + repeats each live event under its own `h3` with `Start Date` / `End Date` rows and a paragraph of + prose; those corroborate the dates and supply the blurb the flat table lacks. + *(Persona 5: The Phantom X)* -Shapes 1, 2, 4 and 5 are handled. Before assuming a new Game8 page will work, dump its heading/table +Shapes 1, 2, 4, 5, 6 and 7 are handled. Before assuming a new Game8 page will work, dump its heading/table structure and check which shape it uses — and check **every** table, not just the obvious one. Endfield was written off as undatable on a first pass that only inspected its `Duration` rows; its two real events were in a table further down. +**Check what ends a section, too.** Headings decide inclusion and the level is not consistent: Persona +5 puts its whole finished back catalogue behind an `

Finished Events

` inside a collapsed +accordion, so a reader that ignores `h4` publishes fifty dead events. Genshin uses `h4` the opposite +way — for sub-headings *within* one event ("Availability Period") — so an unrecognised `h4` gates the +section but must never claim the event title. + ## Stage 1 — fetch - Send `If-None-Match` / `If-Modified-Since` from `sources.etag` / `last_modified`. A `304` ends