From 8c2ec19c22e8d0ea471a0ebe216bec5cf1665520 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Mon, 17 Aug 2026 17:21:19 +0200 Subject: [PATCH] docs: record the two new sources and the shapes they added MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nine sources across eight games now, two more Game8 page shapes, two more date formats, and the h4 rule that neither game's page can be read without. Also notes what the two new games do *not* have: no `resetOffsets` entry, so both take the regional default. Neither source states a server map, and `resetOffsets` feeds `dayKey`, which is a localStorage key — so that is recorded as an assumption to confirm rather than left to look like a verified fact. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/agents/adapter-author.md | 2 +- CLAUDE.md | 19 ++++++++++++------- docs/DATA-MODEL.md | 8 +++++++- docs/INGESTION.md | 25 +++++++++++++++++++++++-- 4 files changed, 43 insertions(+), 11 deletions(-) diff --git a/.claude/agents/adapter-author.md b/.claude/agents/adapter-author.md index f79e08f..0918d98 100644 --- a/.claude/agents/adapter-author.md +++ b/.claude/agents/adapter-author.md @@ -25,7 +25,7 @@ Fetch the page exactly once. **3. Decide whether an existing parser covers it.** Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game. Two exist: `game8` -(six sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first: +(eight sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first: - **Existing parser handles it** → add one entry to `SOURCES` in `adapters/index.ts`. No new parsing code. This is the common case and should be the first thing you try. diff --git a/CLAUDE.md b/CLAUDE.md index 707e495..895be3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,8 +8,8 @@ A web app that aggregates live and upcoming events across popular gacha games, p calendar, sorts them by end date or by what the reader is partway through, tracks day-by-day progress on events that repeat daily, and lets a user mark events completed. -**Status: working app, refreshing itself on a schedule.** Schema, two parsers, seven sources across -six games, the full interface, offline support, a static server, a Docker image and CI all exist and +**Status: working app, refreshing itself on a schedule.** Schema, two parsers, nine sources across +eight games, the full interface, offline support, a static server, a Docker image and CI all exist and are tested. The refresh runner (`bun run refresh`) fetches, caches raw snapshots and rebuilds the feed; `.github/workflows/refresh.yml` runs it twice a day and commits only when a page actually changed. The SQLite layer and the review queue are still specified in `docs/` but not built, so the @@ -76,7 +76,7 @@ re-verify a sample against the live page afterward. ``` src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts -src/ingest/ html.ts, dates.ts (six formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts +src/ingest/ html.ts, dates.ts (eight formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts parsers/ game8.ts, wikigg.ts — keyed by SITE, not game adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam src/client/ React app, service worker, manifest @@ -84,7 +84,7 @@ src/client/ React app, service worker, manifest lens.ts — who sees which rows (focus, outstanding, next-to-expire); pure scripts/ build-feed.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches) serve.ts static server + /api/health -test/ 318 tests +test/ 348 tests fixtures// raw HTML + .expected.json per source — pinned, kept forever snapshots/ current page per source, rewritten by refresh — see its README ``` @@ -116,14 +116,19 @@ These come from how gacha games actually schedule things, and they cause most bu - **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a recoverable disappointment; a confidently wrong date is the failure this product exists to prevent. -- **Parsers are keyed by site, not game.** One `game8` parser serves six sources; `wikigg` serves +- **Parsers are keyed by site, not game.** One `game8` parser serves eight sources; `wikigg` serves one. Adding a source for a known site is one `SOURCES` entry; a new site is a parser module. -- **Game8 has no single template.** Five shapes are known and a page may mix them: label/value +- **Game8 has no single template.** Seven shapes are known and a page may mix them: label/value detail tables, column tables, image-grid schedules (unsupportable), combined label+range+blurb - cells, and rowspan Start/End pairs. Full table in `docs/INGESTION.md`. Before assuming a new + cells, rowspan Start/End pairs, labelled `Start: … End: …` cells, and `
`-separated date pairs. + Full table in `docs/INGESTION.md`. Before assuming a new Game8 page will work, dump its structure and check **every** table — Endfield was written off as undatable on a pass that only inspected its `Duration` rows, and its real events were further down the page. +- **Check what fences a section off.** Inclusion is decided by headings, and the level varies: Persona + 5 hides fifty finished events behind nothing but an `

Finished Events

` in a collapsed + accordion, while Genshin uses `h4` for sub-headings *inside* one event. So `h4` gates sections but + never names one — an unrecognised `h4` must leave the current event title alone. - **Prefer a source that states machine-readable times.** wiki.gg emits ISO timestamps with a timer per server region, which is the only reason `regionEnds` carries real data anywhere. - **Silent drops are the dangerous failure.** A date format the parser does not recognise makes diff --git a/docs/DATA-MODEL.md b/docs/DATA-MODEL.md index ba45953..b117e10 100644 --- a/docs/DATA-MODEL.md +++ b/docs/DATA-MODEL.md @@ -9,7 +9,7 @@ import { z } from "zod"; export const GameId = z.enum([ - "genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", + "genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", ]); export const EventType = z.enum([ @@ -254,9 +254,15 @@ reader cannot see. | Game | Reset (server local) | Server offset | Reset (UTC) | Copenhagen, summer / winter | |---|---|---|---|---| | Genshin, Star Rail, ZZZ, Wuwa, NTE | 04:00 | region (EU = UTC+1) | 03:00 | 05:00 / 04:00 | +| Infinity Nikki, P5X | 04:00 | region (assumed) | 03:00 | 05:00 / 04:00 | | Endfield, Europe | 04:00 | UTC-5 (on the Americas server) | 09:00 | 11:00 / 10:00 | | Endfield, Asia / Americas | 04:00 | regional default | 20:00 / 09:00 | — | +Infinity Nikki and P5X carry **no `resetOffsets` entry**, so they take the regional default. That is +an assumption, not a verified server map — neither source states one. Confirm it against the games +before relying on it, and note that adding an override later re-labels the game-day of ticks readers +have already logged, which is the change this table warns about below. + These server offsets are **fixed and do not observe DST**, so the reader's local reset time moves by an hour across the European clock change while the UTC instant stays put. diff --git a/docs/INGESTION.md b/docs/INGESTION.md index 0403a80..9bb1861 100644 --- a/docs/INGESTION.md +++ b/docs/INGESTION.md @@ -45,7 +45,7 @@ Consequences worth internalising: | Parser | Site | Sources using it | |---|---|---| -| `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE | +| `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE, Infinity Nikki, Persona 5: The Phantom X | | `wikigg` | wiki.gg MediaWiki `mp-event` templates | Endfield | `wikigg` is the better shape by a distance: it emits ISO timestamps with one timer per server @@ -63,11 +63,19 @@ All live in `src/ingest/dates.ts`, each returning null rather than inferring any | `parseFullRange` | `Aug. 14, 2026 - Aug. 24, 2026` (a year each side) | Star Rail, Wuthering Waves | | `parseShortSlashRange` | `08/09/26 - 08/30/26` | Endfield | | `parseSlashDateTimeRange` | `2021/01/16 04:00 - 2021/01/31 03:59` | Genshin past events | +| `parseLabelledStartEnd` | `Start: January 24, 2025 End: Permanent` | Infinity Nikki | +| `parseAdjacentFullRange` | `July 30, 2026 August 13, 2026` (halves split by an `
`) | Persona 5: The Phantom X | | `parseOpenRange` | `Jul. 24, 2026 - End of 4.6`, `July 10, 2026 - Permanent` | Star Rail, Wuthering Waves | `parseOpenRange` is tried last because it is the most permissive — it accepts any leading full date and reports no end. +The last two are anchored at both ends and require a year on each half, which is what keeps them from +eating prose. `August 12, 2026 Day 3 rewards are doubled` would otherwise read "Day 3" as an end, and +`June 25, 2026 July 16/30, 2026` names *two* candidate ends — so it takes neither, and the leftover is +not shown as a summary either (a date the parser refused to trust must not reappear dressed as +information). + ### The parser interface ```ts @@ -125,12 +133,25 @@ Game8 uses at least four page templates and a game's page may use any of them: (`Period: 08/09/26 - 08/30/26 During the event...`). *(Arknights: Endfield)* 5. **Rowspan Start/End pairs** — the event name spans two rows, so a flat cell reader sees `[title, "Start", date]` then `["End", date]`. *(Zenless Zone Zero)* +6. **Labelled cells** — one cell holding `Start: ` and `End: ` split by a `
`, where + the end half is often the word `Permanent`. *(Infinity Nikki)* +7. **`
`-separated pairs** — a `Event | Duration` table whose two dates are divided by a rule + rather than a dash, so a tag-stripping reader sees only whitespace between them. The same page + repeats each live event under its own `h3` with `Start Date` / `End Date` rows and a paragraph of + prose; those corroborate the dates and supply the blurb the flat table lacks. + *(Persona 5: The Phantom X)* -Shapes 1, 2, 4 and 5 are handled. Before assuming a new Game8 page will work, dump its heading/table +Shapes 1, 2, 4, 5, 6 and 7 are handled. Before assuming a new Game8 page will work, dump its heading/table structure and check which shape it uses — and check **every** table, not just the obvious one. Endfield was written off as undatable on a first pass that only inspected its `Duration` rows; its two real events were in a table further down. +**Check what ends a section, too.** Headings decide inclusion and the level is not consistent: Persona +5 puts its whole finished back catalogue behind an `

Finished Events

` inside a collapsed +accordion, so a reader that ignores `h4` publishes fifty dead events. Genshin uses `h4` the opposite +way — for sub-headings *within* one event ("Availability Period") — so an unrecognised `h4` gates the +section but must never claim the event title. + ## Stage 1 — fetch - Send `If-None-Match` / `If-Modified-Since` from `sources.etag` / `last_modified`. A `304` ends