docs: record the FGO source, and the trap it fell into first

The adapter shipped with no docs at all, which left the one thing worth
writing down unwritten: this wiki publishes two servers' schedules on
two cross-linked pages, and the Japanese one parses just as cleanly as
ours. That now has a row in the assessed-sources table, a section under
Fandom, and a line in the ingestion pitfalls table alongside the akwiki
CN column and the bawiki JP tab — three sources bitten the same way is
a pattern, not a coincidence.

Also records what resetOffsets cannot express, since fgo is the first
game to need it: a server clock that shifts with daylight saving.

DATA-MODEL's GameId listing was two games behind the schema, missing
holodori as well as fgo.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-18 21:30:38 +02:00
co-authored by Claude Opus 5
parent 9f0bb62c98
commit efae87ae59
3 changed files with 59 additions and 8 deletions
+35 -5
View File
@@ -115,7 +115,7 @@ src/client/ React app, service worker, manifest
zoom.ts — the timeline's scale ladder; pure zoom.ts — the timeline's scale ladder; pure
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches) scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
serve.ts static server + /api/health serve.ts static server + /api/health
test/ 579 tests test/ 591 tests
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
snapshots/ current page per source, rewritten by refresh — see its README snapshots/ current page per source, rewritten by refresh — see its README
``` ```
@@ -201,10 +201,15 @@ Two more key spaces have the same property, for the same reason:
at 05:00, not 04:00, so its day rolls at 10:00 UTC on its single UTC-5 server. Do not encode that at 05:00, not 04:00, so its day rolls at 10:00 UTC on its single UTC-5 server. Do not encode that
as a bent `resetOffsets` value: shifting a game's stated server offset to land the right instant as a bent `resetOffsets` value: shifting a game's stated server offset to land the right instant
would misreport the server clock to everything else that asks for it. Both fields are absent for would misreport the server clock to everything else that asks for it. Both fields are absent for
every game that takes the default, which is why adding the second one moved nobody's day keys. Every day-key function takes an every game that takes the default, which is why adding the second one moved nobody's day keys.
optional `game`**anything reading or writing a tick must pass it**, or it writes under one Neither field can express a server whose offset *shifts*: Fate/Grand Order's English server runs
clock and reads under another. A day that drops out of `dailyDays` renders no pip, so a tick on it on US Pacific, which observes daylight saving, and one fixed number is wrong for half the year in
becomes unreachable; check real fixture windows before changing an offset. either direction — so `fgo` takes the default and `games.ts` says why. Reaching for a value anyway
would re-label day keys twice a year, which is the one thing this whole section exists to prevent.
Every day-key function takes an optional `game` — **anything reading or writing a tick must pass
it**, or it writes under one clock and reads under another. A day that drops out of `dailyDays`
renders no pip, so a tick on it becomes unreachable; check real fixture windows before changing an
offset.
The sanitizer at the ingest boundary recomputes an event ID only when a sanitized title actually The sanitizer at the ingest boundary recomputes an event ID only when a sanitized title actually
changed *and* the ID was minted the standard way. If a change to it starts moving IDs on real changed *and* the ID was minted the standard way. If a change to it starts moving IDs on real
@@ -253,6 +258,7 @@ A source whose ToS forbids automated access does not get an adapter. Flag it and
| `reverse1999.fandom.com` | **Built** (2026-08-17), via `api.php`, not the wiki page — see § Fandom below | | `reverse1999.fandom.com` | **Built** (2026-08-17), via `api.php`, not the wiki page — see § Fandom below |
| `bluearchive.fandom.com` | **Declined.** Fetches and parses fine; the page is the problem. Its `Event/Event_List` is a JP-server archive whose newest entry ended 2026-02-18, so all 88 rows are history and it yields **zero** live or upcoming events. An adapter would put an empty lane on the calendar and, because the runner rejects a body that parses to nothing, report a broken source forever. Same failure as the Infinity Nikki Game8 page, further along | | `bluearchive.fandom.com` | **Declined.** Fetches and parses fine; the page is the problem. Its `Event/Event_List` is a JP-server archive whose newest entry ended 2026-02-18, so all 88 rows are history and it yields **zero** live or upcoming events. An adapter would put an empty lane on the calendar and, because the runner rejects a body that parses to nothing, report a broken source forever. Same failure as the Infinity Nikki Game8 page, further along |
| `bluearchive.wiki` | **Built** (2026-08-17), from the rendered `/wiki/Events` page — see § Blue Archive below | | `bluearchive.wiki` | **Built** (2026-08-17), from the rendered `/wiki/Events` page — see § Blue Archive below |
| `fategrandorder.fandom.com` | **Built** (2026-08-18), via `api.php` like Reverse: 1999 — but off `Event_List_(US)`, **not** `Event_List`, which is the Japanese server. See § Fandom below |
| `holodori.wiki` | **Built** (2026-08-18), from the rendered `/wiki/Events` page. Miraheze again, so the same call as Blue Archive; CC BY-SA 4.0, no `Content-Signal`, no `Crawl-delay` for `*` | | `holodori.wiki` | **Built** (2026-08-18), from the rendered `/wiki/Events` page. Miraheze again, so the same call as Blue Archive; CC BY-SA 4.0, no `Content-Signal`, no `Crawl-delay` for `*` |
| `prydwen.gg`, `gametora.com` | **Cleared, unbuilt.** `User-agent: *` allows the paths we would want. prydwen sets `Crawl-delay: 10`, far below our one-per-6h | | `prydwen.gg`, `gametora.com` | **Cleared, unbuilt.** `User-agent: *` allows the paths we would want. prydwen sets `Crawl-delay: 10`, far below our one-per-6h |
@@ -288,6 +294,30 @@ broken build — `skipped_robots` does not touch the failure streak, and the run
falls back to the checked-in fixture. Refreshing it means running `bun run refresh` from an address falls back to the checked-in fixture. Refreshing it means running `bun run refresh` from an address
Fandom serves, which is how its first snapshot was taken. Fandom serves, which is how its first snapshot was taken.
**Two Fandom sources now, and the second one's page is chosen, not obvious.**
`fategrandorder.fandom.com` publishes two schedules: `Event_List` opens "This page lists all Events
in Fate/Grand Order Japan", and `Event_List_(US)` is the English server. They run months apart, each
links the other, and reading the Japanese one on an English calendar is the `akwiki` CN column again
— it was how this source first landed, and every date it published was a JP date. The adapter is
pointed at `page=Event_List_(US)` and a test asserts it; `parsers/fandom.ts` carries the reasoning.
Three more things about that page, all of them ways to publish or lose a date:
- **Its sections are fenced by pictures.** `ONGOING EVENTS`, `FUTURE EVENTS` and `PAST EVENTS` are
banner images with the label drawn in a positioned `<div>` over them — no heading, no id. Only the
ongoing section is parsed, and `canParse` asserts both of the dividers that bound it, so a
redesign fails the source rather than emptying the lane.
- **The other two sections cannot be dated, and that is the whole reason they are skipped.**
`FUTURE EVENTS` gives an ETA of `August 2026` — a month with no day, and a day is half an event
ID. `PAST EVENTS` is 111 monthly tables that state no year anywhere; the *Japanese* page's
equivalents carry it in a `MMYYYY` table id, which is a difference easily assumed away.
- **Every duration names a zone and no clock** — `August 12, 2026 ~ August 26, 2026 PDT`. So the
boundaries stay on the day the page states rather than being shifted into UTC: there is no time of
day to anchor a conversion to, and the start's day is part of the event ID. That `PDT` is also the
evidence that the English server is one machine on US Pacific — see `games.ts`, where it does
*not* become a `resetOffsets` entry, because Pacific observes daylight saving and that field holds
one fixed number.
**Blue Archive: the page, never the API — the opposite call to Fandom.** `bluearchive.wiki` is a **Blue Archive: the page, never the API — the opposite call to Fandom.** `bluearchive.wiki` is a
Miraheze wiki, and Miraheze's `robots.txt` **disallows** `/w/` and `/*?action=`. So the route Miraheze wiki, and Miraheze's `robots.txt` **disallows** `/w/` and `/*?action=`. So the route
`parsers/fandom.ts` takes is the one that is closed here, and the rendered `/wiki/Events` page is the `parsers/fandom.ts` takes is the one that is closed here, and the rendered `/wiki/Events` page is the
+1 -1
View File
@@ -10,7 +10,7 @@ import { z } from "zod";
export const GameId = z.enum([ export const GameId = z.enum([
"genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", "r1999", "genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", "r1999",
"ba", "ba", "fgo", "holodori",
]); ]);
export const EventType = z.enum([ export const EventType = z.enum([
+23 -2
View File
@@ -51,7 +51,7 @@ Consequences worth internalising:
| `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE, Infinity Nikki, Persona 5: The Phantom X | | `game8` | game8.co article calendars | Genshin, Star Rail, Wuthering Waves, ZZZ, Endfield, NTE, Infinity Nikki, Persona 5: The Phantom X |
| `wikigg` | wiki.gg MediaWiki `mp-event` templates | Endfield | | `wikigg` | wiki.gg MediaWiki `mp-event` templates | Endfield |
| `akwiki` | arknights.wiki.gg's `mrfz-wtable` "Ongoing/upcoming" table | Arknights | | `akwiki` | arknights.wiki.gg's `mrfz-wtable` "Ongoing/upcoming" table | Arknights |
| `fandom` | Fandom wikis via the MediaWiki `action=parse` API — `Event \| Time Period \| Version` wikitables | Reverse: 1999, Fate/Grand Order | | `fandom` | Fandom wikis via the MediaWiki `action=parse` API — two page templates: `Event \| Time Period \| Version` wikitables, and FGO's picture-fenced `ONGOING EVENTS` blocks | Reverse: 1999, Fate/Grand Order |
| `bawiki` | bluearchive.wiki's rendered `/wiki/Events` — a JP/Global tabber over `Name (EN) \| Start date \| End date \| Notes` wikitables | Blue Archive | | `bawiki` | bluearchive.wiki's rendered `/wiki/Events` — a JP/Global tabber over `Name (EN) \| Start date \| End date \| Notes` wikitables | Blue Archive |
| `holodoriwiki` | holodori.wiki's rendered `/wiki/Events``Current Events` and `Past Events` wikitables over `Event \| Type \| Start Date \| End Date` | hololive Dreams | | `holodoriwiki` | holodori.wiki's rendered `/wiki/Events``Current Events` and `Past Events` wikitables over `Event \| Type \| Start Date \| End Date` | hololive Dreams |
@@ -75,6 +75,26 @@ rather than parse to zero events. Two page facts drive the rest of it:
anchor on. Inclusion is therefore decided against `ctx.now`, the one parser here that does so; anchor on. Inclusion is therefore decided against `ctx.now`, the one parser here that does so;
`akwiki` and `game8` can gate on a heading instead, and should where one exists. `akwiki` and `game8` can gate on a heading instead, and should where one exists.
The second Fandom source, Fate/Grand Order, shares the envelope and nothing else, so `fandom` is one
parser over two templates the way `game8` is one over seven. `canParse` and the parse branch both
route on the same check, and the differences are worth knowing before touching either:
- **The page is a choice.** `fategrandorder.fandom.com` publishes `Event_List` (Japanese server) and
`Event_List_(US)` (English), months apart and cross-linked. The adapter reads the `(US)` one; a
test asserts the URL, because this source shipped once off the Japanese page and every date it
published was wrong by a server. AGENTS.md § Fandom has the rest.
- **Sections are fenced by pictures, not headings.** `ONGOING EVENTS`, `FUTURE EVENTS` and
`PAST EVENTS` are banner images with the label in a positioned `<div>`. Only the ongoing section
is read, and `canParse` asserts both dividers bounding it.
- **The other two sections are undatable, which is why they are skipped rather than filtered.**
Upcoming rows give a month and no day; the 111 past tables state no year at all — unlike the
Japanese page's, which carry it in a `MMYYYY` table id.
- **Durations name a zone but no clock** (`August 12, 2026 ~ August 26, 2026 PDT`), so the stated
calendar day is kept as-is: there is no time of day for a UTC conversion to anchor to, and the
start's day is half the event ID.
- **`(US)` is stripped from the title and kept in the URL.** It disambiguates the English article
from the Japanese one, so it belongs to the article's name and not to the event's.
`bawiki` is the mirror image of `fandom`: same MediaWiki software, opposite conclusion about which `bawiki` is the mirror image of `fandom`: same MediaWiki software, opposite conclusion about which
surface to read. bluearchive.wiki is Miraheze, whose `robots.txt` disallows `/w/` and `/*?action=`, surface to read. bluearchive.wiki is Miraheze, whose `robots.txt` disallows `/w/` and `/*?action=`,
so the API is closed and the rendered `/wiki/Events` page is what `*` is allowed — and it serves our so the API is closed and the rendered `/wiki/Events` page is what `*` is allowed — and it serves our
@@ -111,7 +131,7 @@ All live in `src/ingest/dates.ts`, each returning null rather than inferring any
|---|---|---| |---|---|---|
| `parseMonthDayYear` | `August 12, 2026` | Genshin detail rows | | `parseMonthDayYear` | `August 12, 2026` | Genshin detail rows |
| `parseMonthDayRange` | `August 12 - September 21, 2026` (year on the end only) | Genshin, NTE | | `parseMonthDayRange` | `August 12 - September 21, 2026` (year on the end only) | Genshin, NTE |
| `parseFullRange` | `Aug. 14, 2026 - Aug. 24, 2026` (a year each side) | Star Rail, Wuthering Waves | | `parseFullRange` | `Aug. 14, 2026 - Aug. 24, 2026` (a year each side) | Star Rail, Wuthering Waves, Fate/Grand Order |
| `parseShortSlashRange` | `08/09/26 - 08/30/26` | Endfield | | `parseShortSlashRange` | `08/09/26 - 08/30/26` | Endfield |
| `parseSlashDateTimeRange` | `2021/01/16 04:00 - 2021/01/31 03:59` | Genshin past events | | `parseSlashDateTimeRange` | `2021/01/16 04:00 - 2021/01/31 03:59` | Genshin past events |
| `parseLabelledStartEnd` | `Start: January 24, 2025 End: Permanent` | Infinity Nikki | | `parseLabelledStartEnd` | `Start: January 24, 2025 End: Permanent` | Infinity Nikki |
@@ -177,6 +197,7 @@ produced it.
| Label/value or column tables with full dates including a year | Good — an existing parser may already handle it | | Label/value or column tables with full dates including a year | Good — an existing parser may already handle it |
| Dates without a year, or no end date at all | **Unsupportable** — yields nothing rather than guessing | | Dates without a year, or no end date at all | **Unsupportable** — yields nothing rather than guessing |
| Free-form prose with no table structure | Find a different source | | Free-form prose with no table structure | Find a different source |
| One wiki, two servers' schedules | **Read which one before writing anything.** Three sources here publish both: `akwiki` in a CN column, `bawiki` in a JP tab, `fategrandorder.fandom.com` on a whole separate page that says so in its first sentence. Every one of them parses cleanly and every one of them is months wrong for our readers |
| A clean table whose newest row is months old | **Not a source, an archive.** Check the *latest* date before writing anything: `bluearchive.fandom.com` parses perfectly and yields zero live events, which shows up as an empty lane and a permanently rejected snapshot rather than as an error | | A clean table whose newest row is months old | **Not a source, an archive.** Check the *latest* date before writing anything: `bluearchive.fandom.com` parses perfectly and yields zero live events, which shows up as an empty lane and a permanently rejected snapshot rather than as an error |
Game8 uses at least seven page templates, a game's page may use any of them, and one page may mix Game8 uses at least seven page templates, a game's page may use any of them, and one page may mix