feat: add Umamusume, from Game8's banner list

uma.moe stays declined — its API sits behind a Cloudflare Turnstile proof
header, and an adapter would mean defeating a deliberate access control. This
is the surface that is simply open, and it needed the parser change in the
commit before this one to be readable at all.

The stable URL matters: Game8 also publishes monthly "August 2026 Release
Schedule" pages whose id changes every month, which a static registry cannot
follow. "List of All Banners" does not move.

Four current banners, both columns of the side-by-side table, and none of the
Previous Banners below them. The tenth game8 source, so it carries the same CI
blindness as the other nine and only a manual refresh moves it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-19 03:52:02 +02:00
co-authored by Claude Opus 5
parent 220e73ff7a
commit 04c69fd3df
10 changed files with 1928 additions and 15 deletions
+11 -10
View File
@@ -37,8 +37,8 @@ A web app that aggregates live and upcoming events across popular gacha games, p
calendar, sorts them by end date or by what the reader is partway through, tracks day-by-day
progress on events that repeat daily, and lets a user mark events completed.
**Status: working app, refreshing itself on a schedule.** Schema, eight parsers, seventeen sources across
sixteen games, the full interface, offline support, a static server, a Docker image and CI all exist and
**Status: working app, refreshing itself on a schedule.** Schema, eight parsers, eighteen sources across
seventeen games, the full interface, offline support, a static server, a Docker image and CI all exist and
are tested. The refresh runner (`bun run refresh`) fetches, caches raw snapshots and rebuilds the
feed; `.github/workflows/refresh.yml` runs it twice a day and commits only when a page actually
changed. The SQLite layer and the review queue are still specified in `docs/` but not built, so the
@@ -119,7 +119,7 @@ src/client/ React app, service worker, manifest
theme.ts — dark or light, and what a game hue reads as on each
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
serve.ts static server + /api/health
test/ 674 tests
test/ 683 tests
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
snapshots/ current page per source, rewritten by refresh — see its README
```
@@ -164,7 +164,7 @@ These come from how gacha games actually schedule things, and they cause most bu
- **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing
year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a
recoverable disappointment; a confidently wrong date is the failure this product exists to prevent.
- **Parsers are keyed by site, not game.** One `game8` parser serves nine sources and `fandom` two;
- **Parsers are keyed by site, not game.** One `game8` parser serves ten sources and `fandom` two;
`wikigg`, `akwiki`, `bawiki`, `holodoriwiki`, `iopwiki` and `stellasorawiki` serve one each — the first two share a host family
and have entirely different templates, and the last two are both Miraheze wikis whose page
templates have nothing in common. Adding a source for a known site is one `SOURCES` entry; a new
@@ -256,9 +256,9 @@ Sources are community wikis. Treat them as a guest would:
- Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
- One request per source per refresh cycle, minimum 6 hours apart.
- **Space requests to one host**, honouring its `Crawl-delay` and defaulting to 2s. Nine of the
seventeen sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
as nine back-to-back requests to a single site — which is the shape an edge network throttles, and
- **Space requests to one host**, honouring its `Crawl-delay` and defaulting to 2s. Ten of the
eighteen sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
as ten back-to-back requests to a single site — which is the shape an edge network throttles, and
what a burst looks like from the far end regardless of our intent.
- Send `If-None-Match` / `If-Modified-Since`; treat `304` as "skip, unchanged".
- Cache raw snapshots so re-parsing never re-fetches. **Iterate against fixtures, not the network.**
@@ -270,9 +270,9 @@ training, and no `User-agent: *` rule applies to our paths. Keep it that way: do
rate, and do not add an LLM that consumes page content.
**game8.co does not answer a GitHub Actions runner** (confirmed 2026-08-17). Its edge returns
`202 Accepted` with a bot-management body to every one of the nine game8 sources, from the first
`202 Accepted` with a bot-management body to every one of the ten game8 sources, from the first
scheduled cycle onward — `last confirmed: never` — while the same URLs return `200` and parse
cleanly from a normal address. So `robots.txt` permits us and the network does not, and those nine
cleanly from a normal address. So `robots.txt` permits us and the network does not, and those ten
games have only ever been built from checked-in fixtures in CI.
The per-host spacing above does not fix this and was not meant to: a 202 on the very first request
@@ -300,6 +300,7 @@ re-litigated each pass:
| `iopwiki.com` | **Built** (2026-08-19), Girls' Frontline 2 — see § IOP Wiki below. `robots.txt` is two lines, `User-agent: *` and `Crawl-Delay: 20`, no `Disallow` anywhere |
| `stellasora.miraheze.org` | **Built** (2026-08-19), from the front page's `Current Banners` module and **not** `/wiki/Banner_List` — see § Stella Sora below |
| `game8.co/games/Chaos-Zero-Nightmare` | **Built** (2026-08-19). Zero parser work — the existing `game8` parser reads it. The ninth game8 source, so fixture-backed in CI from day one |
| `game8.co/games/Umamusume-Pretty-Derby` | **Built** (2026-08-19), off the stable `List of All Banners` page, not the monthly release-schedule pages whose URL changes every month. Cost a widening of `game8.ts`'s section and column vocabulary — see § Working on parsers |
`.github/ISSUE_TEMPLATE/feature_request.yml` points readers at that table by heading, so a source
request can be checked against it before anyone writes it up — the loudest feedback on the first
@@ -625,6 +626,6 @@ to an open page). Four things hold it up:
(PRD F7). `freshness()` in `src/shared/feed.ts` is the one definition: it takes the newest
`lastSuccessAt` and **never `generatedAt`**, which is a build stamp that would call a
fixture-backed calendar minutes old, and it treats a game as only as fresh as its *oldest* source,
so one live wiki cannot vouch for a stalled sibling. Given that nine sources cannot be fetched
so one live wiki cannot vouch for a stalled sibling. Given that ten sources cannot be fetched
from CI at all (§ Scraping conduct), this disclosure is the only thing standing between a reader and
a confidently stale calendar — do not let a future change source it from the build clock.