diff --git a/AGENTS.md b/AGENTS.md index dc04508..bae0842 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,7 @@ between repairing a rule and rediscovering it the expensive way. | `docs/INGESTION.md` | The six pipeline stages, the parser/adapter/merge layering, date formats, the review gate | Adding a source, writing or repairing a parser, or changing the fetch runner | | `docs/ARCHITECTURE.md` | Process shape, file layout, request paths, offline and update mechanics | Moving files, adding a route, or changing the service worker | | `docs/FEEDBACK.md` | What readers actually said about the first release, and the work it argues for | Deciding what to build next | +| `docs/SOURCES.md` | Which sites publish a usable schedule for the games we still do not cover, and what is wrong with the ones that do not | Picking the next game to add, or assessing a source request | Two rules that follow from that: diff --git a/docs/SOURCES.md b/docs/SOURCES.md new file mode 100644 index 0000000..044229d --- /dev/null +++ b/docs/SOURCES.md @@ -0,0 +1,425 @@ +# SOURCES.md — event sources for the games still missing + +A source assessment for the games named in `docs/FEEDBACK.md` § P1 that still have no adapter. +Written 2026-08-18. Nothing here has been built: no fixture was captured, no `SOURCES` entry added, +no parser written. This is the reconnaissance that decides what is worth one pass of +**add-game-source** and what is worth writing off. + +Verdicts here are proposals. Once one is acted on — built or declined — the *decision* belongs in +`AGENTS.md` § Scraping conduct, whose table exists so a source is not re-litigated every pass and +which `.github/ISSUE_TEMPLATE/feature_request.yml` points readers at. This file is the working, not +the ruling. + +## Method + +Every page below was fetched from this machine with the refresh runner's own `User-Agent` +(`gacha-event-tracker/1.0 (+https://github.com/StereotypicalCat/gacha-event-tracker)`), one or two +requests per host, `robots.txt` first. No browser-shaped headers, no proxy, no JS execution — +anything that only answers a browser is treated as closed, per the `uma.moe` precedent. Where an +existing parser could plausibly read a page, it was run against the fetched bytes offline +(`src/ingest/parsers/index.ts`) rather than guessed at. + +Two things this method cannot tell you, and both matter: + +- **Whether a host answers the GitHub Actions runner.** game8.co does not (`AGENTS.md` § Scraping + conduct). The only hard evidence we hold is what CI has actually committed: `git log` on + `snapshots/` shows `github-actions[bot]` landing **arknights.wiki.gg**, **endfield.wiki.gg** and + **bluearchive.wiki** (Miraheze). Fandom sources have never refreshed in CI, because `robots.txt` + itself is challenged from a datacentre address and the gate fails closed. +- **Whether a page will still look like this in six weeks.** Each recommendation below names the + assertion `canParse` should make, so a redesign fails the source loudly instead of emptying a lane. + +## Where P1 stands + +| Game | Status | +|---|---| +| Arknights, Reverse: 1999, Blue Archive, Persona 5X | **Built** | +| Azur Lane, Umamusume | Declined on conduct (`azurlane.koumakan.jp` `ai-input=no`; `uma.moe` Turnstile) | +| **Girls' Frontline 2, Nikke, Chaos Zero Nightmare, Stella Sora, Punishing: Gray Raven, Guardian Tales, Aether Gazer** | **No source — assessed below** | +| Silver Palace | Unreleased | + +## Summary of findings + +| Game | Best source found | Parser | Refreshes in CI? | Verdict | +|---|---|---|---|---| +| Girls' Frontline 2 | `iopwiki.com/wiki/GFL2_Events` | new (`iopwiki`) | likely (permissive robots, unproven) | **Build first** — best data quality of anything here | +| Stella Sora | `stellasora.miraheze.org/wiki/Main_Page` | new (or extend `bawiki`) | yes (Miraheze proven) | **Build** — banners only, and read the main page, not the banner list | +| Nikke | `nikke-…-international.fandom.com/api.php?…page=Event` | new shape | **no** — Fandom robots fails closed | **Build**, eyes open: fixture-backed in CI like r1999 | +| Chaos Zero Nightmare | `game8.co/games/Chaos-Zero-Nightmare/archives/559899` | **existing** `game8` | **no** — game8 202s the runner | **Build** if you want a cheap win; it is a fixture-only lane | +| Umamusume | `game8.co/games/Umamusume-Pretty-Derby/archives/536311` | `game8` + two header words | **no** | Possible, but it edits a parser eight sources share | +| Punishing: Gray Raven | `grayravens.com/wiki/Events` | — | — | **Decline for now** — one dated string on the whole page | +| Guardian Tales | `guardian-tales.fandom.com/wiki/Events` | — | — | **Decline** — wiki stopped dating events in 2025 | +| Azur Lane | none | — | — | **Still declined** — the Fandom alternatives are 2021 archives | +| Aether Gazer | — | — | — | **Do not build. The game is shutting down.** | +| Silver Palace | — | — | — | Unreleased; beta only | + +--- + +## 1. Girls' Frontline 2: Exilium — build this one first + +**Source:** `https://iopwiki.com/wiki/GFL2_Events` (IOP Wiki, the Girls' Frontline universe wiki) + +**Conduct.** `robots.txt` is two lines — `User-agent: *` / `Crawl-Delay: 20` — with no `Disallow` +anywhere and no named-agent block. No `Content-Signal` header. Our 6-hour floor is three orders of +magnitude inside a 20s crawl delay. The response carries `Last-Modified`, so `If-Modified-Since` +will do real work here. + +**Shape.** An `