lucasw89/gacha-event-tracker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: record what evaluating the existing sources turned up

Browse Source 
Every one of the fourteen sources was fetched and parsed. Twelve are healthy.
The two findings that are not about a game we just added:

Fandom's posture tightened on 2026-08-19 and now covers every wiki. Yesterday
blhx.fandom.com served robots.txt a 200, which is what the r1999 permission
argument leaned on; today every Fandom wiki tried answers 403 to our fetcher
and gives a real browser a challenge that never resolves. So r1999 and fgo are
permanently fixture-backed, and no new Fandom source can be added at all —
which is what blocks Nikke, whose data is the best of anything unbuilt. It is
blocked rather than declined: one person on an address Fandom serves can read
that file and unblock it.

Infinity Nikki's source is stale and there is nowhere to move it. The Game8
page was last updated 31 August 2025, mentions 2026 zero times, and publishes
five events with endsAt: null that read as live indefinitely — worse than an
empty lane, through a source that looks healthy to the runner because a stale
page is not a broken one. Every replacement was checked and none works. That
one is a decision for the owner, not a fix: retiring the lane drops a GameId
that prefixes every completion key its readers hold.

SOURCES.md carries each game's outcome and the reasoning behind the declines,
so the next pass does not re-derive any of it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-19 03:54:05 +02:00
co-authored by Claude Opus 5
parent 83b6522952
commit c9e9359808
3 changed files with 145 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+37
View File
@@ -301,6 +301,30 @@ re-litigated each pass:
| `stellasora.miraheze.org` | **Built** (2026-08-19), from the front page's `Current Banners` module and **not** `/wiki/Banner_List` — see § Stella Sora below |
| `game8.co/games/Chaos-Zero-Nightmare` | **Built** (2026-08-19). Zero parser work — the existing `game8` parser reads it. The ninth game8 source, so fixture-backed in CI from day one |
| `game8.co/games/Umamusume-Pretty-Derby` | **Built** (2026-08-19), off the stable `List of All Banners` page, not the monthly release-schedule pages whose URL changes every month. Cost a widening of `game8.ts`'s section and column vocabulary — see § Working on parsers |
| `nikke-…-international.fandom.com` | **Blocked, not declined** (2026-08-19). The API answers our User-Agent with a 200 and the data is the best of any unbuilt source — story events *and* dated pickup banners, with an evidenced 05:00 UTC+9 reset. But this wiki's `robots.txt` cannot be read from here **at all**: 403 to our fetcher and an unresolving Cloudflare challenge to a real browser. The permission has to be in writing before the adapter exists, so this waits on someone reading that file from an address Fandom serves |
| `infinitynikki.miraheze.org` | **Declined.** Exists and serves `robots.txt`, but the wiki is abandoned — front page last edited 11 February 2025 and `/wiki/Events` returns a permission error. Checked as a replacement for the stale Infinity Nikki Game8 page |
| `prydwen.gg/infinity-nikki` | **Declined.** 404 — prydwen does not cover Infinity Nikki |
| `grayravens.com` (Punishing: Gray Raven) | **Declined.** Conduct is fine; the data is not. The whole 626 KB `/wiki/Events` page contains exactly one date range, written as prose, one event per six-week patch |
| `guardian-tales.fandom.com` | **Declined.** Parses fine and contains no 2026 date at all — newest dated entry is 2025. The `bluearchive.fandom.com` failure again: parses cleanly to nothing live |
| `blhx.fandom.com`, `azurlane-archive.fandom.com` | **Declined.** The two Fandom alternatives to the declined koumakan wiki are dead archives — `Event_Calendar` stops in **2021**, and the archive wiki's headings have nothing under them. Azur Lane still has no source |
| Aether Gazer | **Do not build.** The developer confirmed no further content updates after 23 July 2026, with store listings removed 17 October 2026. The wiki dates nothing anyway — `Event_Guide_List` is an image gallery. A lane that will be empty by winter |
**The Infinity Nikki lane is knowingly stale, and there is nowhere to move it** (checked
2026-08-19). `game8.co/games/Infinity-Nikki/archives/487445` fetches, parses and passes every test —
and its page says `Last updated on: August 31, 2025`. It mentions the year 2026 zero times. Seven
events parse out of it, of which **five carry `endsAt: null`** and so read as live-with-unknown-end
forever, on a calendar whose whole purpose is telling a reader what is still on. That is worse than
an empty lane, and it is the failure this product exists to prevent — arriving through a source that
looks perfectly healthy to the runner, because a stale page is not a broken one.
Every replacement was checked and none works: `infinitynikki.wiki.gg` 401s (no such wiki),
`prydwen.gg` does not cover the game, `infinitynikki.miraheze.org` is abandoned (front page last
edited February 2025, `/wiki/Events` a permission error), Game8's own `Banners` page for the game is
staler still (May 2025) and its whole Infinity Nikki wiki is unmaintained, and the Fandom wiki is
behind the blanket 403 above. **This is a decision for the repository owner, not a bug to fix in
code**: either the lane stays and the footer's freshness disclosure carries it, or the game is
retired. Retiring it is not free — `nikki` is the first segment of every completion key that game's
readers have, so dropping the `GameId` orphans them with no server-side recovery (§ Event IDs).
`.github/ISSUE_TEMPLATE/feature_request.yml` points readers at that table by heading, so a source
request can be checked against it before anyone writes it up — the loudest feedback on the first
@@ -327,6 +351,19 @@ own headers, on a path the site put in writing. The only namespaces `*` is refus
`User:`, `Template:` and `Help:`, none of which we want; `parsers/fandom.ts` skips `Special:` links
for that reason.
**Fandom's posture tightened on 2026-08-19, and it now covers every wiki.** On 2026-08-18 the
standard `robots.txt` was still readable from a plain address — `blhx.fandom.com` served it `200`,
which is how the permission above was confirmed. As of 2026-08-19 **every** Fandom wiki tried
(`reverse1999`, `fategrandorder`, `nikke-…-international`, `infinitynikki`, `blhx`) answers `403`
to our fetcher, and a real headless browser gets a Cloudflare managed challenge that never resolves.
Two consequences, and neither is a licence to work around it:
- The two built Fandom sources now report `skipped_robots` on **every** run, from any address we
have, so `r1999` and `fgo` are permanently fixture-backed until someone refreshes them from an
address Fandom serves. `fgo` has never had a snapshot at all.
- A **new** Fandom source cannot be added at all, because the permission cannot be read. That is
what blocks Nikke, whose data is otherwise the best of any unbuilt source.
One consequence to keep in mind: because `/robots.txt` is unreadable from a challenged address, the
robots gate **fails closed there and the source is skipped**. That is a warning line rather than a
broken build — `skipped_robots` does not touch the failure streak, and the run only hard-fails if
docs/FEEDBACK.md
+2 -1
View File
@@ -305,8 +305,9 @@ diagnosis in each item still holds — what changed is whether it has been acted
| P1b cap the long list | **Done** (2026-08-18). Both sections cap at six with "show all N" — truncation of the view, not a re-sort |
| P1b persist `view` | **Done** (2026-08-18). `prefs.view`, and the first run now asks which one to open on (PRD F8) |
| P1b Calendar → Timeline | **Done.** The tab reads "Timeline" |
| P1c more games | **Done, four of them.** Infinity Nikki, Persona 5: The Phantom X, Reverse: 1999, Blue Archive — one commit each. Of the games named in the thread, Azur Lane and Umamusume are **declined on conduct** rather than pending, and the declined and cleared-but-unbuilt candidates are recorded in `AGENTS.md` § Scraping conduct so they are not re-litigated |
| P1c more games | **Done, nine of them.** Infinity Nikki, Persona 5: The Phantom X, Reverse: 1999, Blue Archive, then Fate/Grand Order and hololive Dreams, then Girls' Frontline 2, Stella Sora, Chaos Zero Nightmare and Umamusume (2026-08-19). Of the games named in this thread, **Umamusume is now built** — via Game8, not the declined `uma.moe` — and Azur Lane and Aether Gazer stay declined. **Nikke is blocked, not declined**: its data is the best of anything unbuilt and its wiki's `robots.txt` can no longer be read from any client here, so the permission cannot be established. See `docs/SOURCES.md` § 3 |
| P1c roadmap in the colophon | **Not done** |
| P1 games still unserved | Nikke (blocked on a readable `robots.txt`), Azur Lane, Punishing: Gray Raven, Guardian Tales, Aether Gazer (shutting down), Silver Palace (unreleased). Separately, **Infinity Nikki's source is stale** and has no reachable replacement — `docs/SOURCES.md` § 11 |
| P2 colour and screenshot | **Layout done** (2026-08-18), colour untouched. The palette was left alone deliberately — the two-axis system was not what made the screenshot read as unfinished, a dense single column on a wide screen was. Past `lg` the page is now a pinned deadline rail beside the lists, the timeline is a board with its axis and names pinned, and the footer and settings are columns. The screenshot still wants reshooting |
| P2 custom events | **Done.** PRD F13 first, then the code; `mygame:` / `myevent:` key spaces, in the export |
docs/SOURCES.md
+106 -30
View File
@@ -1,9 +1,18 @@
# SOURCES.md — event sources for the games still missing
A source assessment for the games named in `docs/FEEDBACK.md` § P1 that still have no adapter.
Written 2026-08-18. Nothing here has been built: no fixture was captured, no `SOURCES` entry added,
no parser written. This is the reconnaissance that decides what is worth one pass of
**add-game-source** and what is worth writing off.
Written 2026-08-18. **Acted on 2026-08-19**: four of the games below were built, one turned out to be
blocked rather than merely awkward, and the rest were declined. Each section keeps its original
reconnaissance and carries its outcome at the top, because the reasoning is what stops the next pass
re-deriving it.
What changed on 2026-08-19, in one paragraph. **Built:** Girls' Frontline 2 (`iopwiki`, new parser),
Stella Sora (`stellasorawiki`, new parser), Chaos Zero Nightmare (`game8`, no parser work) and
Umamusume (`game8`, after widening its section and column vocabulary). **Blocked:** Nikke — its
`robots.txt` cannot be read from here at all any more, so the permission this file required before
building could not be obtained. **Declined:** Punishing: Gray Raven, Guardian Tales, Azur Lane and
Aether Gazer, all as proposed. One thing not on the original list also came out of the pass: the
**Infinity Nikki source is stale** and has no reachable replacement — see § 11.
Verdicts here are proposals. Once one is acted on — built or declined — the *decision* belongs in
`AGENTS.md` § Scraping conduct, whose table exists so a source is not re-litigated every pass and
@@ -25,7 +34,11 @@ Two things this method cannot tell you, and both matter:
conduct). The only hard evidence we hold is what CI has actually committed: `git log` on
`snapshots/` shows `github-actions[bot]` landing **arknights.wiki.gg**, **endfield.wiki.gg** and
**bluearchive.wiki** (Miraheze). Fandom sources have never refreshed in CI, because `robots.txt`
itself is challenged from a datacentre address and the gate fails closed.
itself is challenged from a datacentre address and the gate fails closed — and as of 2026-08-19
that challenge covers every Fandom wiki from *any* address tried here, not just CI's.
Of the 2026-08-19 additions, **Stella Sora** is Miraheze and so is the one likely to refresh in CI;
**IOP Wiki** is permissive but unproven; **Chaos Zero Nightmare** and **Umamusume** are game8 and
therefore blind in CI by construction.
- **Whether a page will still look like this in six weeks.** Each recommendation below names the
assertion `canParse` should make, so a redesign fails the source loudly instead of emptying a lane.
@@ -33,20 +46,22 @@ Two things this method cannot tell you, and both matter:
| Game | Status |
|---|---|
| Arknights, Reverse: 1999, Blue Archive, Persona 5X | **Built** |
| Azur Lane, Umamusume | Declined on conduct (`azurlane.koumakan.jp` `ai-input=no`; `uma.moe` Turnstile) |
| **Girls' Frontline 2, Nikke, Chaos Zero Nightmare, Stella Sora, Punishing: Gray Raven, Guardian Tales, Aether Gazer** | **No source — assessed below** |
| Arknights, Reverse: 1999, Blue Archive, Persona 5X | **Built** (2026-08-17/18) |
| **Girls' Frontline 2, Stella Sora, Chaos Zero Nightmare, Umamusume** | **Built** (2026-08-19) |
| Nikke | **Blocked.** Best unbuilt data here; `robots.txt` unreadable, so the permission cannot be established — see § 3 |
| Azur Lane | Declined on conduct (`azurlane.koumakan.jp` `ai-input=no`), and both Fandom alternatives are dead archives |
| Punishing: Gray Raven, Guardian Tales, Aether Gazer | Declined — see §§ 6, 7, 9 |
| Silver Palace | Unreleased |
## Summary of findings
| Game | Best source found | Parser | Refreshes in CI? | Verdict |
|---|---|---|---|---|
| Girls' Frontline 2 | `iopwiki.com/wiki/GFL2_Events` | new (`iopwiki`) | likely (permissive robots, unproven) | **Build first** — best data quality of anything here |
| Stella Sora | `stellasora.miraheze.org/wiki/Main_Page` | new (or extend `bawiki`) | yes (Miraheze proven) | **Build** — banners only, and read the main page, not the banner list |
| Nikke | `nikke-…-international.fandom.com/api.php?…page=Event` | new shape | **no** — Fandom robots fails closed | **Build**, eyes open: fixture-backed in CI like r1999 |
| Chaos Zero Nightmare | `game8.co/games/Chaos-Zero-Nightmare/archives/559899` | **existing** `game8` | **no** — game8 202s the runner | **Build** if you want a cheap win; it is a fixture-only lane |
| Umamusume | `game8.co/games/Umamusume-Pretty-Derby/archives/536311` | `game8` + two header words | **no** | Possible, but it edits a parser eight sources share |
| Girls' Frontline 2 | `iopwiki.com/wiki/GFL2_Events` | new (`iopwiki`) | likely (permissive robots, unproven) | **BUILT** 2026-08-19 — best data quality of anything here |
| Stella Sora | `stellasora.miraheze.org/wiki/Main_Page` | new (`stellasorawiki`) | yes (Miraheze proven) | **BUILT** 2026-08-19 — banners only, from the main page |
| Nikke | `nikke-…-international.fandom.com/api.php?…page=Event` | new shape | **no** — Fandom robots fails closed | **BLOCKED** — the robots.txt precondition could not be met from any client here |
| Chaos Zero Nightmare | `game8.co/games/Chaos-Zero-Nightmare/archives/559899` | **existing** `game8` | **no** — game8 202s the runner | **BUILT** 2026-08-19 — no parser work, fixture-only lane |
| Umamusume | `game8.co/games/Umamusume-Pretty-Derby/archives/536311` | `game8` + section and column vocabulary | **no** | **BUILT** 2026-08-19 — cost more than "two header words"; see § 5 |
| Punishing: Gray Raven | `grayravens.com/wiki/Events` | — | — | **Decline for now** — one dated string on the whole page |
| Guardian Tales | `guardian-tales.fandom.com/wiki/Events` | — | — | **Decline** — wiki stopped dating events in 2025 |
| Azur Lane | none | — | — | **Still declined** — the Fandom alternatives are 2021 archives |
@@ -55,7 +70,7 @@ Two things this method cannot tell you, and both matter:
---
## 1. Girls' Frontline 2: Exilium — build this one first
## 1. Girls' Frontline 2: Exilium — BUILT 2026-08-19
**Source:** `https://iopwiki.com/wiki/GFL2_Events` (IOP Wiki, the Girls' Frontline universe wiki)
@@ -114,7 +129,7 @@ zero unparsed, which is the count check `AGENTS.md` § Silent drops asks for. La
---
## 2. Stella Sora — build the main page, not the banner list
## 2. Stella Sora — BUILT 2026-08-19, from the main page and not the banner list
**Source:** `https://stellasora.miraheze.org/` — a Miraheze wiki, so the same call as Blue Archive and
hololive Dreams: `/wiki/` is the surface `*` is allowed, `/w/` and `/*?action=` are disallowed. CC
@@ -150,7 +165,21 @@ source immediately — full coverage, no template dependency.
---
## 3. Nikke — the best data, on the surface CI cannot reach
## 3. Nikke — blocked on a permission that can no longer be read
> **Outcome (2026-08-19): not built, and not declined either.** The precondition this section set —
> "read this wiki's `robots.txt` in a browser and confirm it is the same file" — could not be met.
> From this machine that file answers **403** to our fetcher, and a real headless browser gets a
> Cloudflare managed challenge that never resolves. Worse, the same is now true of *every* Fandom
> wiki tried, including `blhx.fandom.com`, which served it `200` on 2026-08-18 and is what this
> section's permission argument was leaning on. Nothing about the data changed — it is still the
> richest schedule of anything unbuilt — but the permission has to be in writing before the adapter
> exists, and it cannot currently be obtained. **Unblocking it is one action by someone on an
> address Fandom serves:** open that URL, confirm the standard file (allows `/api.php?action=`,
> disallows only `Special:`/`User:`/`Template:`/`Help:`), and paste it into this file. Everything
> below then applies unchanged, including the free `resetOffsets` win.
**Source:** `https://nikke-goddess-of-victory-international.fandom.com/api.php?action=parse&page=Event&prop=text&formatversion=2&format=json`
@@ -205,7 +234,7 @@ a moment's care in `games.ts`, `schema.ts` and the fixture directory name.
---
## 4. Chaos Zero Nightmare — the cheapest adapter available, and a ninth blind source
## 4. Chaos Zero Nightmare — BUILT 2026-08-19; the cheapest adapter available, and a ninth blind source
**Source:** `https://game8.co/games/Chaos-Zero-Nightmare/archives/559899` ("List of All Events")
@@ -246,7 +275,28 @@ is stale (its "latest events" are April–May 2026); `czn.gg` is a WordPress sit
---
## 5. Umamusume — reachable, but it edits a parser eight sources share
## 5. Umamusume — built, and the estimate below was wrong
> **Outcome (2026-08-19): BUILT**, off the stable `List of All Banners` page as recommended. Two
> corrections to what follows, both worth having before the next Game8 page is assessed:
>
> - **"A few characters" was wrong.** Teaching `COL_TITLE` the word `Banner` and `COL_RANGE` the word
> `Availability` gets you **zero events**. The page's schedule sits under `List of All Banners` →
> `All Current Banners`, which the *section* vocabulary did not match either, and its back
> catalogue sits under `Previous Banners`, which `previous events` does not match — so without
> that exclusion the finished rows would have gone straight onto the calendar. Beyond that, the
> current-banners table lays the Standard and Paid schedules side by side inside one `<table>`
> under a spanning label row, and that row is *plausible* enough to resolve both columns at
> indices no data row has. `readColumnTable` now decides the header by what it produces, trying
> row 1 only when row 0 produced nothing.
> - **The blast radius was measurable, and it was zero.** The worry below — that widening a parser
> eight sources share starts matching tables it currently ignores — is right in principle and was
> settled by measurement rather than by regenerating `.expected.json` until the tests agreed: every
> pinned fixture *and* every live snapshot was parsed before and after, and no existing source's
> output changed by a single event. Do that diff on any future change to `game8.ts`; it costs one
> script and it is the only thing that distinguishes a safe widening from a silent one.
`uma.moe` stays declined; nothing about the Turnstile gate has changed. Two other surfaces exist.
@@ -365,6 +415,32 @@ projections point at late 2026 / early 2027. Nothing to scrape, as `docs/FEEDBAC
---
## 11. Infinity Nikki — not on the original list, and the most urgent thing found
This game already has an adapter, which is why it was not assessed on 2026-08-18. It should have
been. `game8.co/games/Infinity-Nikki/archives/487445` fetches, parses, and passes every test — and
the page itself says `Last updated on: August 31, 2025`, with **zero** occurrences of the string
`2026`. Seven events parse out of it, five of them with `endsAt: null`, which the app renders as
live-with-unknown-end indefinitely. A year-old event presented as current is precisely the failure
this product exists to prevent, and it arrives through a source that looks perfectly healthy to the
runner: a stale page is not a broken one, so no failure streak, no annotation, no `broken` tier.
Every replacement was checked on 2026-08-19 and none works:
| Candidate | Result |
|---|---|
| `infinitynikki.wiki.gg` | 401 — no such wiki (as with Nikke, Umamusume, Blue Archive, R1999) |
| `infinitynikki.miraheze.org` | Exists, serves robots.txt, **abandoned** — front page last edited 11 Feb 2025, `/wiki/Events` returns a permission error |
| `prydwen.gg/infinity-nikki` | 404 — not a game prydwen covers |
| `game8.co/…/archives/487444` (Banners) | Staler still: last updated 20 May 2025 |
| `infinitynikki.fandom.com` | Behind the blanket Fandom 403 (§ 3) |
So there is nothing to move the lane to. **This is a call for the repository owner**, and both
options cost something: leaving it means a knowingly stale lane carried only by the footer's
freshness disclosure, and retiring it means dropping a `GameId` that is the first segment of every
completion key that game's readers hold, with no server-side recovery. Recorded in `AGENTS.md`
§ Scraping conduct so the next pass does not rediscover it.
## What every one of these costs, beyond the source
Adding a game is never only a `SOURCES` entry:
@@ -381,19 +457,19 @@ Adding a game is never only a `SOURCES` entry:
## Recommended order
1. **Girls' Frontline 2** — best dates in the list (explicit UTC, both boundaries exact), permissive
host, live event today. Costs a parser module; buys the strongest lane since wiki.gg.
2. **Stella Sora** — Miraheze, which is the one family we know refreshes in CI. Banners only, and
read the main page's `<time>` elements rather than the unzoned banner list.
3. **Nikke** — richest schedule of the three (story events *and* dated pickup banners, plus an
evidenced reset clock), but confirm the `robots.txt` allowance in a browser first, and accept that
CI will never refresh it.
4. **Chaos Zero Nightmare** — zero parser work, but a ninth blind game8 source. Do it when someone is
willing to run the refresh by hand, or not at all.
5. **Umamusume** — after a separate, carefully re-verified commit widening `game8.ts`'s column
vocabulary.
6. Decline **Azur Lane**, **Punishing: Gray Raven**, **Guardian Tales** and **Aether Gazer**, with the
reasons above written into `AGENTS.md` § Scraping conduct so they are not researched a third time.
Steps 1, 2, 4 and 5 were done on 2026-08-19, and step 6's declines are now written into `AGENTS.md`
§ Scraping conduct. What is left is not a queue of sources any more — it is two decisions:
1. **Nikke** (§ 3) — the richest schedule of anything unbuilt, needing one action from someone on an
address Fandom serves: read that wiki's `robots.txt` and record it here. Everything else about
the adapter is specified above, including the evidenced `resetOffsets`/`resetHourLocal` pair that
must ship in the same commit as the game.
2. **Infinity Nikki** (§ 11) — decide whether a lane with no reachable source stays or is retired.
Not a research task; there is nothing left to find.
Below that, in descending order of value: re-check `grayravens.com` if it ever puts its schedule in
a table, and re-check Azur Lane if `koumakan` ever drops `ai-input=no`. Neither is worth a pass
until something changes upstream.
## Appendix — reproducing the checks