Commit Graph
66 Commits
Author SHA1 Message Date
Lucas WintherandClaude Opus 5.5 57406979c2 fix(sources): a pass page between runs says so, rather than reading as broken
When a pass's last listed run ends before the wiki adds the next one, the
history table parses to zero. ZZZ's City Fund would have done that on
19 Oct and been counted as a failed source. statesNoEvents now answers
for the page: its pass table is there and dates its runs, all of them
past. Found in the whole-branch review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:35:33 +02:00
Lucas WintherandClaude Opus 5.5 1b116218e3 fix(sources): Mira Journey ends at Asia's reset after the version's last day
The Version table prints the version's last day as its End Date. Read as
an ordinary day-precision end, the event left the calendar 9 to 24 hours
before the pass did. The owner's timer on Europe put the end at about
20:00 UTC on 29 Oct: Asia's 04:00 reset opening the 30th, for every
server. The source's entry now says so, with that evidence, and the
event ends at exactly 2026-10-29T20:00Z. Found in the whole-branch
review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:35:02 +02:00
Lucas WintherandClaude Opus 5.5 b6cc3b1f51 revert: drop Genshin's Miliastra Pass source
Fandom's Event page, an existing source, already lists the pass as
"Silvery Moonlight Chronicle" with the same dates, and the merge folded
the new copy into it. The feed was unchanged, so a second source for one
event is cost with nothing to show. This reverts 9f8150f.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:28:51 +02:00
Lucas WintherandClaude Opus 5.5 ec669675ed feat(sources): Infinity Nikki's Mira Journey window, from its version
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:24:40 +02:00
Lucas WintherandClaude Opus 5.5 a6a8efa661 feat(sources): ZZZ's City Fund pass windows from Fandom
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:22:05 +02:00
Lucas WintherandClaude Opus 5.5 9f8150fda7 feat(sources): Genshin's Miliastra Pass windows from Fandom
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:20:22 +02:00
Lucas WintherandClaude Opus 5.5 0a4be4e339 feat(sources): Star Rail's Nameless Honor pass windows from Fandom
New `fandomtable` parser (src/ingest/parsers/fandomTable.ts) reads Fandom's
dated pass-history table by its Name/Start/End headings and emits runs not
yet ended by ctx.now, at day precision. Titles come from a per-source
PASS_SOURCES table so the next pass pages are one entry each.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:17:00 +02:00
Lucas WintherandClaude Opus 5.5 40d0dbef5a fix(build): pick a source's own fixture, not a sibling page's on the same site
build-feed globbed fixtures by site (`fandom-*`), so a second Fandom page
for a game, such as Genshin's Miliastra Pass beside its event page, would
have handed its fixture to the events parser. pickFixture matches the
source's whole name followed by a date. Every fixture already follows
that name, and the built feed is unchanged (206 events, same ids).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 06:10:51 +02:00
Lucas WintherandClaude Opus 5.5 16c277b403 feat(merge): pair two copies when one's summary names the other
Game8 and Fandom name Genshin's anniversary selectors with no word in common, and both run the same window, so the calendar showed each selector twice. Game8's own blurb names the Fandom event ("…limited 5-star selector, Tabletop Troupe: A Gathering on Adventure's Eve!"), which is how a person paired them, so merge now reads it the same way.

The rule is narrow on purpose: the whole title of two or more words, both boundaries within 24h, and never for a copy that already has a twin by title in the other source, since then the blurb is naming a neighbour such as a festival's shop. Measured on every two-source page held, it fires on the two selector pairs and otherwise only where titles already matched. Dates alone never pair: patch day puts about twenty Genshin events on one window. aliases.ts is the recorded-pair fallback for copies no page links; it is empty.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 17:49:26 +02:00
Lucas WintherandClaude Opus 5.5 1f6b8fd1d6 feat(merge): pair a title with a qualifier in front of it
titleExtends caught a source appending words ("Bedazzling Dawnstar Sign-In") but not prepending them, which is the shape Game8's "Battle Pass - …" label took. The same qualifier at the other end now pairs too. The two-word floor holds at both ends, because a one-word base would fuse half of a patch day.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 17:48:53 +02:00
Lucas WintherandClaude Opus 5.5 2924605bd4 feat(feed): name the ids merge absorbed, and carry marks off them
When merge fuses two sources' copies, the loser's id leaves the calendar, and it may already hold a reader's ticks from a day it was published alone. Until now that needed a hand-edited RETIRED_IDS entry per fusion.

The survivor now lists what it absorbed in an optional mergedIds field. When a feed arrives the client turns those into carry-forward pairs, skipping any id the feed still publishes, and an import takes the same table. Optional, so an older cached feed still validates and an older bundle drops it unread. RETIRED_IDS stays for parser changes, whose old ids no feed will name again.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 17:48:15 +02:00
Lucas WintherandClaude Opus 5.5 59403ca2cc fix(game8): drop the Battle Pass label Fandom does not print
Game8 heads Genshin's pass "Battle Pass - Moontrace"; Fandom lists "Moontrace" and files the category in a column. The label minted a second id for one event, so every battle pass since the Fandom source landed has been on the calendar twice, and a reader reported it.

Both sources now mint the same id and merge fuses them by id. The two Game8 ids that moved, Frostfarer and Moontrace (the only ones git history shows ever shipped), go in RETIRED_IDS so readers' marks follow.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 17:47:28 +02:00
Lucas WintherandClaude Opus 5.5 ae701de74e feat(refresh): record a lapsed source instead of failing every run on it
holodori.wiki still answers 200 and still parses, but its editors stopped
adding events after 19 Sep 2026, so every cycle parsed to zero, was rejected,
and after three cycles turned the refresh workflow red over something no code
here can fix. No fetchable replacement exists.

A registry entry can now carry `lapsed: { recorded, reason }`, written by a
person. Refresh reports such a source's empty 200s as a warning rather than
broken; any other status is still broken, and the first cycle that lists
events again warns that the record should come out. Readers see no change:
the held snapshot keeps ageing and the staleness footer still says so.

The 2026-10-14 page is kept as a fixture, pinning that the template still
matches and only the current table is empty.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-14 02:34:45 +02:00
Lucas Winther 84441b6d6d fix(snapshots): persist lastConfirmedAt into snapshot metadata upon pull confirmation
CI / Typecheck, test, feed (push) Successful in 20s
CI / Build site (push) Successful in 11s
CI / Container image (push) Successful in 41s
Record lastConfirmedAt in snapshot metadata when a source is confirmed (200 or 304). This ensures git-committed snapshot metadata reflects when sources like wiki.gg were successfully pulled even when page HTML did not change, avoiding false pull overdue notices on downstream builds like GitHub Pages.
2026-09-18 10:37:37 +02:00
Lucas Winther 157b9fe20c fix(feed): reflect content updates in source health confirmation and freshness
Ensure lastConfirmedAt never precedes contentChangedAt for fetched snapshots and calculate game-level lastConfirmedAt using effective pull times.
2026-09-18 09:30:31 +02:00
Lucas Winther 6c09accc20 feat(feed): record crawler check and content update timestamps in SourceHealth
Add lastConfirmedAt and contentChangedAt to SourceHealth and enrich freshness() to track per-source check and content update times.
2026-09-18 08:55:10 +02:00
Lucas Winther 2bc61e7d12 fix(ingest): support redesigned Blue Archive wiki event table
CI / Typecheck, test, feed (push) Successful in 12s
CI / Build site (push) Successful in 7s
CI / Container image (push) Successful in 29s
The Blue Archive Wiki (bluearchive.wiki/wiki/Events) redesigned its
schedule, replacing the per-version MediaWiki tabber with a unified
#eventtable displaying JP and Global periods side-by-side.

- Parse #eventtable resolving GL Period, event titles, and article links
  from .event-region-gl, with fallback to .event-region-jp.
- Maintain backwards compatibility with legacy tabber schedule format.
- Add pinned fixture fixtures/ba/bawiki-events-2026-09-17 and tests.
- Update snapshot, AGENTS.md, ARCHITECTURE.md, and INGESTION.md.
2026-09-17 10:53:05 +02:00
Lucas WintherandClaude Opus 5 f32eb93289 docs: record that CI, the registry and the deploy moved to Gitea
CI / Typecheck, test, feed (push) Successful in 17s
CI / Build site (push) Successful in 9s
CI / Container image (push) Failing after 18s
The pipeline change made a set of sentences false, and they are the only
record of why things are the way they are, so they move with it: the CI
section and deploy target in README, the cron and BASE_PATH notes in
ARCHITECTURE, the documented User-Agent in INGESTION, and the contact
URL in SOURCES.

Two comments in source were false in the same way and are fixed here
rather than left for the next reader. `serve.ts` said compression was
"the whole difference between the Docker image and the deployed site" —
the image *is* the deployed site now, so that negotiation is the only
compression any reader gets. `robots.ts` justified product-token
matching with "our contact URL contains the string StereotypicalCat",
which it no longer does; the hazard is the arbitrary substring, not that
one, so the comment now says so and the regression test keeps the URL
that actually caused it.

What is deliberately NOT rewritten: game8.co's refusal stays in the past
tense with the Gitea result marked unverified, in AGENTS.md, README,
SOURCES and FEEDBACK alike. A new runner address is the variable that
failure turns on, so it may well be served — but "may well be" is not
evidence, and the four places that would have to change are named
together so they change together once a cycle has actually run.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-16 04:32:51 +02:00
Lucas Winther 4382ed35ee feat: add Punishing: Gray Raven with karendar.com source
Add karendar.com as the event source for Punishing: Gray Raven (PGR).
Karendar publishes a dedicated Global event calendar with exact UTC
timestamps in clean server-rendered HTML.

- Register 'pgr' in GameId and GAMES registry (#CC292B, Kuro Games).
- Add parseWeekdayDayMonthYearUtc in src/ingest/dates.ts with unit tests.
- Implement karendarParser in src/ingest/parsers/karendar.ts reading
  'week', 'ongoing', and 'upcoming' sections, mapping indefinite/permanent
  ends to endsAt: null, and categorizing tags to EventType.
- Register pgr-karendar-events in src/ingest/adapters/index.ts.
- Add pinned fixture, expected output, and comprehensive test suites.
- Update documentation in AGENTS.md, README.md, docs/INGESTION.md,
  and docs/SOURCES.md.
2026-09-12 09:10:02 +02:00
Lucas Winther 81ac9fb3dd add Genshin Impact Fandom event source
Add genshin-impact.fandom.com as a second source for Genshin Impact,
which has been fixture-backed in CI because Game8 CloudFront-blocks the
Actions runner with 202s.

- Add fifth Fandom template parser in src/ingest/parsers/fandom.ts
  extracting titles from caption display text, stripping dated subpage
  suffixes, and checking currency via latestBoundaryMs.
- Register genshin-fandom-events in src/ingest/adapters/index.ts and
  set genshin-game8-events priority: 10 so existing event IDs survive
  near-match ties, preventing localStorage completion marks from
  orphaning.
- Explicitly sort adapter groups by priority descending in
  scripts/build-feed.ts to align with parseGame.
- Pin fixtures and add test coverage in test/adapters/game8.test.ts.
- Update AGENTS.md, README.md, docs/INGESTION.md, and docs/SOURCES.md.
2026-09-12 05:20:56 +02:00
Lucas WintherandClaude Opus 5 1156c358a7 fix: stop failing the build on a page that says it has no events
Infinity Nikki's wiki prints "There are no Events in this category"
between versions, and the refresh runner has honoured that since
2026-09-03 — it stores the empty parse as the source's real answer
rather than letting a quiet lane reach the broken tier. The feed build
never asked, so the same page arrived at CI as parsedCount 0,
indistinguishable from a parser that has stopped reading a redesigned
page, and brokenSources failed every build while every refresh stayed
green.

The runner's verdict cannot travel on its own: only a parser has seen
the page, and by the time brokenSources runs there is nothing left but
the feed. So the fact rides on SourceHealth, defaulted so an older feed
the service worker cached keeps validating and reads as the strict
answer.

Both ends now ask it the same way — of an empty parse only, from the
page's own words only — because a redesign yields zero rows too, and
excusing that is the silently emptied calendar the gate exists for.

The rule sits in a module rather than in build-feed.ts, which writes
public/ and so runs a build if a test imports it. That is how the two
ends drifted apart with nothing to catch it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-06 18:02:16 +02:00
Lucas WintherandClaude Opus 5 a73f35b7cd Identify the Infinity Nikki page by its headings, not by finding a row
The wiki emptied both its Current and Upcoming tables on 2026-09-03 — 2.7's
events had ended and 2.8 was not listed yet — and said so in words: "There
are no Events in this category". Nothing else moved; Past Events still
carries twenty tables of the same shape.

`isInfinityNikkiEventPage` looked for a *populated* table, so it read that
as a redesign. The source spent four cycles reporting "the source has
likely been redesigned" at a page nobody had touched, reached the broken
tier, and went on serving a snapshot whose every row expired on 27 August.

A check that reads data cannot tell a rewrite from a quiet week. It now
reads the section headings, which an empty table does not take with it —
either heading rather than both, since requiring the pair would fail the
source over a renamed heading it does not even read, and one already
separates this page from the other three Fandom templates. docs/INGESTION.md
asked for structural checks all along; this one had drifted into content.

`statesNoEvents` then reports the page's own declaration, so the runner can
tell an answer from a failure. It insists on the statement rather than an
empty table, because a redesign yields an empty table too.

The emptied page is pinned as a fixture: an empty answer is a shape, and it
is the one that used to read as a redesign.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-03 18:45:24 +02:00
Lucas WintherandClaude Opus 5 0809fa1122 Let a parser report that a page states it lists no events
A count of zero has two causes and nothing downstream can tell them apart:
a source whose selectors all broke, and a game that is simply between
patches. Both parse to nothing, so `canParse` plus a row count cannot
separate them — and the pipeline currently has to assume the worst, which
is right for a redesign and wrong for a quiet week.

`statesNoEvents` is the seam for the one thing that can settle it: the
page's own words. Optional, because most pages say nothing either way, and
absent means the strict gate stands.

No parser implements it yet and nothing reads it — the contract widens here
so the parser and the refresh gate can land as their own changes.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-03 18:44:51 +02:00
Lucas WintherandClaude Opus 5 c272d9e93d Publish Honkai Impact 3rd events from arustats.com
Sixteen events, from a page whose URL never needs editing: /en-us/hi3/timeline
answers 307 to the live version, so the site names its own current version
server-side and the runner follows it. That is the stable route the
marisaimpact.com page was declined for lacking, and pinning a version here
would publish a finished schedule as current the day the game moves on — a
test asserts the registered URL carries no version.

The boundaries are estimated week-bucket edges rather than announced dates,
which is the objection marisaimpact.com was declined on and is taken here
deliberately: the game had no source at all, and approximate six-week windows
were judged better than an empty lane. Confidence records it in the data.

Verified against the live page rather than only against the expected file:
all sixteen rendered grid-column spans agree with the JSON, which independently
confirms the week mapping and that endWeek is exclusive, and sixteen bars on
the page yield sixteen events with nothing silently dropped. The two 7-Day
Login events get their own test, since collapsing them into one ID is the
failure this parser is most likely to regress into.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-27 01:27:49 +02:00
Lucas WintherandClaude Opus 5 8735222adc Add a parser for arustats.com's version timeline
First parser here that reads no markup: the site is Next.js and server-renders
its whole schedule into a __NEXT_DATA__ blob, so reading that JSON is both
simpler and safer than the rendered grid, whose bars carry their geometry in
inline grid-column styles and whose titles arrive HTML-escaped.

The dates it yields are estimates, which nothing else in this project
publishes. The page schedules by week bucket — events carry startWeek/endWeek
integers, no event states a date of its own, and the grid is headed
"ESTIMATED WEEK". ESTIMATE_CONFIDENCE (0.4) is what carries that fact into the
data rather than leaving it in a comment: mergeEvents prefers the higher
number, so a source that states real dates outranks this one automatically and
nobody has to remember to retire it.

Two details that would otherwise cost data. endWeek is exclusive and runs one
past the grid to mean "to the end of the version"; an index beyond that is
skipped rather than clamped, because pinning an unreadable bar to the version's
edge would invent the boundary. And titleTop/titleMid must be joined when the
first ends in a colon — v9.0 runs two "7-Day Login:" events from week one, and
taking titleTop alone gives both the same title, the same start and therefore
the same event ID.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-27 01:27:25 +02:00
Lucas WintherandClaude Opus 5 0307d24b92 fix(ingest): keep an unannounced end out of the event blurb
The open-end vocabulary is written down twice and the two copies have to agree.
`parseOpenRange` decides whether a row is datable; `RANGE_PREFIX` strips the
range off the front of the cell so `proseAfterDates` can recover the blurb
behind it. A word the first knows and the second does not still yields the
event — with the leftover end tacked onto the summary a reader actually sees.

That is what happened to all eight Endfield rows the previous commit recovered:
`- TBA Sign-in to get extra pulls for Typhoeus!`. Adding `tba` beside `tbd`
fixes it, and a comment now says why the lists travel together, since the next
one of these will be found the same way — by reading generated fixture output.

Pins the 1.5 page beside the 1.4 one rather than replacing it. Fixtures are
permanent and these are different table shapes, not different data: 1.4 is a
single dated row, 1.5 is nine rows of open ends. The group name gains the
fixture path now that one adapter has two.

Nine rows, eight events. `Ridgeline Flows of Autumn Sign-In` reads
`Period: TBA` with no start at all, so it stays undatable and dropped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-23 01:07:21 +02:00
Lucas WintherandClaude Opus 5 843e27b4f1 fix(ingest): read an open range written in slash dates
`parseOpenRange` only ever read a month-name start, because it delegates to
`parseMonthDayYear`. So `Jul. 24, 2026 - End of 4.6` parsed and
`09/02/2026 - TBA` did not.

Game8 re-cut Endfield's upcoming table from version 1.4 to 1.5 and wrote the
whole new schedule in the second notation. All nine rows became undatable and
were dropped in silence: the source went from one event to zero, the parse gate
kept the previous snapshot, and the run reported a shape change on a page whose
shape had not changed. Nine announced events were invisible with no error
anywhere — the failure § Working on parsers calls the dangerous one.

Which notation a page uses is house style, not a statement about how certain the
source is, so both are read. The year stays mandatory in either: this is the
last reader tried, so nothing downstream would catch a guess, and Game8's
year-less summary rows (`08/12 - 08/24`) must keep returning null rather than
becoming confidently dated events.

The two-digit year pivot moves out of `parseShortSlashRange` into `pivotYear`
rather than being written a second time.

Diffed across every pinned fixture and all nineteen live snapshots: no event
moved.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-23 01:06:52 +02:00
Lucas WintherandClaude Opus 5 67bbeaf28c fix(ingest): read the Game8 block that holds the event blurbs
Every Wuthering Waves event published with no description, and two of them
did not publish at all. The page has always carried both: below the Ongoing
and Upcoming tables it repeats each live event as its own sub-section,
titled by an h4, over an `Event Duration` row and a paragraph of prose —
the shape Persona 5 uses, one heading level down. Two things kept the whole
block unreadable, and the second is why the first was never noticed.

`RANGE_LABEL` matched a bare `duration` and not `Event Duration`, so
`readLabelledDates` returned null for all ten of those tables and no
candidate was ever pushed — which is also why `summaryAfter`, whose only
job is that trailing paragraph, never ran. The label is now qualified the
way START_LABEL and END_LABEL already allow.

The other was the h4 rule. An unrecognised h4 could never name an event,
because Genshin uses h4 for sub-headings *inside* one ("Availability
Period") and letting a label claim the title renames the event to whatever
sits above its date table. But an h4 is all Wuthering Waves offers. Both
pages are served by the narrower rule: an h4 may fill an empty title, never
take one. Genshin's h3 has already claimed the slot before its labels
arrive; here the block sits under a section heading, so the slot is empty.
That only holds if the heading is recognised rather than fallen through,
since an unrecognised one is read as an event name and would spend the
title on the section — hence `events summary` in the vocabulary.

`dedupe` needed nothing. It already blends a missing summary onto the
better-dated copy for Persona 5 and never blends a date, which is what
keeps Gifts of Starpath's end on August 19 despite the block spelling it
"Auguts".

The fixture goes from ten events to twelve — `Recaptured: Action
Highlights` and `Mingshen Notices` are dated nowhere else on the page — and
nine of the ten blanks fill in. Ascendant Aces stays blank because
`Unlocked by default` is the only prose Game8 ever wrote for it, which is
what `isRequirementOnly` is for, and a test pins it there. Diffed across
every pinned fixture and all nine live Game8 snapshots: the only other
movement is Gifts of Drifting Mist gaining its blurb, and no date, no
count and no other source moved.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-20 22:38:41 +02:00
Lucas WintherandClaude Opus 5 23ad2ad797 fix: reject a year Date.UTC would silently move
`iso()` read back the month and the day to catch February 30 rolling over to
March 2, and took the year on trust. `Date.UTC` maps years 0–99 into the 1900s,
so a four-digit "0050" came back as 1950 with the month and day intact — past
both existing checks, because nothing about the resulting date is impossible.

Same class of failure as the rollover the guard was written for: a boundary the
source never stated, published as though it had. Reading the year back too is
the skip-rather-than-guess rule this module runs on, applied to the one field
that was not checked. It narrows only — `parseShortSlashRange`'s two-digit pivot
produces full years and is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-20 06:40:58 +02:00
Lucas WintherandClaude Opus 5 47967a3136 robots: tell a challenged 403 from a refusal
`--assume-robots-on-403` stands in for rules a person read in a browser, for a
host that will not serve us the file. It always claimed it would not override a
host that had turned us away — but it could not tell the two 403s apart, so it
excused both. A managed challenge ("we cannot tell what you are") is a question
our fetcher cannot answer and the operators never asked, which is what makes a
human reading the rules a fair substitute. A bare 403 is the site itself saying
no, and nothing recorded on our side may talk over that.

`isInterstitialChallenge` now decides, on Cloudflare's own `cf-mitigated` header
with the challenge page's markers as a fallback; an unreadable body counts as
unclassifiable rather than challenged. This only ever narrows what the flag
opens, so nothing that passed the gate before stops passing it.

Every 403 is classified whether or not the flag is set, and the kind goes into
the reason string the run reports. `robots.txt returned 403` read identically
whether the answer was to refresh by hand on the recorded permission or to stop
fetching the source, and the summary is where somebody has to decide that.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 23:37:44 +02:00
Lucas WintherandClaude Opus 5 f67118913a feat(refresh): --assume-robots-on-403, for a host that hides only its robots.txt
Fandom answers our fetcher 403 on /robots.txt itself, from every address we
have, while api.php?action=parse answers the same User-Agent with a 200. The
gate fails closed on the unreadable file, so r1999, fgo, nikke and nikki can
never refresh — even though their rules are not unknown: a person read them in
a browser and wrote them into AGENTS.md, verbatim, which is how three of those
four were cleared in the first place.

This flag is that recorded permission, and nothing wider. What it deliberately
does not do is most of the design:

  - 403 only. A 401, a 5xx, a timeout or a soft 404 still mean we do not know
    what the site permits, and unknown is still not permission.
  - It never overrides a robots.txt we could read. A file that answers and
    disallows us is an answer, and it still wins. So this is no use for game8.co
    — whose robots.txt reads fine and welcomes us while its edge refuses the
    pages — and it must not become one.
  - Refused under CI. It stands in for a human having read a file this morning,
    and there is no human on a runner. A scheduled job asserting a permission
    nobody re-checked is exactly how "temporary" stops being temporary.
  - Loud. Every host it applied to is warned about by name, in the run log and
    in the summary, with a line saying to go re-read the file. An override that
    reports nothing is one nobody withdraws.

Nothing else about being a guest relaxes: one request per source, six hours
apart, spaced per host, conditional headers, no retries.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 04:45:54 +02:00
Lucas WintherandClaude Opus 5 c782a265ca fix(ingest): retire a day-precision row on the clock the app reads it on
Three parsers decide currency against ctx.now, because their page has no
"ongoing" heading worth trusting: bawiki.ts, and the Fate/Grand Order and
Infinity Nikki branches of fandom.ts. All three compared Date.parse(endsAt) to
now, and all three publish day-precision ends — so the comparison was against
the 00:00Z placeholder, which § Domain rules says is not an instant and nothing
may read literally.

clockFor already knows that and resolves such a boundary to the reset opening
that game-day on the reader's server. The parsers did not, so the feed retired
a row up to nine hours before the app, the countdown and the game all agreed it
was over. On the pinned Infinity Nikki fixture, "Inspiration Burst" left the
feed at 2026-08-22T01:00Z while clockFor still called it live for an American
reader until 09:00Z, and the page said it ran to 03:59 server time. The reader
does not see a stale row; they see the deadline they were counting down to
disappear on its last day, which is the silent drop this codebase treats as the
dangerous failure.

latestBoundaryMs answers clockFor's question for the last region rather than
one reader's, so a row is history only once it is history everywhere. It follows
a game's own server map and reset hour for the same reason clockFor does —
Endfield and Reverse: 1999 both move. Nothing stored changes: this is one
comparison, not a resolved boundary written into the feed, and no expected.json
moves.

Being generous by nine hours costs an expired row at the bottom of a list.
Being strict costs a live one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 04:35:19 +02:00
Lucas WintherandClaude Opus 5 2985484883 feat: rebuild Infinity Nikki on a live source, at day precision
The Game8 page stopped being updated on 31 August 2025 and mentions 2026 zero
times, and it had been publishing five year-old events with endsAt: null — which
the app renders as live-with-unknown-end indefinitely. That is worse than an
empty lane, and nothing in the pipeline catches it, because a stale page is not
a broken one: it fetches, it parses, it passes every test.

The replacement is the game's Fandom wiki, permitted by the same standard
robots.txt that cleared Nikke, maintained to within two days, and shaped as
Current / Upcoming / Permanent / Past tables of Event | Duration | Description
| Type.

Its durations state a wall clock on both sides and no zone anywhere for that
column, so parseZonelessClockRange reads the clock and discards it, publishing
the printed date at day precision. That invents nothing and treats these cells
exactly as every Game8 date already is. Converting instead would mean picking an
offset, and the offset moves the day: July 16 20:00 read as UTC-7 is July 17,
and the start's day is half of every event ID this game will ever have.

The Game8 entry is removed rather than kept as a second opinion, since a source
whose every row is wrong is not corroboration. Its fixture stays: it is the only
page here carrying the labelled "Start: ... End: Permanent" shape, and the test
now drives it through the parser directly.

Titles come from a link's title attribute, which never passes through text(), so
they are entity-decoded by hand — otherwise "Alison&#39;s Travel Shop" becomes a
slug, and a slug is a localStorage key. The ingest sanitiser caught exactly that
during the build.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 04:23:24 +02:00
Lucas WintherandClaude Opus 5 bd385855c0 feat: add Nikke, the third Fandom template
Its robots.txt was read in a browser and is the standard Fandom file: no
Disallow: / for *, /api.php?action= explicitly allowed, and only the namespaces
we never want refused. The AI crawlers it blocks by name are not us. That was
the one thing missing — the API itself has always answered our own User-Agent
with a 200, and it is only robots.txt that a datacentre address cannot read.

The zone lives in the column header, Start(UTC+9) / End(UTC+9), and no date in
any cell carries an offset. So the header is the safety property: a table that
stops naming its zone is refused rather than read as UTC, which is the Blue
Archive hazard arriving one column to the left.

Two shapes worth knowing. Every title is an image read from <a title>, and the
newest row is the one whose logo has not been uploaded yet — it renders as a
red link reading "File:Persona on Frontline logo.png", so a reader that only
understood the link title would silently drop today's live event. And story
events state a clock only on the end, so the start keeps the day the page
printed rather than being shifted nine hours into the previous one; the start's
day is half an event ID. That is the FGO rule applied to the opposite gap.

resetOffsets and resetHourLocal ship with the game because the page evidences
both: events end 04:59:59 and the banner replacing them starts 05:00:00, one
second later, on a stated UTC+9. Adding that later would re-label day keys
readers had already logged ticks under.

Seconds now survive the date helpers, since rounding 04:59:59 to the minute
would make an event overlap the banner that succeeds it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 04:11:03 +02:00
Lucas WintherandClaude Opus 5 6b00d9a9c0 fix(merge): fuse near matches only across sources
Near matching exists to reconcile two sources describing one event under
different titles. Within one source it has no such job: the page has already
told us these are two rows, the parser has already dropped repeats of the same
id, and fusing them overrules a distinction the publisher made on purpose.

The loose 0.80 threshold that makes the rule useful across sources makes it
actively wrong within one. Umamusume's banner list runs "3 Star Guaranteed 1.5
Anniversary Scout (Character)" and "(Support)" concurrently — titles differing
by a single parenthetical, starting the same day — and fusing them dropped a
live banner off the calendar with nothing anywhere reporting it. A silent drop
is the failure this codebase ranks as the dangerous one.

Same-id fusion is untouched, so nothing duplicates.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:52:57 +02:00
Lucas WintherandClaude Opus 5 04c69fd3df feat: add Umamusume, from Game8's banner list
uma.moe stays declined — its API sits behind a Cloudflare Turnstile proof
header, and an adapter would mean defeating a deliberate access control. This
is the surface that is simply open, and it needed the parser change in the
commit before this one to be readable at all.

The stable URL matters: Game8 also publishes monthly "August 2026 Release
Schedule" pages whose id changes every month, which a static registry cannot
follow. "List of All Banners" does not move.

Four current banners, both columns of the side-by-side table, and none of the
Previous Banners below them. The tenth game8 source, so it carries the same CI
blindness as the other nine and only a manual refresh moves it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:52:02 +02:00
Lucas WintherandClaude Opus 5 220e73ff7a fix(game8): decide a column table's header by what it produces
Game8's banner pages lay two schedules side by side inside one <table> under a
spanning label row: "Standard Banners | Banner | Rating | Availability | Paid
Banners | ...". That row is not merely unhelpful, it is plausible — it contains
both column words, so it resolves and puts the range at index 3, which no
three-cell data row has. Every row then fails to date and the table yields
nothing at all, with no error anywhere. So readColumnTable now falls back to
row 1, and only when row 0 produced nothing, which is what keeps every page
that parses today parsing identically.

Those pages also schedule banners rather than events and head their sections
accordingly, so the vocabulary learns "List of All Banners", "All Current
Banners" and — the one that matters — "Previous Banners", whose rows are dated
identically to the live ones directly above them.

This parser serves nine sources, so the widening was measured rather than
trusted: every pinned fixture and every live snapshot was parsed before and
after, and no existing source's output changed by a single event.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:51:48 +02:00
Lucas WintherandClaude Opus 5 6c7369db10 feat: add Chaos Zero Nightmare, which the game8 parser already reads
No parser work at all — a GameId, a games entry, a SOURCES line, a fixture and
a test. The page lists six current events and the parser publishes four; the
two it skips print "Start Date: -", and no start means no event ID, so that is
the skip-never-guess rule working rather than a silent drop.

The cost is worth stating plainly: this is the ninth game8 source, and game8's
edge answers the Actions runner with a 202 and a bot-management body, so the
lane is built from a checked-in fixture in CI from day one and only a manual
`bun run refresh` moves it. freshness() discloses that in the footer, which is
what it is for.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:50:33 +02:00
Lucas WintherandClaude Opus 5 ecbf9c0422 feat: add Stella Sora, from the wiki's front page rather than its banner list
This wiki publishes the same schedule twice and the fuller surface is the worse
one. /wiki/Banner_List has 55 clean rows with full wall clocks and states no
timezone anywhere on the page; the front page's Current Banners module emits
the same instants as real <time datetime="...-07:00"> elements. The two agree
exactly, which is strong evidence the list is UTC and is still only evidence —
so we read the surface that says what it means and pay four live banners
instead of a full history for it. If an editor ever states the zone on
Banner_List, that page becomes the better source immediately.

Two ways this could have failed silently. The template writes its BEM
underscores as &#95;&#95;, so a selector written against the class name a
browser shows matches nothing at all; and banner names are red links to
?action=edit&redlink=1, which Miraheze's robots.txt disallows, so a href
carrying a query is refused and the page URL stands in.

No resetOffsets, and here the source did state an offset: -07:00, which is US
Pacific and therefore shifts twice a year. That is the Fate/Grand Order gap
arriving through a page that looks like it answered the question.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:50:20 +02:00
Lucas WintherandClaude Opus 5 4dff14087f feat: add Girls' Frontline 2, from IOP Wiki
The best date material this project has after wiki.gg: every row states an
exact instant on both boundaries and names the zone, so nothing is converted
and nothing is assumed. `parseIsoClockRangeUtc` requires that `(UTC)` rather
than defaulting to it — a row that ever loses it should drop out instead of
landing hours off a boundary the reader is watching in-game.

The hazard here is the Server column. CN, EN and JP rows share one table and
the Chinese schedule runs about a year ahead, which is the akwiki CN-column
problem verbatim: publish EN, skip the rest. Betas are fenced off separately,
being dated exactly like everything else and playable by nobody, and the page
is an archive back to 2023, so currency is decided against ctx.now as in
bawiki. That leaves one live event today — thin, and true.

No resetOffsets: the EN boundaries land on three different clocks, which is a
patch window rather than a reset hour.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 03:50:05 +02:00
Lucas WintherandClaude Opus 5 e42ac84b40 feat(ingest): read a summoning campaign as a banner
Every game names its gacha something else, and inferType only knew the
five words the game8 sources use. Fate/Grand Order calls its banners
"Summoning Campaign" — the whole of that page's banner vocabulary — so
without this every FGO pull lands as "other" and the type filter stops
meaning anything for that lane.

Nothing on the current English page carries the word, so no checked-in
expected output moves; the unit case is what pins the behaviour until
the next refresh brings a banner in.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 21:30:09 +02:00
Lucas WintherandClaude Opus 5 4698ebc0c7 feat(fgo): read the English server, not the Japanese one
The source landed pointed at Event_List, whose first sentence is "This
page lists all Events in Fate/Grand Order Japan". Every date it
published was a JP date on a calendar whose other twelve games are
global — the akwiki CN column again, and the failure this product
exists to prevent. Event_List_(US) is the English schedule and is what
the adapter reads now; a test asserts the URL so it cannot drift back.

The page is a different template from the Reverse: 1999 one, so the
parser branches, the way game8 branches over seven shapes. Its sections
are fenced by banner images with the label drawn in a positioned div —
no heading, no id — and only ONGOING EVENTS is read. FUTURE EVENTS
states a month with no day, and PAST EVENTS states no year at all, so
neither can be dated and neither is guessed at. canParse asserts both
dividers and the Duration label, which turns a redesign into a stalled
source rather than a lane that quietly empties.

The rewrite drops what the JP branch needed and this page does not: the
hand-rolled month/year rollover (parseFullRange already reads both
stated years), the any-typed locals, and a heading-to-duration regex
whose nested lazy quantifiers hung for minutes on a 620KB body the
moment one anchor stopped matching. Durations name a zone but no clock,
so the stated calendar day is kept rather than shifted into UTC —
there is nothing to anchor a conversion to, and the start's day is half
the event ID.

Also reverts two things done to quiet tests rather than answer them.
The parser no longer normalises NFKC: that was pre-empting the
sanitizer so "real fixtures need no repair at all" would pass, and the
seam is where repairs belong. And renderedHtml no longer reaches into
a legacy {"*": …} envelope, which every source here rules out by
pinning formatversion=2.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 21:29:59 +02:00
Lucas WintherandClaude Opus 5 7ab801f409 feat(holodori): track hololive Dreams from its own wiki
New holodoriwiki parser plus the holodori-holodoriwiki-events source,
giving the game its two live and upcoming events at exact precision.

holodori.wiki is Miraheze, so this is the same call bawiki makes and for
the same reason: robots.txt disallows /w/ and /*?action=, which closes the
API route and leaves the rendered /wiki/Events page as the surface * is
allowed. It serves our own User-Agent a 200, sets no Content-Signal and no
Crawl-delay for us, and the wiki is CC BY-SA 4.0.

What makes this source unusually good is that it states its timezone on
every cell — 08/17/2026 8:00PM (JST) — so it is the only wiki source here
publishing exact precision on both boundaries without a per-region timer.
parseSlashClockZone requires that zone rather than defaulting to UTC: a row
that ever loses it drops out instead of silently landing nine hours off,
and an abbreviation absent from the table is refused rather than guessed,
which is why the table holds only JST and not the plausible-looking
candidates that are not fixed offsets. The 12-hour clock gets its own
handling and its own tests, because 12PM and 12AM are the two readings a
naive parser gets wrong while still producing a valid-looking instant.

Inclusion is fenced by the Current Events <h2> and bounded by the next one.
That matters more than it looks: Past Events sits directly below with
identical columns, so a reader taking every wikitable would put the back
catalogue on the calendar with nothing to mark it. Rows are still checked
against ctx.now on top of the heading, because Current is maintained by
hand and goes stale before anyone moves a row down.

Two rows are deliberately missing. Beginner Mission runs Game Launch →
Unknown, and no start means no event ID — a permanent tutorial chore is not
what a calendar of deadlines is for. An Unknown end is kept as endsAt null,
and unlike bawiki this parser does not drop a started-but-undated row: the
heading has already said the event is running, which is the fact bawiki
lacks when reading an archive.

Every event title is still a red link — the wiki has no article for any of
them yet — so each points at ?action=edit&redlink=1, a create-page form on
a path this robots.txt disallows. Refusing a href with a query keeps those
out and falls back to the events page; the articles get linked when they
exist, with no change here.

holodori gets resetOffsets of UTC+9 across all three regions, evidenced
rather than assumed the way Arknights' is: the game launched worldwide
simultaneously on one service, every boundary is stated in JST, and
Training Support Missions ends at 3:59AM JST, one minute before a 04:00
local reset. Only the offset is overridden; the hour is the default.
Setting it now is free, because no reader has a day key for a game the app
has never shipped — the same override a year from now would re-label ticks
already logged.

Both boundaries and the exclusions are asserted against an independent
extraction off the fixture, computed through a timezone library rather than
by the parser's own arithmetic.

Several counts in AGENTS.md and PRD.md were already stale before this — the
Fate/Grand Order source did not update them — and are corrected to what the
tree now holds rather than to what it held yesterday.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-18 03:39:49 +02:00
Lucas Winther c2179e29e0 fix: failing test case. 2026-08-18 02:47:35 +02:00
Lucas Winther 5cc46a0c9a feat: add fgo to games. 2026-08-18 02:43:56 +02:00
Lucas WintherandClaude Opus 5 d843c7cae7 feat(ba): track Blue Archive from its own wiki
New bawiki parser plus the ba-bawiki-events source, giving the game its
two live and upcoming events at day precision.

It reads the rendered /wiki/Events page, which is the opposite call to the
Fandom source next door and for the same kind of reason: bluearchive.wiki
is Miraheze, whose robots.txt disallows /w/ and /*?action=, so there the
API is the closed route and the page is the surface * is allowed. It
serves our own User-Agent a 200, sets no Content-Signal and no Crawl-delay
for us. The Fandom wiki was declined earlier as a JP archive yielding
nothing live; this is the live source that assessment pointed at.

Three page facts shape the parser, each a way to publish a confidently
wrong date. The schedule is a JP/Global tabber and the Japanese version
runs four to nine months ahead, so only Global is published — the akwiki
hazard. The Global tab's nav button carries the id
tabber-Global_version-label and precedes both panels, so slicing from the
first matching id reads the Japanese schedule while believing it read
ours; the first version of this parser did exactly that and published a
JP-only event. And three tabs on the page are named Global, the schedule
plus Mini-Event and Joint Firing Drill, so the schedule is found by its
Name (EN) header rather than by position, with canParse asserting the same
lookup — a renamed tab or column fails the run instead of emptying the
lane.

The page states no time of day and no timezone anywhere, which is why the
dates are day precision and why ba gets no resetOffsets. It also settles
the five other schedule tables, which do carry a wall clock but name no
zone and mostly do not say which server: reading those as UTC would invent
the fact that matters most, and rounding to a day would not save it,
because a 04:00 local boundary falls either side of UTC midnight depending
on the offset assumed and the start's day is part of the event ID. They
are left unparsed deliberately.

The two-event count is asserted against an independent extraction off the
fixture, and both starts corroborate against the per-event infoboxes
elsewhere on the page — GL 2026-08-04 and GL 2026-09-01, where the JP
reruns those tabs would have given are 2026-04-01 and 2026-05-06.

Three comment counts were already stale before this and are corrected to
what the tree now holds rather than to what it held yesterday.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 23:24:11 +02:00
Lucas WintherandClaude Opus 5 77921e88f5 feat(dates): read a bare ISO date as one boundary
bluearchive.wiki gives each boundary its own column and writes it as a
plain 2026-08-04, so unlike every reader here there is no range to split
and no field order to infer. Day precision, because the page states no
time of day.

Anchored at both ends, which matters more for this shape than for any
other in the module: it is the least distinctive one, and unanchored it
would happily find a date inside an article slug or a version string.

Also names the two readers the paragraph below the table was describing.
"The last two" stopped being true when rows were appended after them, and
the examples belong to parseAdjacentFullRange either way.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 23:23:06 +02:00
Lucas WintherandClaude Opus 5 3377fcb211 feat(ingest): add the Reverse: 1999 event source via Fandom's API
New `fandom` parser plus the r1999-fandom-events source, giving the game
six live and upcoming events at exact precision.

It fetches api.php?action=parse, not /wiki/Events. The rendered page
answers a non-browser client with a Cloudflare challenge, and getting past
that would be defeating an access control — the reason uma.moe was
declined. The wiki's robots.txt instead allows /api.php?action= for *, and
that endpoint serves our real User-Agent a 200, so this reads the
sanctioned surface with our own headers and no impersonation. The body is
therefore JSON, which is what canParse checks first: a challenge page or
an error payload must fail loudly, not parse to zero events.

Two page facts shape the parser. Titles come from each row's <b>, because
a missing banner image renders as a red link reading "File:<Event>
Banner.png" that a cell-text reader would publish as the event name. And
the page is an archive of 154 rows since v1.1 with no ongoing section to
gate on, so inclusion is decided against ctx.now — the six-event count is
asserted against an independent extraction off the fixture, per
docs/INGESTION.md § Testing.

Because robots.txt is unreadable from a challenged address, the gate fails
closed in CI and the source is skipped there — a warning, not a broken
build. Refreshing it means running `bun run refresh` from an address
Fandom serves.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 22:46:47 +02:00
Lucas WintherandClaude Opus 5 7926a70850 feat(dates): read ordinal ranges with a stated UTC offset
Adds parseOrdinalDateTimeRange for "November 9th, 05:00 - December 4th,
2023, 04:59 (UTC-5)". Ordinal days anchor the pattern at both ends so it
cannot claim text the looser readers should get, and the year rolls back
across New Year as parseMonthDayRange does.

The offset is required rather than defaulted. parseSlashDateTimeRange has
to read its wall-clock times as UTC because its source states no zone;
here the zone is part of the format, so a cell without one returns null
instead of being silently read five hours early. Calendar validation runs
on the stated local fields, before the shift, or "February 30th, 23:00
(UTC-5)" would convert into a real instant in March.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 22:44:38 +02:00
Lucas WintherandClaude Opus 5 025605aa09 docs: move the working agreement to AGENTS.md
The guidance in here is not Claude-specific — it is what this project is,
what it refuses to do, and the rules that are invisible from the code. Any
agent working here needs it, and AGENTS.md is the name they all look for.

CLAUDE.md stays as a pointer so Claude Code still finds it, and holds no
guidance of its own; two copies would drift and the wrong one would be read.
Every cross-reference in the source comments, docs, workflow and skills now
names AGENTS.md, and the image ignores both.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 22:16:40 +02:00