feat(ba): track Blue Archive from its own wiki

New bawiki parser plus the ba-bawiki-events source, giving the game its
two live and upcoming events at day precision.

It reads the rendered /wiki/Events page, which is the opposite call to the
Fandom source next door and for the same kind of reason: bluearchive.wiki
is Miraheze, whose robots.txt disallows /w/ and /*?action=, so there the
API is the closed route and the page is the surface * is allowed. It
serves our own User-Agent a 200, sets no Content-Signal and no Crawl-delay
for us. The Fandom wiki was declined earlier as a JP archive yielding
nothing live; this is the live source that assessment pointed at.

Three page facts shape the parser, each a way to publish a confidently
wrong date. The schedule is a JP/Global tabber and the Japanese version
runs four to nine months ahead, so only Global is published — the akwiki
hazard. The Global tab's nav button carries the id
tabber-Global_version-label and precedes both panels, so slicing from the
first matching id reads the Japanese schedule while believing it read
ours; the first version of this parser did exactly that and published a
JP-only event. And three tabs on the page are named Global, the schedule
plus Mini-Event and Joint Firing Drill, so the schedule is found by its
Name (EN) header rather than by position, with canParse asserting the same
lookup — a renamed tab or column fails the run instead of emptying the
lane.

The page states no time of day and no timezone anywhere, which is why the
dates are day precision and why ba gets no resetOffsets. It also settles
the five other schedule tables, which do carry a wall clock but name no
zone and mostly do not say which server: reading those as UTC would invent
the fact that matters most, and rounding to a day would not save it,
because a 04:00 local boundary falls either side of UTC midnight depending
on the offset assumed and the start's day is part of the event ID. They
are left unparsed deliberately.

The two-event count is asserted against an independent extraction off the
fixture, and both starts corroborate against the per-event infoboxes
elsewhere on the page — GL 2026-08-04 and GL 2026-09-01, where the JP
reruns those tabs would have given are 2026-04-01 and 2026-05-06.

Three comment counts were already stale before this and are corrected to
what the tree now holds rather than to what it held yesterday.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-17 23:24:11 +02:00
co-authored by Claude Opus 5
parent 77921e88f5
commit d843c7cae7
17 changed files with 5583 additions and 24 deletions
+39 -10
View File
@@ -36,8 +36,8 @@ A web app that aggregates live and upcoming events across popular gacha games, p
calendar, sorts them by end date or by what the reader is partway through, tracks day-by-day
progress on events that repeat daily, and lets a user mark events completed.
**Status: working app, refreshing itself on a schedule.** Schema, four parsers, eleven sources across
ten games, the full interface, offline support, a static server, a Docker image and CI all exist and
**Status: working app, refreshing itself on a schedule.** Schema, five parsers, twelve sources across
eleven games, the full interface, offline support, a static server, a Docker image and CI all exist and
are tested. The refresh runner (`bun run refresh`) fetches, caches raw snapshots and rebuilds the
feed; `.github/workflows/refresh.yml` runs it twice a day and commits only when a page actually
changed. The SQLite layer and the review queue are still specified in `docs/` but not built, so the
@@ -105,8 +105,8 @@ re-verify a sample against the live page afterward.
```
src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts
custom.ts — reader-authored games and events, and their key spaces
src/ingest/ html.ts, dates.ts (ten formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts
parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts — keyed by SITE, not game
src/ingest/ html.ts, dates.ts (eleven formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts
parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts, bawiki.ts — keyed by SITE, not game
adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam
src/client/ React app, service worker, manifest
state/ progress, daily log, ignores, prefs, sort — all localStorage
@@ -114,7 +114,7 @@ src/client/ React app, service worker, manifest
lens.ts — who sees which rows (focus, outstanding, next-to-expire); pure
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
serve.ts static server + /api/health
test/ 494 tests
test/ 517 tests
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
snapshots/ current page per source, rewritten by refresh — see its README
```
@@ -147,9 +147,9 @@ These come from how gacha games actually schedule things, and they cause most bu
year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a
recoverable disappointment; a confidently wrong date is the failure this product exists to prevent.
- **Parsers are keyed by site, not game.** One `game8` parser serves eight sources; `wikigg`,
`akwiki` and `fandom` serve one each — the first two share a host family and have entirely
different templates. Adding a source for a known site is one `SOURCES` entry; a new site is a
parser module.
`akwiki`, `fandom` and `bawiki` serve one each — the first two share a host family and have
entirely different templates. Adding a source for a known site is one `SOURCES` entry; a new site is
a parser module.
- **A source may publish more than one region's schedule.** Arknights' wiki lists CN and Global on
every row, five months apart. Publish the one our readers are on and skip the row that lacks it —
a CN date on a Global calendar is a confidently wrong date, not a near miss.
@@ -215,7 +215,7 @@ Sources are community wikis. Treat them as a guest would:
- Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
- One request per source per refresh cycle, minimum 6 hours apart.
- **Space requests to one host**, honouring its `Crawl-delay` and defaulting to 2s. Eight of the
eleven sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
twelve sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
as eight back-to-back requests to a single site — which is the shape an edge network throttles, and
what a burst looks like from the far end regardless of our intent.
- Send `If-None-Match` / `If-Modified-Since`; treat `304` as "skip, unchanged".
@@ -250,7 +250,8 @@ A source whose ToS forbids automated access does not get an adapter. Flag it and
| `uma.moe` | **Declined.** Data comes from an API behind a Cloudflare Turnstile proof header; an adapter would mean defeating a deliberate access control. The `robots.txt` is permissive, but the gate is not in `robots.txt` |
| `reverse1999.fandom.com` | **Built** (2026-08-17), via `api.php`, not the wiki page — see § Fandom below |
| `bluearchive.fandom.com` | **Declined.** Fetches and parses fine; the page is the problem. Its `Event/Event_List` is a JP-server archive whose newest entry ended 2026-02-18, so all 88 rows are history and it yields **zero** live or upcoming events. An adapter would put an empty lane on the calendar and, because the runner rejects a body that parses to nothing, report a broken source forever. Same failure as the Infinity Nikki Game8 page, further along |
| `bluearchive.wiki`, `prydwen.gg`, `gametora.com` | **Cleared, unbuilt.** `User-agent: *` allows the paths we would want. prydwen sets `Crawl-delay: 10`, far below our one-per-6h. `bluearchive.wiki` is the live Blue Archive source worth building — `Name (EN) \| Name (JP) \| Start date \| End date` with ISO dates, current to 2026-09-15 — but note it is Miraheze and its `robots.txt` **disallows** `/w/` and `/*?action=`, so the API route is closed there and only the rendered `/wiki/Events` page is permitted. It also lists JP and Global in separate tables, which is the akwiki hazard: publishing the wrong server's date |
| `bluearchive.wiki` | **Built** (2026-08-17), from the rendered `/wiki/Events` page — see § Blue Archive below |
| `prydwen.gg`, `gametora.com` | **Cleared, unbuilt.** `User-agent: *` allows the paths we would want. prydwen sets `Crawl-delay: 10`, far below our one-per-6h |
`.github/ISSUE_TEMPLATE/feature_request.yml` points readers at that table by heading, so a source
request can be checked against it before anyone writes it up — the loudest feedback on the first
@@ -284,6 +285,34 @@ broken build — `skipped_robots` does not touch the failure streak, and the run
falls back to the checked-in fixture. Refreshing it means running `bun run refresh` from an address
Fandom serves, which is how its first snapshot was taken.
**Blue Archive: the page, never the API — the opposite call to Fandom.** `bluearchive.wiki` is a
Miraheze wiki, and Miraheze's `robots.txt` **disallows** `/w/` and `/*?action=`. So the route
`parsers/fandom.ts` takes is the one that is closed here, and the rendered `/wiki/Events` page is the
surface `*` is allowed — it answers our own `User-Agent` with a `200`, no `Content-Signal`, and no
`Crawl-delay` for us. `Special:` is disallowed too, which is why `parsers/bawiki.ts` skips those links
exactly as the Fandom one does.
Three things about that page are worth knowing before touching it, all of them ways to publish a
confidently wrong date:
- **It states JP and Global in separate tabs, and the Japanese one runs four to nine months ahead.**
Same hazard as the CN column on `akwiki`, same answer: publish Global only. The tab's *nav button*
carries the id `tabber-Global_version-label` and sits above **both** panels, so a reader that slices
from the first id match reads the Japanese schedule while believing it read ours.
- **There are three Global tabs, not one** — the schedule, plus Mini-Event and Joint Firing Drill
further down, whose ids are the same name with `_2` and `_3`. The parser finds the schedule by its
`Name (EN)` header rather than by position, and `canParse` asserts that lookup, so a renamed tab or
column fails the run instead of quietly emptying the lane.
- **The page states no time of day and no timezone anywhere.** The schedule's dates are bare
`YYYY-MM-DD`, which is honest day precision. Its five other tables (Mini-Event, Reward campaigns,
Attendance bonuses, Guide missions, Joint Firing Drill) *do* carry a wall clock — `08/12/2026 11:00`
— but name no zone for it, and three of the five do not say which server they describe. Those are
deliberately unparsed: reading them as UTC invents the fact that matters, and rounding to a day does
not save it, because a 04:00 local boundary lands either side of UTC midnight depending on the
offset assumed and the start's day is part of the event ID. Attendance bonuses would be a real
dailies source if a zone is ever stated. For the same reason `ba` has no `resetOffsets`: Blue
Archive Global does run one worldwide server, but nothing in this source says on what clock.
`scripts/refresh-sources.ts` enforces all of the above in code — the 6h floor, one request, no
retries, conditional headers, per-host spacing, robots (failing closed when `robots.txt` cannot be
read). Anything that would make it fetch more often is a change to this section first.
+1
View File
@@ -76,6 +76,7 @@ src/
wikigg.ts wiki.gg mp-event templates ✓ built
akwiki.ts arknights.wiki.gg mrfz-wtable ✓ built
fandom.ts Fandom via the action=parse API ✓ built
bawiki.ts bluearchive.wiki JP/Global tabber ✓ built
index.ts parser registry ✓ built
adapters/
types.ts Adapter interface, ParseContext ✓ built
+9 -5
View File
@@ -10,6 +10,7 @@ import { z } from "zod";
export const GameId = z.enum([
"genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", "r1999",
"ba",
]);
export const EventType = z.enum([
@@ -254,7 +255,7 @@ reader cannot see.
| Game | Reset (server local) | Server offset | Reset (UTC) | Copenhagen, summer / winter |
|---|---|---|---|---|
| Genshin, Star Rail, ZZZ, Wuwa, NTE | 04:00 | region (EU = UTC+1) | 03:00 | 05:00 / 04:00 |
| Infinity Nikki, P5X | 04:00 | region (assumed) | 03:00 | 05:00 / 04:00 |
| Infinity Nikki, P5X, Blue Archive | 04:00 | region (assumed) | 03:00 | 05:00 / 04:00 |
| Arknights, all regions | 04:00 | UTC-7 (one Global server) | 11:00 | 13:00 / 12:00 |
| Endfield, Europe | 04:00 | UTC-5 (on the Americas server) | 09:00 | 11:00 / 10:00 |
| Endfield, Asia / Americas | 04:00 | regional default | 20:00 / 09:00 | — |
@@ -266,10 +267,13 @@ server for every region we model. The offset is read off the source rather than
ending event on arknights.wiki.gg carries an exact end of `10:59:59Z`, which is `03:59:59` at UTC-7,
one second before a 04:00 reset.
Infinity Nikki and P5X carry **no `resetOffsets` entry**, so they take the regional default. That is
an assumption, not a verified server map — neither source states one. Confirm it against the games
before relying on it, and note that adding an override later re-labels the game-day of ticks readers
have already logged, which is the change this table warns about below.
Infinity Nikki, P5X and Blue Archive carry **no `resetOffsets` entry**, so they take the regional
default. That is an assumption, not a verified server map — none of the three sources states one.
Blue Archive is the case where the assumption is most likely wrong and still the right entry: the
game does run one worldwide server, but `bluearchive.wiki` states no time of day and no timezone
anywhere on the page, so there is no offset to read off it. Confirm these against the games before
relying on them, and note that adding an override later re-labels the game-day of ticks readers have
already logged, which is the change this table warns about below.
**Reverse: 1999 is the one game that rolls on a different hour**, and it needs its own field rather
than a bent offset. It runs a single global server on UTC-5 and resets at **05:00**, so its day rolls
+2 -2
View File
@@ -121,7 +121,7 @@ Named across the thread, excluding the six already covered:
| Arknights | 2 | **`GameId`, hue and `dailyTasks` already exist in `games.ts` with no source registered.** Cheapest possible win. |
| Reverse: 1999 | 1 | Named as the one thing between a bookmark and a habit. |
| Azur Lane | 1 | OP judged doable |
| Blue Archive | 1 | OP judged doable |
| Blue Archive | 1 | OP judged doable. **Done** (2026-08-17) — `bluearchive.wiki`, not the Fandom wiki, which is a JP archive yielding nothing live |
| Umamusume | 1 | OP judged doable |
| Persona 5X | 1 | OP judged doable |
| Nikke | 1 | |
@@ -305,7 +305,7 @@ diagnosis in each item still holds — what changed is whether it has been acted
| P1b cap the long list | **Not done.** No "show all N" expander |
| P1b persist `view` | **Not done.** Still `useState` in `App.tsx` |
| P1b Calendar → Timeline | **Done.** The tab reads "Timeline" |
| P1c more games | **Done, three of them.** Infinity Nikki, Persona 5: The Phantom X, Reverse: 1999 — one commit each. The declined and cleared-but-unbuilt candidates are recorded in `AGENTS.md` § Scraping conduct so they are not re-litigated |
| P1c more games | **Done, four of them.** Infinity Nikki, Persona 5: The Phantom X, Reverse: 1999, Blue Archive — one commit each. Of the games named in the thread, Azur Lane and Umamusume are **declined on conduct** rather than pending, and the declined and cleared-but-unbuilt candidates are recorded in `AGENTS.md` § Scraping conduct so they are not re-litigated |
| P1c roadmap in the colophon | **Not done** |
| P2 colour and screenshot | **Not done** |
| P2 custom events | **Done.** PRD F13 first, then the code; `mygame:` / `myevent:` key spaces, in the export |
+16
View File
@@ -52,6 +52,7 @@ Consequences worth internalising:
| `wikigg` | wiki.gg MediaWiki `mp-event` templates | Endfield |
| `akwiki` | arknights.wiki.gg's `mrfz-wtable` "Ongoing/upcoming" table | Arknights |
| `fandom` | Fandom wikis via the MediaWiki `action=parse` API — `Event \| Time Period \| Version` wikitables | Reverse: 1999 |
| `bawiki` | bluearchive.wiki's rendered `/wiki/Events` — a JP/Global tabber over `Name (EN) \| Start date \| End date \| Notes` wikitables | Blue Archive |
`wikigg` is the better shape by a distance: it emits ISO timestamps with one timer per server
region, so its events carry exact precision and real `regionEnds`. Prefer a source like that over a
@@ -73,6 +74,21 @@ rather than parse to zero events. Two page facts drive the rest of it:
anchor on. Inclusion is therefore decided against `ctx.now`, the one parser here that does so;
`akwiki` and `game8` can gate on a heading instead, and should where one exists.
`bawiki` is the mirror image of `fandom`: same MediaWiki software, opposite conclusion about which
surface to read. bluearchive.wiki is Miraheze, whose `robots.txt` disallows `/w/` and `/*?action=`,
so the API is closed and the rendered `/wiki/Events` page is what `*` is allowed — and it serves our
own `User-Agent` a `200`. Three page facts drive the parser (all three in AGENTS.md § Blue Archive):
- **JP and Global are separate tabs, months apart**, so only Global is published — the `akwiki`
hazard. The Global tab's nav *button* precedes both panels, so slicing from the first matching id
reads the Japanese schedule.
- **Three tabs are named Global**, the schedule plus Mini-Event and Joint Firing Drill. The schedule
is identified by its `Name (EN)` header rather than by position, and `canParse` runs the same
lookup so a rename fails the run rather than emptying the lane.
- **Nothing on the page states a time of day or a timezone.** The schedule's bare ISO dates are day
precision; the five tables that do carry a wall clock name no zone for it and are left unparsed
rather than read as UTC.
`akwiki` shares a host family with `wikigg` and nothing else — arknights.wiki.gg has no `mp-event`
cards, so the two are separate modules rather than one parser with a branch. Two things about that
page shape are worth knowing before touching it:
+4 -4
View File
@@ -46,13 +46,13 @@ event, because a missing event sends them to a wiki while a wrong one makes them
| Neverness to Everness | `nte` |
Added since launch, on the strength of the release thread (`docs/FEEDBACK.md` § P1): Infinity Nikki
(`nikki`), Persona 5: The Phantom X (`p5x`), Reverse: 1999 (`r1999`). **`GameId` in
`src/shared/schema.ts` is the live answer** and `SOURCES` says which of them actually have a source —
this table is the launch scope, not a roster to keep in sync.
(`nikki`), Persona 5: The Phantom X (`p5x`), Reverse: 1999 (`r1999`), Blue Archive (`ba`).
**`GameId` in `src/shared/schema.ts` is the live answer** and `SOURCES` says which of them actually
have a source — this table is the launch scope, not a roster to keep in sync.
Adding a game must require no change to `GachaEvent` — only a `GameId` entry, its `games.ts`
metadata, and a source registration. That is the test of whether the data model is right, and it has
held: the ten games here have cost the event schema nothing. Per-game *metadata* does occasionally
held: the eleven games here have cost the event schema nothing. Per-game *metadata* does occasionally
grow (Reverse: 1999 needed `resetHourLocal` for a 05:00 reset), which is a different file and moves
no stored key. A game may have several sources; see `docs/INGESTION.md` § Three layers.
@@ -0,0 +1,44 @@
[
{
"id": "ba:code-box-the-shadow-approaching-millennium:2026-08-04",
"game": "ba",
"title": "Code: BOX - The Shadow Approaching Millennium",
"type": "rerun",
"summary": "Rerun",
"startsAt": "2026-08-04T00:00:00.000Z",
"startPrecision": "day",
"endsAt": "2026-08-18T00:00:00.000Z",
"endPrecision": "day",
"regionScoped": false,
"regionEnds": null,
"sourceUrl": "https://bluearchive.wiki/wiki/Code:_BOX_Shadow_Looming_Over_Millennium_%EF%BD%9EOne_Question_and_Two_Answers%EF%BD%9E/Rerun",
"sourceId": "ba-bawiki-events",
"status": "published",
"confidence": 0.85,
"extractionMethod": "parser",
"version": 1,
"firstSeenAt": "2026-08-14T00:00:00.000Z",
"updatedAt": "2026-08-14T00:00:00.000Z"
},
{
"id": "ba:pray-ball-swing-for-the-grand-slam:2026-09-01",
"game": "ba",
"title": "Pray-Ball! Swing for the Grand Slam!",
"type": "rerun",
"summary": "Rerun",
"startsAt": "2026-09-01T00:00:00.000Z",
"startPrecision": "day",
"endsAt": "2026-09-15T00:00:00.000Z",
"endPrecision": "day",
"regionScoped": false,
"regionEnds": null,
"sourceUrl": "https://bluearchive.wiki/wiki/Pray-Ball!_%EF%BD%9E_Aim_for_a_Grand_Slam_Home_Run_%EF%BD%9E/Rerun",
"sourceId": "ba-bawiki-events",
"status": "published",
"confidence": 0.85,
"extractionMethod": "parser",
"version": 1,
"firstSeenAt": "2026-08-14T00:00:00.000Z",
"updatedAt": "2026-08-14T00:00:00.000Z"
}
]
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -15,7 +15,7 @@
* second request, and the next cycle is minutes-cheap compared to being a
* bad guest.
* - requests to one host are spaced, honouring its `Crawl-delay`. Eight of the
* ten sources are game8.co pages, so without this one cycle is eight
* twelve sources are game8.co pages, so without this one cycle is eight
* back-to-back requests to a single site — inside the per-source floor and
* still the behaviour an edge network throttles.
* - conditional requests always, so an unchanged page costs the wiki a 304.
+1 -1
View File
@@ -215,7 +215,7 @@ export function EventForm({
</select>
</label>
{/* Otherwise the list is nine games we track and no sign of why none of
{/* Otherwise the list is eleven games we track and no sign of why none of
them is theirs — the reader who came here to add a game we don't cover
has no way of knowing they need to make it first. */}
{Object.keys(customGames).length === 0 && (
+10
View File
@@ -96,6 +96,16 @@ const SOURCES: SourceSpec[] = [
url: "https://reverse1999.fandom.com/api.php?action=parse&page=Events&prop=text&formatversion=2&format=json",
parserId: "fandom",
},
{
id: "ba-bawiki-events",
game: "ba",
// The rendered page, deliberately, and the opposite call to the Fandom
// source above: this wiki is Miraheze, whose robots.txt disallows `/w/` and
// `/*?action=`, so the API route is the one that is closed here and
// `/wiki/Events` is the surface the site permits.
url: "https://bluearchive.wiki/wiki/Events",
parserId: "bawiki",
},
];
function toAdapter(spec: SourceSpec): Adapter {
+263
View File
@@ -0,0 +1,263 @@
import { eventId, type GachaEvent } from "../../shared/schema.ts";
import { parseIsoDay, type ParsedInstant } from "../dates.ts";
import { text } from "../html.ts";
import type { ParseContext } from "../adapters/types.ts";
import { inferType } from "./game8.ts";
import type { SourceParser } from "./types.ts";
/**
* The Blue Archive wiki's event schedule (`bluearchive.wiki`, a Miraheze wiki).
*
* The rendered page, not the API: Miraheze's `robots.txt` disallows `/w/` and
* `/*?action=`, so the route `fandom.ts` takes is closed here and `/wiki/Events`
* is the surface this site permits. It answers our own User-Agent with a 200,
* so nothing has to be pretended.
*
* The schedule is a tabber with one panel per game version:
*
* <article id="tabber-Japanese_version">
* <table class="wikitable"><tr><th>Name (EN)<th>Name (JP)<th>Start date …
* <article id="tabber-Global_version">
* <table class="wikitable">
* <tr><th>Name (EN)</th><th>Start date</th><th>End date</th><th>Notes</th>
* <tr><td><a href="/wiki/SLUG">TITLE</a></td>
* <td>2026-08-04</td><td>2026-08-18</td><td>Rerun</td>
*
* Four things about this page decide the code below.
*
* **Only the Global panel is ours.** The Japanese version runs four to nine
* months ahead — the same hazard as the CN column on `akwiki.ts`, and the same
* answer: a JP date on a Global calendar is a confidently wrong date, not a near
* miss. The two panels are also shaped differently (JP carries an extra
* `Name (JP)` column), so a reader that wandered into the wrong one would read
* a Japanese title where it expected a start date.
*
* **Columns are resolved from the header row, never counted.** That difference
* between the panels is exactly how a silent drop starts: if the Global table
* ever gains a `Name (JP)` column, fixed indices would hand a title to the date
* reader, every row would fail to parse, and the game's lane would empty with no
* error anywhere.
*
* **This is an archive, not a schedule.** 98 Global rows go back to 2021, of
* which two had not ended when the fixture was captured. Inclusion is therefore
* decided against `ctx.now`, as in `fandom.ts`; there is no "ongoing" heading to
* gate on.
*
* **The page states no time of day and no timezone — anywhere.** Every boundary
* here is a bare date, so both are day precision. That also settles the five
* other schedule tables further down the page (Mini-Event, Reward campaigns,
* Attendance bonuses, Guide missions, Joint Firing Drill): those *do* state wall
* clock times, `08/12/2026 11:00` style, but the page names no zone for them and
* three of the five do not even say which server they describe. Reading those as
* UTC would invent the fact that matters most, and rounding them to a day would
* not save it — a 04:00 local boundary lands on either side of UTC midnight
* depending on the offset assumed, and the start's day is part of the event ID.
* So they are left unparsed deliberately. If a table there ever states its zone,
* it is worth adding; until then it is a missing fact like any other.
*/
/**
* Every element whose id names the Global tab — the panel *and* the tab button
* in the nav above it, which carries `id="tabber-Global_version-label"` and,
* being part of the header, comes first in the document.
*
* That button is the trap this parser exists to avoid walking into: slicing from
* the first match runs from the nav through the Japanese panel, and the Japanese
* schedule is four to nine months ahead of ours. The `-label` suffix is what
* tells the two apart.
*/
const PANEL = /id="(tabber-Global[^"]*)"/gi;
const TABLE = /<table\b[^>]*>([\s\S]*?)<\/table>/i;
const ROW = /<tr\b[^>]*>([\s\S]*?)<\/tr>/gi;
const CELL = /<t([dh])\b[^>]*>([\s\S]*?)<\/t\1>/gi;
/**
* The event's own article. `Special:` is excluded because this wiki's
* robots.txt disallows it, and it is the wrong page to send a reader to.
*/
const ARTICLE_LINK = /<a\b[^>]*href="(\/wiki\/(?!Special:)[^"#?]+)"/i;
interface Columns {
title: number;
start: number;
end: number;
notes: number | undefined;
}
/**
* Where each column sits, read off the header row.
*
* Returns null when the table is not a schedule at all, which is what keeps the
* per-event infoboxes elsewhere on the page from being read as one.
*/
function columns(headers: string[]): Columns | null {
const at = (match: (h: string) => boolean) => {
const i = headers.findIndex((h) => match(h));
return i < 0 ? undefined : i;
};
// "Name (EN)", and deliberately not a plain "Name": that is how the page's
// four other schedule tables head the same column, and telling this table
// apart from those is the whole job of this function.
const title = at((h) => h === "name (en)");
const start = at((h) => h === "start date");
const end = at((h) => h === "end date");
if (title === undefined || start === undefined || end === undefined) {
return null;
}
return { title, start, end, notes: at((h) => h === "notes") };
}
/**
* The Global version panel's schedule table, and where its columns sit.
*
* There are three Global panels on this page, not one: the schedule, plus the
* Mini-Event and Joint Firing Drill tabbers further down, whose ids are the same
* name with `_2` and `_3` appended. So the right panel is found by the shape of
* the table inside it rather than by its position — the schedule is the one
* headed `Name (EN)`, and the other two head that column `Name`. Position would
* work today and would quietly start reading a different section the day the
* page is reordered or the schedule tabber is renumbered.
*/
function globalSchedule(
html: string,
): { cols: Columns; body: string } | null {
for (const m of html.matchAll(PANEL)) {
// The nav button that names the tab, not the tab itself. It sits in the
// header above *both* panels, so slicing from it runs straight through the
// Japanese schedule — four to nine months ahead of ours.
if ((m[1] ?? "").endsWith("-label")) continue;
// Bounded at whichever comes first: the end of this panel, or the start of
// the next tab. Either alone would do today — the panel is an <article> and
// Global is the last tab in its tabber — and neither alone survives the
// tabs being reordered or the extension emitting a different element.
const from = m.index;
const bounds = [
html.indexOf("</article>", from + 1),
html.indexOf('id="tabber-', from + 1),
].filter((i) => i >= 0);
const section =
bounds.length === 0
? html.slice(from)
: html.slice(from, Math.min(...bounds));
const body = TABLE.exec(section)?.[1];
if (body === undefined) continue;
const headers = [...body.matchAll(/<th\b[^>]*>([\s\S]*?)<\/th>/gi)].map(
(h) => text(h[1] ?? "").toLowerCase(),
);
const cols = columns(headers);
if (cols !== null) return { cols, body };
}
return null;
}
export function parseBlueArchiveWikiEventsPage(
html: string,
ctx: ParseContext,
): GachaEvent[] {
const schedule = globalSchedule(html.replace(/\s+/g, " "));
if (schedule === null) return [];
const { cols } = schedule;
const nowMs = Date.parse(ctx.now);
const out: GachaEvent[] = [];
for (const row of schedule.body.matchAll(ROW)) {
const cells = [...(row[1] ?? "").matchAll(CELL)].map((c) => ({
tag: c[1] ?? "",
html: c[2] ?? "",
}));
if (cells.length === 0 || cells.some((c) => c.tag === "h")) continue;
const titleCell = cells[cols.title]?.html ?? "";
const title = text(titleCell);
if (title.length === 0) continue;
const start = parseIsoDay(text(cells[cols.start]?.html ?? ""));
if (start === null) continue;
const end: ParsedInstant | null = parseIsoDay(
text(cells[cols.end]?.html ?? ""),
);
if (end === null) {
// A row this reader cannot date on the end is publishable only while its
// start is still ahead: that is a real, announced event whose end the wiki
// has not filled in, and `endsAt: null` is the honest way to say so. Once
// the start has passed, the end is the only thing separating a live event
// from any of the ninety-odd finished rows above it, and without one there
// is no way to tell — so that row yields nothing rather than a guess.
if (Date.parse(start.iso) < nowMs) continue;
} else {
if (end.iso <= start.iso) continue;
// Live and upcoming only. Everything else is history the page keeps and
// the calendar does not want.
if (Date.parse(end.iso) < nowMs) continue;
}
// The source's own annotation: "Rerun", "Collaboration Event",
// "Special Operation Part 1". A better type signal than the title alone,
// which for a rerun is simply the original event's name again.
const notes =
cols.notes === undefined ? "" : text(cells[cols.notes]?.html ?? "");
const href = ARTICLE_LINK.exec(titleCell)?.[1];
// Same scoring as the other parsers here: a day-precision boundary is
// weaker evidence than an exact one, and an unannounced end weaker still.
let confidence = 0.95 - 0.05;
confidence -= end === null ? 0.15 : 0.05;
out.push({
id: eventId(ctx.game, title, start.iso),
game: ctx.game,
title,
type: inferType(`${title} ${notes}`),
summary: notes.length > 0 ? notes : null,
startsAt: start.iso,
startPrecision: start.precision,
endsAt: end === null ? null : end.iso,
endPrecision: end === null ? "unknown" : end.precision,
// The panels here are game *versions* — Japanese and Global are separate
// releases on separate schedules — not our asia/america/europe regions.
// Blue Archive Global is one worldwide server and this page draws no
// distinction inside it, so there is nothing region-scoped to report.
regionScoped: false,
regionEnds: null,
sourceUrl:
href === undefined
? ctx.sourceUrl
: new URL(href, ctx.sourceUrl).toString(),
sourceId: ctx.sourceId,
status: "published",
confidence: Math.round(confidence * 100) / 100,
extractionMethod: "parser",
version: 1,
firstSeenAt: ctx.now,
updatedAt: ctx.now,
});
}
return out.sort((a, b) =>
a.startsAt === b.startsAt
? a.id.localeCompare(b.id)
: a.startsAt.localeCompare(b.startsAt),
);
}
export const blueArchiveWikiParser: SourceParser = {
id: "bawiki",
label: "Blue Archive Wiki",
canParse(html: string): boolean {
// The same lookup `parse` does, which is the point: this page has sixty-odd
// wikitables and three Global panels, so "a table exists" proves nothing
// about the one that matters. Asserting the schedule is *findable* is what
// makes a renamed tab or a renamed column fail the run loudly instead of
// emptying the lane, which reads downstream as "Blue Archive has nothing
// on" rather than as a failure.
return globalSchedule(html.replace(/\s+/g, " ")) !== null;
},
parse: parseBlueArchiveWikiEventsPage,
};
+3
View File
@@ -1,4 +1,5 @@
import { arknightsWikiParser } from "./akwiki.ts";
import { blueArchiveWikiParser } from "./bawiki.ts";
import { fandomParser } from "./fandom.ts";
import { game8Parser } from "./game8.ts";
import { wikiGgParser } from "./wikigg.ts";
@@ -14,6 +15,7 @@ export const PARSERS: SourceParser[] = [
wikiGgParser,
arknightsWikiParser,
fandomParser,
blueArchiveWikiParser,
];
export function parserById(id: string): SourceParser | undefined {
@@ -22,6 +24,7 @@ export function parserById(id: string): SourceParser | undefined {
export type { SourceParser } from "./types.ts";
export { arknightsWikiParser } from "./akwiki.ts";
export { blueArchiveWikiParser } from "./bawiki.ts";
export { fandomParser } from "./fandom.ts";
export { game8Parser } from "./game8.ts";
export { wikiGgParser } from "./wikigg.ts";
+7
View File
@@ -87,6 +87,13 @@ export const GAMES: Record<GameId, GameMeta> = {
// state `(UTC-5)`, and every one of them starts at 05:00 and ends at 04:59 —
// an event ending one minute before the reset that the next one begins on.
r1999: { id: "r1999", name: "Reverse: 1999", short: "R1999", hue: "#C9A227" , studio: "Bluepoch", dailyTasks: "Daily missions", resetOffsets: { asia: -5, america: -5, europe: -5 }, resetHourLocal: 5 },
// No `resetOffsets` and no `resetHourLocal`, for the reason p5x has none: the
// source states no time of day anywhere — every date on bluearchive.wiki's
// Global schedule is a bare `YYYY-MM-DD`. Blue Archive Global does run one
// worldwide server, so an override is probably owed here eventually, but it
// has to come from a source that states the clock. Inventing one moves real
// readers' day keys, and a wrong guess ticks the wrong box every night.
ba: { id: "ba", name: "Blue Archive", short: "Blue Archive", hue: "#3FCBDD" , studio: "Nexon Games", dailyTasks: "Daily missions, AP" },
};
export const GAME_LIST: GameMeta[] = Object.values(GAMES);
+1
View File
@@ -11,6 +11,7 @@ export const GameId = z.enum([
"nikki", // Infinity Nikki
"p5x", // Persona 5: The Phantom X
"r1999", // Reverse: 1999
"ba", // Blue Archive
]);
export type GameId = z.infer<typeof GameId>;
+188
View File
@@ -3,6 +3,7 @@ import { adapterById } from "../../src/ingest/adapters/index.ts";
import type { Adapter } from "../../src/ingest/adapters/types.ts";
import { parseOrdinalDateTimeRange } from "../../src/ingest/dates.ts";
import { arknightsWikiParser } from "../../src/ingest/parsers/akwiki.ts";
import { blueArchiveWikiParser } from "../../src/ingest/parsers/bawiki.ts";
import { fandomParser, renderedHtml } from "../../src/ingest/parsers/fandom.ts";
import { inferType } from "../../src/ingest/parsers/game8.ts";
import { GachaEvent, type EventType } from "../../src/shared/schema.ts";
@@ -37,6 +38,7 @@ const CASES: Array<{ adapter: Adapter; fixture: string }> = [
// action API, and `snapshots/` names every stored body `<id>.html` whatever
// its content type. The fixture is the bytes the fetcher would store.
{ adapter: adapter("r1999-fandom-events"), fixture: "fixtures/r1999/fandom-events-2026-08-17" },
{ adapter: adapter("ba-bawiki-events"), fixture: "fixtures/ba/bawiki-events-2026-08-17" },
];
async function runAdapter(adapter: Adapter, fixture: string) {
@@ -516,3 +518,189 @@ describe("fandom parser", () => {
expect(parseOrdinalDateTimeRange("February 20th, 05:00 - March 27th, 04:59 (UTC-5)")).toBeNull();
});
});
describe("blue archive wiki", () => {
const fixture = "fixtures/ba/bawiki-events-2026-08-17";
const ba = adapter("ba-bawiki-events");
/** A stand-in page with the tabber shape this parser navigates. */
function tabbed(japanese: string, global: string): string {
return `<div class="tabber"><header><nav>
<a class="tabber__tab" id="tabber-Japanese_version-label" href="#tabber-Japanese_version">Japanese version</a>
<a class="tabber__tab" id="tabber-Global_version-label" href="#tabber-Global_version">Global version</a>
</nav></header><section>
<article id="tabber-Japanese_version"><table class="wikitable">${japanese}</table></article>
<article id="tabber-Global_version"><table class="wikitable">${global}</table></article>
</section></div>`;
}
const GLOBAL_HEADER =
"<tr><th>Name (EN)</th><th>Start date</th><th>End date</th><th>Notes</th></tr>";
function parse(html: string) {
return ba.parse(html, {
now: NOW,
sourceUrl: ba.url,
sourceId: ba.id,
game: ba.game,
});
}
test("publishes the Global schedule and never the Japanese one", async () => {
// Both panels are on the page and the Japanese one runs months ahead — the
// same hazard as the CN column on the Arknights wiki. Counted independently
// off the fixture: 98 Global rows, of which two had not ended at the pinned
// clock. 104 Japanese rows are on the same page and none of them is ours.
const events = await runAdapter(ba, fixture);
expect(events).toHaveLength(2);
expect(events.map((e) => e.title)).toEqual([
"Code: BOX - The Shadow Approaching Millennium",
"Pray-Ball! Swing for the Grand Slam!",
]);
// Live on Global for the fixture's clock.
expect(events[0]?.startsAt).toBe("2026-08-04T00:00:00.000Z");
expect(events[0]?.endsAt).toBe("2026-08-18T00:00:00.000Z");
// "Pandemic Hazard ~ Miracle Pancake ~" runs 2026-08-12 to 2026-08-26 on
// the *Japanese* server and has not been scheduled on Global at all. It is
// what this parser publishes if it slices from the Global tab's nav button
// — which precedes the Japanese panel — instead of from the panel itself.
for (const e of events) expect(e.title).not.toMatch(/Pandemic Hazard/);
});
test("the back catalogue stays off the calendar", async () => {
// The table is an archive going back to 2021 with no "ongoing" heading to
// gate on, so inclusion is decided against ctx.now.
const events = await runAdapter(ba, fixture);
for (const e of events) {
expect(e.endsAt).not.toBeNull();
expect(Date.parse(e.endsAt!)).toBeGreaterThanOrEqual(Date.parse(NOW));
}
});
test("states day precision, because the page states no time of day", async () => {
// Every boundary on this page is a bare `YYYY-MM-DD`. Day precision is the
// honest reading, and it costs 0.05 of confidence on each side.
for (const e of await runAdapter(ba, fixture)) {
expect(e.startPrecision).toBe("day");
expect(e.endPrecision).toBe("day");
expect(e.startsAt.endsWith("T00:00:00.000Z")).toBe(true);
expect(e.confidence).toBe(0.85);
}
});
test("reports one global end rather than inventing per-region ones", async () => {
// The panels are game *versions*, not our asia/america/europe regions —
// Blue Archive Global is a single worldwide server and this page draws no
// distinction inside it.
for (const e of await runAdapter(ba, fixture)) {
expect(e.regionScoped).toBe(false);
expect(e.regionEnds).toBeNull();
}
});
test("links each event to its own article, never to Special:", async () => {
for (const e of await runAdapter(ba, fixture)) {
expect(e.sourceUrl.startsWith("https://bluearchive.wiki/wiki/")).toBe(true);
// robots.txt disallows Special:, and it is the wrong page to send a
// reader to besides.
expect(e.sourceUrl).not.toMatch(/Special:/);
}
});
test("takes the Notes column as the summary and types from it", async () => {
const events = await runAdapter(ba, fixture);
// A rerun reuses the original's title verbatim, so the title alone cannot
// say what kind of event it is. The Notes column can.
expect(events.every((e) => e.summary === "Rerun")).toBe(true);
expect(events.every((e) => e.type === "rerun")).toBe(true);
});
test("resolves columns from the header row rather than counting them", () => {
// The Japanese table carries an extra `Name (JP)` column. If the Global one
// ever gains it, fixed indices would feed a title to the date reader and
// every row would fail to parse — emptying the lane with no error anywhere.
const events = parse(
tabbed(
"",
`<tr><th>Name (EN)</th><th>Name (JP)</th><th>Start date</th><th>End date</th></tr>
<tr><td><a href="/wiki/Widened">Widened Table</a></td><td></td>
<td>2026-09-02</td><td>2026-09-16</td></tr>`,
),
);
expect(events).toHaveLength(1);
expect(events[0]?.title).toBe("Widened Table");
expect(events[0]?.startsAt).toBe("2026-09-02T00:00:00.000Z");
});
test("an upcoming row with no end date publishes as endsAt null", () => {
// The source genuinely has not announced one. Inventing a plausible end is
// the failure this product exists to prevent; a null end renders as such.
const events = parse(
tabbed(
"",
`${GLOBAL_HEADER}
<tr><td><a href="/wiki/Soon">Announced, Undated</a></td>
<td>2026-09-02</td><td>TBA</td><td></td></tr>`,
),
);
expect(events).toHaveLength(1);
expect(events[0]?.endsAt).toBeNull();
expect(events[0]?.endPrecision).toBe("unknown");
expect(events[0]?.summary).toBeNull();
expect(events[0]?.confidence).toBe(0.75); // held under the 0.8 gate
});
test("a started row with no end date yields nothing", () => {
// On a page that is 98 rows of history, the end date is the only thing
// separating a live event from a finished one. Without it there is no way
// to tell, so the row is not publishable.
const events = parse(
tabbed(
"",
`${GLOBAL_HEADER}
<tr><td><a href="/wiki/Old">Started, Undated</a></td>
<td>2021-03-11</td><td></td><td></td></tr>`,
),
);
expect(events).toEqual([]);
});
test("finds the schedule by its shape, not by its position", async () => {
// Three Global panels are on this page: the schedule, and the Mini-Event
// and Joint Firing Drill tabbers whose ids are the same name with `_2` and
// `_3` appended. Taking the first one works only until the page is
// reordered — and those two head their name column `Name`, which is why
// `Name (EN)` is what identifies the right table.
const html = await Bun.file(`${fixture}.html`).text();
expect(html).toContain('id="tabber-Global_version_2"');
expect(html).toContain('id="tabber-Global_version_3"');
const events = await runAdapter(ba, fixture);
// Those two tables publish reward campaigns and firing drills. Nothing from
// them may appear, and nothing from them can: their dates carry a wall
// clock the page never gives a timezone for, so this reader takes neither.
for (const e of events) expect(e.title).not.toMatch(/Exercise|rewards/i);
});
test("fails loudly if the wiki renames the tab or the columns", async () => {
const html = await Bun.file(`${fixture}.html`).text();
expect(blueArchiveWikiParser.canParse(html)).toBe(true);
// A renamed panel leaves the two Global tabbers further down the page still
// matching, so this is a real test of the header check rather than of the
// id: neither of those is headed `Name (EN)`.
expect(
blueArchiveWikiParser.canParse(
html.replace(/id="tabber-Global_version"/g, 'id="tabber-Worldwide"'),
),
).toBe(false);
// A renamed column is the more likely change, and the more dangerous one:
// without this it costs no error at all, just an empty Blue Archive lane.
expect(
blueArchiveWikiParser.canParse(html.replace(/Name \(EN\)/g, "Event")),
).toBe(false);
});
});
+1 -1
View File
@@ -561,7 +561,7 @@ describe("a page that is not UTF-8", () => {
});
describe("requests to one host are spaced", () => {
// Eight of the ten sources are game8.co pages. Each one is inside the
// Eight of the twelve sources are game8.co pages. Each one is inside the
// six-hour-per-source floor and the burst is still the shape a CDN throttles.
const twoOnOneHost = () => [
adapter({ id: "genshin-game8-events", url: "https://game8.co/a" }),