feat: add Girls' Frontline 2, from IOP Wiki
The best date material this project has after wiki.gg: every row states an exact instant on both boundaries and names the zone, so nothing is converted and nothing is assumed. `parseIsoClockRangeUtc` requires that `(UTC)` rather than defaulting to it — a row that ever loses it should drop out instead of landing hours off a boundary the reader is watching in-game. The hazard here is the Server column. CN, EN and JP rows share one table and the Chinese schedule runs about a year ahead, which is the akwiki CN-column problem verbatim: publish EN, skip the rest. Betas are fenced off separately, being dated exactly like everything else and playable by nobody, and the page is an archive back to 2023, so currency is decided against ctx.now as in bawiki. That leaves one live event today — thin, and true. No resetOffsets: the EN boundaries land on three different clocks, which is a patch window rather than a reset hour. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
co-authored by
Claude Opus 5
Claude Opus 5
parent
c1cd1148d9
commit
4dff14087f
@@ -37,8 +37,8 @@ A web app that aggregates live and upcoming events across popular gacha games, p
|
||||
calendar, sorts them by end date or by what the reader is partway through, tracks day-by-day
|
||||
progress on events that repeat daily, and lets a user mark events completed.
|
||||
|
||||
**Status: working app, refreshing itself on a schedule.** Schema, six parsers, fourteen sources across
|
||||
thirteen games, the full interface, offline support, a static server, a Docker image and CI all exist and
|
||||
**Status: working app, refreshing itself on a schedule.** Schema, seven parsers, fifteen sources across
|
||||
fourteen games, the full interface, offline support, a static server, a Docker image and CI all exist and
|
||||
are tested. The refresh runner (`bun run refresh`) fetches, caches raw snapshots and rebuilds the
|
||||
feed; `.github/workflows/refresh.yml` runs it twice a day and commits only when a page actually
|
||||
changed. The SQLite layer and the review queue are still specified in `docs/` but not built, so the
|
||||
@@ -106,8 +106,9 @@ re-verify a sample against the live page afterward.
|
||||
```
|
||||
src/shared/ schema.ts (the contract), time.ts, daily.ts, effort.ts, games.ts, feed.ts
|
||||
custom.ts — reader-authored games and events, and their key spaces
|
||||
src/ingest/ html.ts, dates.ts (twelve formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts
|
||||
parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts, bawiki.ts, holodori.ts — keyed by SITE, not game
|
||||
src/ingest/ html.ts, dates.ts (thirteen formats), merge.ts, sanitize.ts, robots.ts, snapshots.ts
|
||||
parsers/ game8.ts, wikigg.ts, akwiki.ts, fandom.ts, bawiki.ts, holodori.ts,
|
||||
iopwiki.ts — keyed by SITE, not game
|
||||
adapters/ index.ts — SOURCES registry binding url+game+parser, and the sanitize seam
|
||||
src/client/ React app, service worker, manifest
|
||||
state/ progress, daily log, ignores, prefs, sort — all localStorage
|
||||
@@ -118,7 +119,7 @@ src/client/ React app, service worker, manifest
|
||||
theme.ts — dark or light, and what a game hue reads as on each
|
||||
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
|
||||
serve.ts static server + /api/health
|
||||
test/ 620 tests
|
||||
test/ 641 tests
|
||||
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
|
||||
snapshots/ current page per source, rewritten by refresh — see its README
|
||||
```
|
||||
@@ -164,7 +165,7 @@ These come from how gacha games actually schedule things, and they cause most bu
|
||||
year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a
|
||||
recoverable disappointment; a confidently wrong date is the failure this product exists to prevent.
|
||||
- **Parsers are keyed by site, not game.** One `game8` parser serves eight sources and `fandom` two;
|
||||
`wikigg`, `akwiki`, `bawiki` and `holodoriwiki` serve one each — the first two share a host family
|
||||
`wikigg`, `akwiki`, `bawiki`, `holodoriwiki` and `iopwiki` serve one each — the first two share a host family
|
||||
and have entirely different templates, and the last two are both Miraheze wikis whose page
|
||||
templates have nothing in common. Adding a source for a known site is one `SOURCES` entry; a new
|
||||
site is a parser module.
|
||||
@@ -242,7 +243,7 @@ Sources are community wikis. Treat them as a guest would:
|
||||
- Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
|
||||
- One request per source per refresh cycle, minimum 6 hours apart.
|
||||
- **Space requests to one host**, honouring its `Crawl-delay` and defaulting to 2s. Eight of the
|
||||
fourteen sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
|
||||
fifteen sources are game8.co pages, so the per-source floor alone still permits one cycle to arrive
|
||||
as eight back-to-back requests to a single site — which is the shape an edge network throttles, and
|
||||
what a burst looks like from the far end regardless of our intent.
|
||||
- Send `If-None-Match` / `If-Modified-Since`; treat `304` as "skip, unchanged".
|
||||
@@ -269,7 +270,8 @@ refresh from an address game8 will serve, or to find those games another source.
|
||||
|
||||
A source whose ToS forbids automated access does not get an adapter. Flag it and ask.
|
||||
|
||||
**Sources assessed and declined** (2026-08-17), so these are not re-litigated each pass:
|
||||
**Sources assessed and declined** (2026-08-17, extended 2026-08-19), so these are not
|
||||
re-litigated each pass:
|
||||
|
||||
| Source | Verdict |
|
||||
|---|---|
|
||||
@@ -281,6 +283,7 @@ A source whose ToS forbids automated access does not get an adapter. Flag it and
|
||||
| `fategrandorder.fandom.com` | **Built** (2026-08-18), via `api.php` like Reverse: 1999 — but off `Event_List_(US)`, **not** `Event_List`, which is the Japanese server. See § Fandom below |
|
||||
| `holodori.wiki` | **Built** (2026-08-18), from the rendered `/wiki/Events` page. Miraheze again, so the same call as Blue Archive; CC BY-SA 4.0, no `Content-Signal`, no `Crawl-delay` for `*` |
|
||||
| `prydwen.gg`, `gametora.com` | **Cleared, unbuilt.** `User-agent: *` allows the paths we would want. prydwen sets `Crawl-delay: 10`, far below our one-per-6h |
|
||||
| `iopwiki.com` | **Built** (2026-08-19), Girls' Frontline 2 — see § IOP Wiki below. `robots.txt` is two lines, `User-agent: *` and `Crawl-Delay: 20`, no `Disallow` anywhere |
|
||||
|
||||
`.github/ISSUE_TEMPLATE/feature_request.yml` points readers at that table by heading, so a source
|
||||
request can be checked against it before anyone writes it up — the loudest feedback on the first
|
||||
@@ -391,6 +394,26 @@ calendar of deadlines is for. An `Unknown` **end** is kept, though — that is `
|
||||
unlike `bawiki.ts` this parser does not drop a started-but-undated row, because the heading has
|
||||
already said the event is running.
|
||||
|
||||
**IOP Wiki: the Server column is the whole safety story.** `iopwiki.com/wiki/GFL2_Events` is the
|
||||
best date material here after wiki.gg — every row states an exact instant on both boundaries *and*
|
||||
names the zone (`2026-08-06 13:00 - 2026-08-26 22:59 (UTC)`), so `parseIsoClockRangeUtc` converts
|
||||
nothing and both sides are `exact`. Three things about it:
|
||||
|
||||
- **CN, EN and JP rows share one table**, and the Chinese schedule runs about a year ahead. This is
|
||||
the `akwiki` CN-column hazard verbatim and gets the same answer: publish `EN`, skip the rest. It
|
||||
would be wrong by *months* on a row that otherwise looks perfect.
|
||||
- **`Betas` is a section, not an event type.** Closed beta rows are dated exactly like everything
|
||||
else and would parse cleanly onto a calendar of things nobody can play. Fenced on the `<h2>`.
|
||||
- **The page is an archive**, 145 rows back to 2023, so inclusion is decided against `ctx.now` as in
|
||||
`bawiki.ts`. The lane is therefore thin by design — one live event on a quiet week is the truth,
|
||||
not a gap.
|
||||
|
||||
The zone requirement is deliberate: `parseIsoClockRangeUtc` refuses a row that loses its `(UTC)`
|
||||
rather than assuming it, exactly as `parseSlashClockZone` does. GFL2 takes no `resetOffsets`: its EN
|
||||
boundaries land on three different clocks (22:59, 08:59 and 02:59 UTC), which is a patch window
|
||||
rather than a reset hour — Arknights and Reverse: 1999 each earned an override from a single
|
||||
boundary their whole page agreed on.
|
||||
|
||||
`scripts/refresh-sources.ts` enforces all of the above in code — the 6h floor, one request, no
|
||||
retries, conditional headers, per-host spacing, robots (failing closed when `robots.txt` cannot be
|
||||
read). Anything that would make it fetch more often is a change to this section first.
|
||||
|
||||
+11
-3
@@ -10,7 +10,7 @@ import { z } from "zod";
|
||||
|
||||
export const GameId = z.enum([
|
||||
"genshin", "hsr", "zzz", "wuwa", "arknights", "endfield", "nte", "nikki", "p5x", "r1999",
|
||||
"ba", "fgo", "holodori",
|
||||
"ba", "fgo", "holodori", "gfl2",
|
||||
]);
|
||||
|
||||
export const EventType = z.enum([
|
||||
@@ -308,8 +308,16 @@ nothing to add, which is the point worth carrying to the next game: an override
|
||||
the game moves nobody, while the same override added a year later re-labels every tick already
|
||||
logged under the old clock. Get it right at introduction or accept a migration.
|
||||
|
||||
Infinity Nikki, P5X and Blue Archive carry **no `resetOffsets` entry**, so they take the regional
|
||||
default. That is an assumption, not a verified server map — none of the three sources states one.
|
||||
Infinity Nikki, P5X and Blue Archive carry **no `resetOffsets`
|
||||
entry**, so they take the regional default. That is an assumption, not a verified server map — none
|
||||
of those sources states one. The 2026-08-19 addition below is worth separating out, because it is
|
||||
silent for a different reason than the usual one:
|
||||
|
||||
- **Girls' Frontline 2** states an exact UTC instant on every boundary, so the data is there and
|
||||
still settles nothing: its EN events end at 22:59 (33 of them), 08:59 (11) and 02:59 (5). Arknights
|
||||
and Reverse: 1999 each earned an override from a single boundary the whole page agreed on; three
|
||||
of them is a patch window, not a reset hour.
|
||||
|
||||
Blue Archive is the case where the assumption is most likely wrong and still the right entry: the
|
||||
game does run one worldwide server, but `bluearchive.wiki` states no time of day and no timezone
|
||||
anywhere on the page, so there is no offset to read off it. Confirm these against the games before
|
||||
|
||||
+2
-8
@@ -54,6 +54,7 @@ Consequences worth internalising:
|
||||
| `fandom` | Fandom wikis via the MediaWiki `action=parse` API — two page templates: `Event \| Time Period \| Version` wikitables, and FGO's picture-fenced `ONGOING EVENTS` blocks | Reverse: 1999, Fate/Grand Order |
|
||||
| `bawiki` | bluearchive.wiki's rendered `/wiki/Events` — a JP/Global tabber over `Name (EN) \| Start date \| End date \| Notes` wikitables | Blue Archive |
|
||||
| `holodoriwiki` | holodori.wiki's rendered `/wiki/Events` — `Current Events` and `Past Events` wikitables over `Event \| Type \| Start Date \| End Date` | hololive Dreams |
|
||||
| `iopwiki` | iopwiki.com's `gf-table event-period` tables — `Title \| Period (start/end) \| Server \| Type \| Comment`, one table per event and one row per server | Girls' Frontline 2 |
|
||||
|
||||
`wikigg` is the better shape by a distance: it emits ISO timestamps with one timer per server
|
||||
region, so its events carry exact precision and real `regionEnds`. Prefer a source like that over a
|
||||
@@ -140,6 +141,7 @@ All live in `src/ingest/dates.ts`, each returning null rather than inferring any
|
||||
| `parseOrdinalDateTimeRange` | `November 9th, 05:00 - December 4th, 2023, 04:59 (UTC-5)` (ordinal days, stated offset) | Reverse: 1999 |
|
||||
| `parseIsoDay` | `2026-08-04` (one boundary per column, so nothing to split) | Blue Archive |
|
||||
| `parseSlashClockZone` | `08/17/2026 8:00PM (JST)` (one boundary per column, 12-hour clock, **named** zone) | hololive Dreams |
|
||||
| `parseIsoClockRangeUtc` | `2026-08-06 13:00 - 2026-08-26 22:59 (UTC)` (whole range in one cell, **zone required**, nothing converted) | Girls' Frontline 2 |
|
||||
| `parseOpenRange` | `Jul. 24, 2026 - End of 4.6`, `July 10, 2026 - Permanent` | Star Rail, Wuthering Waves |
|
||||
|
||||
**A day-precision result is 00:00Z, and that is a placeholder rather than a time.** Every reader
|
||||
@@ -338,14 +340,6 @@ start-date proximity check is the actual guard; the title threshold is deliberat
|
||||
that "Stygian Onslaught" and "Stygian Onslaught Event" collapse into one row rather than showing
|
||||
the user a duplicate.
|
||||
|
||||
**Agreement raises confidence (+0.10) only across different `sourceId`s.** The same row seen twice
|
||||
in one document is not corroboration.
|
||||
|
||||
**Disagreement is surfaced, never averaged.** Two sources whose `endsAt` differ by more than 24
|
||||
hours produce a `conflicts` entry; the pipeline routes those to quarantine. Splitting the difference
|
||||
between two dates would produce a value neither source asserts — the worst possible answer for a
|
||||
product whose promise is date accuracy.
|
||||
|
||||
## Stage 4 — validate
|
||||
|
||||
Zod parse against `GachaEvent`, then calendar sanity rules. Anything failing a hard rule goes to
|
||||
|
||||
+2
-2
@@ -59,7 +59,7 @@ have a source — this table is the launch scope, not a roster to keep in sync.
|
||||
|
||||
Adding a game must require no change to `GachaEvent` — only a `GameId` entry, its `games.ts`
|
||||
metadata, and a source registration. That is the test of whether the data model is right, and it has
|
||||
held: the thirteen games here have cost the event schema nothing. Per-game *metadata* does occasionally
|
||||
held: the fourteen games here have cost the event schema nothing. Per-game *metadata* does occasionally
|
||||
grow (Reverse: 1999 needed `resetHourLocal` for a 05:00 reset), which is a different file and moves
|
||||
no stored key. A game may have several sources; see `docs/INGESTION.md` § Three layers.
|
||||
|
||||
@@ -90,7 +90,7 @@ default `"game"` — the board every existing reader already has). Merged, the o
|
||||
`endingSoonestFirst`, the same comparator behind the list's "Ending soonest" (F2), so the two views
|
||||
cannot mean different things by the same words and an unannounced end still sorts behind every dated
|
||||
one rather than claiming a place in the queue. Merged there is also no lane heading to say whose
|
||||
event a bar is, and hue alone cannot answer that once thirteen games share a stack — so each bar
|
||||
event a bar is, and hue alone cannot answer that once fourteen games share a stack — so each bar
|
||||
carries its game's short name, with the full name in its tooltip. A bar too narrow for a tag and a
|
||||
title both keeps the title: a chopped game name reads as a broken word, and the hue and the tooltip
|
||||
still answer it.
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
[
|
||||
{
|
||||
"id": "gfl2:moonshroud-requiem:2026-08-06",
|
||||
"game": "gfl2",
|
||||
"title": "Moonshroud Requiem",
|
||||
"type": "banner",
|
||||
"summary": null,
|
||||
"startsAt": "2026-08-06T13:00:00.000Z",
|
||||
"startPrecision": "exact",
|
||||
"endsAt": "2026-08-26T22:59:00.000Z",
|
||||
"endPrecision": "exact",
|
||||
"regionScoped": false,
|
||||
"regionEnds": null,
|
||||
"sourceUrl": "https://iopwiki.com/wiki/GFL2_Events",
|
||||
"sourceId": "gfl2-iopwiki-events",
|
||||
"status": "published",
|
||||
"confidence": 0.95,
|
||||
"extractionMethod": "parser",
|
||||
"version": 1,
|
||||
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||
}
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -123,6 +123,16 @@ const SOURCES: SourceSpec[] = [
|
||||
url: "https://holodori.wiki/wiki/Events",
|
||||
parserId: "holodoriwiki",
|
||||
},
|
||||
{
|
||||
id: "gfl2-iopwiki-events",
|
||||
game: "gfl2",
|
||||
// IOP Wiki, the Girls' Frontline universe wiki. Its robots.txt is two lines
|
||||
// — `User-agent: *` and `Crawl-Delay: 20` — with no Disallow anywhere, and
|
||||
// the page answers our own User-Agent with a 200 and a real `Last-Modified`,
|
||||
// so the conditional request below costs it a 304 on an unchanged day.
|
||||
url: "https://iopwiki.com/wiki/GFL2_Events",
|
||||
parserId: "iopwiki",
|
||||
},
|
||||
];
|
||||
|
||||
function toAdapter(spec: SourceSpec): Adapter {
|
||||
|
||||
@@ -479,3 +479,40 @@ export function parseSlashClockZone(input: string): ParsedInstant | null {
|
||||
);
|
||||
return value === null ? null : { iso: value, precision: "exact" };
|
||||
}
|
||||
|
||||
/**
|
||||
* "2025-01-16 17:00 - 2025-02-06 02:59 (UTC)" → both instants, exact precision.
|
||||
*
|
||||
* IOP Wiki's GFL2 event tables put the whole period in one cell and — unusually
|
||||
* for a community wiki — state the zone on every one of them. That makes this
|
||||
* the only range reader here that converts nothing: the source has already done
|
||||
* it, so both boundaries are exact and no offset is assumed anywhere.
|
||||
*
|
||||
* **The zone is required**, for the reason `parseSlashClockZone` requires its
|
||||
* own: a cell that states a wall clock and no zone is a missing fact, not an
|
||||
* invitation to read it as UTC. All 145 rows on the page carry `(UTC)` today,
|
||||
* so the day one loses it that row should drop out rather than land hours off
|
||||
* on a boundary the reader is standing in the game watching.
|
||||
*
|
||||
* Anchored at the start so a range cannot be found inside prose or a slug; left
|
||||
* open at the end because the cell also carries an ICS export widget, whose
|
||||
* markup the caller strips but whose container survives as trailing whitespace.
|
||||
*/
|
||||
export function parseIsoClockRangeUtc(
|
||||
input: string,
|
||||
): { start: ParsedInstant; end: ParsedInstant } | null {
|
||||
const re =
|
||||
/^\s*(\d{4})-(\d{2})-(\d{2})\s+(\d{1,2}):(\d{2})\s*[-–—]\s*(\d{4})-(\d{2})-(\d{2})\s+(\d{1,2}):(\d{2})\s*\(UTC\)/i;
|
||||
const m = re.exec(input);
|
||||
if (!m) return null;
|
||||
|
||||
const n = (i: number) => Number(m[i]);
|
||||
const startIso = iso(n(1), n(2), n(3), n(4), n(5));
|
||||
const endIso = iso(n(6), n(7), n(8), n(9), n(10));
|
||||
if (startIso === null || endIso === null) return null;
|
||||
|
||||
return {
|
||||
start: { iso: startIso, precision: "exact" },
|
||||
end: { iso: endIso, precision: "exact" },
|
||||
};
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ import { blueArchiveWikiParser } from "./bawiki.ts";
|
||||
import { fandomParser } from "./fandom.ts";
|
||||
import { game8Parser } from "./game8.ts";
|
||||
import { holodoriWikiParser } from "./holodori.ts";
|
||||
import { iopWikiParser } from "./iopwiki.ts";
|
||||
import { wikiGgParser } from "./wikigg.ts";
|
||||
import type { SourceParser } from "./types.ts";
|
||||
|
||||
@@ -18,6 +19,7 @@ export const PARSERS: SourceParser[] = [
|
||||
fandomParser,
|
||||
blueArchiveWikiParser,
|
||||
holodoriWikiParser,
|
||||
iopWikiParser,
|
||||
];
|
||||
|
||||
export function parserById(id: string): SourceParser | undefined {
|
||||
@@ -30,4 +32,5 @@ export { blueArchiveWikiParser } from "./bawiki.ts";
|
||||
export { fandomParser } from "./fandom.ts";
|
||||
export { game8Parser } from "./game8.ts";
|
||||
export { holodoriWikiParser } from "./holodori.ts";
|
||||
export { iopWikiParser } from "./iopwiki.ts";
|
||||
export { wikiGgParser } from "./wikigg.ts";
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
import { eventId, type EventType, type GachaEvent } from "../../shared/schema.ts";
|
||||
import { parseIsoClockRangeUtc } from "../dates.ts";
|
||||
import { scanDocument, type DocNode, type TableNode } from "../html.ts";
|
||||
import type { ParseContext } from "../adapters/types.ts";
|
||||
import { inferType } from "./game8.ts";
|
||||
import type { SourceParser } from "./types.ts";
|
||||
|
||||
/**
|
||||
* IOP Wiki's Girls' Frontline 2 event schedule (`iopwiki.com/wiki/GFL2_Events`).
|
||||
*
|
||||
* The page is an `<h3>` per event, each followed by one
|
||||
* `<table class="gf-table event-period">` whose rows are that event's runs on
|
||||
* each server:
|
||||
*
|
||||
* Title | Period (start/end) | Server | Type | Comment
|
||||
* Amidst Wings of Gray | 2025-01-16 17:00 - 2025-02-06 02:59 (UTC) | EN | …
|
||||
*
|
||||
* Three things about it decide the code below.
|
||||
*
|
||||
* **The Server column is the whole safety story.** CN, EN and JP rows sit in
|
||||
* the same table, and the Chinese schedule runs roughly a year ahead of ours.
|
||||
* This is the `akwiki` CN-column hazard verbatim and gets the same answer:
|
||||
* publish `EN` and skip every other row. A CN date on a Global calendar is a
|
||||
* confidently wrong date, not a near miss — and here it would be wrong by
|
||||
* months, on a row that otherwise looks perfect.
|
||||
*
|
||||
* **The zone is stated on every row**, which is what makes this the best date
|
||||
* material in the project after wiki.gg: `exact` precision on both boundaries
|
||||
* with nothing converted or assumed. `parseIsoClockRangeUtc` requires the
|
||||
* `(UTC)` marker rather than defaulting to it, so a row that ever loses it
|
||||
* drops out instead of landing hours off.
|
||||
*
|
||||
* **`Betas` is a section, not an event type.** The page's `<h2>`s are
|
||||
* `Main Events`, `Minor Events`, `Betas` and `References`; closed beta rows are
|
||||
* dated exactly like everything else and would parse cleanly onto a calendar of
|
||||
* things nobody can do. Inclusion is fenced on the heading, as it is in
|
||||
* `holodori.ts`.
|
||||
*
|
||||
* **This is an archive, not a schedule.** 145 rows go back to 2023, 51 of them
|
||||
* ours, of which one had not ended when the fixture was captured. So inclusion
|
||||
* is decided against `ctx.now` exactly as it is in `bawiki.ts` — there is no
|
||||
* "ongoing" heading to gate on, and nothing downstream drops a finished event.
|
||||
* That makes the lane thin by design: the wiki publishes a patch when it is
|
||||
* announced, so a quiet week is one live event, which is the truth rather than
|
||||
* a gap.
|
||||
*
|
||||
* Titles come from the row's own `Title` cell and never from the `<h3>` above
|
||||
* it: those headings carry the CN and EN names as one slash pair
|
||||
* ("Exotic Cadence/Amidst Wings of Gray"), so a parser that took the heading
|
||||
* would publish both names for every event on the calendar. The row cell is
|
||||
* already localised to its own server.
|
||||
*
|
||||
* The period cell also carries an ICS export widget — a `<script>` restating
|
||||
* the same two instants as `icsStart`/`icsEnd`. That is a useful independent
|
||||
* check when verifying a fixture by hand, but it is script text rather than
|
||||
* content, so it is stripped before reading rather than parsed: `html.ts`'s
|
||||
* `text()` strips tags without stripping what a `<script>` contains, and a
|
||||
* date reader anchored at the start of the cell would otherwise be reading the
|
||||
* range with a page of JavaScript stuck to the end of it.
|
||||
*/
|
||||
|
||||
/** Script and style bodies are markup we never want as text. */
|
||||
const SCRIPTS = /<(script|style)\b[^>]*>[\s\S]*?<\/\1>/gi;
|
||||
|
||||
/** Sections whose rows are dated but not playable. */
|
||||
const EXCLUDED_SECTIONS = new Set(["betas", "references"]);
|
||||
|
||||
/**
|
||||
* The source's own event taxonomy, mapped onto ours.
|
||||
*
|
||||
* Reading the stated `Type` beats inferring one from the title: "Moonshroud
|
||||
* Requiem" says nothing about being a character banner, and the page says so
|
||||
* outright. Anything unlisted falls back to `inferType` over the title and the
|
||||
* stated type together, so a new value the wiki invents gets word-inferred
|
||||
* rather than silently flattened to "other".
|
||||
*/
|
||||
const TYPES: Record<string, EventType> = {
|
||||
"character event": "banner",
|
||||
"main story event": "story",
|
||||
"combat event": "challenge",
|
||||
"missions event": "other",
|
||||
"special event": "other",
|
||||
"warm-up event": "other",
|
||||
"popularity contest": "other",
|
||||
};
|
||||
|
||||
interface Columns {
|
||||
title: number;
|
||||
period: number;
|
||||
server: number;
|
||||
type: number;
|
||||
comment: number | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where each column sits, read off the header row rather than counted.
|
||||
*
|
||||
* Returns null for every table that is not an event period table, which is what
|
||||
* keeps the page's layout tables — and its navigation — from being read as a
|
||||
* schedule. It is also the check `canParse` makes: if the header row is ever
|
||||
* renamed the source fails loudly instead of emptying the lane.
|
||||
*/
|
||||
function columns(headers: string[]): Columns | null {
|
||||
const at = (match: (h: string) => boolean) => {
|
||||
const i = headers.findIndex((h) => match(h.toLowerCase().trim()));
|
||||
return i < 0 ? undefined : i;
|
||||
};
|
||||
|
||||
const title = at((h) => h === "title");
|
||||
// "Period (start/end)" — matched on the word rather than the whole string so
|
||||
// an editor adding or dropping the parenthetical does not empty the lane.
|
||||
const period = at((h) => h.startsWith("period"));
|
||||
const server = at((h) => h === "server");
|
||||
const type = at((h) => h === "type");
|
||||
if (
|
||||
title === undefined ||
|
||||
period === undefined ||
|
||||
server === undefined ||
|
||||
type === undefined
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return { title, period, server, type, comment: at((h) => h === "comment") };
|
||||
}
|
||||
|
||||
/** The server whose schedule our readers are on. */
|
||||
const OUR_SERVER = "en";
|
||||
|
||||
export function parseIopWikiEventsPage(
|
||||
html: string,
|
||||
ctx: ParseContext,
|
||||
): GachaEvent[] {
|
||||
const nodes: DocNode[] = scanDocument(html.replace(SCRIPTS, " "));
|
||||
const out: GachaEvent[] = [];
|
||||
const seen = new Set<string>();
|
||||
let section = "";
|
||||
|
||||
for (const node of nodes) {
|
||||
if (node.kind === "h2") {
|
||||
section = node.text.toLowerCase();
|
||||
continue;
|
||||
}
|
||||
if (node.kind !== "table") continue;
|
||||
if (EXCLUDED_SECTIONS.has(section)) continue;
|
||||
|
||||
const cols = columns(node.headers);
|
||||
if (cols === null) continue;
|
||||
|
||||
for (const event of readRows(node, cols, ctx)) {
|
||||
// One event can be listed twice on the page — a run and its rerun share a
|
||||
// title, and the ID separates them by start date. Two rows that really do
|
||||
// agree on both are the same run stated twice, and the second is dropped
|
||||
// rather than merged: they are keyed identically downstream anyway, and
|
||||
// one completion mark for two rows is the collision the ID scheme exists
|
||||
// to avoid.
|
||||
if (seen.has(event.id)) continue;
|
||||
seen.add(event.id);
|
||||
out.push(event);
|
||||
}
|
||||
}
|
||||
|
||||
return out.sort((a, b) =>
|
||||
a.startsAt === b.startsAt
|
||||
? a.id.localeCompare(b.id)
|
||||
: a.startsAt.localeCompare(b.startsAt),
|
||||
);
|
||||
}
|
||||
|
||||
function readRows(
|
||||
table: TableNode,
|
||||
cols: Columns,
|
||||
ctx: ParseContext,
|
||||
): GachaEvent[] {
|
||||
const out: GachaEvent[] = [];
|
||||
|
||||
for (const row of table.rows) {
|
||||
const cell = (i: number | undefined) =>
|
||||
i === undefined ? "" : (row[i] ?? "").trim();
|
||||
|
||||
// The header row comes back as cells like any other; it identifies itself.
|
||||
if (cell(cols.server).toLowerCase() === "server") continue;
|
||||
if (cell(cols.server).toLowerCase() !== OUR_SERVER) continue;
|
||||
|
||||
const title = cell(cols.title);
|
||||
if (title.length === 0) continue;
|
||||
|
||||
const range = parseIsoClockRangeUtc(cell(cols.period));
|
||||
if (range === null) continue;
|
||||
if (range.end.iso <= range.start.iso) continue;
|
||||
// Live and upcoming only. Everything above is history the wiki keeps and a
|
||||
// calendar of deadlines does not want — see the archive note above.
|
||||
if (Date.parse(range.end.iso) < Date.parse(ctx.now)) continue;
|
||||
|
||||
const stated = cell(cols.type);
|
||||
const comment = cell(cols.comment);
|
||||
|
||||
out.push({
|
||||
id: eventId(ctx.game, title, range.start.iso),
|
||||
game: ctx.game,
|
||||
title,
|
||||
type: TYPES[stated.toLowerCase()] ?? inferType(`${title} ${stated}`),
|
||||
summary: comment.length > 0 ? comment : null,
|
||||
startsAt: range.start.iso,
|
||||
startPrecision: range.start.precision,
|
||||
endsAt: range.end.iso,
|
||||
endPrecision: range.end.precision,
|
||||
// GFL2's EN build is a single worldwide server and this page states one
|
||||
// instant per run, in UTC, with no per-region column anywhere. There is
|
||||
// nothing region-scoped here to report — the Server column splits game
|
||||
// *releases* (CN/EN/JP), not our asia/america/europe regions.
|
||||
regionScoped: false,
|
||||
regionEnds: null,
|
||||
sourceUrl: ctx.sourceUrl,
|
||||
sourceId: ctx.sourceId,
|
||||
status: "published",
|
||||
// The strongest evidence this codebase accepts: both boundaries exact,
|
||||
// both stated in UTC by the source itself. Only wiki.gg scores here.
|
||||
confidence: 0.95,
|
||||
extractionMethod: "parser",
|
||||
version: 1,
|
||||
firstSeenAt: ctx.now,
|
||||
updatedAt: ctx.now,
|
||||
});
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
export const iopWikiParser: SourceParser = {
|
||||
id: "iopwiki",
|
||||
label: "IOP Wiki",
|
||||
canParse(html: string): boolean {
|
||||
// At least one event period table, found the same way `parse` finds it.
|
||||
// "A table exists" would prove nothing on a page with 47 of them, most of
|
||||
// them layout; asserting that the Title/Period/Server header row is still
|
||||
// findable is what turns a redesign into a failed source rather than a
|
||||
// silently empty lane.
|
||||
return scanDocument(html.replace(SCRIPTS, " ")).some(
|
||||
(n) => n.kind === "table" && columns(n.headers) !== null,
|
||||
);
|
||||
},
|
||||
parse: parseIopWikiEventsPage,
|
||||
};
|
||||
@@ -114,6 +114,15 @@ export const GAMES: Record<GameId, GameMeta> = {
|
||||
// reader has a day key for a game the app has never shipped, whereas adding
|
||||
// the same override later would re-label ticks they had already logged.
|
||||
holodori: { id: "holodori", name: "hololive Dreams", short: "holodori", hue: "#5FD3F3", studio: "QualiArts / COVER", dailyTasks: "Daily missions, stamina", resetOffsets: { asia: 9, america: 9, europe: 9 } },
|
||||
// No `resetOffsets`, and unusually the source is not silent here — it is
|
||||
// ambiguous, which comes to the same answer. Every IOP Wiki row states an
|
||||
// exact UTC instant, but the EN boundaries land on three different clocks:
|
||||
// 33 events end at 22:59, 11 at 08:59 and 5 at 02:59. Arknights and
|
||||
// Reverse: 1999 each earned an override from a single boundary the whole page
|
||||
// agreed on, one minute or second before a 04:00 local reset; three of them
|
||||
// is a patch window, not a reset hour. So this game takes the regional
|
||||
// default until something states its server clock outright.
|
||||
gfl2: { id: "gfl2", name: "Girls' Frontline 2: Exilium", short: "GFL2", hue: "#A9C23F", studio: "Sunborn", dailyTasks: "Daily missions" },
|
||||
};
|
||||
|
||||
export const GAME_LIST: GameMeta[] = Object.values(GAMES);
|
||||
|
||||
@@ -14,6 +14,7 @@ export const GameId = z.enum([
|
||||
"ba", // Blue Archive
|
||||
"fgo", // Fate/Grand Order
|
||||
"holodori", // hololive Dreams
|
||||
"gfl2", // Girls' Frontline 2: Exilium
|
||||
]);
|
||||
export type GameId = z.infer<typeof GameId>;
|
||||
|
||||
|
||||
@@ -42,6 +42,7 @@ const CASES: Array<{ adapter: Adapter; fixture: string }> = [
|
||||
{ adapter: adapter("ba-bawiki-events"), fixture: "fixtures/ba/bawiki-events-2026-08-17" },
|
||||
{ adapter: adapter("fgo-fandom-events"), fixture: "fixtures/fgo/fandom-events-2026-08-18" },
|
||||
{ adapter: adapter("holodori-holodoriwiki-events"), fixture: "fixtures/holodori/holodoriwiki-events-2026-08-18" },
|
||||
{ adapter: adapter("gfl2-iopwiki-events"), fixture: "fixtures/gfl2/iopwiki-events-2026-08-19" },
|
||||
];
|
||||
|
||||
async function runAdapter(adapter: Adapter, fixture: string) {
|
||||
@@ -1059,3 +1060,127 @@ describe("Fate/Grand Order wiki", () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("IOP Wiki (Girls' Frontline 2)", () => {
|
||||
const fixture = "fixtures/gfl2/iopwiki-events-2026-08-19";
|
||||
const gfl2 = adapter("gfl2-iopwiki-events");
|
||||
|
||||
function parse(html: string) {
|
||||
return gfl2.parse(html, {
|
||||
now: NOW,
|
||||
sourceUrl: gfl2.url,
|
||||
sourceId: gfl2.id,
|
||||
game: gfl2.game,
|
||||
});
|
||||
}
|
||||
|
||||
const HEADER =
|
||||
"<tr><th>Title</th><th>Period (start/end)</th><th>Server</th><th>Type</th><th>Comment</th></tr>";
|
||||
|
||||
const row = (title: string, period: string, server: string, type = "Character Event") =>
|
||||
`<tr><td>${title}</td><td>${period}</td><td>${server}</td><td>${type}</td><td></td></tr>`;
|
||||
|
||||
/** A stand-in page with the heading-fenced shape this parser navigates. */
|
||||
const paged = (main: string, betas = "") =>
|
||||
`<h2>Main Events</h2><h3>Event</h3><table class="gf-table event-period">${main}</table>
|
||||
<h2>Betas</h2><table class="gf-table event-period">${betas}</table>`;
|
||||
|
||||
test("publishes only the EN server's rows", async () => {
|
||||
// The hazard this source has, and the only one that could put a
|
||||
// confidently wrong date on the calendar: CN, EN and JP rows share one
|
||||
// table and the Chinese schedule runs about a year ahead. Counted off the
|
||||
// fixture: 145 rows, 51 of them EN.
|
||||
const events = parse(
|
||||
paged(
|
||||
HEADER +
|
||||
row("异乡乐徽", "2027-08-06 10:00 - 2027-08-26 09:59 (UTC)", "CN") +
|
||||
row("Moonshroud Requiem", "2026-08-06 13:00 - 2026-08-26 22:59 (UTC)", "EN") +
|
||||
row("失意の翼の中で", "2026-11-06 13:00 - 2026-11-26 22:59 (UTC)", "JP"),
|
||||
),
|
||||
);
|
||||
expect(events.map((e) => e.title)).toEqual(["Moonshroud Requiem"]);
|
||||
});
|
||||
|
||||
test("fences out the Betas section", async () => {
|
||||
// Closed beta rows are dated exactly like everything else and parse
|
||||
// cleanly — onto a calendar of things nobody can play.
|
||||
const events = parse(
|
||||
paged(
|
||||
HEADER + row("Moonshroud Requiem", "2026-08-06 13:00 - 2026-08-26 22:59 (UTC)", "EN"),
|
||||
HEADER +
|
||||
row("Closed Beta Test (Sunborn)", "2026-08-10 12:00 - 2026-08-19 08:00 (UTC)", "EN", "Open Beta"),
|
||||
),
|
||||
);
|
||||
expect(events.map((e) => e.title)).toEqual(["Moonshroud Requiem"]);
|
||||
});
|
||||
|
||||
test("takes both boundaries exact, in the UTC the page states", async () => {
|
||||
const events = await runAdapter(gfl2, fixture);
|
||||
expect(events).toHaveLength(1);
|
||||
const [live] = events;
|
||||
expect(live?.title).toBe("Moonshroud Requiem");
|
||||
expect(live?.startsAt).toBe("2026-08-06T13:00:00.000Z");
|
||||
expect(live?.endsAt).toBe("2026-08-26T22:59:00.000Z");
|
||||
expect(live?.startPrecision).toBe("exact");
|
||||
expect(live?.endPrecision).toBe("exact");
|
||||
// Nothing was converted or assumed, which is what earns the top score.
|
||||
expect(live?.confidence).toBe(0.95);
|
||||
});
|
||||
|
||||
test("drops a row that loses its zone rather than reading it as UTC", async () => {
|
||||
// The page states `(UTC)` on all 145 rows. If one ever stops, that row is
|
||||
// a wall clock with no zone — a missing fact, and the same call
|
||||
// `parseSlashClockZone` makes on the hololive wiki.
|
||||
const events = parse(
|
||||
paged(HEADER + row("Zoneless", "2026-08-06 13:00 - 2026-08-26 22:59", "EN")),
|
||||
);
|
||||
expect(events).toEqual([]);
|
||||
});
|
||||
|
||||
test("prefers the stated Type over a guess from the title", async () => {
|
||||
// "Moonshroud Requiem" says nothing about being a character banner; the
|
||||
// page's own Type column does.
|
||||
const events = parse(
|
||||
paged(
|
||||
HEADER +
|
||||
row("Moonshroud Requiem", "2026-08-06 13:00 - 2026-08-26 22:59 (UTC)", "EN") +
|
||||
row("Corposant Part 1", "2026-08-07 13:00 - 2026-08-27 22:59 (UTC)", "EN", "Main Story Event") +
|
||||
row("Endless Projections", "2026-08-08 13:00 - 2026-08-28 22:59 (UTC)", "EN", "Combat Event"),
|
||||
),
|
||||
);
|
||||
expect(events.map((e) => e.type)).toEqual(["banner", "story", "challenge"]);
|
||||
});
|
||||
|
||||
test("word-infers a Type the wiki has not used before", async () => {
|
||||
// An unmapped value must not silently flatten to "other" — that is how a
|
||||
// taxonomy change goes unnoticed.
|
||||
const events = parse(
|
||||
paged(
|
||||
HEADER +
|
||||
row("Anniversary Login", "2026-08-06 13:00 - 2026-08-26 22:59 (UTC)", "EN", "Sign-in Event"),
|
||||
),
|
||||
);
|
||||
expect(events[0]?.type).toBe("login");
|
||||
});
|
||||
|
||||
test("publishes nothing from the back catalogue", async () => {
|
||||
// 145 rows go back to 2023 and nothing downstream drops a finished event,
|
||||
// so the currency gate is this parser's job.
|
||||
for (const e of await runAdapter(gfl2, fixture)) {
|
||||
expect(e.endsAt === null || e.endsAt > NOW).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test("a redesign fails the source instead of emptying the lane", () => {
|
||||
// 47 tables on this page, most of them layout. "A table exists" proves
|
||||
// nothing; that the Title/Period/Server header row is still findable does.
|
||||
expect(() =>
|
||||
gfl2.parse("<h2>Main Events</h2><table><tr><th>Name</th><th>When</th></tr></table>", {
|
||||
now: NOW,
|
||||
sourceUrl: gfl2.url,
|
||||
sourceId: gfl2.id,
|
||||
game: gfl2.game,
|
||||
}),
|
||||
).toThrow(/redesigned/);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
parseSlashClockZone,
|
||||
parseSlashDateTimeRange,
|
||||
parseYearFirstSlashRange,
|
||||
parseIsoClockRangeUtc,
|
||||
} from "../src/ingest/dates.ts";
|
||||
|
||||
describe("parseMonthDayYear", () => {
|
||||
@@ -416,3 +417,58 @@ describe("parseSlashClockZone", () => {
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseIsoClockRangeUtc", () => {
|
||||
test("reads both boundaries exact, converting nothing", () => {
|
||||
// IOP Wiki states the zone itself, so this is the one range reader here
|
||||
// that assumes no offset anywhere.
|
||||
const range = parseIsoClockRangeUtc(
|
||||
"2026-08-06 13:00 - 2026-08-26 22:59 (UTC)",
|
||||
);
|
||||
expect(range?.start.iso).toBe("2026-08-06T13:00:00.000Z");
|
||||
expect(range?.end.iso).toBe("2026-08-26T22:59:00.000Z");
|
||||
expect(range?.start.precision).toBe("exact");
|
||||
expect(range?.end.precision).toBe("exact");
|
||||
});
|
||||
|
||||
test("accepts the en dash and the non-breaking spaces the page emits", () => {
|
||||
// The wiki writes ` - `, which `text()` decodes to spaces.
|
||||
expect(
|
||||
parseIsoClockRangeUtc("2025-01-16 17:00 – 2025-02-06 02:59 (UTC)")?.end
|
||||
.iso,
|
||||
).toBe("2025-02-06T02:59:00.000Z");
|
||||
});
|
||||
|
||||
test("requires the stated zone rather than defaulting to UTC", () => {
|
||||
// The failure this prevents is silent and hours wide: a wall clock with no
|
||||
// zone is a missing fact, exactly as it is in `parseSlashClockZone`.
|
||||
expect(parseIsoClockRangeUtc("2026-08-06 13:00 - 2026-08-26 22:59")).toBeNull();
|
||||
expect(
|
||||
parseIsoClockRangeUtc("2026-08-06 13:00 - 2026-08-26 22:59 (UTC+8)"),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
test("rejects an impossible calendar day", () => {
|
||||
expect(
|
||||
parseIsoClockRangeUtc("2026-02-30 13:00 - 2026-03-26 22:59 (UTC)"),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseIsoClockRangeUtc("2026-08-06 25:00 - 2026-08-26 22:59 (UTC)"),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
test("is anchored at the start, so it cannot find a range inside prose", () => {
|
||||
expect(
|
||||
parseIsoClockRangeUtc("Runs 2026-08-06 13:00 - 2026-08-26 22:59 (UTC)"),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
test("tolerates the ICS widget's leftovers after the zone", () => {
|
||||
// The period cell also carries an export widget; the parser strips its
|
||||
// markup but the container survives as trailing whitespace.
|
||||
expect(
|
||||
parseIsoClockRangeUtc("2026-08-06 13:00 - 2026-08-26 22:59 (UTC) ")
|
||||
?.start.iso,
|
||||
).toBe("2026-08-06T13:00:00.000Z");
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user