feat: add wiki.gg as a second Endfield source
First non-Game8 parser, and the first source that states region-scoped ends: wiki.gg emits ISO timestamps with one timer per server region, so Endfield events now carry exact times and separate Asia / Americas-Europe ends. That gap is up to 13 hours, which is what regionEnds was built for. Two sources for one game also exercised merge for real. It caught both overlaps — including "Bedazzling Dawnstar" against Game8's "Bedazzling Dawnstar Sign-In" — and flagged that the two disagree on the end date by 70 hours rather than averaging them into a date neither states. Matching an appended qualifier needed a prefix test, not a subset test: a subset would also fuse "Gold Clash" with "Gold Rush Clash Royale". Also fixes build-feed picking fixtures by game rather than by source, which handed the wiki.gg page to the Game8 parser once Endfield had two. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f4b133274a
commit
dd596b5a43
@@ -82,13 +82,18 @@ date makes you miss content. Given the choice, this ships nothing rather than a
|
|||||||
| Wuthering Waves | Game8 | 10 |
|
| Wuthering Waves | Game8 | 10 |
|
||||||
| Zenless Zone Zero | Game8 | 12 |
|
| Zenless Zone Zero | Game8 | 12 |
|
||||||
| Neverness to Everness | Game8 | 13 |
|
| Neverness to Everness | Game8 | 13 |
|
||||||
| Arknights: Endfield | Game8 | 2 |
|
| Arknights: Endfield | Game8 + wiki.gg | 6 |
|
||||||
|
|
||||||
Game8 uses a different page template for almost every game — label/value detail tables, column
|
Game8 uses a different page template for almost every game — label/value detail tables, column
|
||||||
tables, rowspan Start/End pairs — and four different date formats between them. One parser handles
|
tables, rowspan Start/End pairs — and four different date formats between them. One parser handles
|
||||||
all of it; each game costs a registry entry.
|
all of it; each game costs a registry entry.
|
||||||
|
|
||||||
Endfield yields only two events because most of its page genuinely has no dates — every `Duration`
|
Endfield is the first game with two sources. wiki.gg publishes machine-readable ISO timestamps with
|
||||||
|
one timer per server region, so its events carry exact times and per-region ends — it outranks Game8
|
||||||
|
and wins when they disagree. Merge caught two real disagreements between them, each 70 hours apart on
|
||||||
|
the end date; those are flagged rather than averaged.
|
||||||
|
|
||||||
|
Its Game8 page yields only two events because most of it genuinely has no dates — every `Duration`
|
||||||
row reads "Permanently Available" and its version grid shows `07/16` with no year. The two dated
|
row reads "Permanently Available" and its version grid shows `07/16` with no year. The two dated
|
||||||
events hide in a combined cell (`Period: 08/09/26 - 08/30/26 During the event...`), which is where
|
events hide in a combined cell (`Period: 08/09/26 - 08/30/26 During the event...`), which is where
|
||||||
the `MM/DD/YY` parser earns its place.
|
the `MM/DD/YY` parser earns its place.
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"id": "endfield:heat-rage-mega-arena:2026-07-30",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "HEAT RAGE! MEGA ARENA!",
|
||||||
|
"type": "challenge",
|
||||||
|
"summary": "Challenge Event",
|
||||||
|
"startsAt": "2026-07-30T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-08-12T20:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": true,
|
||||||
|
"regionEnds": {
|
||||||
|
"asia": "2026-08-12T20:00:00.000Z",
|
||||||
|
"america": "2026-08-13T09:00:00.000Z",
|
||||||
|
"europe": "2026-08-13T09:00:00.000Z"
|
||||||
|
},
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/HEAT_RAGE!_MEGA_ARENA!",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "endfield:monumental-etching-beastly-howl:2026-08-06",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "Monumental Etching: Beastly Howl",
|
||||||
|
"type": "challenge",
|
||||||
|
"summary": "Challenge Event",
|
||||||
|
"startsAt": "2026-08-06T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-08-19T22:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": false,
|
||||||
|
"regionEnds": null,
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/Monumental_Etching:_Beastly_Howl",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "endfield:bedazzling-dawnstar:2026-08-09",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "Bedazzling Dawnstar",
|
||||||
|
"type": "login",
|
||||||
|
"summary": "Sign-In Event",
|
||||||
|
"startsAt": "2026-08-09T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-09-01T22:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": false,
|
||||||
|
"regionEnds": null,
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/Bedazzling_Dawnstar",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "endfield:like-a-star-streaking-through-the-boundaries:2026-08-09",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "Like a Star Streaking Through the Boundaries",
|
||||||
|
"type": "other",
|
||||||
|
"summary": "Narrative Event",
|
||||||
|
"startsAt": "2026-08-09T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-09-01T22:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": false,
|
||||||
|
"regionEnds": null,
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/Like_a_Star_Streaking_Through_the_Boundaries",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "endfield:season-of-delirating:2026-08-09",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "Season of Delirating",
|
||||||
|
"type": "other",
|
||||||
|
"summary": "Echoes of War Event",
|
||||||
|
"startsAt": "2026-08-09T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-09-01T22:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": false,
|
||||||
|
"regionEnds": null,
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/Season_of_Delirating",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "endfield:the-rooted-realm:2026-08-09",
|
||||||
|
"game": "endfield",
|
||||||
|
"title": "The Rooted Realm",
|
||||||
|
"type": "other",
|
||||||
|
"summary": "Fun & Games Event",
|
||||||
|
"startsAt": "2026-08-09T04:00:00.000Z",
|
||||||
|
"startPrecision": "exact",
|
||||||
|
"endsAt": "2026-09-01T22:00:00.000Z",
|
||||||
|
"endPrecision": "exact",
|
||||||
|
"regionScoped": false,
|
||||||
|
"regionEnds": null,
|
||||||
|
"sourceUrl": "https://endfield.wiki.gg/wiki/The_Rooted_Realm",
|
||||||
|
"sourceId": "endfield-wikigg-events",
|
||||||
|
"status": "published",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"extractionMethod": "parser",
|
||||||
|
"version": 1,
|
||||||
|
"firstSeenAt": "2026-08-14T00:00:00.000Z",
|
||||||
|
"updatedAt": "2026-08-14T00:00:00.000Z"
|
||||||
|
}
|
||||||
|
]
|
||||||
File diff suppressed because one or more lines are too long
+11
-4
@@ -14,13 +14,20 @@ import type { GachaEvent, GameId } from "../src/shared/schema.ts";
|
|||||||
|
|
||||||
const OUT = "public/data/events.v1.json";
|
const OUT = "public/data/events.v1.json";
|
||||||
|
|
||||||
/** Newest fixture per adapter. */
|
/**
|
||||||
|
* Newest fixture for one *source*, not one game.
|
||||||
|
*
|
||||||
|
* A game can have several sources, and fixtures are named `<site>-events-<date>`
|
||||||
|
* against adapter ids of `<game>-<site>-events`. Globbing by game alone hands
|
||||||
|
* one site's page to another site's parser.
|
||||||
|
*/
|
||||||
async function latestFixture(adapterId: string, game: GameId) {
|
async function latestFixture(adapterId: string, game: GameId) {
|
||||||
const glob = new Bun.Glob(`fixtures/${game}/*.html`);
|
const site = adapterId.replace(`${game}-`, "").replace(/-events$/, "");
|
||||||
const files = [...glob.scanSync(".")].sort();
|
const pattern = `fixtures/${game}/${site}-*.html`;
|
||||||
|
const files = [...new Bun.Glob(pattern).scanSync(".")].sort();
|
||||||
const file = files.at(-1);
|
const file = files.at(-1);
|
||||||
if (file === undefined) {
|
if (file === undefined) {
|
||||||
throw new Error(`no fixture found for ${adapterId} (fixtures/${game}/*.html)`);
|
throw new Error(`no fixture found for ${adapterId} (${pattern})`);
|
||||||
}
|
}
|
||||||
return { file, html: await Bun.file(file).text() };
|
return { file, html: await Bun.file(file).text() };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -50,6 +50,15 @@ const SOURCES: SourceSpec[] = [
|
|||||||
url: "https://game8.co/games/Arknights-Endfield/archives/535443",
|
url: "https://game8.co/games/Arknights-Endfield/archives/535443",
|
||||||
parserId: "game8",
|
parserId: "game8",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
id: "endfield-wikigg-events",
|
||||||
|
game: "endfield",
|
||||||
|
url: "https://endfield.wiki.gg/wiki/Event",
|
||||||
|
parserId: "wikigg",
|
||||||
|
// Exact timestamps and per-region ends beat Game8's day-precision prose,
|
||||||
|
// so this source wins when the two disagree.
|
||||||
|
priority: 10,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
id: "nte-game8-events",
|
id: "nte-game8-events",
|
||||||
game: "nte",
|
game: "nte",
|
||||||
|
|||||||
+34
-8
@@ -123,12 +123,36 @@ function isSameEvent(
|
|||||||
toleranceHours: number,
|
toleranceHours: number,
|
||||||
): boolean {
|
): boolean {
|
||||||
if (a.id === b.id) return true;
|
if (a.id === b.id) return true;
|
||||||
if (titleSimilarity(a.title, b.title) < titleThreshold) return false;
|
|
||||||
|
// Overlap alone misses a source that appends a qualifier: "Bedazzling
|
||||||
|
// Dawnstar" vs "Bedazzling Dawnstar Sign-In" scores 0.67, well under any
|
||||||
|
// safe threshold, yet is plainly one event.
|
||||||
|
const similar =
|
||||||
|
titleSimilarity(a.title, b.title) >= titleThreshold ||
|
||||||
|
titleExtends(a.title, b.title);
|
||||||
|
if (!similar) return false;
|
||||||
|
|
||||||
// Similar titles are not enough — reruns reuse names. Require the start dates
|
// Similar titles are not enough — reruns reuse names. Require the start dates
|
||||||
// to be close before treating two entries as one event.
|
// to be close before treating two entries as one event.
|
||||||
return hoursBetween(a.startsAt, b.startsAt) <= toleranceHours;
|
return hoursBetween(a.startsAt, b.startsAt) <= toleranceHours;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* True when one title is the other with words appended — the shape a source
|
||||||
|
* qualifier actually takes ("Bedazzling Dawnstar" → "… Sign-In").
|
||||||
|
*
|
||||||
|
* Deliberately a prefix test, not a subset test. Subset matching would also
|
||||||
|
* fuse "Gold Clash" with "Gold Rush Clash Royale", which are different events.
|
||||||
|
* Two words is the floor: a one-word title would swallow half the calendar.
|
||||||
|
*/
|
||||||
|
export function titleExtends(a: string, b: string): boolean {
|
||||||
|
const ta = tokenList(a);
|
||||||
|
const tb = tokenList(b);
|
||||||
|
const [small, large] = ta.length <= tb.length ? [ta, tb] : [tb, ta];
|
||||||
|
if (small.length < 2 || small.length === large.length) return false;
|
||||||
|
return small.every((word, i) => large[i] === word);
|
||||||
|
}
|
||||||
|
|
||||||
function findConflict(
|
function findConflict(
|
||||||
a: GachaEvent,
|
a: GachaEvent,
|
||||||
b: GachaEvent,
|
b: GachaEvent,
|
||||||
@@ -163,12 +187,14 @@ export function titleSimilarity(a: string, b: string): number {
|
|||||||
return (2 * shared) / (ta.size + tb.size);
|
return (2 * shared) / (ta.size + tb.size);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function tokenList(title: string): string[] {
|
||||||
|
return title
|
||||||
|
.toLowerCase()
|
||||||
|
.replace(/[^a-z0-9\s]/g, " ")
|
||||||
|
.split(/\s+/)
|
||||||
|
.filter((w) => w.length > 0);
|
||||||
|
}
|
||||||
|
|
||||||
function tokens(title: string): Set<string> {
|
function tokens(title: string): Set<string> {
|
||||||
return new Set(
|
return new Set(tokenList(title));
|
||||||
title
|
|
||||||
.toLowerCase()
|
|
||||||
.replace(/[^a-z0-9\s]/g, " ")
|
|
||||||
.split(/\s+/)
|
|
||||||
.filter((w) => w.length > 0),
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { game8Parser } from "./game8.ts";
|
import { game8Parser } from "./game8.ts";
|
||||||
|
import { wikiGgParser } from "./wikigg.ts";
|
||||||
import type { SourceParser } from "./types.ts";
|
import type { SourceParser } from "./types.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -6,7 +7,7 @@ import type { SourceParser } from "./types.ts";
|
|||||||
* an entry in `adapters/index.ts`; adding a new *site* means a parser module
|
* an entry in `adapters/index.ts`; adding a new *site* means a parser module
|
||||||
* here and one line below.
|
* here and one line below.
|
||||||
*/
|
*/
|
||||||
export const PARSERS: SourceParser[] = [game8Parser];
|
export const PARSERS: SourceParser[] = [game8Parser, wikiGgParser];
|
||||||
|
|
||||||
export function parserById(id: string): SourceParser | undefined {
|
export function parserById(id: string): SourceParser | undefined {
|
||||||
return PARSERS.find((p) => p.id === id);
|
return PARSERS.find((p) => p.id === id);
|
||||||
@@ -14,3 +15,4 @@ export function parserById(id: string): SourceParser | undefined {
|
|||||||
|
|
||||||
export type { SourceParser } from "./types.ts";
|
export type { SourceParser } from "./types.ts";
|
||||||
export { game8Parser } from "./game8.ts";
|
export { game8Parser } from "./game8.ts";
|
||||||
|
export { wikiGgParser } from "./wikigg.ts";
|
||||||
|
|||||||
@@ -0,0 +1,142 @@
|
|||||||
|
import {
|
||||||
|
eventId,
|
||||||
|
Region,
|
||||||
|
type GachaEvent,
|
||||||
|
type Precision,
|
||||||
|
} from "../../shared/schema.ts";
|
||||||
|
import { text } from "../html.ts";
|
||||||
|
import type { ParseContext } from "../adapters/types.ts";
|
||||||
|
import { inferType } from "./game8.ts";
|
||||||
|
import type { SourceParser } from "./types.ts";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* wiki.gg event pages (MediaWiki with the `mp-event` template).
|
||||||
|
*
|
||||||
|
* A markedly better source than a prose wiki: each event carries machine
|
||||||
|
* readable ISO timestamps, one timer per server region:
|
||||||
|
*
|
||||||
|
* <div class="mp-event">
|
||||||
|
* <div class="mp-event-header">[<span class="mp-event-name">TITLE</span>]
|
||||||
|
* <span class="mp-event-type">Challenge Event</span></div>
|
||||||
|
* <div class="mp-event-image"><a href="/wiki/SLUG" …></div>
|
||||||
|
* <ul class="mp-event-timers">
|
||||||
|
* <li data-start="…Z" data-end="…Z"><span class="mp-server-name">Asia:</span>…
|
||||||
|
*
|
||||||
|
* This is the first source that states region-scoped ends, which is what
|
||||||
|
* `regionScoped` / `regionEnds` exist for — the difference is up to 13 hours.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const EVENT_BLOCK = /<div class="mp-event">([\s\S]*?)<ul class="mp-event-timers">([\s\S]*?)<\/ul>/gi;
|
||||||
|
const NAME = /<span class="mp-event-name">([\s\S]*?)<\/span>/i;
|
||||||
|
const TYPE = /<span class="mp-event-type">([\s\S]*?)<\/span>/i;
|
||||||
|
const LINK = /<a href="(\/wiki\/[^"]+)"/i;
|
||||||
|
const TIMER =
|
||||||
|
/<li[^>]*class="[^"]*mp-event-timer[^"]*"[^>]*data-start="([^"]+)"[^>]*data-end="([^"]+)"[\s\S]*?<span class="[^"]*mp-server-name[^"]*">([\s\S]*?)<\/span>/gi;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Server labels vary in wording ("Americas / Europe:", "Asia:"), so match on
|
||||||
|
* the words present rather than expecting one label per region. A single timer
|
||||||
|
* can legitimately cover two regions.
|
||||||
|
*/
|
||||||
|
function regionsFor(label: string): Region[] {
|
||||||
|
const l = label.toLowerCase();
|
||||||
|
const out: Region[] = [];
|
||||||
|
if (/asia/.test(l)) out.push("asia");
|
||||||
|
if (/america/.test(l)) out.push("america");
|
||||||
|
if (/europe|eu\b/.test(l)) out.push("europe");
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Timer {
|
||||||
|
regions: Region[];
|
||||||
|
start: number;
|
||||||
|
end: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseWikiGgEventsPage(
|
||||||
|
html: string,
|
||||||
|
ctx: ParseContext,
|
||||||
|
): GachaEvent[] {
|
||||||
|
const flat = html.replace(/\s+/g, " ");
|
||||||
|
const out: GachaEvent[] = [];
|
||||||
|
|
||||||
|
for (const block of flat.matchAll(EVENT_BLOCK)) {
|
||||||
|
const head = block[1] ?? "";
|
||||||
|
const timersHtml = block[2] ?? "";
|
||||||
|
|
||||||
|
const title = text(NAME.exec(head)?.[1] ?? "");
|
||||||
|
if (title.length === 0) continue;
|
||||||
|
|
||||||
|
const timers: Timer[] = [];
|
||||||
|
for (const t of timersHtml.matchAll(TIMER)) {
|
||||||
|
const start = Date.parse(t[1] ?? "");
|
||||||
|
const end = Date.parse(t[2] ?? "");
|
||||||
|
if (Number.isNaN(start) || Number.isNaN(end) || end <= start) continue;
|
||||||
|
timers.push({ regions: regionsFor(text(t[3] ?? "")), start, end });
|
||||||
|
}
|
||||||
|
if (timers.length === 0) continue;
|
||||||
|
|
||||||
|
const regionEnds: Partial<Record<Region, string>> = {};
|
||||||
|
for (const timer of timers) {
|
||||||
|
for (const region of timer.regions) {
|
||||||
|
regionEnds[region] = new Date(timer.end).toISOString();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const covered = Object.keys(regionEnds).length;
|
||||||
|
// Region-scoped only when the source actually distinguishes regions and
|
||||||
|
// they genuinely differ. One timer for everyone is a global instant.
|
||||||
|
const distinctEnds = new Set(Object.values(regionEnds)).size;
|
||||||
|
const regionScoped = covered > 0 && distinctEnds > 1;
|
||||||
|
|
||||||
|
// Canonical start is the earliest any region sees it; the canonical end is
|
||||||
|
// the earliest any region loses it. `endsAt` is only a fallback for a
|
||||||
|
// region the source did not list, so the conservative choice is right.
|
||||||
|
const startsAt = new Date(
|
||||||
|
Math.min(...timers.map((t) => t.start)),
|
||||||
|
).toISOString();
|
||||||
|
const endsAt = new Date(Math.min(...timers.map((t) => t.end))).toISOString();
|
||||||
|
|
||||||
|
const href = LINK.exec(block[0] ?? "")?.[1];
|
||||||
|
const typeLabel = text(TYPE.exec(head)?.[1] ?? "");
|
||||||
|
const precision: Precision = "exact";
|
||||||
|
|
||||||
|
out.push({
|
||||||
|
id: eventId(ctx.game, title, startsAt),
|
||||||
|
game: ctx.game,
|
||||||
|
title,
|
||||||
|
type: inferType(`${title} ${typeLabel}`),
|
||||||
|
summary: typeLabel.length > 0 ? typeLabel : null,
|
||||||
|
startsAt,
|
||||||
|
startPrecision: precision,
|
||||||
|
endsAt,
|
||||||
|
endPrecision: precision,
|
||||||
|
regionScoped,
|
||||||
|
regionEnds: regionScoped ? (regionEnds as Record<Region, string>) : null,
|
||||||
|
sourceUrl: href === undefined ? ctx.sourceUrl : new URL(href, ctx.sourceUrl).toString(),
|
||||||
|
sourceId: ctx.sourceId,
|
||||||
|
status: "published",
|
||||||
|
// Exact timestamps straight from the source, no inference anywhere.
|
||||||
|
confidence: 0.95,
|
||||||
|
extractionMethod: "parser",
|
||||||
|
version: 1,
|
||||||
|
firstSeenAt: ctx.now,
|
||||||
|
updatedAt: ctx.now,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return out.sort((a, b) =>
|
||||||
|
a.startsAt === b.startsAt
|
||||||
|
? a.id.localeCompare(b.id)
|
||||||
|
: a.startsAt.localeCompare(b.startsAt),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export const wikiGgParser: SourceParser = {
|
||||||
|
id: "wikigg",
|
||||||
|
label: "wiki.gg",
|
||||||
|
canParse(html: string): boolean {
|
||||||
|
return /class="mp-event"/.test(html) && /mp-event-timer/.test(html);
|
||||||
|
},
|
||||||
|
parse: parseWikiGgEventsPage,
|
||||||
|
};
|
||||||
@@ -26,6 +26,7 @@ const CASES: Array<{ adapter: Adapter; fixture: string }> = [
|
|||||||
{ adapter: adapter("wuwa-game8-events"), fixture: "fixtures/wuwa/game8-events-2026-08-14" },
|
{ adapter: adapter("wuwa-game8-events"), fixture: "fixtures/wuwa/game8-events-2026-08-14" },
|
||||||
{ adapter: adapter("zzz-game8-events"), fixture: "fixtures/zzz/game8-events-2026-08-14" },
|
{ adapter: adapter("zzz-game8-events"), fixture: "fixtures/zzz/game8-events-2026-08-14" },
|
||||||
{ adapter: adapter("endfield-game8-events"), fixture: "fixtures/endfield/game8-events-2026-08-14" },
|
{ adapter: adapter("endfield-game8-events"), fixture: "fixtures/endfield/game8-events-2026-08-14" },
|
||||||
|
{ adapter: adapter("endfield-wikigg-events"), fixture: "fixtures/endfield/wikigg-events-2026-08-14" },
|
||||||
];
|
];
|
||||||
|
|
||||||
async function runAdapter(adapter: Adapter, fixture: string) {
|
async function runAdapter(adapter: Adapter, fixture: string) {
|
||||||
@@ -224,3 +225,34 @@ describe("endfield", () => {
|
|||||||
expect(rooted?.summary).not.toMatch(/^Period:/);
|
expect(rooted?.summary).not.toMatch(/^Period:/);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("wiki.gg parser", () => {
|
||||||
|
test("reads exact per-region timers", async () => {
|
||||||
|
// The first source that states region-scoped ends. Asia and the Americas
|
||||||
|
// differ by hours, which is precisely what regionEnds exists to carry.
|
||||||
|
const events = await runAdapter(
|
||||||
|
adapter("endfield-wikigg-events"),
|
||||||
|
"fixtures/endfield/wikigg-events-2026-08-14",
|
||||||
|
);
|
||||||
|
expect(events).toHaveLength(6);
|
||||||
|
|
||||||
|
const heat = events.find((e) => e.title === "HEAT RAGE! MEGA ARENA!");
|
||||||
|
expect(heat?.startPrecision).toBe("exact");
|
||||||
|
expect(heat?.regionScoped).toBe(true);
|
||||||
|
expect(heat?.regionEnds?.asia).toBe("2026-08-12T20:00:00.000Z");
|
||||||
|
expect(heat?.regionEnds?.america).toBe("2026-08-13T09:00:00.000Z");
|
||||||
|
// endsAt is the fallback for a region the source did not list, so it takes
|
||||||
|
// the earliest — never promise more time than some region actually gets.
|
||||||
|
expect(heat?.endsAt).toBe("2026-08-12T20:00:00.000Z");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("links each event to its own wiki page", async () => {
|
||||||
|
const events = await runAdapter(
|
||||||
|
adapter("endfield-wikigg-events"),
|
||||||
|
"fixtures/endfield/wikigg-events-2026-08-14",
|
||||||
|
);
|
||||||
|
for (const e of events) {
|
||||||
|
expect(e.sourceUrl).toStartWith("https://endfield.wiki.gg/wiki/");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
+58
-1
@@ -1,5 +1,9 @@
|
|||||||
import { describe, expect, test } from "bun:test";
|
import { describe, expect, test } from "bun:test";
|
||||||
import { mergeEvents, titleSimilarity } from "../src/ingest/merge.ts";
|
import {
|
||||||
|
mergeEvents,
|
||||||
|
titleExtends,
|
||||||
|
titleSimilarity,
|
||||||
|
} from "../src/ingest/merge.ts";
|
||||||
import type { GachaEvent } from "../src/shared/schema.ts";
|
import type { GachaEvent } from "../src/shared/schema.ts";
|
||||||
|
|
||||||
const NOW = "2026-08-14T00:00:00.000Z";
|
const NOW = "2026-08-14T00:00:00.000Z";
|
||||||
@@ -134,3 +138,56 @@ describe("titleSimilarity", () => {
|
|||||||
expect(titleSimilarity("Gold Clash", "Fishing Frenzy")).toBeLessThan(0.3);
|
expect(titleSimilarity("Gold Clash", "Fishing Frenzy")).toBeLessThan(0.3);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("titleExtends", () => {
|
||||||
|
test("matches a title with an appended qualifier", () => {
|
||||||
|
// Real case: Game8 says "Bedazzling Dawnstar Sign-In", wiki.gg says
|
||||||
|
// "Bedazzling Dawnstar". Token overlap scores 0.67 — under any safe
|
||||||
|
// threshold — but it is plainly one event.
|
||||||
|
expect(titleExtends("Bedazzling Dawnstar", "Bedazzling Dawnstar Sign-In")).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("refuses a single shared word", () => {
|
||||||
|
// A one-word title would otherwise swallow everything containing it.
|
||||||
|
expect(titleExtends("Rerun", "Overflowing Abundance Rerun")).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("refuses a shared-word subset that is not a prefix", () => {
|
||||||
|
// Subset matching would fuse these; they are different events.
|
||||||
|
expect(titleExtends("Gold Clash", "Gold Rush Clash Royale")).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("refuses two titles of equal length", () => {
|
||||||
|
expect(titleExtends("Gold Clash", "Gold Rush")).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("merging two sources for one game", () => {
|
||||||
|
test("collapses an appended-qualifier duplicate and flags the date gap", () => {
|
||||||
|
const g8 = event({
|
||||||
|
id: "endfield:bedazzling-dawnstar-sign-in:2026-08-09",
|
||||||
|
game: "endfield",
|
||||||
|
title: "Bedazzling Dawnstar Sign-In",
|
||||||
|
startsAt: "2026-08-09T00:00:00.000Z",
|
||||||
|
endsAt: "2026-08-30T00:00:00.000Z",
|
||||||
|
sourceId: "endfield-game8-events",
|
||||||
|
confidence: 0.85,
|
||||||
|
});
|
||||||
|
const wiki = event({
|
||||||
|
id: "endfield:bedazzling-dawnstar:2026-08-09",
|
||||||
|
game: "endfield",
|
||||||
|
title: "Bedazzling Dawnstar",
|
||||||
|
startsAt: "2026-08-09T04:00:00.000Z",
|
||||||
|
endsAt: "2026-09-01T22:00:00.000Z",
|
||||||
|
sourceId: "endfield-wikigg-events",
|
||||||
|
confidence: 0.95,
|
||||||
|
});
|
||||||
|
const { events, conflicts } = mergeEvents([[g8], [wiki]]);
|
||||||
|
expect(events).toHaveLength(1);
|
||||||
|
// The better-sourced copy wins, and the disagreement is surfaced rather
|
||||||
|
// than averaged into a date neither source states.
|
||||||
|
expect(events[0]?.sourceId).toBe("endfield-wikigg-events");
|
||||||
|
expect(conflicts).toHaveLength(1);
|
||||||
|
expect(conflicts[0]?.field).toBe("endsAt");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user