fix(ingest): retire a day-precision row on the clock the app reads it on
Three parsers decide currency against ctx.now, because their page has no "ongoing" heading worth trusting: bawiki.ts, and the Fate/Grand Order and Infinity Nikki branches of fandom.ts. All three compared Date.parse(endsAt) to now, and all three publish day-precision ends — so the comparison was against the 00:00Z placeholder, which § Domain rules says is not an instant and nothing may read literally. clockFor already knows that and resolves such a boundary to the reset opening that game-day on the reader's server. The parsers did not, so the feed retired a row up to nine hours before the app, the countdown and the game all agreed it was over. On the pinned Infinity Nikki fixture, "Inspiration Burst" left the feed at 2026-08-22T01:00Z while clockFor still called it live for an American reader until 09:00Z, and the page said it ran to 03:59 server time. The reader does not see a stale row; they see the deadline they were counting down to disappear on its last day, which is the silent drop this codebase treats as the dangerous failure. latestBoundaryMs answers clockFor's question for the last region rather than one reader's, so a row is history only once it is history everywhere. It follows a game's own server map and reset hour for the same reason clockFor does — Endfield and Reverse: 1999 both move. Nothing stored changes: this is one comparison, not a resolved boundary written into the feed, and no expected.json moves. Being generous by nine hours costs an expired row at the bottom of a list. Being strict costs a live one. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
0f9c141ea5
commit
c782a265ca
@@ -119,7 +119,7 @@ src/client/ React app, service worker, manifest
|
|||||||
theme.ts — dark or light, and what a game hue reads as on each
|
theme.ts — dark or light, and what a game hue reads as on each
|
||||||
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
|
scripts/ build-feed.ts, build-static.ts, parse-fixture.ts (offline), refresh-sources.ts (fetches)
|
||||||
serve.ts static server + /api/health
|
serve.ts static server + /api/health
|
||||||
test/ 706 tests
|
test/ 711 tests
|
||||||
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
|
fixtures/<game>/ raw HTML + .expected.json per source — pinned, kept forever
|
||||||
snapshots/ current page per source, rewritten by refresh — see its README
|
snapshots/ current page per source, rewritten by refresh — see its README
|
||||||
```
|
```
|
||||||
@@ -154,6 +154,12 @@ These come from how gacha games actually schedule things, and they cause most bu
|
|||||||
*reading* of the printed date, not an invented time, and it changes nothing stored: the feed,
|
*reading* of the printed date, not an invented time, and it changes nothing stored: the feed,
|
||||||
every event ID and the parsers are untouched, so it is one resolution at the point where the
|
every event ID and the parsers are untouched, so it is one resolution at the point where the
|
||||||
region is finally known.
|
region is finally known.
|
||||||
|
The same clock governs the *other* end of the pipeline. A parser whose page has no trustworthy
|
||||||
|
"ongoing" heading decides currency against `ctx.now` itself, and comparing the 00:00Z placeholder
|
||||||
|
to `now` retires a row hours before `clockFor` calls it over for anybody — the reader watches a
|
||||||
|
deadline they were counting down to vanish on its last day. So `latestBoundaryMs` answers the same
|
||||||
|
question for the *last* region, and `bawiki.ts` and two branches of `fandom.ts` ask it. Nothing
|
||||||
|
stored changes: it is one comparison, not a resolved boundary written to the feed.
|
||||||
- **Patch cycles are ~6 weeks.** Any event over 180 days is a parse error, not a long event. The
|
- **Patch cycles are ~6 weeks.** Any event over 180 days is a parse error, not a long event. The
|
||||||
validator and the tests both reject it.
|
validator and the tests both reject it.
|
||||||
|
|
||||||
|
|||||||
+14
-3
@@ -152,9 +152,20 @@ All live in `src/ingest/dates.ts`, each returning null rather than inferring any
|
|||||||
above returns `precision: "day"` when the source printed no clock, and stores the date at UTC
|
above returns `precision: "day"` when the source printed no clock, and stores the date at UTC
|
||||||
midnight because it has to store *something*. It is not a statement that the event begins or ends
|
midnight because it has to store *something*. It is not a statement that the event begins or ends
|
||||||
then, and nothing may count down to it literally: `clockFor` resolves a day-precision boundary to
|
then, and nothing may count down to it literally: `clockFor` resolves a day-precision boundary to
|
||||||
that game-day's server reset for the reader's region (`docs/DATA-MODEL.md` § Field notes). The
|
that game-day's server reset for the reader's region (`docs/DATA-MODEL.md` § Field notes).
|
||||||
parsers are unaffected by this and must stay so — resolving here would need a region the parser does
|
|
||||||
not have, and would bake one reader's server into the stored feed.
|
**No parser may store a resolved boundary, and three of them must read one to decide inclusion.**
|
||||||
|
The stored value stays the printed day at 00:00Z: resolving it here would need a region the parser
|
||||||
|
does not have, and would bake one reader's server into the feed everybody downloads. But a parser
|
||||||
|
whose page carries no "ongoing" heading it can trust decides currency against `ctx.now` itself —
|
||||||
|
`bawiki.ts`, and the Fate/Grand Order and Infinity Nikki branches of `fandom.ts` — and comparing the
|
||||||
|
placeholder to `now` retires a row at UTC midnight, hours before `clockFor` calls it over for
|
||||||
|
anybody. The reader does not see a stale row; they watch the deadline they were counting down to
|
||||||
|
disappear on its last day, which is the silent drop AGENTS.md § Working on parsers calls the
|
||||||
|
dangerous failure. So those three ask `latestBoundaryMs` (`src/shared/time.ts`) when the boundary is
|
||||||
|
day-precision: the last region's reset, and therefore the instant the row is history for every
|
||||||
|
reader rather than for the earliest of them. Being generous by nine hours costs one expired row at
|
||||||
|
the bottom of a list; being strict costs a live one.
|
||||||
|
|
||||||
`parseOpenRange` is tried last because it is the most permissive — it accepts any leading full date
|
`parseOpenRange` is tried last because it is the most permissive — it accepts any leading full date
|
||||||
and reports no end.
|
and reports no end.
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { eventId, type GachaEvent } from "../../shared/schema.ts";
|
import { eventId, type GachaEvent } from "../../shared/schema.ts";
|
||||||
|
import { latestBoundaryMs } from "../../shared/time.ts";
|
||||||
import { parseIsoDay, type ParsedInstant } from "../dates.ts";
|
import { parseIsoDay, type ParsedInstant } from "../dates.ts";
|
||||||
import { text } from "../html.ts";
|
import { text } from "../html.ts";
|
||||||
import type { ParseContext } from "../adapters/types.ts";
|
import type { ParseContext } from "../adapters/types.ts";
|
||||||
@@ -189,12 +190,19 @@ export function parseBlueArchiveWikiEventsPage(
|
|||||||
// the start has passed, the end is the only thing separating a live event
|
// the start has passed, the end is the only thing separating a live event
|
||||||
// from any of the ninety-odd finished rows above it, and without one there
|
// from any of the ninety-odd finished rows above it, and without one there
|
||||||
// is no way to tell — so that row yields nothing rather than a guess.
|
// is no way to tell — so that row yields nothing rather than a guess.
|
||||||
if (Date.parse(start.iso) < nowMs) continue;
|
if (latestBoundaryMs(start.iso, start.precision, ctx.game) < nowMs) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
if (end.iso <= start.iso) continue;
|
if (end.iso <= start.iso) continue;
|
||||||
// Live and upcoming only. Everything else is history the page keeps and
|
// Live and upcoming only. Everything else is history the page keeps and
|
||||||
// the calendar does not want.
|
// the calendar does not want.
|
||||||
if (Date.parse(end.iso) < nowMs) continue;
|
//
|
||||||
|
// Every boundary on this page is day precision, so both checks resolve
|
||||||
|
// through `latestBoundaryMs` rather than reading the stored UTC midnight
|
||||||
|
// as an instant: that placeholder retires a row hours before the reader's
|
||||||
|
// own reset does, which loses a live event on the day it ends.
|
||||||
|
if (latestBoundaryMs(end.iso, end.precision, ctx.game) < nowMs) continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
// The source's own annotation: "Rerun", "Collaboration Event",
|
// The source's own annotation: "Rerun", "Collaboration Event",
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { eventId, type GachaEvent } from "../../shared/schema.ts";
|
import { eventId, type GachaEvent } from "../../shared/schema.ts";
|
||||||
|
import { latestBoundaryMs } from "../../shared/time.ts";
|
||||||
import {
|
import {
|
||||||
parseDayMonthYearClock,
|
parseDayMonthYearClock,
|
||||||
parseFullRange,
|
parseFullRange,
|
||||||
@@ -229,7 +230,14 @@ function parseFgoOngoingEvents(
|
|||||||
|
|
||||||
// "Ongoing" is maintained by hand and goes stale before anyone moves a row,
|
// "Ongoing" is maintained by hand and goes stale before anyone moves a row,
|
||||||
// so the heading vouching for an event is not enough on its own.
|
// so the heading vouching for an event is not enough on its own.
|
||||||
if (Date.parse(range.end.iso) < nowMs) continue;
|
//
|
||||||
|
// `latestBoundaryMs`, not `Date.parse`: these ends are day precision, and
|
||||||
|
// the raw value is UTC midnight — a placeholder the countdown resolves to
|
||||||
|
// each reader's own reset. Retiring the row on the placeholder drops it
|
||||||
|
// while the app still shows it as live.
|
||||||
|
if (latestBoundaryMs(range.end.iso, range.end.precision, ctx.game) < nowMs) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
out.push({
|
out.push({
|
||||||
id: eventId(ctx.game, title, range.start.iso),
|
id: eventId(ctx.game, title, range.start.iso),
|
||||||
@@ -536,8 +544,12 @@ function parseInfinityNikkiEvents(
|
|||||||
if (range.end.iso <= range.start.iso) continue;
|
if (range.end.iso <= range.start.iso) continue;
|
||||||
|
|
||||||
// "Current" is maintained by hand and goes stale before anyone moves a
|
// "Current" is maintained by hand and goes stale before anyone moves a
|
||||||
// row, so currency is checked rather than taken on trust.
|
// row, so currency is checked rather than taken on trust — on the same
|
||||||
if (Date.parse(range.end.iso) < nowMs) continue;
|
// clock the countdown reads a day-precision end on, not on the UTC
|
||||||
|
// midnight placeholder stored for it.
|
||||||
|
if (latestBoundaryMs(range.end.iso, range.end.precision, ctx.game) < nowMs) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
const id = eventId(ctx.game, title, range.start.iso);
|
const id = eventId(ctx.game, title, range.start.iso);
|
||||||
if (seen.has(id)) continue;
|
if (seen.has(id)) continue;
|
||||||
|
|||||||
+35
-1
@@ -1,6 +1,7 @@
|
|||||||
import type { DisplayEvent, LaneId } from "./custom.ts";
|
import type { DisplayEvent, LaneId } from "./custom.ts";
|
||||||
import { GAMES } from "./games.ts";
|
import { GAMES } from "./games.ts";
|
||||||
import type { GameId, Precision, Region } from "./schema.ts";
|
import { Region } from "./schema.ts";
|
||||||
|
import type { GameId, Precision } from "./schema.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Time is this product's entire subject, so the vocabulary lives in one place:
|
* Time is this product's entire subject, so the vocabulary lives in one place:
|
||||||
@@ -196,6 +197,39 @@ function boundaryMs(
|
|||||||
return dayStartMs(iso.slice(0, 10), region, event.game);
|
return dayStartMs(iso.slice(0, 10), region, event.game);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The instant a printed boundary has passed for **every** reader, whatever
|
||||||
|
* region they are on.
|
||||||
|
*
|
||||||
|
* `boundaryMs` above answers the question for one reader; this answers it for
|
||||||
|
* the last of them. The two must agree, because they are read at opposite ends
|
||||||
|
* of the same pipeline: an ingest parser deciding whether a row is still worth
|
||||||
|
* publishing, and the countdown deciding whether to call it over.
|
||||||
|
*
|
||||||
|
* They did not. A parser comparing `Date.parse(endsAt)` against `now` retires a
|
||||||
|
* day-precision end at UTC midnight — the placeholder, not an instant (§ Domain
|
||||||
|
* rules) — while the app keeps the event live until the reset that opens that
|
||||||
|
* game-day on the reader's own server. For a default server map that is 09:00Z
|
||||||
|
* in the Americas, so the feed drops an event nine hours before the app, the
|
||||||
|
* countdown and the game all agree it is over. The reader does not see a stale
|
||||||
|
* row; they see the deadline they were counting down to vanish on its last day,
|
||||||
|
* which is the silent drop AGENTS.md § Working on parsers calls the dangerous
|
||||||
|
* failure.
|
||||||
|
*
|
||||||
|
* So a row is history only once it is history everywhere. Being generous by a
|
||||||
|
* few hours costs an expired row at the bottom of a list; being strict costs a
|
||||||
|
* live one.
|
||||||
|
*/
|
||||||
|
export function latestBoundaryMs(
|
||||||
|
iso: string,
|
||||||
|
precision: Precision,
|
||||||
|
game?: LaneId,
|
||||||
|
): number {
|
||||||
|
if (precision !== "day") return Date.parse(iso);
|
||||||
|
const day = iso.slice(0, 10);
|
||||||
|
return Math.max(...Region.options.map((r) => dayStartMs(day, r, game)));
|
||||||
|
}
|
||||||
|
|
||||||
export type Urgency = "expired" | "critical" | "soon" | "near" | "calm";
|
export type Urgency = "expired" | "critical" | "soon" | "near" | "calm";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -1588,6 +1588,29 @@ describe("Infinity Nikki wiki (the fourth Fandom template)", () => {
|
|||||||
`<h2><span class="mw-headline" id="x">${heading}</span></h2>
|
`<h2><span class="mw-headline" id="x">${heading}</span></h2>
|
||||||
<table class="article-table">${HEAD}${rows}</table>`;
|
<table class="article-table">${HEAD}${rows}</table>`;
|
||||||
|
|
||||||
|
test("keeps a day-precision end until it has passed in every region", async () => {
|
||||||
|
// The row is dated "August 15, 2026 04:00 - August 22, 2026 03:59" and the
|
||||||
|
// clock is discarded, so `endsAt` is the 00:00Z placeholder. Retiring the
|
||||||
|
// row on that placeholder drops it nine hours before `clockFor` calls it
|
||||||
|
// over for an American reader — the reader watches the deadline they were
|
||||||
|
// counting down to vanish on its last day, which is a silent drop.
|
||||||
|
const html = await Bun.file(`${fixture}.html`).text();
|
||||||
|
const at = (now: string) =>
|
||||||
|
nikki
|
||||||
|
.parse(html, {
|
||||||
|
now,
|
||||||
|
sourceUrl: nikki.url,
|
||||||
|
sourceId: nikki.id,
|
||||||
|
game: nikki.game,
|
||||||
|
})
|
||||||
|
.some((e) => e.title === "Inspiration Burst");
|
||||||
|
|
||||||
|
expect(at("2026-08-22T01:00:00.000Z")).toBe(true);
|
||||||
|
// 04:00 on the last server to roll, UTC-5. Past that it is over everywhere.
|
||||||
|
expect(at("2026-08-22T08:59:00.000Z")).toBe(true);
|
||||||
|
expect(at("2026-08-22T09:01:00.000Z")).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
test("takes the printed date at day precision and drops the clock", async () => {
|
test("takes the printed date at day precision and drops the clock", async () => {
|
||||||
// The page states a wall clock on both sides and names no zone for it
|
// The page states a wall clock on both sides and names no zone for it
|
||||||
// anywhere. Publishing an instant would mean picking an offset, and the
|
// anywhere. Publishing an instant would mean picking an offset, and the
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import {
|
|||||||
endingSoonestFirst,
|
endingSoonestFirst,
|
||||||
formatRemaining,
|
formatRemaining,
|
||||||
HOUR,
|
HOUR,
|
||||||
|
latestBoundaryMs,
|
||||||
urgency,
|
urgency,
|
||||||
} from "../src/shared/time.ts";
|
} from "../src/shared/time.ts";
|
||||||
|
|
||||||
@@ -232,3 +233,46 @@ describe("dayStartMs", () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("latestBoundaryMs", () => {
|
||||||
|
/**
|
||||||
|
* The ingest half of the rule above. `clockFor` resolves a day-precision
|
||||||
|
* boundary per reader; a parser deciding whether a row is still worth
|
||||||
|
* publishing has no reader, so it has to answer for the last of them — and
|
||||||
|
* before this existed it answered for none of them, comparing the stored UTC
|
||||||
|
* midnight placeholder against `now` and retiring rows the app still showed.
|
||||||
|
*/
|
||||||
|
const DAY_END = "2026-08-19T00:00:00.000Z";
|
||||||
|
|
||||||
|
test("is the last region's reset, not UTC midnight", () => {
|
||||||
|
// The Americas server is the last to roll: 04:00 on UTC-5.
|
||||||
|
expect(latestBoundaryMs(DAY_END, "day", "genshin")).toBe(
|
||||||
|
Date.parse("2026-08-19T09:00:00.000Z"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("is never earlier than any region's own reading of the same date", () => {
|
||||||
|
for (const game of [undefined, "genshin", "endfield", "r1999"] as const) {
|
||||||
|
const latest = latestBoundaryMs(DAY_END, "day", game);
|
||||||
|
for (const region of ["asia", "america", "europe"] as const) {
|
||||||
|
expect(latest).toBeGreaterThanOrEqual(dayStartMs("2026-08-19", region, game));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("follows a game that states its own server map or reset hour", () => {
|
||||||
|
// Endfield's Europe sits on the Americas machine, so no region rolls later
|
||||||
|
// than 09:00Z; Reverse: 1999 is one UTC-5 server rolling at 05:00.
|
||||||
|
expect(latestBoundaryMs(DAY_END, "day", "endfield")).toBe(
|
||||||
|
Date.parse("2026-08-19T09:00:00.000Z"),
|
||||||
|
);
|
||||||
|
expect(latestBoundaryMs(DAY_END, "day", "r1999")).toBe(
|
||||||
|
Date.parse("2026-08-19T10:00:00.000Z"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("leaves an exact boundary exactly where the source put it", () => {
|
||||||
|
const exact = "2026-08-19T10:59:59.000Z";
|
||||||
|
expect(latestBoundaryMs(exact, "exact", "genshin")).toBe(Date.parse(exact));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user