Tell a broken source apart from a stale one

CI failed on Infinity Nikki yielding no events, and was wrong to. Its
snapshot parses to six events; every one of them had ended by the morning
the build ran. The parser is fine — `--now 2026-08-14` gives six, today
gives none — and the page simply has nothing current left on it.

eventCount is counted after expired events are dropped, so "this parser has
stopped reading a redesigned page" and "this page's events have all
finished" arrived as the same zero. Only the first means our code is wrong,
and only the first should redden a build. So the feed now records what each
document yields parsed as of its own capture date, before expiry, and the
check fails on that instead.

Nikki cannot refresh itself out of this, either: docs/SOURCES.md records
that Fandom refuses the Actions runner. A lane with an empty calendar is a
real problem, but it is a refresh problem, so it is reported on the build
log and left visible rather than thrown.

parsedCount is nullable and defaulted, never required: the client validates
the whole feed with safeParse and the service worker serves the last feed it
downloaded, so a required field would have made every cached feed fail
validation and taken the offline promise with it. Null also covers a source
whose bytes were never confirmed live — there is no date to parse "as of",
and a build is not failed on missing information.

The rule moved to shared/feed.ts. A test did pin the old one, by grepping
the workflow for the string — which proved the check existed, never that it
was right.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

.github/workflows/ci.yml

@@ -72,14 +72,25 @@ jobs:
             }
             // Per source, not just in total: nine healthy sources hide a tenth
             // that has gone to zero, and the total stays comfortably over the
-            // floor while one game shows an empty calendar. A live snapshot
-            // that parsed to nothing is refused upstream and never stored, so
-            // zero here means the fixture stopped parsing — a regression in our
-            // code, not a quiet week for that game.
-            const empty = feed.sources.filter((s) => s.eventCount === 0);
-            if (empty.length > 0) {
+            // floor while one game shows an empty calendar.
+            //
+            // Which zero it is decides whether this build should fail, and
+            // the rule lives in shared/feed.ts rather than here. It was
+            // inline, and a test did pin it — by grepping this file for the
+            // string. That proved the check existed, never that it was right,
+            // and it was not: it read eventCount, which is counted after
+            // expiry, so a page whose events had all simply ended reddened the
+            // build. Behaviour belongs where behaviour can be exercised.
+            const { brokenSources, staleSources } = await import("./src/shared/feed.ts");
+            for (const s of staleSources(feed.sources)) {
+              console.log(
+                `  note: ${s.sourceId} parsed ${s.parsedCount} events, all of them ended — stale page, not a fault`,
+              );
+            }
+            const broken = brokenSources(feed.sources);
+            if (broken.length > 0) {
               throw new Error(
-                `sources yielding no events: ${empty.map((s) => s.sourceId).join(", ")}`,
+                `sources parsing to nothing: ${broken.map((s) => s.sourceId).join(", ")}`,
               );
             }
           '

README.md

@@ -341,10 +341,16 @@ Until then the `pages` job is skipped and the pipeline stays green. Pages is una
 repositories on the free plan. Both steps are done here, and the deploy lands at
 <https://stereotypicalcat.github.io/gacha-event-tracker/>.
 
-The feed job fails if the event count collapses. A source that quietly stops yielding events is the
-failure mode a parser-only pipeline is most prone to, and nothing else would surface it. Tests run
-offline against checked-in fixtures, so a red pipeline always means the code changed rather than a
-wiki being down.
+The feed job fails if the event count collapses, or if any single source parses to nothing — nine
+healthy sources hide a tenth that has gone quiet, and the total stays comfortably over the floor
+while one game shows an empty calendar. That is the failure mode a parser-only pipeline is most
+prone to, and nothing else would surface it.
+
+It distinguishes that from a source whose events have all simply *ended*, which is a stale page
+rather than a broken parser and is reported instead of thrown. The two used to be the same zero,
+because the count was taken after expired events were dropped — so Infinity Nikki reddened the
+build the morning its last event finished. Tests run offline against checked-in fixtures, so a red
+pipeline always means the code changed rather than a wiki being down.
 
 ### Refreshing the data
 

scripts/build-feed.ts

@@ -69,6 +69,25 @@ for (const adapter of ADAPTERS) {
     game: adapter.game,
   });
 
+  // Parsed a second time as of the document's own capture date, when nothing
+  // in it had expired yet. That figure is what separates "this parser has
+  // stopped reading the page" from "this page's events have all finished
+  // since it was captured" — the two are the same zero once expiry has been
+  // applied, and only the first means our code is wrong.
+  // Null when we do not know when these bytes were current: there is no date
+  // to parse "as of", and inventing one would manufacture a figure the check
+  // then trusts. Unknown is a real answer here, and `brokenSources` declines
+  // to fail a build on it.
+  const parsedCount =
+    at === null
+      ? null
+      : adapter.parse(html, {
+          now: at,
+          sourceUrl: adapter.url,
+          sourceId: adapter.id,
+          game: adapter.game,
+        }).length;
+
   const groups = byGame.get(adapter.game) ?? [];
   groups.push(events);
   byGame.set(adapter.game, groups);
@@ -81,9 +100,18 @@ for (const adapter of ADAPTERS) {
     // this source has never been refreshed.
     lastSuccessAt: at,
     eventCount: events.length,
+    parsedCount,
   });
 
-  console.log(`  ${adapter.id.padEnd(24)} ${String(events.length).padStart(3)} events  ← ${file}`);
+  // A source that parsed events and then lost them all to the calendar says
+  // so on the build log, because a bare "0 events" reads as a fault.
+  const note =
+    events.length === 0 && parsedCount !== null && parsedCount > 0
+      ? `  (all ${parsedCount} have ended — stale page)`
+      : "";
+  console.log(
+    `  ${adapter.id.padEnd(24)} ${String(events.length).padStart(3)} events  ← ${file}${note}`,
+  );
 }
 
 const events: GachaEvent[] = [];

src/shared/feed.ts

@@ -15,7 +15,25 @@ export const SourceHealth = z.object({
   game: GameId,
   url: z.string().url(),
   lastSuccessAt: z.string().datetime().nullable(),
+  /** Events this source contributed to the feed — after expired ones are dropped. */
   eventCount: z.number().int().nonnegative(),
+  /**
+   * Events the document yields when parsed as of its own capture date, before
+   * anything is dropped for having ended.
+   *
+   * The pair is what separates a broken source from a stale one. `eventCount`
+   * alone cannot: a parser that has stopped reading a redesigned page and a
+   * page whose every event has since finished both report zero, and only the
+   * first means our code is wrong.
+   *
+   * **Nullable and defaulted, never required.** The client validates the whole
+   * feed with `EventFeed.safeParse`, and the service worker serves the last
+   * feed it downloaded — so a required field here would make every cached feed
+   * fail validation and take the offline promise with it. Null means an older
+   * feed that never recorded this, which is an absence of information rather
+   * than evidence of a fault.
+   */
+  parsedCount: z.number().int().nonnegative().nullable().default(null),
 });
 
 export const EventFeed = z.object({
@@ -88,3 +106,36 @@ export function freshness(
 
   return { refreshedAt, stale };
 }
+
+/**
+ * Sources whose document yielded nothing at all.
+ *
+ * This is the failure a parser-only pipeline is most prone to and that nothing
+ * else would surface: a page is redesigned, the parser reads it as empty, and
+ * one game's calendar goes blank while the total stays comfortably healthy.
+ * Worth failing a build over.
+ *
+ * A source whose events have merely all ended is not this, and CI said it was
+ * — the check read `eventCount`, which is measured after expiry, so a stale
+ * page and a broken parser arrived as the same zero. Only an explicit zero
+ * counts here; a null is an older feed that never recorded the figure, and
+ * failing on missing information would be the same mistake in a new place.
+ */
+export function brokenSources(sources: readonly SourceHealth[]): SourceHealth[] {
+  return sources.filter((s) => s.parsedCount === 0);
+}
+
+/**
+ * Sources that parsed fine but have nothing current left to show.
+ *
+ * A real problem — that lane renders an empty calendar — but a refresh
+ * problem rather than a code one, and some of these cannot be refreshed from
+ * CI at all (`docs/SOURCES.md` records which hosts refuse the runner). So it
+ * is reported and left visible rather than thrown, the same way the app shows
+ * a stale timestamp rather than pretending the calendar is current.
+ */
+export function staleSources(sources: readonly SourceHealth[]): SourceHealth[] {
+  return sources.filter(
+    (s) => s.parsedCount !== null && s.parsedCount > 0 && s.eventCount === 0,
+  );
+}

test/custom-ui.test.tsx

@@ -238,6 +238,8 @@ describe("Colophon freshness notice (PRD F7)", () => {
     url: "https://game8.co/games/Genshin-Impact/archives/301601",
     lastSuccessAt: new Date(NOW - 3 * HOUR).toISOString(),
     eventCount: 9,
+
+    parsedCount: 9,
   };
 
   test("states when the data was refreshed, unprompted", () => {

test/feed.test.ts

@@ -1,6 +1,8 @@
 import { describe, expect, test } from "bun:test";
 import {
+  brokenSources,
   freshness,
+  staleSources,
   STALE_AFTER_MS,
   type SourceHealth,
 } from "../src/shared/feed.ts";
@@ -30,6 +32,8 @@ function source(
     url: "https://example.test/events",
     lastSuccessAt,
     eventCount: 3,
+
+    parsedCount: 3,
   };
 }
 
@@ -114,3 +118,57 @@ describe("freshness", () => {
     expect(freshness([], NOW)).toEqual({ refreshedAt: null, stale: [] });
   });
 });
+
+describe("telling a broken source from a stale one", () => {
+  // CI failed on Infinity Nikki yielding nothing, and the check was wrong to.
+  // Its snapshot parses to seven events; every one of them had simply ended by
+  // the day the build ran. `eventCount` is measured after expiry, so "our
+  // parser broke" and "this source has nothing current left" arrived as the
+  // same zero — and only the first is a reason to redden a build.
+  const health = (over: Partial<SourceHealth>): SourceHealth => ({
+    sourceId: "nikki-fandom-events",
+    game: "nikki" as GameId,
+    url: "https://example.test/nikki",
+    lastSuccessAt: "2026-08-19T00:00:00.000Z",
+    eventCount: 0,
+    parsedCount: 7,
+    ...over,
+  });
+
+  test("a source that parsed nothing is broken", () => {
+    // The failure this check exists for: a page changed shape and the parser
+    // now reads it as empty. Nothing to publish and nothing to expire.
+    expect(brokenSources([health({ parsedCount: 0 })]).map((s) => s.sourceId)).toEqual([
+      "nikki-fandom-events",
+    ]);
+  });
+
+  test("a source whose events have all ended is not broken", () => {
+    // The Nikki case exactly. The parser did its job; the calendar moved past
+    // everything the page still lists.
+    expect(brokenSources([health({})])).toEqual([]);
+  });
+
+  test("a healthy source is neither", () => {
+    const ok = health({ eventCount: 5, parsedCount: 5 });
+    expect(brokenSources([ok])).toEqual([]);
+    expect(staleSources([ok])).toEqual([]);
+  });
+
+  test("a source with nothing current left is reported as stale", () => {
+    // Worth saying out loud — a lane showing an empty calendar is a real
+    // problem — but it is a refresh problem, not a code one, so it is
+    // reported rather than thrown.
+    expect(staleSources([health({})]).map((s) => s.sourceId)).toEqual([
+      "nikki-fandom-events",
+    ]);
+  });
+
+  test("a feed that never recorded the count is not called broken", () => {
+    // An older feed — one the service worker cached before this field existed
+    // — says nothing either way, and absence of information is not evidence of
+    // a fault.
+    expect(brokenSources([health({ parsedCount: null })])).toEqual([]);
+    expect(staleSources([health({ parsedCount: null })])).toEqual([]);
+  });
+});

test/issue-templates.test.tsx

@@ -166,6 +166,8 @@ describe("the app's links into them", () => {
             url: "https://game8.co/games/Genshin-Impact/archives/301601",
             lastSuccessAt: new Date(NOW - 3 * 60 * 60 * 1000).toISOString(),
             eventCount: 9,
+
+            parsedCount: 9,
           },
         ]}
         now={NOW}

test/refresh.test.ts

@@ -1047,12 +1047,16 @@ describe("the workflows that drive the refresh", () => {
     expect(refresh.slice(health)).toContain("exit 1");
   });
 
-  test("ci.yml fails when any one source yields no events", async () => {
+  test("ci.yml fails a source that parsed nothing, not one whose events ended", async () => {
     // The total-event floor is blind to one source going to zero while nine
     // others hold the number up, which shows the reader an empty calendar for
-    // that game.
+    // that game. But the old rule was `eventCount === 0`, counted after
+    // expiry, so a page whose events had all merely finished failed the build
+    // too. Grepping this file can only say which rule is wired up; whether it
+    // is the right one is exercised in test/feed.test.ts.
     const ci = await read("ci.yml");
-    expect(ci).toContain("eventCount === 0");
+    expect(ci).toContain("brokenSources");
+    expect(ci).not.toContain("eventCount === 0");
   });
 });