fix(refresh): only 200 is a page, so a 202 challenge names itself

The health check earned its keep on the first run: six sources broken, four cycles each, "last confirmed never" — and a last status of **202**. That is game8.co's edge answering a GitHub runner with a bot-management body instead of the wiki. `response.ok` admitted it. A 202 therefore passed the status gate as a document, reached the parser, yielded nothing, and was reported as "kept previous snapshot; new body yielded 0 events" — an accurate description of the symptom that never mentioned the status explaining it, which is why this looked like a parser problem for days. Require 200: 202 means the request was accepted for processing, 204 has no body, 206 is a fragment, and none of them is a page. The rejection note already names the Server header, so the next cycle reads `HTTP 202 (AkamaiGHost)` instead. Recorded in CLAUDE.md § Scraping conduct, including that this must not be worked around. The per-host spacing added alongside does not help and was never going to — a 202 on a cycle's first request is address reputation, not rate. Browser -shaped headers or a residential egress would be defeating an access control, which is the ground uma.moe was declined on, and game8's robots.txt is permissive, which makes doing it there worse rather than better. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-17 22:05:05 +02:00
parent 988f72a56f
commit 54052d50c4
4 changed files with 51 additions and 1 deletions
@@ -214,6 +214,30 @@ describe("one request per source per six hours", () => {
    expect(calls).toHaveLength(1);
  });

+  test("a 2xx that is not 200 is not a page", async () => {
+    // game8.co's edge answers a GitHub runner with 202 and a bot-management
+    // body. `response.ok` admitted it, so it reached the parser and was reported
+    // as "yielded 0 events" — the symptom, while the status that explained it
+    // was never named. Six sources read that way for days.
+    await seed("<html><event></event></html>", "2026-08-01T00:00:00.000Z", 1);
+    const { opts } = options({
+      responder: () =>
+        new Response("<html>checking your browser</html>", {
+          status: 202,
+          headers: { Server: "AkamaiGHost" },
+        }),
+    });
+    const summary = await runRefresh(opts);
+
+    expect(summary.outcomes[0]?.result).toBe("failed");
+    expect(summary.outcomes[0]?.note).toContain("HTTP 202");
+    expect(summary.outcomes[0]?.note).toContain("AkamaiGHost");
+    // The page we already hold is still the page.
+    expect((await store.read("genshin-game8-events"))?.html).toBe(
+      "<html><event></event></html>",
+    );
+  });
+
  test("never retries a failure inside the same cycle", async () => {
    const { opts, calls } = options({
      responder: () => new Response("nope", { status: 500 }),