From 240e4d59d16d2d9a2a400f1456c8df33fd853ba3 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Wed, 19 Aug 2026 04:52:34 +0200 Subject: [PATCH] docs: all four Fandom sources have a snapshot now MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit landed the first live pages for nikke and nikki, which retires the last version of a sentence that has been wrong twice in one session — it counted which of the four had ever been fetched, and that number kept moving underneath it. So it no longer counts. What matters about these four is not how many have a snapshot but that every one was taken by hand and always will be: their freshness is exactly as old as the last person who ran the refresh themselves, because the schedule cannot read Fandom's robots.txt. Points at --assume-robots-on-403, which is what makes that run possible. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7e49026..5a9963a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -367,11 +367,11 @@ to our fetcher, and a real headless browser gets a Cloudflare managed challenge Two consequences, and neither is a licence to work around it: - **All four** built Fandom sources report `skipped_robots` on **every** run, from any address we - have, so `r1999`, `fgo`, `nikke` and `nikki` are permanently fixture-backed until someone - refreshes them from an address Fandom serves. `r1999` and `fgo` have snapshots, both taken by hand - on 2026-08-19; `nikke` and `nikki` have never had one and are served from their fixtures. That is - not a source being down: nothing is broken, and nothing will ever update any of the four on a - schedule. + have, so `r1999`, `fgo`, `nikke` and `nikki` only ever move when someone refreshes them from an + address Fandom serves. All four have a snapshot as of 2026-08-19 and every one of them was taken + by hand. That is not a source being down — nothing is broken — but nothing will ever update these + four on a schedule either, so their freshness is exactly as old as the last person who ran + `bun run refresh` themselves. `--assume-robots-on-403` below is what makes that run possible. - A **new** Fandom source can still be added, but only once someone reads that wiki's `robots.txt` from an address Fandom serves and records it here. That is exactly how Nikke was cleared on 2026-08-19: the file was read in a browser, is the standard Fandom file — no `Disallow: /` for