refresh: ask who authorised a run, not whether it is CI

Both overrides stand in for a person deciding something, and the gate that
enforced it asked the coarser question. `isCi()` got the case that matters
wrong: a `workflow_dispatch` sets CI=true, so somebody clicking "Run workflow"
was refused exactly as the cron was, which made both flags unreachable from the
workflow at all. A dispatch is a person, and a better-evidenced one than a local
shell — GitHub records which one, and the run log keeps it.

`runAttendance` draws the line where it belongs: a local shell is a person, a
dispatch is a person and names the actor, and a schedule — or any other runner
event, erring that way on purpose — is not. The run now prints which override
was used and who authorised it, because a local shell leaves no trace anybody
else can read.

The schedule stays refused, and for the robots override that is not ceremony.
From an address that gets a challenge we never receive robots.txt at all, so a
cron standing on the recorded permission has no way to notice the host
withdrawing it, and that permission has no expiry. The challenge-or-refusal
narrowing does not close this: a plain 403 still stops us, but a robots.txt
edited to disallow us would be invisible, because a challenge arrives instead of
a file. A person re-reading it is the only thing that ever re-validates it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

AGENTS.md

@@ -262,7 +262,7 @@ Sources are community wikis. Treat them as a guest would:
 
 - Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
 - One request per source per refresh cycle, minimum 6 hours apart. **`--force` sets that floor
-  aside for one run, and only a person at a keyboard may pass it** — it is refused under CI, because
+  aside for one run, and only a person may pass it** — it is refused on an unattended run, because
   a schedule that forces every cycle is just a shorter interval with extra steps, and the interval is
   the obligation. What makes it defensible is what it does *not* change: conditional headers still go
   out, so a page that has not moved costs the host a `304` rather than a re-serve, and per-host
@@ -470,11 +470,27 @@ that helps: `bun run refresh --assume-robots-on-403` treats **an interstitial ch
 `/robots.txt` itself as the permission recorded above rather than failing closed. It is not a
 workaround for a host that turned us away — it never overrides a `robots.txt` we could read, so a
 file that disallows us still says no, and it does nothing at all for game8.co, whose robots.txt reads
-fine and welcomes us while its edge refuses the pages. It is refused under CI, because what it stands
+fine and welcomes us while its edge refuses the pages. Every host it applied to is named in the run's
-in for is a person having read a file in a browser, and there is no person on a runner. Every host it
+warnings, so it stays a thing somebody decided this morning rather than a default. Nothing else
-applied to is named in the run's warnings, so it stays a thing somebody decided this morning rather
+relaxes: one request per source, six hours apart, spaced per host, no retries.
-than a default. Nothing else relaxes: one request per source, six hours apart, spaced per host, no
+
-retries.
+**Who may pass it is a question about the person, not the machine.** Both overrides are refused on an
+*unattended* run and available to a person, and `runAttendance` (`scripts/refresh-sources.ts`) draws
+that line: a local shell is a person, a `workflow_dispatch` is a person and GitHub records which one
+in `GITHUB_ACTOR`, and a `schedule` — or any other runner event — is not. This replaced a blanket
+`isCi()` check, which asked the coarser question and got the case that matters wrong: a dispatch sets
+`CI=true`, so somebody clicking "Run workflow" was refused exactly as the cron was, and the overrides
+were unreachable from the workflow at all. The run prints which override was used and who authorised
+it.
+
+**The cron stays refused, and for the robots override that is not ceremony.** From an address that
+gets a challenge we never receive `robots.txt` at all — so a schedule standing on the recorded
+permission has no way to notice the host withdrawing it. The recorded permission has no expiry, and a
+person re-reading the file in a browser is the only thing that ever re-validates it. A twice-daily job
+asserting it forever would be fetching on a snapshot of consent taken on 2026-08-19. Note what the
+challenge-vs-refusal narrowing does and does not cover here: a plain `403` still stops us, but a
+`robots.txt` *edited* to disallow us would be invisible, because we get a challenge instead of a file.
+That gap is the whole reason a person has to be the one asking.
 
 **A `403` is two answers wearing one status code, and only one of them is covered.** A managed
 challenge means "we cannot tell what you are" — the question a human answers by reading the file in a
@@ -661,7 +677,7 @@ Fate/Grand Order problem arriving through a source that looks like it answered t
 `scripts/refresh-sources.ts` enforces all of the above in code — the 6h floor (except under the
 opt-in `--force` above), one request, no retries, conditional headers, per-host spacing, robots
 (failing closed when `robots.txt` cannot be read, except under the opt-in `--assume-robots-on-403`
-described in § Fandom, which covers a challenged `403` and never a plain refusal). Both overrides are interactive-only, refused under CI, and reported by name
+described in § Fandom, which covers a challenged `403` and never a plain refusal). Both overrides are refused on an unattended run — a `schedule` gets neither, a person or a `workflow_dispatch` may pass both (`runAttendance`) — and reported by name
 in the run's warnings — an override that reports nothing is one nobody withdraws. Anything that would make it fetch more often is a change to this section first.
 
 **A source down is a warning; a source down for days is a broken build.** One wiki failing must

docs/INGESTION.md

@@ -292,7 +292,12 @@ section but must never claim the event title.
     challenge page's markers; a `403` whose body cannot be read is unclassifiable and therefore not
     excused.
   - It never overrides a `robots.txt` we *could* read, so a file that disallows us still says no.
-  - It is refused under CI, because it stands in for a human and there is none on a runner.
+  - **It is refused on an unattended run, not on every runner.** `runAttendance` asks who asked: a
+    local shell and a `workflow_dispatch` are both a person (the latter better evidenced — GitHub
+    names the actor), while a `schedule` or any other runner event is not. The `schedule` refusal is
+    load-bearing rather than ceremonial: from a challenged address `robots.txt` never arrives, so a
+    cron standing on the recorded permission could never see the host withdraw it, and the recorded
+    permission has no expiry.
 
   Every host it applied to is named in the run's warnings and in `summary.assumedRobots` — an
   override that reports nothing is one nobody withdraws. It changes no other obligation: still one

scripts/refresh-sources.ts

@@ -676,19 +676,51 @@ export async function rebuildFeedViaScript(): Promise<void> {
   if (code !== 0) throw new Error(`build-feed exited ${code}`);
 }
 
+/** Whether a person authorised this run, and who, for the override gates. */
+export type Attendance =
+  | { readonly attended: true; readonly by: string }
+  | { readonly attended: false; readonly why: string };
+
 /**
- * Are we on a runner rather than at somebody's keyboard?
+ * Did a person ask for this run, or did a schedule?
  *
- * Both variables, because `CI` is the convention every runner sets and
+ * The two overrides below stand in for somebody deciding something — that a
- * `GITHUB_ACTIONS` is the one this repo's workflow guarantees. Erring towards
+ * page has moved and they want it now, or that they have read a robots.txt in a
- * "yes" is the safe direction: the only thing it costs is refusing an
+ * browser. What must never happen is a *schedule* asserting either one every
- * interactive-only flag to a human whose shell exports `CI`.
+ * cycle, because an override on every run is not an override, it is the new
+ * default with extra steps (AGENTS.md § Scraping conduct).
+ *
+ * This used to be `isCi()`, which asked the coarser question and got the wrong
+ * answer for the case that matters: a `workflow_dispatch` sets `CI=true`, so a
+ * human clicking "Run workflow" was refused exactly like the cron was. A
+ * dispatch is a person, and a better-evidenced one than a local shell — GitHub
+ * records who in `GITHUB_ACTOR`, and the run log keeps it.
+ *
+ * Erring towards "unattended" stays the safe direction: any runner event that is
+ * not a dispatch is treated as a schedule, so a `push`- or `repository_dispatch`-
+ * triggered run gets no overrides either.
  */
-function isCi(): boolean {
+export function runAttendance(
-  return (
+  env: Record<string, string | undefined> = process.env,
-    process.env["CI"] !== undefined && process.env["CI"] !== "" ||
+): Attendance {
-    process.env["GITHUB_ACTIONS"] === "true"
+  const onRunner =
-  );
+    (env["CI"] !== undefined && env["CI"] !== "") ||
+    env["GITHUB_ACTIONS"] === "true";
+  if (!onRunner) return { attended: true, by: "a local shell" };
+
+  const event = env["GITHUB_EVENT_NAME"];
+  if (event === "workflow_dispatch") {
+    const actor = env["GITHUB_ACTOR"];
+    return {
+      attended: true,
+      by:
+        actor === undefined || actor === ""
+          ? "a manual dispatch"
+          : `a manual dispatch by ${actor}`,
+    };
+  }
+
+  return { attended: false, why: event === undefined || event === "" ? "CI" : event };
 }
 
 interface Args {
@@ -811,33 +843,47 @@ async function main(): Promise<number> {
     return 2;
   }
 
-  // The flag stands in for a human having read a robots.txt in a browser and
+  // Both overrides are a person deciding something — that a page has moved and
-  // written it down. There is no human on a runner, and a scheduled job quietly
+  // they want it now, or that they have read a robots.txt in a browser and
-  // asserting a permission nobody re-checked is how "temporary" becomes
+  // written it down. Neither may be asserted by a *schedule*: an override on
-  // permanent — so CI is refused the option outright rather than trusted not to
+  // every cycle is not an override, it is the new default with extra steps, and
-  // pass it. AGENTS.md § Scraping conduct is the argument.
+  // for the robots one it would be worse than that. From an address that gets a
-  // Same reasoning as the robots override: this is a person deciding, once,
+  // challenge we never receive the file at all, so a cron standing on a recorded
-  // that a page has moved and they want it now. A schedule deciding that every
+  // permission has no way to notice the host withdrawing it — the recorded
-  // run is just a shorter interval with extra steps, and the interval is the
+  // permission has no expiry, and a person re-reading it is the only thing that
-  // obligation.
+  // ever re-validates it. AGENTS.md § Scraping conduct is the argument.
-  if (args.force && isCi()) {
+  const attendance = runAttendance();
+
+  if (args.force && !attendance.attended) {
     console.error(
-      "--force is interactive-only and refused under CI.\n" +
+      `--force needs a person to ask for it; this run came from ` +
+        `${attendance.why}.\n` +
         "The 6h floor is what the scheduled runner is for; change the schedule, " +
-        "not the floor.",
+        "not the floor. Dispatch the workflow by hand to pass it.",
     );
     return 2;
   }
 
-  if (args.assumeRobotsOn403 && isCi()) {
+  if (args.assumeRobotsOn403 && !attendance.attended) {
     console.error(
-      "--assume-robots-on-403 is interactive-only and refused under CI.\n" +
+      `--assume-robots-on-403 needs a person to ask for it; this run came from ` +
-        "It asserts a permission a person read by hand; run the refresh from a " +
+        `${attendance.why}.\n` +
-        "machine the host serves instead.",
+        "It asserts a permission a person read by hand, and a schedule cannot " +
+        "re-read it. Dispatch the workflow by hand to pass it.",
     );
     return 2;
   }
 
+  // Who authorised the override is part of the record, not a detail. A local
+  // shell leaves no trace anybody else can read; a dispatch names the actor.
+  if (attendance.attended && (args.force || args.assumeRobotsOn403)) {
+    const used = [
+      args.force ? "--force" : null,
+      args.assumeRobotsOn403 ? "--assume-robots-on-403" : null,
+    ].filter((f): f is string => f !== null);
+    console.log(`  override:   ${used.join(", ")} — authorised by ${attendance.by}`);
+  }
+
   const store = new SnapshotStore(args.root);
   const robots = new RobotsCache({
     userAgent: args.userAgent,

test/refresh.test.ts

@@ -8,6 +8,7 @@ import {
   DEFAULT_HOST_GAP_MS,
   outputs,
   parseArgs,
+  runAttendance,
   runRefresh,
   stepSummary,
   type RefreshOptions,
@@ -1076,3 +1077,65 @@ describe("flags", () => {
     );
   });
 });
+
+/**
+ * The override gates ask who authorised the run, not whether a runner is
+ * present. That distinction is the whole reason the workflow can offer these
+ * toggles at all: a `workflow_dispatch` sets `CI=true`, so the old `isCi()`
+ * check refused a human clicking "Run workflow" exactly as it refused the cron.
+ *
+ * What must not regress is the other half — a *schedule* must never be able to
+ * assert either override, because an override on every cycle is the new default
+ * with extra steps, and for the robots one a cron cannot re-read the permission
+ * it would be standing on.
+ */
+describe("runAttendance", () => {
+  test("a local shell is a person", () => {
+    const a = runAttendance({});
+    expect(a.attended).toBe(true);
+    expect(a.attended && a.by).toBe("a local shell");
+  });
+
+  test("a scheduled run is not, and says so", () => {
+    const a = runAttendance({ CI: "true", GITHUB_EVENT_NAME: "schedule" });
+    expect(a.attended).toBe(false);
+    expect(!a.attended && a.why).toBe("schedule");
+  });
+
+  test("a manual dispatch is a person, and names the actor", () => {
+    const a = runAttendance({
+      CI: "true",
+      GITHUB_ACTIONS: "true",
+      GITHUB_EVENT_NAME: "workflow_dispatch",
+      GITHUB_ACTOR: "StereotypicalCat",
+    });
+    expect(a.attended).toBe(true);
+    expect(a.attended && a.by).toBe("a manual dispatch by StereotypicalCat");
+  });
+
+  test("a dispatch with no actor recorded is still a person", () => {
+    const a = runAttendance({ CI: "true", GITHUB_EVENT_NAME: "workflow_dispatch" });
+    expect(a.attended).toBe(true);
+    expect(a.attended && a.by).toBe("a manual dispatch");
+  });
+
+  test("any other runner event counts as unattended", () => {
+    // Erring towards "no person" — a push- or repository_dispatch-triggered run
+    // gets no overrides either, so a schedule cannot be dressed up as one.
+    for (const event of ["push", "repository_dispatch", "pull_request", ""]) {
+      const a = runAttendance({ CI: "true", GITHUB_EVENT_NAME: event });
+      expect(`${event}: ${a.attended}`).toBe(`${event}: false`);
+    }
+  });
+
+  test("GITHUB_ACTIONS alone is enough to count as a runner", () => {
+    const a = runAttendance({ GITHUB_ACTIONS: "true", GITHUB_EVENT_NAME: "schedule" });
+    expect(a.attended).toBe(false);
+  });
+
+  test("an empty CI variable is not a runner", () => {
+    // `CI=` is how a shell unsets it in practice; treating it as a runner would
+    // refuse a person their own flags.
+    expect(runAttendance({ CI: "" }).attended).toBe(true);
+  });
+});