refresh: ask who authorised a run, not whether it is CI

Both overrides stand in for a person deciding something, and the gate that
enforced it asked the coarser question. `isCi()` got the case that matters
wrong: a `workflow_dispatch` sets CI=true, so somebody clicking "Run workflow"
was refused exactly as the cron was, which made both flags unreachable from the
workflow at all. A dispatch is a person, and a better-evidenced one than a local
shell — GitHub records which one, and the run log keeps it.

`runAttendance` draws the line where it belongs: a local shell is a person, a
dispatch is a person and names the actor, and a schedule — or any other runner
event, erring that way on purpose — is not. The run now prints which override
was used and who authorised it, because a local shell leaves no trace anybody
else can read.

The schedule stays refused, and for the robots override that is not ceremony.
From an address that gets a challenge we never receive robots.txt at all, so a
cron standing on the recorded permission has no way to notice the host
withdrawing it, and that permission has no expiry. The challenge-or-refusal
narrowing does not close this: a plain 403 still stops us, but a robots.txt
edited to disallow us would be invisible, because a challenge arrives instead of
a file. A person re-reading it is the only thing that ever re-validates it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
2026-08-20 00:01:11 +02:00
co-authored by Claude Opus 5
parent d0b0f2e00c
commit 2fb944135d
4 changed files with 164 additions and 34 deletions
+72 -26
View File
@@ -676,19 +676,51 @@ export async function rebuildFeedViaScript(): Promise<void> {
if (code !== 0) throw new Error(`build-feed exited ${code}`);
}
/** Whether a person authorised this run, and who, for the override gates. */
export type Attendance =
| { readonly attended: true; readonly by: string }
| { readonly attended: false; readonly why: string };
/**
* Are we on a runner rather than at somebody's keyboard?
* Did a person ask for this run, or did a schedule?
*
* Both variables, because `CI` is the convention every runner sets and
* `GITHUB_ACTIONS` is the one this repo's workflow guarantees. Erring towards
* "yes" is the safe direction: the only thing it costs is refusing an
* interactive-only flag to a human whose shell exports `CI`.
* The two overrides below stand in for somebody deciding something — that a
* page has moved and they want it now, or that they have read a robots.txt in a
* browser. What must never happen is a *schedule* asserting either one every
* cycle, because an override on every run is not an override, it is the new
* default with extra steps (AGENTS.md § Scraping conduct).
*
* This used to be `isCi()`, which asked the coarser question and got the wrong
* answer for the case that matters: a `workflow_dispatch` sets `CI=true`, so a
* human clicking "Run workflow" was refused exactly like the cron was. A
* dispatch is a person, and a better-evidenced one than a local shell — GitHub
* records who in `GITHUB_ACTOR`, and the run log keeps it.
*
* Erring towards "unattended" stays the safe direction: any runner event that is
* not a dispatch is treated as a schedule, so a `push`- or `repository_dispatch`-
* triggered run gets no overrides either.
*/
function isCi(): boolean {
return (
process.env["CI"] !== undefined && process.env["CI"] !== "" ||
process.env["GITHUB_ACTIONS"] === "true"
);
export function runAttendance(
env: Record<string, string | undefined> = process.env,
): Attendance {
const onRunner =
(env["CI"] !== undefined && env["CI"] !== "") ||
env["GITHUB_ACTIONS"] === "true";
if (!onRunner) return { attended: true, by: "a local shell" };
const event = env["GITHUB_EVENT_NAME"];
if (event === "workflow_dispatch") {
const actor = env["GITHUB_ACTOR"];
return {
attended: true,
by:
actor === undefined || actor === ""
? "a manual dispatch"
: `a manual dispatch by ${actor}`,
};
}
return { attended: false, why: event === undefined || event === "" ? "CI" : event };
}
interface Args {
@@ -811,33 +843,47 @@ async function main(): Promise<number> {
return 2;
}
// The flag stands in for a human having read a robots.txt in a browser and
// written it down. There is no human on a runner, and a scheduled job quietly
// asserting a permission nobody re-checked is how "temporary" becomes
// permanent — so CI is refused the option outright rather than trusted not to
// pass it. AGENTS.md § Scraping conduct is the argument.
// Same reasoning as the robots override: this is a person deciding, once,
// that a page has moved and they want it now. A schedule deciding that every
// run is just a shorter interval with extra steps, and the interval is the
// obligation.
if (args.force && isCi()) {
// Both overrides are a person deciding something — that a page has moved and
// they want it now, or that they have read a robots.txt in a browser and
// written it down. Neither may be asserted by a *schedule*: an override on
// every cycle is not an override, it is the new default with extra steps, and
// for the robots one it would be worse than that. From an address that gets a
// challenge we never receive the file at all, so a cron standing on a recorded
// permission has no way to notice the host withdrawing it — the recorded
// permission has no expiry, and a person re-reading it is the only thing that
// ever re-validates it. AGENTS.md § Scraping conduct is the argument.
const attendance = runAttendance();
if (args.force && !attendance.attended) {
console.error(
"--force is interactive-only and refused under CI.\n" +
`--force needs a person to ask for it; this run came from ` +
`${attendance.why}.\n` +
"The 6h floor is what the scheduled runner is for; change the schedule, " +
"not the floor.",
"not the floor. Dispatch the workflow by hand to pass it.",
);
return 2;
}
if (args.assumeRobotsOn403 && isCi()) {
if (args.assumeRobotsOn403 && !attendance.attended) {
console.error(
"--assume-robots-on-403 is interactive-only and refused under CI.\n" +
"It asserts a permission a person read by hand; run the refresh from a " +
"machine the host serves instead.",
`--assume-robots-on-403 needs a person to ask for it; this run came from ` +
`${attendance.why}.\n` +
"It asserts a permission a person read by hand, and a schedule cannot " +
"re-read it. Dispatch the workflow by hand to pass it.",
);
return 2;
}
// Who authorised the override is part of the record, not a detail. A local
// shell leaves no trace anybody else can read; a dispatch names the actor.
if (attendance.attended && (args.force || args.assumeRobotsOn403)) {
const used = [
args.force ? "--force" : null,
args.assumeRobotsOn403 ? "--assume-robots-on-403" : null,
].filter((f): f is string => f !== null);
console.log(` override: ${used.join(", ")} — authorised by ${attendance.by}`);
}
const store = new SnapshotStore(args.root);
const robots = new RobotsCache({
userAgent: args.userAgent,