Game8 still lists Y1INABQB6DEX, 2BJ64QRZ7RT8 and PLDXIB886Y1S as active
Genshin codes after the game refused them. CODE_CHECKS records the
owner's in-game check; build-codes stamps the matching code confirmedDead,
and the Codes tab shows it expired: "Confirmed expired in game, 2026-10-20."
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Game8 codes pages for Star Rail, ZZZ, Wuthering Waves, Endfield, NTE,
P5X, Chaos Zero Nightmare and Umamusume, read by one table parser told
each page's active-codes heading. Each page was fetched once and pinned
as a fixture. Infinity Nikki's page is stale (still titled July 2025)
and is left out.
The Codes tab links the publisher's own redeem page where one exists
(Genshin, Star Rail, ZZZ) and gives the in-game route for every game,
and names the server on a redeemed checkbox only when there are two.
Settings uses the width of a large screen: content up to 1320px and the
menu column running the full height.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The new reader app replaces the old UI: Checklist with a rail (focus
pills, Next to expire, today's dailies, pinned routines), Timeline in a
new and a classic look, Routines, Archive and a Settings area, in a
Structured (default) and a Relaxed interface. It follows the approved
Round 6 B and Round 7 mockups, corrected screen by screen with the owner.
Reader data moves to a v2 store (gacha-tracker:v2:*). Existing v1 data
is migrated on first load, a recovery copy is written first, and the v1
keys are never removed, so a rollback finds them unchanged. Redeem codes
for Genshin arrive as their own feed.
Also: live's urgency ramp, editable per theme; completion and
section-complete fireworks; a returning-user tour; the design mockups,
specs, plans, reviews and handoff record.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Records why Miliastra was reverted and why Endfield's Protocol Pass has
no source yet, updates AGENTS.md's parser list and marks sub-project 3
in TODO with the follow-ups.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The owner's rule, recorded so the next agent does not squash a multi-part fix to keep the changelog to one line: one coherent change per commit, each passing alone, ordered so each is safe to ship alone, with a single changelog entry naming the range.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Game8 and Fandom name Genshin's anniversary selectors with no word in common, and both run the same window, so the calendar showed each selector twice. Game8's own blurb names the Fandom event ("…limited 5-star selector, Tabletop Troupe: A Gathering on Adventure's Eve!"), which is how a person paired them, so merge now reads it the same way.
The rule is narrow on purpose: the whole title of two or more words, both boundaries within 24h, and never for a copy that already has a twin by title in the other source, since then the blurb is naming a neighbour such as a festival's shop. Measured on every two-source page held, it fires on the two selector pairs and otherwise only where titles already matched. Dates alone never pair: patch day puts about twenty Genshin events on one window. aliases.ts is the recorded-pair fallback for copies no page links; it is empty.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
When merge fuses two sources' copies, the loser's id leaves the calendar, and it may already hold a reader's ticks from a day it was published alone. Until now that needed a hand-edited RETIRED_IDS entry per fusion.
The survivor now lists what it absorbed in an optional mergedIds field. When a feed arrives the client turns those into carry-forward pairs, skipping any id the feed still publishes, and an import takes the same table. Optional, so an older cached feed still validates and an older bundle drops it unread. RETIRED_IDS stays for parser changes, whose old ids no feed will name again.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Game8 heads Genshin's pass "Battle Pass - Moontrace"; Fandom lists "Moontrace" and files the category in a column. The label minted a second id for one event, so every battle pass since the Fandom source landed has been on the calendar twice, and a reader reported it.
Both sources now mint the same id and merge fuses them by id. The two Game8 ids that moved, Frostfarer and Moontrace (the only ones git history shows ever shipped), go in RETIRED_IDS so readers' marks follow.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
.impeccable/design.json was a derived copy of docs/DESIGN.md and the CSS,
read only by Impeccable's live panel, which this project does not use. It
had drifted two weeks behind while DESIGN.md stayed current, so DESIGN.md
is now the only design reference; /impeccable document regenerates the
sidecar if it is ever wanted.
The layout block's "1,099 tests" was 285 behind; it now names the suite
without a number, which only goes stale.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
SOURCES.md § 16 holds the diagnosis (run 317's rejected zeros, the page
unchanged in shape with nothing current since 19 Sep) and the replacements
assessed and declined: NamuWiki behind a Cloudflare challenge, an official
site with no event posts, no Game8 page. AGENTS.md says when a person may set
`lapsed` and what it does not excuse. TODO ticks the item and notes the
cosmetic "last confirmed never" seen in the CI state.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Giving Genshin's "To Temper Thyself and Journey Far" its five-of-seven goal
meant finding it in the list, opening its sheet, saying it repeats and only
then setting the goal. The strip now has an Edit mode beside Catch up that
lists its events by game with their goals, opens the sheet's goal control in
place, and adds any running event a game at a time. It calls the same
setDaily/setGoal the sheet does, so it can store nothing the sheet would not.
Weekly and monthly tasks are not due tonight, so they move to a row under the
dailies, and each period can be switched off (showWeeklyTasks,
showMonthlyTasks; default on, so nothing disappears on upgrade).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Four lessons were known to one agent's private notes and to nobody else.
They now live where every agent reads: how Gitea Actions differs from
GitHub Actions, the owner's decision to refresh unreachable sources by
hand rather than relax the robots gate, how to look at the built app in
a headless browser and its traps, and one command per shell call.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Anything only in an agent's private memory reaches no one else. AGENTS.md
now says so, and records two lessons from today: push to Gitea only,
because it mirrors to the GitHub copy the changelog links into; and the
image build runs the suite without docs/ or git, so tests must not read
them.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The changelog's own entry names the commit before this one, since a
commit cannot contain its own hash. AGENTS.md now asks every
reader-visible change for an entry, and every multi-commit feature for a
note, so the record keeps up. PRD F18 and the storage and design records
describe what shipped.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The product, storage and design records now cover F17. They say why the
task and amount stores, unlike the custom-event store, never filter on
load, which is the rule a future store would most likely copy wrongly.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The product, storage and design records now say what a goal is, why it
never marks an event done, and why an unreadable goal is kept rather than
stripped. That last rule is the one a future cleanup would most likely
undo by mistake.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Rewrites the reset table for every corrected game, records the rule that a
game's offset must make its day key name the source's calendar day, and
names Game-Time-Master as the evidence for server clocks a source does not
state (transcribed, never vendored: it is GPL-3.0).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
They are reasoning for whoever touches the interface next, which is
what docs/ holds, and the table in AGENTS.md is how an agent finds it.
The design tooling resolves both from docs/ without configuration.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
readableHue returned every dark hue untouched on the claim that all of
them already cleared 4.5:1 there. Five did not, and Fate's navy was
1.8:1, so its name under the headline deadline was nearly invisible.
Those five are now lifted in lightness only, keeping hue and saturation;
the fifteen that pass are unchanged. The owner approved relaxing the
"dark theme does not move" rule to this, and AGENTS.md and PRD F15 say so.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Document the forge responsibilities across AGENTS.md, README.md, ARCHITECTURE.md, INGESTION.md, and the CI workflow headers: GitHub Actions only builds and serves (Pages deployment) without fetching data sources, while Gitea Actions exclusively runs the scheduled refresh and publishes the container image.
The Blue Archive Wiki (bluearchive.wiki/wiki/Events) redesigned its
schedule, replacing the per-version MediaWiki tabber with a unified
#eventtable displaying JP and Global periods side-by-side.
- Parse #eventtable resolving GL Period, event titles, and article links
from .event-region-gl, with fallback to .event-region-jp.
- Maintain backwards compatibility with legacy tabber schedule format.
- Add pinned fixture fixtures/ba/bawiki-events-2026-09-17 and tests.
- Update snapshot, AGENTS.md, ARCHITECTURE.md, and INGESTION.md.
Update README, AGENTS.md, ARCHITECTURE.md, and INGESTION.md to reflect
that the scheduled refresh and container image publishing run on Gitea
Actions under .gitea/workflows/, while GitHub Actions under .github/
builds with BASE_PATH and deploys to GitHub Pages when synced.
The pipeline change made a set of sentences false, and they are the only
record of why things are the way they are, so they move with it: the CI
section and deploy target in README, the cron and BASE_PATH notes in
ARCHITECTURE, the documented User-Agent in INGESTION, and the contact
URL in SOURCES.
Two comments in source were false in the same way and are fixed here
rather than left for the next reader. `serve.ts` said compression was
"the whole difference between the Docker image and the deployed site" —
the image *is* the deployed site now, so that negotiation is the only
compression any reader gets. `robots.ts` justified product-token
matching with "our contact URL contains the string StereotypicalCat",
which it no longer does; the hazard is the arbitrary substring, not that
one, so the comment now says so and the regression test keeps the URL
that actually caused it.
What is deliberately NOT rewritten: game8.co's refusal stays in the past
tense with the Gitea result marked unverified, in AGENTS.md, README,
SOURCES and FEEDBACK alike. A new runner address is the variable that
failure turns on, so it may well be served — but "may well be" is not
evidence, and the four places that would have to change are named
together so they change together once a cycle has actually run.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Add karendar.com as the event source for Punishing: Gray Raven (PGR).
Karendar publishes a dedicated Global event calendar with exact UTC
timestamps in clean server-rendered HTML.
- Register 'pgr' in GameId and GAMES registry (#CC292B, Kuro Games).
- Add parseWeekdayDayMonthYearUtc in src/ingest/dates.ts with unit tests.
- Implement karendarParser in src/ingest/parsers/karendar.ts reading
'week', 'ongoing', and 'upcoming' sections, mapping indefinite/permanent
ends to endsAt: null, and categorizing tags to EventType.
- Register pgr-karendar-events in src/ingest/adapters/index.ts.
- Add pinned fixture, expected output, and comprehensive test suites.
- Update documentation in AGENTS.md, README.md, docs/INGESTION.md,
and docs/SOURCES.md.
Add genshin-impact.fandom.com as a second source for Genshin Impact,
which has been fixture-backed in CI because Game8 CloudFront-blocks the
Actions runner with 202s.
- Add fifth Fandom template parser in src/ingest/parsers/fandom.ts
extracting titles from caption display text, stripping dated subpage
suffixes, and checking currency via latestBoundaryMs.
- Register genshin-fandom-events in src/ingest/adapters/index.ts and
set genshin-game8-events priority: 10 so existing event IDs survive
near-match ties, preventing localStorage completion marks from
orphaning.
- Explicitly sort adapter groups by priority descending in
scripts/build-feed.ts to align with parseGame.
- Pin fixtures and add test coverage in test/adapters/game8.test.ts.
- Update AGENTS.md, README.md, docs/INGESTION.md, and docs/SOURCES.md.
Patch cycles are ~6 weeks, so a span over 180 days previously indicated
a misread year. However, real year-long events exist in this domain:
Genshin's anniversary 5-star selection runs 336 days (2025-10-22 to
2026-09-23), correctly dated on its wiki, and Fire Emblem Heroes runs a
seven-month new-player banner (docs/SOURCES.md § 12b).
Widening the ceiling to 365 days admits genuinely year-long events while
still catching misread years, which place the end 12 months out (span +
~365, or 405+ days for a 6-week event).
Update test/adapters/game8.test.ts duration test, AGENTS.md domain rules,
and docs/INGESTION.md sanity checks.
The build log prints a count per source, and CI now declines to fail on
a page that states its own emptiness — which leaves that source showing
an unexplained 0, reading as exactly the fault the gate just decided it
was not. The zero we are content with is the one that has to say why.
It is also the only line that would ever prompt anyone to ask whether a
lane quiet for a month is a game between patches or a wiki that reworded
the sentence a statesNoEvents check is still matching. Nothing else in
the pipeline can tell those apart.
The test count in the layout block was stale at 898 and moves with these
two commits, so it is corrected here.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Infinity Nikki's wiki prints "There are no Events in this category"
between versions, and the refresh runner has honoured that since
2026-09-03 — it stores the empty parse as the source's real answer
rather than letting a quiet lane reach the broken tier. The feed build
never asked, so the same page arrived at CI as parsedCount 0,
indistinguishable from a parser that has stopped reading a redesigned
page, and brokenSources failed every build while every refresh stayed
green.
The runner's verdict cannot travel on its own: only a parser has seen
the page, and by the time brokenSources runs there is nothing left but
the feed. So the fact rides on SourceHealth, defaulted so an older feed
the service worker cached keeps validating and reads as the strict
answer.
Both ends now ask it the same way — of an empty parse only, from the
page's own words only — because a redesign yields zero rows too, and
excusing that is the silently emptied calendar the gate exists for.
The rule sits in a module rather than in build-feed.ts, which writes
public/ and so runs a build if a test imports it. That is how the two
ends drifted apart with nothing to catch it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The two preceding changes make sentences in docs/ false. INGESTION.md
described `canParse` as a structural check without saying that "structural"
excludes the presence of a row — which is exactly the drift that let this
bug in — and described the Stage 1 gate as rejecting every empty parse.
SOURCES.md § 11 left a reader to conclude that an empty Infinity Nikki lane
means the source has died.
AGENTS.md gets the rule in both places somebody would look: § Working on
parsers, next to the bullet about a source resting on one row, and § Fandom
where the fourth template is described.
The general lesson is worth more than the incident. Any page that can list
nothing can produce this, so the guidance is about what identifies a page
versus what happens to be on it — not about this wiki.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The footer's staleness notice is an instruction, and the only remedy it
offers is "go and check that game's source page". That is not something a
reader can act on for a game they turned off and cannot see a single row
of — and with nineteen lanes, naming eleven they do not play buries the one
they do. A new lane arrives switched off (F8), so an untouched install was
being warned about most of the catalogue.
Scoped in two places, not one: the named list, and the count the
summarising branch measures against. Without the second, that branch would
never fire for a reader with most of the calendar off and they would get a
list where a sentence was the readable answer.
The notice now also states whose games it counted, in the words NextUp
already uses for the same set. Narrowing what the footer measured while
still printing "nothing has refreshed" would turn a claim about four games
into one about eighteen, in the one paragraph on the page whose job is
being trusted about age.
The headline age and the credits stay whole: the first is a fact about the
feed and feeds the bug form, the second is attribution owed to every source
we read regardless of what is on screen.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
SOURCES.md § 13 declined this game's only known source partly because every
boundary was a week bucket, and we have now built a different source with that
same objection standing. Leaving that contradiction unwritten is how a later
pass rediscovers the trade the expensive way, so § 14 records the decision
itself: the conduct evidence, which three of § 13's four objections arustats
answers, which one it does not, and that any source stating a date per event
retires it. § 13 is marked superseded rather than rewritten — its verdict on
marisaimpact.com still stands and was not re-tested.
AGENTS.md gets the same warning where a parser author will hit it, because the
estimate is not visible from the code: the numbers look like every other
day-precision date in the project.
Also corrects the counts this game made false — nine parsers, twenty sources,
nineteen games. The "stopped working at eighteen" passages are left alone: they
date the settings-panel redesign, and bumping them would falsify the history
rather than update a count.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Two things the last two commits taught, written down where the next person
looking at a parser will hit them.
The date table now carries the slash-dated open range, and the prose beside it
says the year is required and why — that reader is tried last, so a guess there
reaches the feed unchallenged. It also names the second copy of the open-end
vocabulary, because the two lists being out of step is not a hypothetical
failure any more.
The second is the one worth having on record. `eventCount` in a snapshot's meta
is not a count of the page. Endfield's Game8 lane read 1 for a week — one row of
an *upcoming* table, while four live events sat in a card grid the parser cannot
read — so the first time the page moved, the lane read 0 and the gate reported a
shape change. Neither number described the page. A count far below what the page
shows is a shape being skipped, and it is worth finding before it reaches zero
and the gate has to guess on your behalf.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Both files stated the h4 rule as a flat prohibition — an unrecognised h4
must never claim the event title — which is no longer what the code does.
They now state the narrower claim: an h4 may fill an empty event title but
never take one, with the reason each page needs it. Genshin's labels arrive
after its h3 has claimed the slot; Wuthering Waves' block sits under a
section heading that leaves the slot empty.
INGESTION.md gains the block as shape 9, and the part that is easy to get
wrong on the way back: its heading has to be *in* the section vocabulary
rather than merely fall through, because an unrecognised heading is read as
an event name and would spend the title on the section.
AGENTS.md also gains a bullet for where a missing blurb hides. Two
templates now put unlock conditions in the cell where a description
belongs, and in both the real prose is in a per-event section further down
the page — under an h3 on one and an h4 on the other.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
They were one group called "Reading", grouped on "both are how do I read
this?" — which summarised as "Europe · Dark", two unrelated answers joined
by a dot, under a name for neither of them, in a panel whose whole premise
is that a closed line answers its own group.
The region is not a reading preference. Region-scoped ends, a date printed
without a time, and every daily streak are all cut on that server's clock,
so it is the one control in this panel that can make a countdown wrong —
and it was filed behind a word for the theme. Split, each line answers one
question, and the region group says what it governs when opened.
The eyebrow above each pill row goes with it, since the group summary now
asks the question the eyebrow was repeating; aria-label keeps the
accessible name, which is what stops six unlabelled buttons in a row from
being all a screen reader hears. The summary assertions are looked up by
group name rather than by index, so splitting a group no longer moves every
assertion below it onto a neighbour.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Looked at on a 1440px screen it was the one block on the page not using
the column it sits in. The rows were held to max-w-3xl inside an 1150px
shell, but the section's own rule spans the shell — so every hairline
inside the panel stopped a third of the column short of the one above it,
between a two-column checklist and a three-column footer that both go
full width. The measure was there to keep a name and its state reading as
one line, but that pairing is what "running now" already does with a
title and its countdown directly above, so full width is the house
pattern rather than a stretch. Prose inside a group keeps its measure.
The game list was the worse half: eighteen rows of four controls at
max-w-md is a thousand pixels of ribbon down the left with two thirds of
the screen empty beside it. It takes two columns past lg once there are
nine or more games, flowing down then across so the numbers still read in
a straight line and an arrow still swaps a row with its neighbour. Below
nine, and on a phone, it is unchanged.
The last group also loses its bottom rule, which was a second hairline a
few pixels above the footer's own.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
`build:js` was shipping React's development build. That cost 222 KB of the
566 KB bundle (61 KB of 164 KB gzipped), ran every element creation through
dev-only validation, and left `StrictMode` double-invoking effects — so
`fetchFeed` fired twice and every reader downloaded the feed twice on every
single load.
On a 4G/4×-CPU profile with the cache cold: first contentful paint 1480ms →
972ms, the feed on screen 1766ms → 1261ms, and two feed requests → one.
The flag is `--production` and not `--define process.env.NODE_ENV='"production"'`,
which is the obvious-looking version and produces a bundle that does not run at
all: the define flips React to its production build while the JSX transform
keeps emitting `jsxDEV`, so the page dies on `z is not a function` with a blank
body. Both variants build clean, typecheck clean and pass all 843 tests, because
nothing in the suite executes the bundle — so AGENTS.md now says to open a
browser before believing this one.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The panel was one open block of everything in two columns. That reads fine at
four games and stops working at eighteen: the game list alone is eighteen rows
of four controls each, and it sat above the checkbox somebody had scrolled down
here to tick. Nothing was findable and everything was in the way.
Collapsing it into groups is only half an answer, though — a group showing
nothing but its name turns "what is my region set to?" into a click, which
trades one kind of friction for another. So each summary carries its group's
current state: "Europe · Dark", "17 of 18 on · A–Z", "plus finished, not
started". Closed, the panel is a five-line report of how the app is configured;
opening one is for changing an answer rather than reading it. The states are
derived from `prefs` at render, never stored, so they cannot drift from the
controls they describe.
Native `<details>`, for the reason the reorder arrows are ordinary buttons:
keyboard and screen reader reach it with no second implementation. `summary` is
not an `a`, `button` or `[tabindex]`, so it needed its own focus-visible rule —
the shared one does not reach it.
Two things fell out along the way. The region and appearance rows had no
accessible name at all, so a screen reader read six unlabelled buttons in a row;
they and the board's split pills are one `PillGroup` now, with the `role="group"`
the board's own controls already carry. And the empty states that point at
"Show events that haven't started" now name the group holding it, which is what
makes shipping the groups closed safe.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This project has no linter and does not want one, but `strict` says nothing
about a binding that is simply never read — and that is the one kind of dead
code a reader cannot tell apart from a deliberate seam. `tsc` already walks
every file, so it may as well answer the question.
Three had accumulated: an unread `type View` import in App, and a
`useGameMeta()` resolver in Controls and YourOwn, both of which draw their
colours from somewhere else — the game-order list has its own resolver, and
YourOwn reads the reader's own records directly. None of them were wrong, they
were just leftovers that read as intent.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The rules a future change could undo without noticing, written where whoever
touches that area next will read them.
Three of them are traps rather than descriptions. An absent `gameOrder` means
the reader has never placed a game, not an empty order — reading it the other
way hands every existing install a blank list, which is the `knownGames`
mistake again. `games` is left in feed order deliberately, because the code
that hides a reader's games diffs it. And the catch-up window bounds display
and never storage: a fortnight's streak exists on one device and nowhere else,
so nothing prunes the log against a window.
The spec that produced the work goes in alongside, including the helper it
specified and the implementation dropped, so the next reader sees why rather
than wondering.
Also corrects the test count in AGENTS.md, which said 772 and was already
stale before this.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Switching refresh.yml back to ubuntu-latest left this file asserting that it
runs on a self-hosted runner "because the address is the legitimate lever". The
lever is still the address; that particular address just was not a better one.
Says so explicitly, because the revert is easy to misread as the idea being
wrong rather than one address not being the one — and the next person here will
otherwise try the same runner again.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The runner move was recorded here as untested for game8 because that cycle
interval-skipped all nine. The next one asked them: 202 CloudFront, every one.
So safe-ip is challenged by Cloudflare and refused by CloudFront alike and
bought nothing by itself — worth stating plainly, since "try a different runner"
is the first idea anybody has here and it has now been tried.
The Fandom half is fixed, by the cron passing --assume-robots-on-403 rather
than by the address: the next cycle put all four through at 200, two with fresh
bytes.
Also records why game8 cannot have the same treatment. Its 202 is the page
itself being withheld, not a permission we hold and cannot re-read, so there is
nothing for a recorded decision to stand on.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Four Fandom sources have skipped on a challenged robots.txt every cycle, so
their calendars are only ever as fresh as the last manual run. For a product
whose promise is a trustworthy end date, four stale lanes are the worse failure
— so the scheduled run now passes --assume-robots-on-403 and the owner re-reads
those files by hand over time.
Kept asymmetric on purpose: --force is still refused on an unattended run.
Forcing every cycle really is just a shorter interval with extra steps, and
nothing about this decision touches that.
The accepted risk is narrower than "crawling against robots.txt", and worth
stating precisely. A plain 403 still fails closed, so a host that turns us away
still stops the run — that is what the challenge-or-refusal split buys. What is
invisible is a robots.txt edited to disallow us, because from a challenged
address a withdrawal looks exactly like the challenge we already expect. No
code can catch that; only the re-read can.
Which makes the per-cycle warning the compensating control rather than a
courtesy, so it is now pinned as one: every host named, and surviving as a
run-page annotation on a completely green cycle where nothing else draws the
eye. A dispatch can set the input to false to see the real state.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Both overrides stand in for a person deciding something, and the gate that
enforced it asked the coarser question. `isCi()` got the case that matters
wrong: a `workflow_dispatch` sets CI=true, so somebody clicking "Run workflow"
was refused exactly as the cron was, which made both flags unreachable from the
workflow at all. A dispatch is a person, and a better-evidenced one than a local
shell — GitHub records which one, and the run log keeps it.
`runAttendance` draws the line where it belongs: a local shell is a person, a
dispatch is a person and names the actor, and a schedule — or any other runner
event, erring that way on purpose — is not. The run now prints which override
was used and who authorised it, because a local shell leaves no trace anybody
else can read.
The schedule stays refused, and for the robots override that is not ceremony.
From an address that gets a challenge we never receive robots.txt at all, so a
cron standing on the recorded permission has no way to notice the host
withdrawing it, and that permission has no expiry. The challenge-or-refusal
narrowing does not close this: a plain 403 still stops us, but a robots.txt
edited to disallow us would be invisible, because a challenge arrives instead of
a file. A person re-reading it is the only thing that ever re-validates it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The runner move to [self-hosted, safe-ip] was recorded here as untested, with
the next cycle to settle it. It ran, and all four Fandom sources came back
403 (an interstitial challenge) — so that address is challenged too, and these
four still only move when a person refreshes them on the recorded permission.
Worth being exact about what the cycle did not settle, because the summary
looks fuller than it is: the nine game8 sources were all skipped_interval, not
due until 23:46 UTC, so nothing asked game8 anything and its 202 is unmeasured
on the new runner.
The reason string is doing the job it was added for one commit ago — the
challenge-or-refusal distinction is what says this is covered by the flag
rather than a host that turned us away, and it answered that from the summary
alone.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This file recorded a 403 as Fandom's posture, holding across every wiki and
every address, which made those four sources permanently unschedulable. It was
measured with curl, and curl is not what fetches: on one address, in the same
minute, curl takes a 403 with a bare, a Chrome and our own User-Agent alike,
while Bun's fetch is served the real robots.txt on all five hosts and the gate
passes with no override.
Two variables were folded into one there, and they are worth holding apart. The
client mattered — but so does the address, and the client does not rescue it:
the 17:46 UTC run on ubuntu-latest still skipped all four Fandom sources using
that same Bun client. So a claim that these refresh on a schedule is a claim
about the address the runner has, and on the new one it is untested rather than
established. The next cycle settles it.
Three traps recorded so the episode is not repeated: verify a fetch gate with
the real RobotsCache from the address that will run it, never a shell client;
the User-Agent is not a lever in either direction, so a 200 that appears when
you drop it is the challenge's probabilistic half and not a header you tuned;
and neither block is a rate limit, since every failure is first contact with
that host, the spacing is already applied, and a challenge carries no
Retry-After. More delay buys nothing.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
`--assume-robots-on-403` stands in for rules a person read in a browser, for a
host that will not serve us the file. It always claimed it would not override a
host that had turned us away — but it could not tell the two 403s apart, so it
excused both. A managed challenge ("we cannot tell what you are") is a question
our fetcher cannot answer and the operators never asked, which is what makes a
human reading the rules a fair substitute. A bare 403 is the site itself saying
no, and nothing recorded on our side may talk over that.
`isInterstitialChallenge` now decides, on Cloudflare's own `cf-mitigated` header
with the challenge page's markers as a fallback; an unreadable body counts as
unclassifiable rather than challenged. This only ever narrows what the flag
opens, so nothing that passed the gate before stops passing it.
Every 403 is classified whether or not the flag is set, and the kind goes into
the reason string the run reports. `robots.txt returned 403` read identically
whether the answer was to refresh by hand on the recorded permission or to stop
fetching the source, and the summary is where somebody has to decide that.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The link sits last in the existing row beside @StereotypicalCat and Source
code, in that row's own muted grey, with the same hover and the same small
mark. No accent colour, no button, no sentence asking for anything, and the
footer still contains no appeal anywhere — a test asserts that. This page earns
its keep by being trusted about dates, and a tip jar competing with the
disclaimer two paragraphs above spends that trust to make an ask. As one more
link it costs nothing and is there for a reader already looking.
The row also moves above "Additional ideas and design from". That row is who
built this and where to find them; the credit below it is other people. Reading
order now follows that split, instead of routing a reader past five strangers'
handles to reach the author's own links.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The interval gate has no override, so a page you know has just changed cannot
be fetched until six hours after the last attempt. The only workaround was
deleting snapshots/<id>.state.json, which also wipes consecutiveFailures and
lastConfirmedAt — resetting the broken-source streak and making the footer age
every source from when its bytes last changed rather than when we last
confirmed them. A flag that says what it means is better than a side effect
nobody documented.
It sets aside the interval and nothing else. Conditional headers still go out,
which is what makes forcing defensible at all: the host is asked, not
re-served, and an unchanged page costs it a 304. Per-host spacing, robots, one
request per source and the no-retry rule all still apply — a source that was
not due and is also disallowed stays skipped, for the reason that matters.
Refused under CI, like --assume-robots-on-403 and for the same reason: a
schedule that forces every cycle is a shorter interval with extra steps, and
the interval is the obligation, not the default. So AGENTS.md § Scraping
conduct is amended rather than left to be quietly contradicted by a flag.
Every source asked early is named in summary.forced and warned about. A run
that was due anyway is never reported as forced — a summary that cried "forced"
on an ordinary run would train the reader to ignore the word.
Also repoints the "unknown flag" test, which used --force as its example and
stopped testing anything the moment --force existed.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
