`lastConfirmedAt` lives in gitignored bookkeeping that only refresh.yml
restored, so the workflow that actually builds and deploys never saw it:
every source reported its last *content change* as its last success, and
the UI flagged anything whose bytes had not moved in two days as stale —
which is most wiki pages most of the time. ci.yml now restores the same
cache read-only before building the feed.
The refresh push was a bare `git push`, so a human push landing in
between made it non-fast-forward: the job failed and threw away pages it
had just fetched, while the bookkeeping had already been saved, so those
sources would not be re-asked for six hours. Rebase and retry instead —
never force.
The cache save key used run_id, which is stable across re-runs, so a
re-run saved nothing and the run after it restored stale bookkeeping.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The feed was generated from checked-in fixtures and only moved when
somebody captured a page by hand. This fetches.
bun run refresh caches each page raw under snapshots/ and rebuilds the
feed from what it cached; build-feed prefers a snapshot and falls back to
the fixture, so a clean checkout and the container build stay offline and
reproducible. The workflow runs it twice a day and commits only when a
page's bytes actually changed — a 304, an identical body or a rejected
parse all leave the tree clean — then dispatches ci.yml, which already
knows how to test, build and deploy.
Scraping conduct is enforced in code rather than left to good
intentions: one request per source per cycle, a six-hour floor checked
per source, conditional requests, a User-Agent with a contact URL, and
robots.txt honoured — failing closed, because a permission we could not
read is not a permission we have. No retries; a retry is a second
request.
A body that yields zero events is rejected and the previous snapshot
kept, so a redesigned wiki shows up as a stale timestamp rather than an
emptied calendar. One source down is a warning; all of them down fails
the run, so a cycle that learned nothing is never committed.
Tested entirely offline against an injected fetch and clock — no request
has ever been made to a live wiki from this code.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
