docs: record that CI, the registry and the deploy moved to Gitea
The pipeline change made a set of sentences false, and they are the only record of why things are the way they are, so they move with it: the CI section and deploy target in README, the cron and BASE_PATH notes in ARCHITECTURE, the documented User-Agent in INGESTION, and the contact URL in SOURCES. Two comments in source were false in the same way and are fixed here rather than left for the next reader. `serve.ts` said compression was "the whole difference between the Docker image and the deployed site" — the image *is* the deployed site now, so that negotiation is the only compression any reader gets. `robots.ts` justified product-token matching with "our contact URL contains the string StereotypicalCat", which it no longer does; the hazard is the arbitrary substring, not that one, so the comment now says so and the regression test keeps the URL that actually caused it. What is deliberately NOT rewritten: game8.co's refusal stays in the past tense with the Gitea result marked unverified, in AGENTS.md, README, SOURCES and FEEDBACK alike. A new runner address is the variable that failure turns on, so it may well be served — but "may well be" is not evidence, and the four places that would have to change are named together so they change together once a cycle has actually run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
5b8109b556
commit
f32eb93289
+10
-6
@@ -37,7 +37,7 @@ are no users.
|
||||
```
|
||||
|
||||
**That is the design, and the scheduler half of it is not built yet.** What runs today is
|
||||
`scripts/refresh-sources.ts` on a GitHub Actions cron, writing snapshots to disk, and a feed built
|
||||
`scripts/refresh-sources.ts` on a Gitea Actions cron, writing snapshots to disk, and a feed built
|
||||
from those files rather than from SQLite — see § Today. The stages, the layering and the review gate
|
||||
are unchanged by that; only what wakes them up and where they land is.
|
||||
|
||||
@@ -230,8 +230,10 @@ gates and publish it.
|
||||
|
||||
Assets resolve against a `<base href>` substituted at build time, the feed URL resolves against
|
||||
`document.baseURI` so deep links work, and the service worker derives its paths from its own
|
||||
registration scope. `BASE_PATH=/gacha-event-tracker/ bun run build` for GitHub Pages; without it a
|
||||
subpath deploy 404s on every asset.
|
||||
registration scope. `BASE_PATH=/gacha-event-tracker/ bun run build` for any host that serves the app
|
||||
from a subpath; without it a subpath deploy 404s on every asset. CI no longer sets it — the container
|
||||
image is what gets deployed and `serve.ts` serves from `/` — but the mechanism stays, because a
|
||||
subpath deploy is a hosting decision rather than a build one.
|
||||
|
||||
### Offline
|
||||
|
||||
@@ -263,9 +265,11 @@ mode of the `fetch()` that follows it, and a mismatch is not a no-op — the bro
|
||||
preload and fetches the feed a second time, which is worse than not preloading. If that link is ever
|
||||
edited, count the feed requests in a real browser; nothing in the test suite can see this.
|
||||
|
||||
Compression is the server's job and both hosts do it: GitHub Pages transparently, and `serve.ts` —
|
||||
which is what the Docker image runs — by negotiating `accept-encoding` per request. It matters more
|
||||
than it sounds: the bundle is 344 KB raw and 100 KB gzipped.
|
||||
Compression is the server's job, and `serve.ts` — which is what the Docker image runs, and so what
|
||||
serves the deployed site — does it by negotiating `accept-encoding` per request. It matters more
|
||||
than it sounds: the bundle is 344 KB raw and 100 KB gzipped. A static host put in front of the built
|
||||
`public/` instead must do its own; GitHub Pages did this transparently while it was the deploy
|
||||
target, which is why nothing here ever had to.
|
||||
|
||||
### The theme, before the bundle arrives
|
||||
|
||||
|
||||
+1
-1
@@ -299,7 +299,7 @@ diagnosis in each item still holds — what changed is whether it has been acted
|
||||
|
||||
| Item | Status |
|
||||
|---|---|
|
||||
| P0 refresh pipeline | **Diagnosed, half acted on.** game8.co answers a GitHub Actions runner with `202` and a bot-management body, so those eight sources have only ever built from fixtures in CI — see `AGENTS.md` § Scraping conduct, including why it is not to be worked around. The `broken` tier now makes a source failing three cycles fail the run. Step 5 (a build assertion on snapshot age) is **not built** |
|
||||
| P0 refresh pipeline | **Diagnosed, half acted on.** game8.co answered a GitHub Actions runner with `202` and a bot-management body, so those eight sources had only ever built from fixtures in CI — see `AGENTS.md` § Scraping conduct, including why it is not to be worked around. CI moved to a Gitea runner on 2026-09-16, which changes the address this failure turns on; whether game8 serves it is **unverified**, and the row stays as-is until a cycle has run. The `broken` tier now makes a source failing three cycles fail the run. Step 5 (a build assertion on snapshot age) is **not built** |
|
||||
| P1a Arknights | **Done.** `arknights-akwiki-events`, via the new `akwiki` parser |
|
||||
| P1b `NextUp` → three | **Done** (2026-08-18). One headline and two behind it, off `nextToExpire` |
|
||||
| P1b cap the long list | **Done** (2026-08-18). Both sections cap at six with "show all N" — truncation of the view, not a re-sort |
|
||||
|
||||
+1
-1
@@ -318,7 +318,7 @@ on the section and leave every `h4` beneath it unable to take it back — publis
|
||||
|
||||
- Send `If-None-Match` / `If-Modified-Since` from `sources.etag` / `last_modified`. A `304` ends
|
||||
the run as `skipped_unchanged`.
|
||||
- `User-Agent: gacha-event-tracker/1.0 (+https://github.com/<owner>/gacha-event-tracker)`.
|
||||
- `User-Agent: gacha-event-tracker/1.0 (+https://gitea.lucaswinther.info/<owner>/gacha-event-tracker)`.
|
||||
- Honor `robots.txt`; cache parsed robots per host for 24h. **Fail closed** — a `robots.txt` that
|
||||
5xxs or times out means "do not fetch", because a permission we could not read is not a
|
||||
permission we have. A 404 means no restrictions.
|
||||
|
||||
+22
-12
@@ -27,23 +27,33 @@ the ruling.
|
||||
## Method
|
||||
|
||||
Every page below was fetched from this machine with the refresh runner's own `User-Agent`
|
||||
(`gacha-event-tracker/1.0 (+https://github.com/StereotypicalCat/gacha-event-tracker)`), one or two
|
||||
requests per host, `robots.txt` first. No browser-shaped headers, no proxy, no JS execution —
|
||||
(`gacha-event-tracker/1.0 (+https://github.com/StereotypicalCat/gacha-event-tracker)` — the contact
|
||||
URL at the time; the repo has since moved to Gitea and the runner now sends that address instead),
|
||||
one or two requests per host, `robots.txt` first. No browser-shaped headers, no proxy, no JS execution —
|
||||
anything that only answers a browser is treated as closed, per the `uma.moe` precedent. Where an
|
||||
existing parser could plausibly read a page, it was run against the fetched bytes offline
|
||||
(`src/ingest/parsers/index.ts`) rather than guessed at.
|
||||
|
||||
Two things this method cannot tell you, and both matter:
|
||||
|
||||
- **Whether a host answers the GitHub Actions runner.** game8.co does not (`AGENTS.md` § Scraping
|
||||
conduct). The only hard evidence we hold is what CI has actually committed: `git log` on
|
||||
`snapshots/` shows `github-actions[bot]` landing **arknights.wiki.gg**, **endfield.wiki.gg** and
|
||||
**bluearchive.wiki** (Miraheze). Fandom sources have never refreshed in CI, because `robots.txt`
|
||||
itself is challenged from a datacentre address and the gate fails closed — and as of 2026-08-19
|
||||
that challenge covers every Fandom wiki from *any* address tried here, not just CI's.
|
||||
Of the 2026-08-19 additions, **Stella Sora** is Miraheze and so is the one likely to refresh in CI;
|
||||
**IOP Wiki** is permissive but unproven; **Chaos Zero Nightmare** and **Umamusume** are game8 and
|
||||
therefore blind in CI by construction.
|
||||
- **Whether a host answers the CI runner.** This is a property of the runner's address, not of the
|
||||
page, so it has to be re-established whenever that address changes — and it just did: CI moved from
|
||||
GitHub Actions to a Gitea runner (`AGENTS.md` § Scraping conduct). **Everything in this bullet
|
||||
below is evidence about the old GitHub address.** Treat it as the prior, not the current state,
|
||||
until a cycle has run on Gitea.
|
||||
|
||||
game8.co did not answer the GitHub runner. The only hard evidence we held was what CI had actually
|
||||
committed: `git log` on `snapshots/` shows `github-actions[bot]` landing **arknights.wiki.gg**,
|
||||
**endfield.wiki.gg** and **bluearchive.wiki** (Miraheze). Fandom sources never refreshed in CI,
|
||||
because `robots.txt` itself is challenged from a datacentre address and the gate fails closed — and
|
||||
as of 2026-08-19 that challenge covered every Fandom wiki from *any* address tried here, not just
|
||||
CI's. Of the 2026-08-19 additions, **Stella Sora** is Miraheze and so was the one likely to refresh
|
||||
in CI; **IOP Wiki** is permissive but unproven; **Chaos Zero Nightmare** and **Umamusume** are
|
||||
game8 and were therefore blind in CI by construction.
|
||||
|
||||
The Gitea runner may well be served where GitHub's was not — that is one of the two remedies
|
||||
`AGENTS.md` allows — but "may well be" is not evidence. The check is the same one that produced the
|
||||
list above: run a cycle, then read `git log` on `snapshots/` for what actually landed.
|
||||
- **Whether a page will still look like this in six weeks.** Each recommendation below names the
|
||||
assertion `canParse` should make, so a redesign fails the source loudly instead of emptying a lane.
|
||||
|
||||
@@ -836,7 +846,7 @@ if it ever prints a year (§ 13). None is worth a pass until something changes u
|
||||
## Appendix — reproducing the checks
|
||||
|
||||
```bash
|
||||
UA='gacha-event-tracker/1.0 (+https://github.com/StereotypicalCat/gacha-event-tracker)'
|
||||
UA='gacha-event-tracker/1.0 (+https://gitea.lucaswinther.info/lucasw89/gacha-event-tracker)'
|
||||
|
||||
# conduct
|
||||
curl -sS -A "$UA" https://iopwiki.com/robots.txt
|
||||
|
||||
Reference in New Issue
Block a user