A review of code that has never made a live request. Each of these was
reproduced before it was fixed.
robots.txt matched a group by testing whether our whole User-Agent
contained the group's name. Our contact URL carries "StereotypicalCat",
so a wiki writing `User-agent: cat` matched us — and because a named
group *replaces* the wildcard group, that silently discarded every rule
under `User-agent: *`. Match the RFC 9309 product token instead. A 200
carrying an HTML "not found" page also parsed to zero rules and read as
permission; it now fails closed, while a genuinely empty body still
means "no restrictions".
The response body was read outside the try that guarded the request, so
one truncated body aborted the whole cycle: later sources were never
fetched, and the failed source never recorded its check, meaning a
re-dispatch would ask that wiki again minutes later.
Bodies were decoded as UTF-8 unconditionally and stored re-encoded. A
page served in a legacy charset became replacement characters with the
original bytes gone — and mojibake in a title flows into slugify and
moves every localStorage key for that source. Decode by the declared
charset, keep the served bytes verbatim, and hash those.
Unchanged bytes skipped the metadata write, so once a server rotated its
ETag we sent a stale validator forever and it served full bodies instead
of 304s.
Also: snapshot writes go through a temp file and a rename, and `--only`
with no value is an argument error rather than "all sources".
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The cut was marked with U+2026, which NFKC decomposes into three dots —
so re-sanitising a truncated string grew it by two characters and re-cut
it at a different word boundary. A title would quietly rewrite itself
every time the event was re-ingested, and the module promises
idempotency in its own docstring.
Append what normalisation would produce instead. The existing corpus
missed this because its only over-length entry has no space in its last
40%, so it happened to re-truncate to the identical string; the new test
uses prose.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
"What runs out first" is the question this app exists for, but it is not
the only one a reader arrives with. The other is "what was I in the
middle of?", and until now the list could not answer it.
Two modes, toggled from the list's own header rather than from settings,
because ordering is something you reach for while looking at a list.
Ticking a day off a daily counts as doing it without the reader also
setting the status — the tick already said so.
Sorting only ever groups. Both modes fall back to endingSoonestFirst
inside a group and live still beats upcoming, so choosing an order can
never cost the reader the deadline order the product is for.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
A login campaign is not one job with a deadline. It is twenty small jobs
on twenty separate deadlines, and a day you miss is gone whatever you do
afterwards — which a single "done" tick cannot express.
Repeating events now get a checklist: today's tick, a strip of every day
in the run showing what you got and what you missed, the streak, and how
many chances are left. Past days stay editable, because people tick up
later and a checklist you cannot correct stops being trusted after the
first mistake. Alongside it sits today's dailies — commissions, sanity,
daily training — one tick per game, keyed `dailies:<game>`, since no
source publishes those and they are the only thing on the page that
expires tonight rather than next patch.
Days roll at 04:00 server time per region, not midnight: finishing at
02:00 is still yesterday, and a naive UTC date would tick the wrong box
for four hours every night.
Dailiness is read off what the source published — a login event type, or
"daily"/"check-in"/"7-day" wording — never from a game's habits or an
event's length. That adds no schema field, so the feed contract and every
event ID are untouched. An unannounced end yields a tick count, not a
checklist of invented length, and a tick is never removed except by the
reader, including ticks outside the window the feed now claims: a source
quietly moving a date must not erase a fortnight's streak that exists
nowhere else.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The feed was generated from checked-in fixtures and only moved when
somebody captured a page by hand. This fetches.
bun run refresh caches each page raw under snapshots/ and rebuilds the
feed from what it cached; build-feed prefers a snapshot and falls back to
the fixture, so a clean checkout and the container build stay offline and
reproducible. The workflow runs it twice a day and commits only when a
page's bytes actually changed — a 304, an identical body or a rejected
parse all leave the tree clean — then dispatches ci.yml, which already
knows how to test, build and deploy.
Scraping conduct is enforced in code rather than left to good
intentions: one request per source per cycle, a six-hour floor checked
per source, conditional requests, a User-Agent with a contact URL, and
robots.txt honoured — failing closed, because a permission we could not
read is not a permission we have. No retries; a retry is a second
request.
A body that yields zero events is rejected and the previous snapshot
kept, so a redesigned wiki shows up as a stale timestamp rather than an
emptied calendar. One source down is a warning; all of them down fails
the run, so a cycle that learned nothing is never committed.
Tested entirely offline against an injected fetch and clock — no request
has ever been made to a live wiki from this code.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Scraped text becomes React content, JSON on disk and eventually SQLite
rows, so it gets cleaned at one seam: the parse wrapper in toAdapter().
Every source passes through it, a source added tomorrow is covered
without its author doing anything, and no parser can opt out — parsers
stay pure readers of one site's markup.
Removes script/style/comment content and residual tags, decodes entities
to a fixed point so an encoded tag cannot resurrect in a later decoder,
NFKC-normalises, strips control, zero-width and bidi-override characters
(an RTL override visually spoofs a title), bounds each field to the cap
the schema already declares, and requires sourceUrl to be absolute
http(s).
Three things it will not do: touch a date, drop an event it could clean
instead, or repair in silence — every repair and drop is logged by
default. Event IDs are localStorage keys, so an ID is recomputed only
when a sanitised title actually changed and the ID was minted the
standard way; all seven fixtures pass through unrepaired and
byte-identical, which is the regression guard.
Also stops decodeEntities throwing RangeError on an out-of-range
numeric reference, which would have taken a whole source's events down.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Four buckets named the way a player would describe them — quick, short, long,
grind — each carrying rough working hours.
The hours exist so effort can answer the question the calendar cannot: can I
still finish this? The same two days is comfortable for a quick event and
hopeless for a grind. The runway heuristic assumes about an hour of play a
day, which is deliberately modest — a warning that only fires once something
is already impossible is useless.
An event with no recorded effort returns "fine" and never warns. Inferring an
estimate in order to warn about it would be fabricating the reader's input.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The image build ran the tests without serve.ts present, so the server the
tests spawn never started and the hook stalled until it timed out. The
runtime stage would have failed on the same missing file immediately after.
The test now bails the moment the process exits and reports its stderr, so
this shows up as "serve.ts exited with 1 before listening" in 53ms rather
than an unexplained hook timeout after five seconds.
serve.ts is also in tsconfig's include now — it was outside it, so a type
error in the file that serves the app would only have surfaced at runtime.
Adding it immediately caught one in the test.
Verified by replaying the build stage against exactly the copied file set.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
They read from public/, which does not exist on a clean checkout — CI runs
bun test before any build, so the four of them failed there while passing
locally against stale build output.
serve.ts now takes PUBLIC_DIR, and the tests serve a temporary tree they
create themselves. They exercise serve.ts rather than the build, which is
what they were always meant to do.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The server is a placeholder for the one in docs/ARCHITECTURE.md — it serves
public/ and a health endpoint, nothing more. Reads are confined to public/ by
resolving the path and checking it stays inside the root; string-matching
".." is not enough, since encodings and URL normalisation both change what
the string looks like and only the resolved path says which file would open.
The image runs typecheck and tests during build, ships no source or
toolchain, and runs unprivileged.
CI's feed job fails if the event count collapses. A source that quietly stops
yielding events is what a parser-only pipeline is most prone to, and nothing
else would surface it. Everything is offline, so a red pipeline always means
the code changed rather than a wiki being down.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
First non-Game8 parser, and the first source that states region-scoped ends:
wiki.gg emits ISO timestamps with one timer per server region, so Endfield
events now carry exact times and separate Asia / Americas-Europe ends. That
gap is up to 13 hours, which is what regionEnds was built for.
Two sources for one game also exercised merge for real. It caught both
overlaps — including "Bedazzling Dawnstar" against Game8's "Bedazzling
Dawnstar Sign-In" — and flagged that the two disagree on the end date by 70
hours rather than averaging them into a date neither states.
Matching an appended qualifier needed a prefix test, not a subset test: a
subset would also fuse "Gold Clash" with "Gold Rush Clash Royale".
Also fixes build-feed picking fixtures by game rather than by source, which
handed the wiki.gg page to the Game8 parser once Endfield had two.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Endfield was previously written off as undatable. That was wrong: a first
pass only inspected its Duration rows (all "Permanently Available") and its
year-less version grid, and missed an "Event | Schedule & Summary" table
whose cells read "Period: 08/09/26 - 08/30/26 During the event...". Two real
events, with a year.
Adds MM/DD/YY range parsing for that shape. Month-first ordering is not
assumed lightly — Endfield's own version grid reads 01/22, 04/17, 07/16 for
versions 1.0, 1.2 and 1.4, chronological only if the month comes first.
Also stops presenting unlock conditions as descriptions. Several templates
put "Reach Union Level 8" where a blurb would go; filling the summary slot
with text that never says what the event is is worse than leaving it empty.
Wuthering Waves correctly drops to zero summaries as a result.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Three more Game8 templates. ZZZ spans each event name across two rows with a
rowspan, so a flat cell reader sees [title, "Start", date] then ["End",
date]; losing that pairing would silently halve the calendar.
Section matching also had to widen — these pages head their sections
"Featured Events", "List of Events and Schedule" and "Ongoing Events".
Verified independently of the parser: HSR's year-less summary table lists
six ongoing events and all six are captured via their detail rows, so
nothing is silently dropped.
28 events added across the three games.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Honkai: Star Rail and Wuthering Waves state a year on each side of a range
("Aug. 14, 2026 - Aug. 24, 2026"), which the year-on-the-end parser cannot
read.
Both also publish genuinely open-ended events — "Jul. 24, 2026 - End of
4.6", "July 10, 2026 - Permanent". Those keep their real start and report no
end, rather than being dropped or given a guessed one.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Time is this product's subject, so the vocabulary lives in one module:
remaining, progress through a window, and urgency. Urgency is derived from
absolute time left, deliberately independent of proportion — a 90-day event
with three hours left is as urgent as a 3-day one.
An unannounced end is never urgent and has no progress: "we don't know" and
"loads of time" are different facts, and conflating them is what this
product exists to avoid.
Games carry a hue for identity only. Urgency is a separate axis, so one
glance answers both "whose event is this?" and "how long have I got?".
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Collapses the same event seen by two sources, matching on ID or on title
similarity plus start-date proximity. Proximity is the real guard against
false positives, since a rerun reuses its name months later.
Agreement from an independent source raises confidence; the same row seen
twice in one document does not. Disagreement on an end date is recorded as
a conflict for the review gate rather than averaged — splitting the
difference would publish a date neither source asserts.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Parsers are keyed by site template, not game: one Game8 parser serves
every Game8 page, and adapters bind a URL and game to a parser. Adding a
source for a known site is one SOURCES entry.
Game8 uses at least three templates. This handles label/value detail
tables (Genshin, 9 events) and column tables (NTE, 13 events). Its
image-grid schedule has no year and no end date, so Arknights: Endfield
is not supportable from that source and gets no adapter.
canParse is a structural tripwire: a site redesign fails the run loudly
instead of publishing an empty calendar that reads as "no events".
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Every date function returns null rather than inferring a missing year,
month or end date. A source that does not state an end yields
endsAt: null, never a plausible guess — a confidently wrong end date is
the failure this product exists to prevent.
Handles the three formats seen on Game8: "August 12, 2026",
"August 12 - September 21, 2026" (year applies to both ends, rolling the
start back when the range crosses New Year), and a slash datetime range.
Abbreviated months parse too, so a source switching to "Apr. 29" does not
silently drop events.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
