docs: rewrite CLAUDE.md for the current codebase
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Lucas Winther
co-authored by
Claude Opus 5
Claude Opus 5
parent
e9f69ea8dc
commit
3b2feadf5d
@@ -4,135 +4,144 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
## What this is
|
||||
|
||||
A web app that aggregates live and upcoming events across popular gacha games (Genshin Impact,
|
||||
Honkai: Star Rail, Zenless Zone Zero, Wuthering Waves, Arknights, Arknights: Endfield), plots them
|
||||
on a calendar, sorts them by end date, and lets a user mark events completed.
|
||||
A web app that aggregates live and upcoming events across popular gacha games, plots them on a
|
||||
calendar, sorts them by end date, and lets a user mark events completed.
|
||||
|
||||
**Status: specification only.** No application code exists yet. `docs/` is the source of truth for
|
||||
what to build; everything below describes the intended system, not an existing one. When you write
|
||||
the first code, follow `docs/ARCHITECTURE.md` and update this file's Commands section with the real
|
||||
commands.
|
||||
**Status: first vertical slice.** The schema, the Game8 parser, and two working adapters (Genshin
|
||||
Impact, Neverness to Everness) exist and are tested. The server, database, and UI do not exist yet —
|
||||
`docs/` specifies them.
|
||||
|
||||
## Two constraints that shape everything
|
||||
## Three constraints that shape everything
|
||||
|
||||
1. **No accounts, no logins, no user records.** Completion state lives in the browser's
|
||||
`localStorage`, keyed by event ID. There is no user table and no session. Any feature request
|
||||
that implies "sync across devices" must be solved with export/import JSON, not a server-side
|
||||
user.
|
||||
2. **A server is allowed, and is where all secrets live.** The Bun server owns scraping, LLM
|
||||
extraction, and the SQLite database. `ANTHROPIC_API_KEY` never reaches the browser. The client
|
||||
only ever calls this app's own `/api/*`.
|
||||
`localStorage`, keyed by event ID. There is no user table and no session. Any request implying
|
||||
"sync across devices" is solved with export/import JSON, not a server-side user.
|
||||
2. **No LLM in the pipeline.** Event data is extracted by deterministic code-based parsers only.
|
||||
There is no Anthropic dependency, no API key, and no per-run inference cost. A source that
|
||||
cannot be parsed deterministically does not get an adapter — see `docs/INGESTION.md` § No LLM.
|
||||
3. **A server is allowed** (Bun) and owns fetching, parsing, and SQLite. The client only ever calls
|
||||
this app's own `/api/*`.
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Choice |
|
||||
|---|---|
|
||||
| Runtime / server / bundler / test runner | Bun (single dependency — `Bun.serve`, `bun:sqlite`, `bun test`, `bun build`) |
|
||||
| Runtime / server / bundler / test runner | Bun 1.3 (`Bun.serve`, `bun:sqlite`, `bun test`, `bun build`) |
|
||||
| UI | React 19 + TypeScript (strict) + Tailwind |
|
||||
| Storage | SQLite via `bun:sqlite` (file is gitignored — `*.sqlite`) |
|
||||
| Storage | SQLite via `bun:sqlite` (gitignored — `*.sqlite`) |
|
||||
| Validation | Zod — one schema module shared by server and client |
|
||||
| LLM | Anthropic TypeScript SDK (`@anthropic-ai/sdk`), model `claude-opus-5` |
|
||||
|
||||
TypeScript runs `strict: true` **and** `noUncheckedIndexedAccess`. Do not add a bundler, test
|
||||
runner, or process manager — Bun covers all three.
|
||||
The only runtime dependency is `zod`. Do not add a bundler, test runner, HTTP client, or HTML
|
||||
parsing library — Bun covers all four. `tsconfig.json` runs `strict` plus
|
||||
`noUncheckedIndexedAccess` and `exactOptionalPropertyTypes`.
|
||||
|
||||
## Architecture in one paragraph
|
||||
## Commands
|
||||
|
||||
A scheduled job inside the Bun process runs one **adapter** per game. Each adapter fetches a source
|
||||
page, cleans it, and hands it to a **deterministic parser** when the source has a stable shape, or
|
||||
to **Claude structured extraction** when it doesn't. Results are validated with Zod plus calendar
|
||||
sanity rules, then either published to the `events` table or held in `events_quarantine` for human
|
||||
review at an unauthenticated `/review` route bound to `127.0.0.1`. The React client fetches
|
||||
`/api/events`, renders a calendar and an ends-soonest list, and stores completion ticks in
|
||||
`localStorage`. Full detail: `docs/ARCHITECTURE.md`.
|
||||
```bash
|
||||
bun install
|
||||
bun test # full suite, offline, no network
|
||||
bun run typecheck # tsc --noEmit
|
||||
|
||||
The important consequence: **the LLM runs at ingestion time, never in a request path.** A page load
|
||||
must never trigger an API call to Anthropic. If you find yourself adding one, the design is wrong.
|
||||
# Run an adapter against a checked-in fixture (offline, free)
|
||||
bun run parse genshin-game8-events fixtures/genshin/game8-events-2026-08-14.html
|
||||
bun run parse nte-game8-events fixtures/nte/game8-events-2026-08-14.html --json
|
||||
|
||||
## Reading order for a new task
|
||||
# Single test file / single test
|
||||
bun test test/dates.test.ts
|
||||
bun test --test-name-pattern "year-less"
|
||||
```
|
||||
|
||||
| Task | Read |
|
||||
|---|---|
|
||||
| Anything at all | `docs/ARCHITECTURE.md` |
|
||||
| Adding/changing an event field | `docs/DATA-MODEL.md` — the schema is versioned and the client depends on it |
|
||||
| Adding a game, fixing a broken adapter | `docs/INGESTION.md`, then invoke the `add-game-source` skill |
|
||||
| Touching prompts, extraction, or cost | `docs/LLM-EXTRACTION.md` |
|
||||
| Product questions (what does the calendar show?) | `docs/PRD.md` |
|
||||
`bun run parse ... --json` is also how `.expected.json` fixtures are regenerated after an
|
||||
intentional parser change. Regenerating them makes the test self-consistent, not correct — always
|
||||
re-verify a sample against the live page afterward.
|
||||
|
||||
## Current state of the code
|
||||
|
||||
```
|
||||
src/shared/schema.ts Zod GachaEvent, GameId, slugify, eventId ← the contract
|
||||
src/ingest/html.ts flat-table HTML reader (no dependency)
|
||||
src/ingest/dates.ts three date formats, null rather than guess
|
||||
src/ingest/adapters/
|
||||
types.ts Adapter interface, ParseContext
|
||||
game8.ts shared Game8 parser — handles 2 table shapes
|
||||
index.ts adapter registry
|
||||
scripts/parse-fixture.ts offline adapter runner
|
||||
test/ 37 tests
|
||||
fixtures/<game>/ raw HTML + .expected.json per source
|
||||
```
|
||||
|
||||
Not yet built: `src/server/**`, `src/client/**`, the SQLite layer, the scheduler, the review UI.
|
||||
|
||||
## Domain rules that are not obvious from the code
|
||||
|
||||
These come from how gacha games actually schedule things, and they are the source of most bugs in
|
||||
this kind of app:
|
||||
These come from how gacha games actually schedule things, and they cause most bugs here:
|
||||
|
||||
- **Store every timestamp as UTC ISO 8601. Never store a local wall-clock time.** Sources publish
|
||||
in a mix of UTC+8, server-local, and "after maintenance".
|
||||
- **Banner ends are usually global and simultaneous; event ends are usually per-region.** Genshin
|
||||
and HSR character banners end at the same instant worldwide, while story/login events end at each
|
||||
region's daily reset (Asia / America / Europe are offset by hours). The `regionScoped` flag and
|
||||
the optional `regionEnds` map exist for exactly this — do not collapse them into one timestamp.
|
||||
- **"Ends after maintenance" and "TBD" are real values.** An event whose end is genuinely unknown
|
||||
gets `endsAt: null` and `endPrecision: "unknown"`. Never invent a plausible date to satisfy a
|
||||
non-null type — that is the single worst failure mode for this app, because the user's whole
|
||||
- **Store every timestamp as UTC ISO 8601.** Sources publish in a mix of UTC+8, server-local, and
|
||||
"after maintenance".
|
||||
- **Banner ends are usually global and simultaneous; event ends are usually per-region.** Character
|
||||
banners end at one instant worldwide; story/login events end at each region's daily reset (Asia /
|
||||
America / Europe differ by hours). `regionScoped` and `regionEnds` exist for this — do not
|
||||
collapse them into one timestamp.
|
||||
- **`endsAt: null` is a correct, expected value.** An event whose end is genuinely unannounced gets
|
||||
`endsAt: null` and `endPrecision: "unknown"`. **Never invent a plausible date to satisfy a
|
||||
non-null type.** This is the worst failure mode this codebase has, because the user's entire
|
||||
reason for visiting is trusting the end date.
|
||||
- **Version 1.x patch cycles are ~6 weeks (42 days), split into two banner phases.** Any extracted
|
||||
event with a duration over 180 days is almost certainly a parse error, not a long event. The
|
||||
validator rejects it.
|
||||
- **Patch cycles are ~6 weeks.** Any event over 180 days is a parse error, not a long event. The
|
||||
validator and the tests both reject it.
|
||||
|
||||
## Working with the LLM extraction layer
|
||||
## Working on parsers
|
||||
|
||||
Read `docs/LLM-EXTRACTION.md` before editing any prompt or request. The rules that will actually
|
||||
bite you:
|
||||
- **Parsers are pure.** No network, no `Date.now()`, no randomness — time arrives as `ctx.now`.
|
||||
This is what makes fixture tests meaningful; a parser that reads the clock cannot be tested.
|
||||
- **Skip, never guess.** Every function in `dates.ts` returns `null` rather than inferring a missing
|
||||
year, month, or end. `readColumnTable` drops a row it cannot date. An omitted event is a
|
||||
recoverable disappointment; a confidently wrong date is the failure this product exists to prevent.
|
||||
- **Game8 has no single template.** Three shapes are known so far, and a page may mix them:
|
||||
1. Label/value detail tables (`Event Start` / `Event End`) — Genshin.
|
||||
2. Column tables (`Event | Duration | Event Details | Rewards`) — NTE.
|
||||
3. Image-grid schedules with a bare `MM/DD` and no end — Arknights: Endfield. **Unsupportable**;
|
||||
yields nothing by design.
|
||||
Before assuming a new Game8 page will work, dump its structure and check which shape it uses.
|
||||
- **Silent drops are the dangerous failure.** A date format the parser does not recognise makes
|
||||
events vanish with no error. Abbreviated months (`Apr. 29 - May 13, 2026`) are supported for
|
||||
exactly this reason. When adding a source, compare the parser's event count against an
|
||||
independent count of the page.
|
||||
|
||||
- **Model is `claude-opus-5`.** That is the exact, complete ID — never append a date suffix.
|
||||
- **Use structured outputs, not prompt-and-parse.** `client.messages.parse()` with
|
||||
`zodOutputFormat(EventExtractionSchema)` from `@anthropic-ai/sdk/helpers/zod`. Read
|
||||
`response.parsed_output`. Do not write a JSON-repair or regex-extraction fallback — if the schema
|
||||
is right, there is nothing to repair.
|
||||
- **Never set `temperature`, `top_p`, or `top_k`.** They are removed on `claude-opus-5` and return
|
||||
a 400. Steer with the prompt.
|
||||
- **Never set `thinking: {type: "enabled", budget_tokens: N}`.** Removed — returns 400. Thinking is
|
||||
on by default; control depth with `output_config.effort`.
|
||||
- **Deterministic parsers come first.** The LLM is for sources whose markup is unstable. A source
|
||||
with a clean JSON API or a stable table must not go through the model.
|
||||
- **Skip unchanged sources by content hash.** This is the main cost lever — most refresh cycles
|
||||
should make zero API calls.
|
||||
## Event IDs are localStorage keys
|
||||
|
||||
## Cost discipline
|
||||
```
|
||||
`${game}:${slugify(title)}:${startsAt.slice(0, 10)}`
|
||||
→ "genshin:mutual-aid-in-bloom-into-the-frostlands:2026-08-12"
|
||||
```
|
||||
|
||||
Every ingestion run should be able to answer "why did this cost anything?" Extraction is billed at
|
||||
`claude-opus-5` rates ($5/MTok input, $25/MTok output). Three levers, in order of impact:
|
||||
|
||||
1. **Content-hash skip** — unchanged source, no call at all.
|
||||
2. **HTML pre-cleaning** — strip `<script>`, `<style>`, nav, and footer before sending. Cuts a
|
||||
typical wiki page from ~15k to ~5k tokens.
|
||||
3. **Batch API for scheduled runs** — 50% off, results within an hour, which is fine for a 6-hour
|
||||
cadence. Use the synchronous API only for the manual `refresh --now` path.
|
||||
|
||||
Prompt caching applies to the shared extraction system prompt (`cache_control: {type: "ephemeral"}`,
|
||||
1-hour TTL). The minimum cacheable prefix on `claude-opus-5` is **512 tokens** — the system prompt
|
||||
is deliberately above it. If you shorten the system prompt below 512 tokens, caching silently stops
|
||||
working and nothing errors; check `usage.cache_read_input_tokens` after any prompt edit.
|
||||
Changing `slugify` or `eventId` in `src/shared/schema.ts` — including seemingly cosmetic changes to
|
||||
the slug rules — **silently orphans every completion mark every user has, with no server-side
|
||||
recovery**, because the server never had the data. If it must change, ship a client-side migration
|
||||
that remaps old keys and keep it for at least a year. Use the **schema-guardian** agent on any such
|
||||
change.
|
||||
|
||||
## Scraping conduct
|
||||
|
||||
Sources are community wikis and official news pages. Treat them as a guest would:
|
||||
Sources are community wikis. Treat them as a guest would:
|
||||
|
||||
- Honor `robots.txt`; set a descriptive `User-Agent` with a contact URL.
|
||||
- One request per source per refresh cycle, minimum 6 hours apart. No parallel hammering.
|
||||
- Send `If-None-Match` / `If-Modified-Since` and treat `304` as "skip, unchanged".
|
||||
- Cache raw snapshots locally so re-running extraction never re-fetches.
|
||||
- One request per source per refresh cycle, minimum 6 hours apart.
|
||||
- Send `If-None-Match` / `If-Modified-Since`; treat `304` as "skip, unchanged".
|
||||
- Cache raw snapshots so re-parsing never re-fetches. **Iterate against fixtures, not the network.**
|
||||
- Record `sourceUrl` on every event and surface attribution in the UI.
|
||||
|
||||
A new source that forbids automated access in its ToS does not get an adapter. Flag it and ask.
|
||||
Note that game8.co disallows `GPTBot` and `Google-Extended` in `robots.txt` — it has opted out of
|
||||
AI-training crawlers. Our use is a low-rate personal aggregator with attribution and no model
|
||||
training, and no `User-agent: *` rule applies to our paths. Keep it that way: do not raise the fetch
|
||||
rate, and do not add an LLM that consumes page content.
|
||||
|
||||
A source whose ToS forbids automated access does not get an adapter. Flag it and ask.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Zod schemas are the single source of truth for types.** Derive TypeScript types with
|
||||
`z.infer<>`; never hand-write an interface that duplicates a schema.
|
||||
- **Event IDs are stable and deterministic**: `${gameId}:${slugify(title)}:${startsAt.slice(0,10)}`.
|
||||
They are the localStorage keys, so **changing the ID scheme silently wipes every user's completion
|
||||
state.** If you must change it, ship a migration in the client that remaps old keys.
|
||||
- **Adapters are pure over their input.** `fetch` is separate from `parse` so parsers can be tested
|
||||
against checked-in HTML fixtures with no network.
|
||||
- Every adapter ships a fixture in `fixtures/<game>/` and a test asserting the parsed output. This
|
||||
is how a source silently changing shape gets caught.
|
||||
- **Zod schemas are the single source of truth for types.** Derive with `z.infer<>`; never
|
||||
hand-write an interface that duplicates a schema.
|
||||
- Every adapter ships a fixture in `fixtures/<game>/` and a test asserting parsed output. This is
|
||||
how a source silently changing shape gets caught.
|
||||
- Keep old fixtures when a source changes shape — the old one is the regression test proving the
|
||||
parser still handles the previous format.
|
||||
|
||||
Reference in New Issue
Block a user