docs: bring every markdown file up to date with the code
The docs had drifted in ways that would mislead: DATA-MODEL documented a localStorage shape the code stopped using (completedAt, no ignored store), INGESTION claimed three Game8 templates when five are known, ARCHITECTURE still listed the whole client and time.ts as unbuilt, and the review-quarantine skill described a pipeline that does not exist yet without saying so. Adds the parser roster and the six date formats as tables, documents the subpath/base-href and offline behaviour, and records the new product surface (first-run picker, ignore, offline, credit) as PRD features. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f2cf9ba0cb
commit
692b5a83d5
@@ -24,8 +24,8 @@ Fetch the page exactly once.
|
||||
|
||||
**3. Decide whether an existing parser covers it.**
|
||||
|
||||
Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game — one Game8 parser serves
|
||||
every Game8 page. Check `PARSERS` first:
|
||||
Parsers live in `src/ingest/parsers/` and are keyed by *site*, not game. Two exist: `game8`
|
||||
(six sources) and `wikigg` (wiki.gg MediaWiki `mp-event` templates). Check `PARSERS` first:
|
||||
|
||||
- **Existing parser handles it** → add one entry to `SOURCES` in `adapters/index.ts`. No new
|
||||
parsing code. This is the common case and should be the first thing you try.
|
||||
@@ -34,6 +34,10 @@ every Game8 page. Check `PARSERS` first:
|
||||
- **Undatable source** (no year, no end date, image-grid schedule) → **stop and report.** There is
|
||||
no LLM fallback in this pipeline by design. Suggest a different source.
|
||||
|
||||
**Check every table before concluding a page is undatable.** Endfield was written off on a pass that
|
||||
only inspected its `Duration` rows; its real events were in a table further down, and a second
|
||||
source (wiki.gg) turned out to publish ISO timestamps with per-region timers.
|
||||
|
||||
**4. Implement the parser (only if step 3 says you need one).**
|
||||
|
||||
- `parse` must be **pure**: no network, no `Date.now()`, no randomness. Time comes from `ctx.now`.
|
||||
@@ -48,9 +52,13 @@ every Game8 page. Check `PARSERS` first:
|
||||
- An unstated end is `endsAt: null` + `endPrecision: "unknown"`. **Never compute a plausible end
|
||||
from typical patch length.** This is the failure mode that makes the product worthless.
|
||||
|
||||
**5. Write the test.** `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` holds the exact
|
||||
expected `GachaEvent[]`. The test runs `parse` + `normalize` against the fixture with a pinned
|
||||
`ctx.now` and asserts deep equality.
|
||||
**5. Write the test.** `fixtures/<game>/<site>-events-<YYYY-MM-DD>.expected.json` holds the exact
|
||||
expected `GachaEvent[]`. Add the source to `CASES` in `test/adapters/game8.test.ts`; the shared
|
||||
assertions (schema, determinism, no backwards intervals, 180-day cap, unique IDs) then apply for
|
||||
free. Regenerate the expected file with `bun run parse <id> <fixture> --json`.
|
||||
|
||||
**Fixture names matter**: `build-feed` selects by `<site>-*` within the game directory, so a game
|
||||
with two sources needs distinct site prefixes.
|
||||
|
||||
**6. Verify.** Run `bun test` and confirm it passes with no network. Then hand-check three or four
|
||||
events against the live page and state in your report that you did — a green test against an
|
||||
|
||||
@@ -35,9 +35,13 @@ only schema edit a new game should require. If it needs more, that is a finding
|
||||
Fetch the source page **once** and save the raw HTML:
|
||||
|
||||
```
|
||||
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
|
||||
fixtures/<game>/<site>-events-<YYYY-MM-DD>.html
|
||||
```
|
||||
|
||||
The `<site>` prefix is load-bearing: `build-feed` picks fixtures by site within the game directory,
|
||||
so a game with two sources whose files share a prefix will hand one site's page to the other's
|
||||
parser.
|
||||
|
||||
Everything after this point works offline against that file. Do not re-fetch while iterating on the
|
||||
parser.
|
||||
|
||||
@@ -77,16 +81,18 @@ report that you did this.
|
||||
`parserId`, and optionally `priority` (higher wins when sources disagree).
|
||||
- Insert the matching `sources` row: id, game, url, parser_id, priority, `min_interval_ms`.
|
||||
|
||||
## 7. First run
|
||||
## 7. Rebuild the feed
|
||||
|
||||
```
|
||||
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
|
||||
bun run build:feed # regenerates public/data/events.v1.json
|
||||
bun run dev # build and serve on :3000
|
||||
```
|
||||
|
||||
Inspect what it *would* publish before letting it write. Then run for real and check the review
|
||||
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
|
||||
on its first pass, because nothing corroborates it yet. That is the gate working, not a bug.
|
||||
Review and approve them.
|
||||
Check the event count and any conflicts the merge reports. A game with two sources will surface
|
||||
disagreements — those are the gate working, not a bug.
|
||||
|
||||
The scheduler, quarantine table and `/review` queue described in `docs/INGESTION.md` are **not built
|
||||
yet**; today the feed is generated offline from fixtures.
|
||||
|
||||
## Checklist
|
||||
|
||||
|
||||
@@ -5,6 +5,11 @@ description: Work through the quarantined-event queue — triage held events by
|
||||
|
||||
# Reviewing the quarantine queue
|
||||
|
||||
> **Not built yet.** The quarantine table, the `/review` route and the ingest scheduler are specified
|
||||
> in `docs/INGESTION.md` but do not exist in the tree. Today the closest equivalent is the conflict
|
||||
> list `bun run build:feed` prints when two sources disagree. Use this skill once the pipeline lands;
|
||||
> until then, treat it as the spec for what that review flow should do.
|
||||
|
||||
Held events are candidates the pipeline declined to publish. Working the queue is both a data task
|
||||
(get these events onto the calendar) and a diagnostic one (**a growing queue means something
|
||||
upstream broke**).
|
||||
|
||||
Reference in New Issue
Block a user