108 lines
4.6 KiB
Markdown
108 lines
4.6 KiB
Markdown
---
|
|
name: add-game-source
|
|
description: End-to-end workflow for adding a new game (or a new source for an existing game) to the event tracker — legal check, fixture capture, adapter, tests, registration, and first ingest run. Use when asked to add a game, add a second source, or wire up a data source for the calendar.
|
|
---
|
|
|
|
# Adding a game source
|
|
|
|
The design goal is that a new game costs an adapter, a fixture, and a test — **no schema change and
|
|
no client change.** If you find yourself editing `src/shared/schema.ts` or a React component to make
|
|
a game fit, stop: either the data model is wrong (raise it) or the game is being forced into a shape
|
|
it does not have.
|
|
|
|
Read `docs/INGESTION.md` and `docs/DATA-MODEL.md` before starting.
|
|
|
|
## 1. Legal and conduct check — a hard gate
|
|
|
|
Before anything else:
|
|
|
|
- Fetch `<host>/robots.txt`. Confirm the target path is not disallowed.
|
|
- Skim the site's terms for a prohibition on automated access or scraping.
|
|
- Prefer an official API or a community wiki with a permissive license over scraping an official
|
|
site directly.
|
|
|
|
If the source forbids automated access, **stop and report it.** Do not write the adapter and do not
|
|
look for a workaround. Suggest an alternative source instead.
|
|
|
|
## 2. Register the game
|
|
|
|
If this is a new game rather than a new source for an existing one, add it to `GameId` in
|
|
`src/shared/schema.ts` and give it a display name and lane color in the games registry. This is the
|
|
only schema edit a new game should require. If it needs more, that is a finding worth reporting.
|
|
|
|
## 3. Capture a fixture
|
|
|
|
Fetch the source page **once** and save the raw HTML:
|
|
|
|
```
|
|
fixtures/<game>/<source-id>-<YYYY-MM-DD>.html
|
|
```
|
|
|
|
Everything after this point works offline against that file. Do not re-fetch while iterating on the
|
|
parser.
|
|
|
|
## 4. Build the adapter
|
|
|
|
Delegate to the **adapter-author** agent, or do it inline for a simple table source. Either way the
|
|
requirements are the same:
|
|
|
|
- `parse` is pure over its input — no network, no `Date.now()`. Time comes from `ctx.now`.
|
|
- Prefer a deterministic parser. Use `llm` strategy only when the markup genuinely cannot be parsed
|
|
reliably, and say why.
|
|
- `normalize` handles source-timezone → UTC, region reset offsets, and ID construction.
|
|
|
|
**The three domain rules that break new adapters**, from `CLAUDE.md`:
|
|
|
|
1. Everything stored as UTC ISO 8601.
|
|
2. Banners are usually one global end instant; story and login events usually end at each region's
|
|
daily reset. Set `regionScoped` and `regionEnds` accordingly.
|
|
3. An unstated end date is `endsAt: null` with `endPrecision: "unknown"`. **Never derive a plausible
|
|
end from typical patch length.** A confidently wrong end date is the failure this product exists
|
|
to prevent.
|
|
|
|
## 5. Test it
|
|
|
|
Write `fixtures/<game>/<source-id>-<YYYY-MM-DD>.expected.json` with the exact expected
|
|
`GachaEvent[]`, and a test asserting deep equality with a pinned `ctx.now`.
|
|
|
|
Run `bun test`. It must pass with no network access.
|
|
|
|
Then **hand-check three or four events against the live page.** The test only proves the parser
|
|
agrees with an expected file you wrote yourself; it does not prove either is right. Say in your
|
|
report that you did this.
|
|
|
|
## 6. Wire it up
|
|
|
|
- Register the adapter in `src/ingest/adapters/index.ts`.
|
|
- Insert the `sources` row: id, game, url, strategy, `min_interval_ms` (default 6h).
|
|
|
|
## 7. First run
|
|
|
|
```
|
|
INGEST_ENABLED=true bun run ingest --source <source-id> --dry-run
|
|
```
|
|
|
|
Inspect what it *would* publish before letting it write. Then run for real and check the review
|
|
queue at `http://127.0.0.1:$ADMIN_PORT/review` — a new source commonly lands events in quarantine
|
|
on its first pass, because nothing corroborates it yet and LLM-extracted events start at 0.70
|
|
confidence. That is the gate working, not a bug. Review and approve them.
|
|
|
|
## Checklist
|
|
|
|
- [ ] robots.txt and ToS permit it
|
|
- [ ] Game registered in `GameId` (new games only)
|
|
- [ ] Fixture captured, page fetched exactly once
|
|
- [ ] Adapter implemented; `parse` pure, no clock access
|
|
- [ ] Timestamps UTC; `regionScoped` correct; unstated ends are `null`
|
|
- [ ] Expected-output file + passing test, offline
|
|
- [ ] Manually spot-checked against the live page
|
|
- [ ] Registered in the adapter index and `sources`
|
|
- [ ] Dry run inspected, then real run, then quarantine reviewed
|
|
|
|
## When something does not fit
|
|
|
|
Report it rather than working around it. A game with a fourth server region, an event type the enum
|
|
lacks, or a source that publishes only relative dates ("starts next Tuesday") are all real
|
|
possibilities the current model does not cover. Those are design questions, not adapter bugs — say
|
|
what you found and what it would take.
|