Lucas WintherandClaude Opus 5.5
baa3476b73
From the final review: proxied archived HTML could run on our origin and read a remembered token; /api/upload accepted any Wayback URL and GIF-prefixed bytes; the origin check broke behind a TLS proxy; one unreadable snapshot could fail a run. Co-Authored-By: Claude Opus 5.5 <[email protected]>
66 lines
5.1 KiB
Markdown
66 lines
5.1 KiB
Markdown
# Architecture
|
|
|
|
How `mangabaka-seeder` is built today. Update in the same commit as any structural change.
|
|
|
|
## Overview
|
|
v0.1 is built: the core (`src/core/`), the self-hosted server, the page and the userscript. `pipeline.ts` composes the stages of spec §3; `session.ts` holds the review state both front ends share; the page and the userscript mount the same markup (`src/web/template.ts`) and wiring (`src/web/app.ts`). The design is in [`specs/001-webtoons-covers.md`](specs/001-webtoons-covers.md):
|
|
one pure TypeScript core (`src/core/`) that turns a MangaBaka series link and a Webtoons series
|
|
link into a cover submission, used by three front ends — a static page served by a small
|
|
self-hosted Bun server (primary), a userscript, and later a Bun CLI.
|
|
|
|
## Map
|
|
| Path | Responsibility |
|
|
|---|---|
|
|
| `src/core/` | Pure pipeline: sources, Wayback history, image inspection, dedupe, seed, MangaBaka client. No DOM, Bun, Node or `GM_*` APIs. |
|
|
| `src/core/session.ts` | Review state shared by the page and the userscript: cards, main cover, rehosting, manual text, token flow (preview → confirm → save for review). |
|
|
| `src/web/` | Built. The page ("Obi Shelf" design, see `DESIGN.md`): `template.ts` markup shared with the userscript panel, `app.ts` wiring (`mountApp`), `render.ts` shelf and timeline views, `view-model.ts` pure helpers, `transport.ts` (direct to MangaBaka/litterbox, everything else via `/api/fetch`), `styles.css` scoped for a shadow root. |
|
|
| `src/server/` | Built. `app.ts`: web-standard handler (`createApp(env)`) — static assets, `/api/config`, `/api/fetch` allowlisted proxy (redirects re-checked, 20 MB cap), `/api/upload` (store or catbox; takes a source URL, never bytes), `/covers/*`. `storage.ts` (interface, in-memory) and `storage-dir.ts` (folder, size cap, 30-day sweep). `config.ts` (env, client IP), `rate-limit.ts`, `assets-path.ts`. `bun.ts`: thin `Bun.serve` entry. Gate and Workers entry deferred. |
|
|
| `src/userscript/` | Built. `gm.ts`: `GM_xmlhttpRequest` transport (no proxy; honest User-Agent for Wayback) and GM storage. `main.ts`: a "Seed covers" launcher on MangaBaka series pages (follows client-side navigation) opening the page's app in a shadow-root panel; optional server URL in its settings. Built by `scripts/build-userscript.ts` into `dist/web/mbseed.user.js`, served at `/mbseed.user.js`. |
|
|
| `src/cli/` | Later: Bun CLI. |
|
|
| `test/fixtures/` | Saved real responses; tests never hit the network. |
|
|
|
|
## Data flow
|
|
`runCovers` (pipeline.ts): links → MangaBaka series, links and images (direct) → `source.resolve` and
|
|
`currentCovers` → `waybackHistory` (optional) → `inspectAll` (bytes, size, sha256, shape) → `dedupeByHash`
|
|
→ `splitExisting` against MangaBaka → `pickDefault`. `Session` (session.ts) turns that into review cards; the
|
|
user edits them, then `rehost` (explicit) → `manualText` for the cover form, or `preview` → confirm → `submit`
|
|
(`save_mode: "review"`).
|
|
|
|
## Running the server
|
|
`bun run build` then `bun run start`. Environment:
|
|
|
|
| Variable | Default | Meaning |
|
|
|---|---|---|
|
|
| `MBSEED_PORT` | `8787` | Listen port. |
|
|
| `MBSEED_HOST` | `127.0.0.1` | Listen address (behind the reverse proxy). |
|
|
| `MBSEED_PUBLIC_URL` | — | Public origin, e.g. `https://mbseed.example.org`; required with storage (stored covers are served from it). |
|
|
| `MBSEED_STORAGE_DIR` | — | Covers folder; unset = no server storage. |
|
|
| `MBSEED_STORAGE_MAX_BYTES` | `1073741824` | Folder size cap; uploads past it get 507. |
|
|
| `MBSEED_TRUST_PROXY` | unset | `1` = rate-limit on the last `X-Forwarded-For` entry. |
|
|
| `MBSEED_ASSETS_DIR` | `dist/web` | Built page and userscript. |
|
|
|
|
Put it behind a reverse proxy that terminates TLS and **appends** the client address to
|
|
`X-Forwarded-For` (nginx `$proxy_add_x_forwarded_for`, Caddy's default), then set `MBSEED_TRUST_PROXY=1`.
|
|
|
|
## External dependencies
|
|
| Service | Reached via | If unavailable |
|
|
|---|---|---|
|
|
| `api.mangabaka.org` | direct (CORS open) | Nothing works. |
|
|
| Webtoons HTML / `swebtoon-phinf.pstatic.net` | server proxy or `GM_xmlhttpRequest` | Nothing works. |
|
|
| Wayback (`web.archive.org`) | same | Warning; current covers only. |
|
|
| Server storage (folder on our server) | server | `server` host hidden; litterbox/catbox/direct still work. |
|
|
| litterbox / catbox | browser directly (litterbox), server or `GM_xmlhttpRequest` (catbox) | That host fails; others still work. |
|
|
|
|
## Invariants
|
|
- `src/core/` never imports from `web/`, `server/`, `userscript/` or `cli/`.
|
|
- The MangaBaka token is only ever sent to `api.mangabaka.org`, never through our server.
|
|
- The covers folder never exceeds `MBSEED_STORAGE_MAX_BYTES`; when full, uploads are refused and
|
|
no file is deleted before its 30 days.
|
|
- Nothing is uploaded or submitted without an explicit user action; saves use `save_mode: "review"`.
|
|
- `/api/fetch` only fetches allowlisted hosts; `/api/upload` only takes a source URL, never bytes.
|
|
- Proxied bodies are inert (`application/octet-stream`, sandbox CSP, attachment), and `/api/*` fetch/upload need
|
|
the `x-mbseed: 1` header; `/api/upload` only rehosts from source image hosts or their Wayback copies.
|
|
|
|
## Known rough edges
|
|
See the open questions in spec §7 and [`TODO.md`](TODO.md).
|