78 lines
7.3 KiB
Markdown
78 lines
7.3 KiB
Markdown
# Architecture
|
|
|
|
How `mangabaka-seeder` is built today. Update in the same commit as any structural change.
|
|
|
|
## Overview
|
|
v0.1 is built: the core (`src/core/`), the self-hosted server, the page and the userscript. `pipeline.ts` composes the stages of spec §3; `session.ts` holds the review state both front ends share; the page and the userscript mount the same markup (`src/web/template.ts`) and wiring (`src/web/app.ts`). The design is in [`specs/001-webtoons-covers.md`](specs/001-webtoons-covers.md):
|
|
one pure TypeScript core (`src/core/`) that turns a MangaBaka series link and a Webtoons series
|
|
link into a cover submission, used by three front ends — a static page served by a small
|
|
self-hosted Bun server (primary), a userscript, and later a Bun CLI.
|
|
|
|
## Map
|
|
| Path | Responsibility |
|
|
|---|---|
|
|
| `src/core/` | Pure pipeline: sources, Wayback history, image inspection, dedupe, seed, MangaBaka client. No DOM, Bun, Node or `GM_*` APIs. |
|
|
| `src/core/session.ts` | Review state shared by the page and the userscript: cards, main cover, rehosting (cards remember host, cache key and expiry), manual text, the download bundle, `restore` from history, token flow (preview → confirm → save for review). |
|
|
| `src/core/zip.ts`, `download.ts` | Store-only ZIP writer (CRC-32, UTF-8 names); the download bundle: file names `NN_role_lang[_YYYY]_WxH.ext` and `covers.txt` (spec 002 §2). |
|
|
| `src/core/saved.ts` | History model (spec 002 §3): `SavedSearch` (plain data, source by id), `toSaved`/`fromSaved` (drops expired links), `carryEdits` (search again), `pruneIds` (30 entries, 300 MB), `HistoryStore` interface and an in-memory one. |
|
|
| `src/core/sources/` | `webtoons/` and `tapas/` (spec 003): each `urls.ts` (links, image URL normalisation, same-series), `parse.ts` (pure HTML parsing, also used on Wayback snapshots), `index.ts` (the `Source`); `registry.ts` lists them and builds the proxy and upload allowlists. |
|
|
| `src/web/` | Built. The page ("Obi Shelf" design, see `DESIGN.md`): `template.ts` markup shared with the userscript panel, `app.ts` wiring (`mountApp`), `render.ts` shelf and timeline views, `view-model.ts` pure helpers, `transport.ts` (direct to MangaBaka/litterbox, everything else via `/api/fetch`), `styles.css` scoped for a shadow root, `save-file.ts` (Blob download, works in the panel's shadow root), `history-db.ts` (IndexedDB `HistoryStore`; null when storage is refused), `history-keeper.ts` (when the series on screen is saved: after a search, debounced edits, marks; flushes before a new search, an open and page hide; removal/clear stop tracking), `recent.ts` (History list and the start screen's recent shelf; the page only), `dark-reader.ts` (switches to the page's own dark theme and locks Dark Reader out when that extension is darkening the page). |
|
|
| `src/server/` | Built. `app.ts`: web-standard handler (`createApp(env)`) — static assets, `/api/config`, `/api/fetch` allowlisted proxy (redirects re-checked, 20 MB cap), `/api/upload` (store or catbox; takes a source URL, never bytes), `/covers/*`. `storage.ts` (interface, in-memory) and `storage-dir.ts` (folder, size cap, 30-day sweep). `config.ts` (env, client IP), `rate-limit.ts`, `assets-path.ts`. `bun.ts`: thin `Bun.serve` entry. Gate and Workers entry deferred. |
|
|
| `src/userscript/` | Built. `gm.ts`: `GM_xmlhttpRequest` transport (no proxy; honest User-Agent for Wayback) and GM storage. `main.ts`: a "Seed covers" launcher on MangaBaka series pages (follows client-side navigation) opening the page's app in a shadow-root panel; optional server URL in its settings. Built by `scripts/build-userscript.ts` into `dist/web/mbseed.user.js`, served at `/mbseed.user.js`. |
|
|
| `src/cli/` | Later: Bun CLI. |
|
|
| `test/fixtures/` | Saved real responses; tests never hit the network. |
|
|
|
|
## Data flow
|
|
`runCovers` (pipeline.ts): links → MangaBaka series, links and images (direct) → `source.resolve` and
|
|
`currentCovers` → `waybackHistory` (optional) → `inspectAll` (bytes, size, sha256, shape) → `dedupeByHash`
|
|
→ `splitExisting` against MangaBaka → `pickDefault`. `Session` (session.ts) turns that into review cards; the
|
|
user edits them, then `bundle` → ZIP download, or `rehost` (explicit) → `manualText` for the cover form, or
|
|
`preview` → confirm → `submit` (`save_mode: "review"`). On the page each search is saved to IndexedDB
|
|
(`toSaved` → `saveAndPrune`) after the run, edits, rehosting, download and filing; opening an entry runs
|
|
`fromSaved` → `Session.restore`, and a new search of a saved series runs `carryEdits`.
|
|
|
|
## Running the server
|
|
`bun run build` then `bun run start`. Environment:
|
|
|
|
| Variable | Default | Meaning |
|
|
|---|---|---|
|
|
| `MBSEED_PORT` | `8787` | Listen port. |
|
|
| `MBSEED_HOST` | `127.0.0.1` | Listen address (behind the reverse proxy). |
|
|
| `MBSEED_PUBLIC_URL` | — | Public origin, e.g. `https://mbseed.example.org`; required with storage (stored covers are served from it). |
|
|
| `MBSEED_STORAGE_DIR` | — | Covers folder; unset = no server storage. |
|
|
| `MBSEED_STORAGE_MAX_BYTES` | `1073741824` | Folder size cap; uploads past it get 507. |
|
|
| `MBSEED_TRUST_PROXY` | unset | `1` = rate-limit on the last `X-Forwarded-For` entry. |
|
|
| `MBSEED_ASSETS_DIR` | `dist/web` | Built page and userscript. |
|
|
|
|
Put it behind a reverse proxy that terminates TLS and **appends** the client address to
|
|
`X-Forwarded-For` (nginx `$proxy_add_x_forwarded_for`, Caddy's default), then set `MBSEED_TRUST_PROXY=1`.
|
|
|
|
**Docker.** `Dockerfile` builds the page in one stage and runs the server as user `bun` (uid 1000) on
|
|
`0.0.0.0:8787`; `MBSEED_ASSETS_DIR=/app/dist/web`. Storage stays off unless `MBSEED_STORAGE_DIR` is set
|
|
(`/data/covers` exists for it). The container builds the userscript at start, so its `@connect` list
|
|
includes `MBSEED_PUBLIC_URL`'s host. `bun.ts` stops on SIGTERM/SIGINT (Bun as PID 1 ignores them otherwise).
|
|
`.gitea/workflows/ci.yml` typechecks, tests and builds, then builds and pushes the image (not for pull
|
|
requests) and smoke-tests it. Example deployment: `docker-compose.example.yml`.
|
|
|
|
## External dependencies
|
|
| Service | Reached via | If unavailable |
|
|
|---|---|---|
|
|
| `api.mangabaka.org` | direct (CORS open) | Nothing works. |
|
|
| Webtoons HTML / `swebtoon-phinf.pstatic.net` | server proxy or `GM_xmlhttpRequest` | Nothing works. |
|
|
| Wayback (`web.archive.org`) | same | Warning; current covers only. |
|
|
| Server storage (folder on our server) | server | `server` host hidden; litterbox/catbox/direct still work. |
|
|
| litterbox / catbox | browser directly (litterbox), server or `GM_xmlhttpRequest` (catbox) | That host fails; others still work. |
|
|
|
|
## Invariants
|
|
- `src/core/` never imports from `web/`, `server/`, `userscript/` or `cli/`.
|
|
- The MangaBaka token is only ever sent to `api.mangabaka.org`, never through our server.
|
|
- The covers folder never exceeds `MBSEED_STORAGE_MAX_BYTES`; when full, uploads are refused and
|
|
no file is deleted before its 30 days.
|
|
- Nothing is uploaded or submitted without an explicit user action; saves use `save_mode: "review"`.
|
|
- `/api/fetch` only fetches allowlisted hosts; `/api/upload` only takes a source URL, never bytes.
|
|
- Proxied bodies are inert (`application/octet-stream`, sandbox CSP, attachment), and `/api/*` fetch/upload need
|
|
the `x-mbseed: 1` header; `/api/upload` only rehosts from source image hosts or their Wayback copies.
|
|
|
|
## Known rough edges
|
|
See the open questions in spec §7 and [`TODO.md`](TODO.md).
|