diff --git a/AGENTS.md b/AGENTS.md index 46ea9d5..f780fa7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # mangabaka-seeder Seeds cover edits into MangaBaka from a Webtoons series. TypeScript on **Bun** (not Node): -a pure core, a static page served by a Cloudflare Worker, and a userscript. No code yet — the +a pure core, a static page served by a small server (Cloudflare Worker or self-hosted Bun), and a userscript. No code yet — the approved design is `docs/specs/001-webtoons-covers.md`. `AGENTS.md` is the single source of truth for agents here. `CLAUDE.md` and `GEMINI.md` only point here. @@ -32,14 +32,16 @@ Use impeccable for the static page (`src/web/`) and the userscript panel — `cr - `src/core/` runs in a browser, a Worker, a userscript and Bun: no DOM, no `Bun.*`, no `node:*`, no `GM_*`. All network goes through the injected `Transport`; hashing uses `crypto.subtle`. -- `core/` never imports from `web/`, `worker/`, `userscript/` or `cli/`. -- The MangaBaka token goes straight to `api.mangabaka.org` (CORS is open) — never through the Worker. +- `core/` never imports from `web/`, `server/`, `userscript/` or `cli/`. `server/app.ts` uses only + web-standard APIs so it runs on Workers and `Bun.serve`; `worker.ts`/`bun.ts` stay thin adapters. +- The MangaBaka token goes straight to `api.mangabaka.org` (CORS is open) — never through our server. - MangaBaka `images` **replaces the whole set**: a save body must include every existing image. Saves always use `save_mode: "review"`. - Never call the MangaBaka save endpoint (`POST /v0/my/submissions/…` without `/preview`) yourself: it files a real edit request. Tests use the fake transport; tests never touch the network. - Fixtures in `test/fixtures/` are captured from real responses, never hand-written. - `/api/fetch` only proxies allowlisted hosts; `/api/upload` takes a source URL, never raw bytes. +- When `MBSEED_PASSWORD` is set the gate covers **every** route except `/login` — static assets too. - Webtoons image URLs: strip the `?type=` query and use the `swebtoon-phinf.pstatic.net` host: ```ts @@ -51,17 +53,19 @@ Use impeccable for the static page (`src/web/`) and the userscript panel — `cr ## Architecture src/core/ -> pure pipeline: sources, Wayback history, inspect, dedupe, seed, MangaBaka client -src/web/ -> static page · src/worker/ -> assets, `/api/fetch`, `/api/upload` → R2 (30-day expiry) +src/web/ -> static page · src/server/ -> gate, assets, `/api/fetch`, `/api/upload` (Workers or Bun) +src/core/hosting/ -> server storage (30 days), litterbox (≤72h), catbox (permanent), direct src/userscript/ -> panel on mangabaka.org, `GM_xmlhttpRequest` · src/cli/ -> later Full picture: `docs/ARCHITECTURE.md` and the spec — read them before your first change in a session. ## Boundaries -Never commit `.env*`, `.dev.vars`, tokens, or Cloudflare credentials. +Never commit `.env*`, `.dev.vars`, tokens, passwords, S3 keys or Cloudflare credentials. Never force-push, rewrite pushed history, or commit directly to `main`. -Ask first: new runtime dependencies · `wrangler deploy` or any Cloudflare/R2 configuration · +Ask first: new runtime dependencies · `wrangler deploy` or any Cloudflare/R2/S3 configuration · +real uploads to litterbox/catbox · anything sent to MangaBaka with a real token · anything destructive (`rm -rf`, `git reset --hard`) · work the approved plan does not cover. diff --git a/README.md b/README.md index d4341de..abae7ef 100644 --- a/README.md +++ b/README.md @@ -15,11 +15,13 @@ poster becomes the main (default) cover for the edition's language. ## Two ways to use it - **Web page** (primary). Open the page, paste the two links, review the - covers. The page is hosted on Cloudflare; a small Worker behind it reads - Webtoons and the Wayback Machine for you, because browsers can't. + covers. A small server behind the page reads Webtoons and the Wayback + Machine for you, because browsers can't. It runs on Cloudflare Workers or on + your own machine with Bun, and can be locked with a password. - **Userscript.** Install `mbseed.user.js` in Tampermonkey or Violentmonkey. On a MangaBaka series page it adds a **Seed covers** panel; paste the - Webtoons link. It reads Webtoons directly from your browser, no proxy needed. + Webtoons link. It reads Webtoons directly from your browser, and with + litterbox or catbox it needs no server at all. A command-line version (Bun) may follow. @@ -39,9 +41,18 @@ A command-line version (Bun) may follow. 4. **Build the submission.** Existing covers are kept and new ones added. The current poster is the default (the square for canvas series, which have no poster). Every new cover gets a note recording where and when it was seen. -5. **Rehost the images** when you click **Rehost selected**. They go to our - storage and **expire after 30 days** — long enough for a MangaBaka review. - If a review takes longer, run the tool again. +5. **Rehost the images** when you click **Rehost selected**, to the host you + pick: + + | Host | Links last | + |---|---| + | the server's own storage (default, if configured) | 30 days | + | [litterbox](https://litterbox.catbox.moe) | 1 hour – 72 hours | + | [catbox.moe](https://catbox.moe) | permanently | + | none (`direct`): submit the Webtoons URL itself | as long as Webtoons keeps it | + + A link must still work when a MangaBaka moderator looks at the request. If + it has expired by then, run the tool again. 6. **Hand it to you.** - **Without API access (most users):** copy the hosted URLs and paste them into MangaBaka's cover form for the series (the page links to it). @@ -51,7 +62,7 @@ A command-line version (Bun) may follow. Nothing is uploaded or sent to MangaBaka without your action, and submissions always go to review. You choose the content rating (default `safe`); titles Webtoons marks as mature show a warning. Your token is sent only to -MangaBaka's API, never to our Worker. +MangaBaka's API, never to our server. MangaBaka's submissions API is only open to contributors, or to users with at least 25 approved submissions. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6fdba53..dbdc45a 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -5,15 +5,16 @@ How `mangabaka-seeder` is built today. Update in the same commit as any structur ## Overview No code exists yet. The design is in [`specs/001-webtoons-covers.md`](specs/001-webtoons-covers.md): one pure TypeScript core (`src/core/`) that turns a MangaBaka series link and a Webtoons series -link into a cover submission, used by three front ends — a static page served by a Cloudflare -Worker (primary), a userscript, and later a Bun CLI. Replace this section with reality as modules land. +link into a cover submission, used by three front ends — a static page served by a small, +optionally password-gated server that runs on Cloudflare Workers or self-hosted under Bun +(primary), a userscript, and later a Bun CLI. Replace this section with reality as modules land. ## Map (planned — spec §4) | Path | Responsibility | |---|---| | `src/core/` | Pure pipeline: sources, Wayback history, image inspection, dedupe, seed, MangaBaka client. No DOM, Bun, Node or `GM_*` APIs. | | `src/web/` | Static page UI. | -| `src/worker/` | Cloudflare Worker: static assets, `/api/fetch` allowlisted proxy, `/api/upload` → R2. | +| `src/server/` | Portable `fetch` handler (`app.ts`) + password gate; entries `worker.ts` (Cloudflare, R2) and `bun.ts` (self-hosted, S3). Static assets, `/api/fetch` allowlisted proxy, `/api/upload`. | | `src/userscript/` | Userscript on mangabaka.org; network via `GM_xmlhttpRequest`. | | `src/cli/` | Later: Bun CLI. | | `test/fixtures/` | Saved real responses; tests never hit the network. | @@ -26,13 +27,15 @@ sha256) → dedupe against MangaBaka → seed → user reviews → rehost (expli | Service | Reached via | If unavailable | |---|---|---| | `api.mangabaka.org` | direct (CORS open) | Nothing works. | -| Webtoons HTML / `swebtoon-phinf.pstatic.net` | Worker proxy or `GM_xmlhttpRequest` | Nothing works. | +| Webtoons HTML / `swebtoon-phinf.pstatic.net` | server proxy or `GM_xmlhttpRequest` | Nothing works. | | Wayback (`web.archive.org`) | same | Warning; current covers only. | -| Cloudflare R2 | Worker binding | Rehost fails; `direct` host still possible. | +| Server storage (R2 binding or S3 bucket) | server | `server` host hidden; litterbox/catbox/direct still work. | +| litterbox / catbox | browser directly (litterbox), server or `GM_xmlhttpRequest` (catbox) | That host fails; others still work. | ## Invariants -- `src/core/` never imports from `web/`, `worker/`, `userscript/` or `cli/`. -- The MangaBaka token is only ever sent to `api.mangabaka.org`, never through the Worker. +- `src/core/` never imports from `web/`, `server/`, `userscript/` or `cli/`. +- The MangaBaka token is only ever sent to `api.mangabaka.org`, never through our server. +- With `MBSEED_PASSWORD` set, every route except `/login` requires a session (or Bearer password). - Nothing is uploaded or submitted without an explicit user action; saves use `save_mode: "review"`. - `/api/fetch` only fetches allowlisted hosts; `/api/upload` only takes a source URL, never bytes. diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index f208c12..ead8d89 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -16,6 +16,25 @@ the repo every session — measurably slower and more tokens). **Consequences:** Rule changes happen in one file; pointer files must stay thin. Plans are working artefacts: kept after merge they go stale and contradict the code, so the spec is the record. +## 2026-10-14 — Also support catbox/litterbox; optional password gate; self-hostable server +**Status:** accepted (extends the two entries below) +**Context:** The user asked for catbox/litterbox support as well, and for a way to password-gate the +static site "such that we could lock the service behind our own server". +**Decision:** Image hosts are `server` (our storage, 30 days, default when configured), `litterbox` +(≤ 72h; uploaded straight from the browser, which works because litterbox sends CORS headers), +`catbox` (permanent; through our server for the page, direct for the userscript) and `direct`. +The server is one web-standard `fetch` handler with two thin entries: Cloudflare Workers (R2 +binding) and `Bun.serve` (S3-compatible bucket) for our own machine. An optional shared password +(`MBSEED_PASSWORD`) gates every route, static assets included, with an HMAC-signed session cookie, +plus `Authorization: Bearer` for the userscript. Self-hosted instances may use the reverse proxy's +auth instead. "Behind our own server" was read as covering both the built-in gate and self-hosting. +**Alternatives:** Cloudflare Access in front of the Worker (free for small teams, but +Cloudflare-only and no help when self-hosting); per-user accounts (not needed for a small group); +Basic auth (can't log out, and the browser prompt is clumsy for the userscript). +**Consequences:** On Workers, assets must run through the Worker (`run_worker_first`) or the gate +leaks the page. The userscript works without any server when using litterbox or catbox. Litterbox +links may die before a slow review; the UI warns about it. + ## 2026-10-14 — Temporary image hosting: Cloudflare R2 with a 30-day expiry **Status:** accepted (supersedes "Rehosting on catbox.moe" below) **Context:** A static page can't upload to catbox.moe (no CORS). The user said temporary hosting is diff --git a/docs/TODO.md b/docs/TODO.md index ff79173..ff1bdc6 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -21,17 +21,27 @@ Design: [specs/001-webtoons-covers.md](specs/001-webtoons-covers.md). `V1_Series_Cover_Image`). - [ ] `core/`: model, transport (rate limit, retries, timeouts, token scoping), links, sources + registry, webtoons (urls, parse, index), history/wayback, images, covers, mangabaka client, - hosting (worker, direct), seed. -- [ ] `worker/`: static assets, `/api/fetch` allowlist, `/api/upload` → R2, per-IP rate limit. -- [ ] `web/`: two-link form, review cards, rehost, copy, token flow (preview → confirm → save). + hosting (server, litterbox, catbox, direct), seed. +- [ ] `server/`: `app.ts` (assets, `/api/config`, `/api/fetch` allowlist, `/api/upload` store|catbox, + per-IP rate limit), `gate.ts` (login, signed cookie, Bearer), storage (R2 binding, S3), + entries `worker.ts` and `bun.ts`. +- [ ] `web/`: login page, two-link form, review cards, host picker with lifetimes, rehost, copy, + token flow (preview → confirm → save). - [ ] `userscript/`: panel on mangabaka.org series pages, `GM_xmlhttpRequest` transport, token in - `GM_setValue`, build to `mbseed.user.js` with metadata banner. + `GM_setValue`, direct litterbox/catbox uploads, optional server URL + password, + build to `mbseed.user.js` with metadata banner. - [ ] Manual smoke test against Tower of God (9999 / title_no 95). - [ ] Update README, `docs/ARCHITECTURE.md` and `PRODUCT.md`/`DESIGN.md` to match reality. ## Verify with real services (once possible) - [ ] Do Webtoons and Wayback serve requests from Cloudflare Worker IPs? -- [ ] Does MangaBaka accept our R2 URLs? Does it accept `swebtoon-phinf.pstatic.net` (`direct`)? +- [ ] Does catbox accept uploads from Cloudflare Worker IPs? +- [ ] Workers static assets: confirm the config key that runs the Worker before assets + (`run_worker_first`) so the gate covers them. +- [ ] Choose the deployment: Cloudflare, our own server, or both; and the gate (password or + reverse-proxy auth). +- [ ] Does MangaBaka accept our storage URLs, litterbox/catbox URLs, and + `swebtoon-phinf.pstatic.net` (`direct`)? - [ ] Does MangaBaka copy images on submission or on approval? - [ ] Is "main cover" really `is_default: true`? Ask on the MangaBaka Discord if unclear. - [ ] Where are personal access tokens created in the MangaBaka UI? Add it to the README. diff --git a/docs/specs/001-webtoons-covers.md b/docs/specs/001-webtoons-covers.md index 759c9ac..356029a 100644 --- a/docs/specs/001-webtoons-covers.md +++ b/docs/specs/001-webtoons-covers.md @@ -20,6 +20,9 @@ This spec records the facts we verified about the external services, the module boundaries and the extension points. Revised 2026-10-14: the user gives both links directly, the product is a static web page first plus a userscript, the runtime is Bun, and images go to a temporary host with a 30-day expiry. +Revised again the same day: catbox.moe and litterbox are also supported as +image hosts, the server can be password-gated, and the same server code runs +on Cloudflare Workers or under Bun on our own machine. --- @@ -31,11 +34,18 @@ the runtime is Bun, and images go to a temporary host with a 30-day expiry. MangaBaka series link and a Webtoons series link. There is no automatic series matching. - **Two front ends on one core:** - 1. a **static web page** (the primary product), served by a Cloudflare Worker - that also proxies the sites browsers cannot read; + 1. a **static web page** (the primary product), served by our small server, + which also proxies the sites browsers cannot read; 2. a **userscript** (Tampermonkey/Violentmonkey) on mangabaka.org, which needs no proxy because `GM_xmlhttpRequest` is not bound by CORS. A Bun CLI on the same core comes later. +- **One server, two deployments.** The server is a single `fetch(Request) → + Response` handler that runs on Cloudflare Workers or under `Bun.serve` on + our own machine (e.g. behind our own reverse proxy) (§4.10). +- **Lockable.** An optional password gates the whole site, the page and the API + alike (§4.11). +- **Choice of image host:** our own storage (30-day expiry), litterbox (up to + 72h), catbox.moe (permanent), or no rehosting (§4.6). - **Find historical covers**, not just the current one. - **Work for everyone.** With a MangaBaka personal access token, submit through the API. Without one, help the user enter the covers by hand. @@ -53,8 +63,10 @@ the runtime is Bun, and images go to a temporary host with a 30-day expiry. submission layer is keyed by MangaBaka "resource" so this can come later. - Following a Webtoons series across language editions. Each `title_no` belongs to one language edition; run the tool once per edition link. -- Permanent image hosting. Rehosted images only need to outlive the MangaBaka - review; they expire after 30 days (§4.6). +- Running our own permanent image host. Our storage only needs to outlive the + MangaBaka review, so it expires after 30 days; catbox.moe is the option for + anyone who wants permanent links (§4.6). +- User accounts. The gate is one shared password, not per-user logins. --- @@ -122,7 +134,7 @@ Verified on 2026-10-14. Treat them as assumptions that the fixtures in `access-control-allow-origin: *` and allows any header (including `x-api-key`) and `POST`, on both public reads and `/v0/my/submissions/…`. So the page and the userscript talk to MangaBaka - **directly**, never through the Worker. + **directly**, never through our server. - **Public reads.** - `GET /v1/series/{id}` and `GET /v1/series/{id}/links` return the series and its links. Links are canonicalised, e.g. @@ -164,10 +176,18 @@ Verified on 2026-10-14. Treat them as assumptions that the fixtures in | Host | Browser CORS | Expiry | Verdict | |---|---|---|---| -| catbox.moe | no | permanent | Not usable from a page, and permanent is more than we need. | -| litterbox.catbox.moe | yes | 1h / 12h / 24h / 72h only | 72h is likely shorter than a review. | +| **Our storage** (R2 or any S3-compatible bucket) | n/a (behind our server) | 30-day lifecycle rule | **Default** when configured. | +| **litterbox.catbox.moe** | **yes** (`access-control-allow-origin: *`) | `time` = `1h` / `12h` / `24h` / `72h` only | Supported; no account; 72h may be shorter than a review. | +| **catbox.moe** | no | permanent | Supported via our server (page) or `GM_xmlhttpRequest` (userscript). | | 0x0.st | unverified | configurable | Not used. | -| **Cloudflare R2** (our bucket) | n/a (behind our Worker) | 30-day lifecycle rule | **Chosen.** | + +- **litterbox API:** `POST https://litterbox.catbox.moe/resources/internals/api.php`, + multipart `reqtype=fileupload`, `time=<1h|12h|24h|72h>`, `fileToUpload`; + the response body is the file URL. +- **catbox API:** `POST https://catbox.moe/user/api.php`, multipart + `reqtype=fileupload`, `fileToUpload`, optional `userhash` (attaches the + upload to a catbox account so it can be deleted later); the response body is + the file URL. Its `OPTIONS` preflight returns 412 with no CORS headers. --- @@ -193,7 +213,7 @@ Verified on 2026-10-14. Treat them as assumptions that the fixtures in ``` Every network call goes through an injected `Transport` (§4.2), so every stage -runs unchanged in the Worker-backed page, the userscript, the CLI and tests. +runs unchanged in the server-backed page, the userscript, the CLI and tests. --- @@ -220,12 +240,19 @@ src/ client.ts typed API client (public reads + submissions) types.ts API types (subset of the OpenAPI spec we use) hosting/ - host.ts ImageHost interface - worker.ts upload via our Worker → R2 (default) + host.ts ImageHost interface + registry + server.ts upload via our server → our storage (30 days) + litterbox.ts litterbox.catbox.moe, 1h–72h (direct; CORS open) + catbox.ts catbox.moe, permanent (via server, or GM in the userscript) direct.ts no rehosting: submit source URLs as-is seed/covers.ts CoverSeed builder: existing + new images → submission body - web/ the static page (HTML + TS bundled by `bun build`) - worker/ Cloudflare Worker: serves web/ as static assets, /api/fetch, /api/upload + web/ the static page (HTML + TS bundled by `bun build`) + login page + server/ + app.ts the portable fetch(Request) handler: gate, assets, /api/* + gate.ts password check, signed session cookie, login rate limit + storage.ts Storage interface: R2 binding or S3-compatible bucket + worker.ts Cloudflare Workers entry (R2 binding, static assets) + bun.ts Bun.serve entry for self-hosting (S3 storage, assets from dist/) userscript/ entry + metadata banner, bundled by `bun build` into mbseed.user.js cli/ later: Bun CLI on the same core test/ @@ -233,9 +260,11 @@ test/ *.test.ts bun test ``` -Dependency direction: `web/`, `userscript/`, `worker/` and `cli/` may import -`core/`; `core/` imports none of them. `worker/` imports only the small part of -`core/` it needs (host allowlist, image sniffing). +Dependency direction: `web/`, `userscript/`, `server/` and `cli/` may import +`core/`; `core/` imports none of them. `server/` imports only the small part of +`core/` it needs (host allowlist, image sniffing). `server/app.ts` and +`server/gate.ts` use only web-standard APIs, so the same code runs in both +entries; `worker.ts` and `bun.ts` are thin adapters. ### 4.1 The `Source` interface (extension point) @@ -270,7 +299,7 @@ export interface Source { /** True if a MangaBaka series link points at this source series. */ isSameSeries(linkUrl: string, series: SourceSeries): boolean; - /** Hosts this source fetches from; the Worker proxy allowlist is built from these. */ + /** Hosts this source fetches from; the server proxy allowlist is built from these. */ readonly hosts: readonly string[]; } @@ -297,7 +326,7 @@ export interface CoverCandidate { **Adding a source** (e.g. Tapas): create `src/core/sources/tapas/` implementing `Source` (HTML extraction in a pure `parse.ts` with fixture tests), add it to `registry.ts`, and map its language codes to MangaBaka codes. Its `hosts` flow -into the Worker allowlist automatically. +into the server allowlist automatically. ### 4.2 Transport (how each front end reaches the network) @@ -311,7 +340,7 @@ export interface Transport { | Front end | MangaBaka API | Webtoons / Wayback / pstatic | |---|---|---| -| Static page | `fetch` directly (CORS open) | `fetch("/api/fetch?url=…")` on our Worker (same origin) | +| Static page | `fetch` directly (CORS open) | `fetch("/api/fetch?url=…")` on our server (same origin; the session cookie rides along) | | Userscript | `fetch` directly | `GM_xmlhttpRequest`, wrapped into a `Response` | | CLI (later) | Bun `fetch` | Bun `fetch` | | Tests | fake transport serving fixtures | fake transport serving fixtures | @@ -323,7 +352,7 @@ injected: - times out after 20 s; - retries twice with backoff on 429/5xx; - never sends the MangaBaka token anywhere except `api.mangabaka.org`. In - particular the token never passes through the Worker. + particular the token never passes through our server. ### 4.3 History @@ -386,30 +415,50 @@ Classification by aspect ratio (h / w): ### 4.6 Image hosting MangaBaka must be able to download each submitted image until a reviewer acts, -which may take weeks. Rehosted images expire after **30 days**; if the review -takes longer, the user reruns the tool. +which may take weeks. The user picks the host per run; the UI shows each +host's expiry next to the button. ```ts export interface ImageHost { - readonly id: string; // "worker" | "direct" | … - /** Rehosts one inspected image; returns a public URL and the sha256 the host stored. */ - upload(image: { sourceUrl: string; sha256: string; contentType: string }, - ctx: HostContext): Promise<{ url: string; sha256: string }>; + readonly id: "server" | "litterbox" | "catbox" | "direct"; + /** Human-readable lifetime, shown in the UI ("30 days", "72 hours", "permanent"). */ + lifetime(opts: HostOptions): string; + /** Rehosts one inspected image; returns a public URL. */ + upload(image: { sourceUrl: string; bytes: Uint8Array; sha256: string; contentType: string }, + ctx: HostContext): Promise<{ url: string }>; } ``` -| Host | Notes | -|---|---| -| `worker` (default) | `POST /api/upload { url }` on our Worker. The Worker fetches the image itself from an allowlisted host, checks it is an image ≤ 20 MB, stores it in R2 at `covers/.`, and returns its public URL and sha256. | -| `direct` | No upload: submits the normalised source URL. Use it if MangaBaka turns out to fetch `swebtoon-phinf.pstatic.net` fine. | +| Host | Page | Userscript | Lifetime | +|---|---|---|---| +| `server` (default when the server has storage) | `POST /api/upload { url, host: "store" }` | same, with the gate password (§4.11) | 30 days (bucket lifecycle rule) | +| `litterbox` (default otherwise) | multipart upload of the inspected bytes **directly from the browser** (CORS open), `time=72h` unless the user picks less | same, via `GM_xmlhttpRequest` | 1h / 12h / 24h / 72h | +| `catbox` | `POST /api/upload { url, host: "catbox", userhash? }` — the server fetches the image and uploads it, because catbox has no CORS | **directly** via `GM_xmlhttpRequest`, no server needed | permanent | +| `direct` | no upload: the normalised source URL | same | as long as Webtoons keeps it | + +- **Server storage** stores at `covers/.` and returns the public + URL. On Cloudflare it is an R2 binding; self-hosted it is any S3-compatible + bucket (R2, B2, MinIO, AWS) via `aws4fetch`. Either way the bucket has a + 30-day lifecycle rule and is publicly readable (custom domain or `r2.dev`). + If the server has no storage configured, `GET /api/config` says so and the + page hides the `server` host. +- **litterbox 72h warning.** The UI warns that a review may take longer than + 72 hours and the links will then be dead. +- **catbox userhash.** Optional, entered by the user, kept in the browser + (`localStorage` only if "remember" is ticked) or in `GM_setValue`. Through + the page it passes through our server, which forwards it to catbox only and + never logs or stores it. Rules: -- **Upload only what we checked.** The client sends the source URL, not bytes, - so the endpoint can't be used to host arbitrary files. The client compares - the returned `sha256` with the one it inspected and rejects a mismatch. -- **Re-uploads are idempotent.** The key is the hash, so rerunning overwrites - the same object (and restarts its 30 days). +- **Upload only what we checked.** For hosts reached through our server, the + client sends the source URL, not bytes, so `/api/upload` can't be used to + host arbitrary files; the server returns the `sha256` of what it fetched and + the client rejects a mismatch. Hosts reached directly get the inspected bytes. +- **Re-uploads are cheap.** Server storage is keyed by hash, so rerunning + overwrites the same object (and restarts its 30 days). For litterbox and + catbox, the page keeps a per-session `sha256 → url` map and the userscript a + persistent one (`GM_setValue`), so the same image isn't uploaded twice. - **No upload without an explicit user action.** - **Failures are per image.** A failed upload is reported and the image left out; if every upload fails, the action aborts. @@ -445,14 +494,15 @@ language: series.language, type, note, content_rating, is_default }`. ### 4.9 Front ends -**Static page** (`src/web/`, served by the Worker): +**Static page** (`src/web/`, served by our server): 1. Two inputs: MangaBaka series link, Webtoons series link. Optional: "include history" (on), history limit. 2. Runs the pipeline and shows one card per new cover (thumbnail, dimensions, suggested type, language, note, content-rating selector, checkbox), plus "already present" and skipped covers. -3. **Rehost selected** uploads through the Worker and replaces the shown URLs. +3. **Rehost selected** uploads to the chosen host (§4.6) and replaces the + shown URLs. 4. **Without a token:** "Copy all selected" and a link to `https://mangabaka.org/data/submissions/series/{id}/images` for manual entry. @@ -469,39 +519,79 @@ language: series.language, type, note, content_rating, is_default }`. panel; the MangaBaka series id comes from the page URL and the user pastes the Webtoons link. - Fetches Webtoons/Wayback/pstatic with `GM_xmlhttpRequest` - (`@connect webtoons.com`, `pstatic.net`, `web.archive.org`, our Worker's - host). Same review cards as the page. -- Rehosts through the same Worker `/api/upload` (no R2 keys in the script). + (`@connect webtoons.com`, `pstatic.net`, `web.archive.org`, `catbox.moe`, + and our server's host). Same review cards as the page. +- Rehosts to litterbox or catbox directly, or to our server's storage + (`/api/upload`, with the gate password; no storage keys in the script). With + litterbox or catbox it works with **no server at all**. - Submits through the API when a token is set (stored with `GM_setValue`). - **Without a token**, on the cover form page it can fill the form fields for the selected covers. Whether the form accepts programmatically filled fields is unverified (§7). -**Worker** (`src/worker/`): +### 4.10 Server (`src/server/`) | Route | Behaviour | |---|---| +| `GET /login`, `POST /login`, `POST /logout` | The gate (§4.11). Always reachable. | | `GET /*` | Static assets of the page. | +| `GET /api/config` | `{ storage: boolean, gated: boolean }`, so the page knows which hosts to offer. | | `GET /api/fetch?url=` | Fetches `url` only if its host is in the allowlist (union of `Source.hosts` plus `web.archive.org`), with a browser User-Agent; streams the response back. Anything else: 403. | -| `POST /api/upload` | §4.6. | +| `POST /api/upload` | `{ url, host: "store" \| "catbox", userhash? }` (§4.6). `url` must pass the same allowlist. | -Both API routes reject other origins (except the userscript, which is not bound -by CORS), cap response size at 20 MB, and are rate-limited per IP. The R2 -bucket has a lifecycle rule deleting objects after 30 days and is publicly -readable through a custom domain or `r2.dev`. +The API routes reject other origins (except the userscript, which is not bound +by CORS), cap response size at 20 MB, and are rate-limited per IP. + +**Two deployments, one handler.** `app.ts` exports +`handle(req: Request, env: ServerEnv): Promise` using only +web-standard APIs (`fetch`, `Request`, `Response`, `crypto.subtle`): + +| | Cloudflare (`worker.ts`) | Self-hosted (`bun.ts`) | +|---|---|---| +| Run | `wrangler dev` / `wrangler deploy` | `bun run src/server/bun.ts` behind our reverse proxy | +| Static assets | Workers static assets, with `run_worker_first = true` so the gate also covers assets (confirm the key when scaffolding) | read from the built `dist/web/` | +| Storage | R2 binding | S3-compatible bucket via `MBSEED_S3_*` env vars, or none | +| Password | `wrangler secret put MBSEED_PASSWORD` | `MBSEED_PASSWORD` env var | +| Client IP for rate limits | `CF-Connecting-IP` | `server.requestIP(req)`, or `X-Forwarded-For` only when `MBSEED_TRUST_PROXY=1` | + +Self-hosting also removes the Cloudflare-egress risk (§7). + +### 4.11 Access gate (`server/gate.ts`) + +An optional shared password locks the **whole** service: page, assets and API. + +- **Off by default.** If `MBSEED_PASSWORD` is unset, the gate is open and + `/api/config` reports `gated: false`. A self-hosted instance can instead rely + on the reverse proxy's own auth (e.g. basic auth or SSO) and leave it unset. +- **Login.** `GET /login` serves a minimal form. `POST /login` compares the + password in constant time, then sets `mbseed_session=.` + (`HttpOnly; Secure; SameSite=Strict; Path=/`, 30 days). The HMAC key is + derived from `MBSEED_PASSWORD`, so changing the password logs everyone out. + Login attempts are rate-limited per IP (5 per minute). +- **Every other route** requires a valid, unexpired session cookie: browser + page requests without one get a 302 to `/login`, and API requests get 401. +- **Userscript.** It can't rely on the cookie, so API calls may instead send + `Authorization: Bearer ` (constant-time compare). The userscript + stores the password with `GM_setValue`. It needs this only for + `/api/upload` with server storage or catbox-via-server; with litterbox or + direct catbox it never calls our server. +- **No accounts.** One password for the group; rotate it to revoke access. --- ## 5. Cross-cutting concerns - **Secrets.** The MangaBaka token stays in the user's browser (or userscript - storage) and is only sent to `api.mangabaka.org`. The Worker holds no - secrets; R2 is a binding. + storage) and is only sent to `api.mangabaka.org`. The server's only secrets + are `MBSEED_PASSWORD` and, when self-hosted, the `MBSEED_S3_*` credentials + (on Cloudflare, R2 is a binding). The catbox userhash is forwarded, never + stored. - **Errors.** The UI shows a per-stage status; Wayback failure is a warning, not a failure. -- **Cost.** Cloudflare free tiers cover expected use: a run is ~60–100 Worker +- **Cost.** On Cloudflare, free tiers cover expected use: a run is ~60–100 Worker requests and ≤ ~6 MB stored for 30 days. Abuse of the proxy is the main cost - risk, hence the allowlist, the size cap and the per-IP rate limit. + risk, hence the allowlist, the size cap, the per-IP rate limit and the + password gate. ## 6. Testing @@ -510,22 +600,30 @@ readable through a custom domain or `r2.dev`. series/links/images/snapshot/preview JSON. - Pure functions under test: link parsing, Webtoons parsing, URL normalisation, classification, dedupe, default selection, seed building, content-rating - resolution, language mapping, the Worker allowlist. -- I/O (`pipeline`, `mangabaka/client`, `history/wayback`, `hosting/worker`, - the Worker routes) with a fake transport. **No network in tests.** + resolution, language mapping, the server allowlist, session cookie + signing/expiry. +- I/O (`pipeline`, `mangabaka/client`, `history/wayback`, every `hosting/*`, + `server/app.ts` routes incl. the gate: no cookie → 302/401, bad password, + expired cookie, Bearer header) with a fake transport and in-memory storage. + **No network in tests.** - Manual smoke test: the page against Tower of God (MangaBaka `9999`, Webtoons `title_no=95`), with "Rehost" and without submitting. ## 7. Open questions and risks 1. **Cloudflare egress.** Webtoons or Wayback may block or throttle Cloudflare - IPs. The userscript does not have this risk (it fetches from the user's IP). + IPs. The userscript and a self-hosted server don't have this risk. 2. **When MangaBaka copies images** (submission or approval) and whether it - accepts our R2 URLs or `swebtoon-phinf.pstatic.net` (`direct`). + accepts our storage URLs, `litter.catbox.moe` / `files.catbox.moe` URLs, or + `swebtoon-phinf.pstatic.net` (`direct`). If it copies only on approval, + litterbox's 72h is likely too short. 3. **"Main cover" = `is_default: true`** is inferred from the API spec. 4. **Where users create a personal access token** in the MangaBaka UI. 5. **Whether the cover form accepts fields filled by a userscript.** -6. **Hosting details:** Cloudflare account, Worker domain, R2 public domain. +6. **Whether catbox accepts uploads from Cloudflare IPs** (catbox-via-server). +7. **Hosting details:** Cloudflare account and domains, or which of our own + servers and which S3-compatible bucket; whether the gate or the reverse + proxy's auth locks it. ## 8. Test data