docs: approve spec 001; self-hosted Bun server, defer password gate

The user signed off the spec. The server runs self-hosted under Bun;
the password gate and the Cloudflare Workers entry stay designed but
deferred until abuse or traffic calls for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas Winther
2026-10-14 22:10:20 +02:00
co-authored by Claude Opus 5.5
parent 54cd626f21
commit dd2d5a94bc
6 changed files with 80 additions and 57 deletions
+7 -7
View File
@@ -1,8 +1,8 @@
# mangabaka-seeder
Seeds cover edits into MangaBaka from a Webtoons series. TypeScript on **Bun** (not Node):
a pure core, a static page served by a small server (Cloudflare Worker or self-hosted Bun), and a userscript. No code yet — the
approved design is `docs/specs/001-webtoons-covers.md`.
a pure core, a static page served by a small self-hosted Bun server, and a userscript.
No code yet — the approved design is `docs/specs/001-webtoons-covers.md`.
`AGENTS.md` is the single source of truth for agents here. `CLAUDE.md` and `GEMINI.md` only point here.
Deeper context on demand: `docs/ARCHITECTURE.md`, `docs/DECISIONS.md`, `docs/TODO.md`.
@@ -30,10 +30,10 @@ Use impeccable for the static page (`src/web/`) and the userscript panel — `cr
## Rules
- `src/core/` runs in a browser, a Worker, a userscript and Bun: no DOM, no `Bun.*`, no `node:*`,
- `src/core/` runs in a browser, a userscript and Bun: no DOM, no `Bun.*`, no `node:*`,
no `GM_*`. All network goes through the injected `Transport`; hashing uses `crypto.subtle`.
- `core/` never imports from `web/`, `server/`, `userscript/` or `cli/`. `server/app.ts` uses only
web-standard APIs so it runs on Workers and `Bun.serve`; `worker.ts`/`bun.ts` stay thin adapters.
web-standard APIs (a Workers entry may come later); `bun.ts` stays a thin adapter.
- The MangaBaka token goes straight to `api.mangabaka.org` (CORS is open) — never through our server.
- MangaBaka `images` **replaces the whole set**: a save body must include every existing image.
Saves always use `save_mode: "review"`.
@@ -41,7 +41,7 @@ Use impeccable for the static page (`src/web/`) and the userscript panel — `cr
it files a real edit request. Tests use the fake transport; tests never touch the network.
- Fixtures in `test/fixtures/` are captured from real responses, never hand-written.
- `/api/fetch` only proxies allowlisted hosts; `/api/upload` takes a source URL, never raw bytes.
- When `MBSEED_PASSWORD` is set the gate covers **every** route except `/login` — static assets too.
- The password gate (spec §4.11) and the Cloudflare entry are deferred: don't build them unasked.
- Webtoons image URLs: strip the `?type=` query and use the `swebtoon-phinf.pstatic.net` host:
```ts
@@ -53,7 +53,7 @@ Use impeccable for the static page (`src/web/`) and the userscript panel — `cr
## Architecture
src/core/ -> pure pipeline: sources, Wayback history, inspect, dedupe, seed, MangaBaka client
src/web/ -> static page · src/server/ -> gate, assets, `/api/fetch`, `/api/upload` (Workers or Bun)
src/web/ -> static page · src/server/ -> assets, `/api/fetch`, `/api/upload` (`Bun.serve`)
src/core/hosting/ -> server storage (30 days), litterbox (≤72h), catbox (permanent), direct
src/userscript/ -> panel on mangabaka.org, `GM_xmlhttpRequest` · src/cli/ -> later
@@ -64,7 +64,7 @@ Full picture: `docs/ARCHITECTURE.md` and the spec — read them before your firs
Never commit `.env*`, `.dev.vars`, tokens, passwords, S3 keys or Cloudflare credentials.
Never force-push, rewrite pushed history, or commit directly to `main`.
Ask first: new runtime dependencies · `wrangler deploy` or any Cloudflare/R2/S3 configuration ·
Ask first: new runtime dependencies · deploying, or any server/bucket configuration ·
real uploads to litterbox/catbox ·
anything sent to MangaBaka with a real token · anything destructive (`rm -rf`, `git reset --hard`) ·
work the approved plan does not cover.
+2 -2
View File
@@ -16,8 +16,8 @@ poster becomes the main (default) cover for the edition's language.
- **Web page** (primary). Open the page, paste the two links, review the
covers. A small server behind the page reads Webtoons and the Wayback
Machine for you, because browsers can't. It runs on Cloudflare Workers or on
your own machine with Bun, and can be locked with a password.
Machine for you, because browsers can't. It runs on your own machine with
Bun.
- **Userscript.** Install `mbseed.user.js` in Tampermonkey or Violentmonkey.
On a MangaBaka series page it adds a **Seed covers** panel; paste the
Webtoons link. It reads Webtoons directly from your browser, and with
+4 -6
View File
@@ -5,16 +5,15 @@ How `mangabaka-seeder` is built today. Update in the same commit as any structur
## Overview
No code exists yet. The design is in [`specs/001-webtoons-covers.md`](specs/001-webtoons-covers.md):
one pure TypeScript core (`src/core/`) that turns a MangaBaka series link and a Webtoons series
link into a cover submission, used by three front ends — a static page served by a small,
optionally password-gated server that runs on Cloudflare Workers or self-hosted under Bun
(primary), a userscript, and later a Bun CLI. Replace this section with reality as modules land.
link into a cover submission, used by three front ends — a static page served by a small
self-hosted Bun server (primary), a userscript, and later a Bun CLI. Replace this section with reality as modules land.
## Map (planned — spec §4)
| Path | Responsibility |
|---|---|
| `src/core/` | Pure pipeline: sources, Wayback history, image inspection, dedupe, seed, MangaBaka client. No DOM, Bun, Node or `GM_*` APIs. |
| `src/web/` | Static page UI. |
| `src/server/` | Portable `fetch` handler (`app.ts`) + password gate; entries `worker.ts` (Cloudflare, R2) and `bun.ts` (self-hosted, S3). Static assets, `/api/fetch` allowlisted proxy, `/api/upload`. |
| `src/server/` | Web-standard `fetch` handler (`app.ts`) run by `bun.ts` (`Bun.serve`, behind our reverse proxy). Static assets, `/api/fetch` allowlisted proxy, `/api/upload`. Gate and Workers entry deferred. |
| `src/userscript/` | Userscript on mangabaka.org; network via `GM_xmlhttpRequest`. |
| `src/cli/` | Later: Bun CLI. |
| `test/fixtures/` | Saved real responses; tests never hit the network. |
@@ -29,13 +28,12 @@ sha256) → dedupe against MangaBaka → seed → user reviews → rehost (expli
| `api.mangabaka.org` | direct (CORS open) | Nothing works. |
| Webtoons HTML / `swebtoon-phinf.pstatic.net` | server proxy or `GM_xmlhttpRequest` | Nothing works. |
| Wayback (`web.archive.org`) | same | Warning; current covers only. |
| Server storage (R2 binding or S3 bucket) | server | `server` host hidden; litterbox/catbox/direct still work. |
| Server storage (S3-compatible bucket) | server | `server` host hidden; litterbox/catbox/direct still work. |
| litterbox / catbox | browser directly (litterbox), server or `GM_xmlhttpRequest` (catbox) | That host fails; others still work. |
## Invariants
- `src/core/` never imports from `web/`, `server/`, `userscript/` or `cli/`.
- The MangaBaka token is only ever sent to `api.mangabaka.org`, never through our server.
- With `MBSEED_PASSWORD` set, every route except `/login` requires a session (or Bearer password).
- Nothing is uploaded or submitted without an explicit user action; saves use `save_mode: "review"`.
- `/api/fetch` only fetches allowlisted hosts; `/api/upload` only takes a source URL, never bytes.
+13
View File
@@ -16,6 +16,19 @@ the repo every session — measurably slower and more tokens).
**Consequences:** Rule changes happen in one file; pointer files must stay thin. Plans are working
artefacts: kept after merge they go stale and contradict the code, so the spec is the record.
## 2026-10-14 — Spec signed off; self-hosted only; password gate deferred
**Status:** accepted (narrows the entry below)
**Context:** The user signed off spec 001 and said the app will run self-hosted, and that
password-gating should wait: "if it gets abused or we see too big amounts of traffic, we will deal
with it then."
**Decision:** First version runs `server/app.ts` under `Bun.serve` on our own machine behind our
reverse proxy. No gate, no Cloudflare entry; both stay designed in the spec (§4.10, §4.11) and are
listed under Someday in `docs/TODO.md`. The handler stays web-standard so either can be added later.
**Alternatives:** Build the gate now (cost without a demonstrated need).
**Consequences:** The site is open: the host allowlist, the 20 MB cap and the per-IP rate limit are
the only protections. The rate limit needs the real client IP from our reverse proxy
(`MBSEED_TRUST_PROXY`).
## 2026-10-14 — Also support catbox/litterbox; optional password gate; self-hostable server
**Status:** accepted (extends the two entries below)
**Context:** The user asked for catbox/litterbox support as well, and for a way to password-gate the
+10 -12
View File
@@ -5,12 +5,11 @@ Small, specific, independently actionable. Remove only when actually done.
Design: [specs/001-webtoons-covers.md](specs/001-webtoons-covers.md).
## Now
- [ ] User signs off the revised spec `docs/specs/001-webtoons-covers.md` (status: draft).
- [ ] Write the implementation plan with `superpowers:writing-plans` into `docs/plans/001-webtoons-covers.md`,
and **show it to the user before implementing** (the user asked for this in the design session).
## Next (rough order; the plan will refine it)
- [ ] **Scaffold with Bun.** `package.json`, `bun.lock`, TypeScript, `wrangler`, `.gitignore`.
- [ ] **Scaffold with Bun.** `package.json`, `bun.lock`, TypeScript, `.gitignore` (already exists).
Scripts for test, typecheck, build (page, Worker, userscript). Then fill the Commands in
`AGENTS.md` and the Verified commands in `docs/PERMISSIONS.md` from runs that worked.
- [ ] **Capture fixtures from real responses:** Webtoons pages (Tower of God, crow-time canvas,
@@ -23,23 +22,19 @@ Design: [specs/001-webtoons-covers.md](specs/001-webtoons-covers.md).
registry, webtoons (urls, parse, index), history/wayback, images, covers, mangabaka client,
hosting (server, litterbox, catbox, direct), seed.
- [ ] `server/`: `app.ts` (assets, `/api/config`, `/api/fetch` allowlist, `/api/upload` store|catbox,
per-IP rate limit), `gate.ts` (login, signed cookie, Bearer), storage (R2 binding, S3),
entries `worker.ts` and `bun.ts`.
- [ ] `web/`: login page, two-link form, review cards, host picker with lifetimes, rehost, copy,
per-IP rate limit), storage, entry `bun.ts`. Decide in the plan: S3-compatible bucket or
local directory with a 30-day sweep (spec §7.7).
- [ ] `web/`: two-link form, review cards, host picker with lifetimes, rehost, copy,
token flow (preview → confirm → save).
- [ ] `userscript/`: panel on mangabaka.org series pages, `GM_xmlhttpRequest` transport, token in
`GM_setValue`, direct litterbox/catbox uploads, optional server URL + password,
`GM_setValue`, direct litterbox/catbox uploads, optional server URL,
build to `mbseed.user.js` with metadata banner.
- [ ] Manual smoke test against Tower of God (9999 / title_no 95).
- [ ] Update README, `docs/ARCHITECTURE.md` and `PRODUCT.md`/`DESIGN.md` to match reality.
## Verify with real services (once possible)
- [ ] Do Webtoons and Wayback serve requests from Cloudflare Worker IPs?
- [ ] Does catbox accept uploads from Cloudflare Worker IPs?
- [ ] Workers static assets: confirm the config key that runs the Worker before assets
(`run_worker_first`) so the gate covers them.
- [ ] Choose the deployment: Cloudflare, our own server, or both; and the gate (password or
reverse-proxy auth).
- [ ] Which of our servers and domain hosts it; which bucket (or local dir) backs `server` storage.
- [ ] Does catbox accept uploads from our server?
- [ ] Does MangaBaka accept our storage URLs, litterbox/catbox URLs, and
`swebtoon-phinf.pstatic.net` (`direct`)?
- [ ] Does MangaBaka copy images on submission or on approval?
@@ -48,6 +43,9 @@ Design: [specs/001-webtoons-covers.md](specs/001-webtoons-covers.md).
- [ ] Does the MangaBaka cover form accept fields filled by a userscript?
## Someday
- [ ] Password gate (spec §4.11) — only if the open site gets abused or traffic grows too large.
- [ ] Cloudflare Workers entry (`server/worker.ts`, R2 binding). If added: Workers serve static
assets before the Worker unless `run_worker_first` is set — needed for the gate to cover them.
- [ ] Bun CLI on the same core.
- [ ] More sources: Tapas, Tappytoon, Lezhin, Manta.
- [ ] Following Webtoons language editions of the same series.
+44 -30
View File
@@ -1,5 +1,5 @@
---
status: draft
status: approved
---
# 001 — Webtoons covers
@@ -23,6 +23,10 @@ the runtime is Bun, and images go to a temporary host with a 30-day expiry.
Revised again the same day: catbox.moe and litterbox are also supported as
image hosts, the server can be password-gated, and the same server code runs
on Cloudflare Workers or under Bun on our own machine.
Signed off by the user 2026-10-14, with two changes applied here: the server
is **self-hosted** (Bun), and the **password gate is deferred** until abuse or
traffic calls for it (§4.11 keeps the design). The Cloudflare entry is
deferred too; the handler stays web-standard so it can be added later.
---
@@ -39,11 +43,10 @@ on Cloudflare Workers or under Bun on our own machine.
2. a **userscript** (Tampermonkey/Violentmonkey) on mangabaka.org, which needs
no proxy because `GM_xmlhttpRequest` is not bound by CORS.
A Bun CLI on the same core comes later.
- **One server, two deployments.** The server is a single `fetch(Request) →
Response` handler that runs on Cloudflare Workers or under `Bun.serve` on
our own machine (e.g. behind our own reverse proxy) (§4.10).
- **Lockable.** An optional password gates the whole site, the page and the API
alike (§4.11).
- **Self-hosted server.** The server is a single web-standard `fetch(Request) →
Response` handler run by `Bun.serve` on our own machine, behind our reverse
proxy (§4.10). Because it is web-standard, a Cloudflare Workers entry can be
added later without touching it.
- **Choice of image host:** our own storage (30-day expiry), litterbox (up to
72h), catbox.moe (permanent), or no rehosting (§4.6).
- **Find historical covers**, not just the current one.
@@ -66,7 +69,10 @@ on Cloudflare Workers or under Bun on our own machine.
- Running our own permanent image host. Our storage only needs to outlive the
MangaBaka review, so it expires after 30 days; catbox.moe is the option for
anyone who wants permanent links (§4.6).
- User accounts. The gate is one shared password, not per-user logins.
- Access control, for now. The site is open; the password gate (§4.11) is
designed but deferred until abuse or traffic calls for it.
- A Cloudflare Workers deployment, for now (§4.10).
- User accounts, ever: if a gate is added, it is one shared password.
---
@@ -249,9 +255,9 @@ src/
web/ the static page (HTML + TS bundled by `bun build`) + login page
server/
app.ts the portable fetch(Request) handler: gate, assets, /api/*
gate.ts password check, signed session cookie, login rate limit
storage.ts Storage interface: R2 binding or S3-compatible bucket
worker.ts Cloudflare Workers entry (R2 binding, static assets)
gate.ts (deferred) password check, signed session cookie, login rate limit
storage.ts Storage interface: S3-compatible bucket (R2 binding later)
worker.ts (deferred) Cloudflare Workers entry (R2 binding, static assets)
bun.ts Bun.serve entry for self-hosting (S3 storage, assets from dist/)
userscript/ entry + metadata banner, bundled by `bun build` into mbseed.user.js
cli/ later: Bun CLI on the same core
@@ -431,7 +437,7 @@ export interface ImageHost {
| Host | Page | Userscript | Lifetime |
|---|---|---|---|
| `server` (default when the server has storage) | `POST /api/upload { url, host: "store" }` | same, with the gate password (§4.11) | 30 days (bucket lifecycle rule) |
| `server` (default when the server has storage) | `POST /api/upload { url, host: "store" }` | same | 30 days (bucket lifecycle rule) |
| `litterbox` (default otherwise) | multipart upload of the inspected bytes **directly from the browser** (CORS open), `time=72h` unless the user picks less | same, via `GM_xmlhttpRequest` | 1h / 12h / 24h / 72h |
| `catbox` | `POST /api/upload { url, host: "catbox", userhash? }` — the server fetches the image and uploads it, because catbox has no CORS | **directly** via `GM_xmlhttpRequest`, no server needed | permanent |
| `direct` | no upload: the normalised source URL | same | as long as Webtoons keeps it |
@@ -522,7 +528,7 @@ language: series.language, type, note, content_rating, is_default }`.
(`@connect webtoons.com`, `pstatic.net`, `web.archive.org`, `catbox.moe`,
and our server's host). Same review cards as the page.
- Rehosts to litterbox or catbox directly, or to our server's storage
(`/api/upload`, with the gate password; no storage keys in the script). With
(`/api/upload`; no storage keys in the script). With
litterbox or catbox it works with **no server at all**.
- Submits through the API when a token is set (stored with `GM_setValue`).
- **Without a token**, on the cover form page it can fill the form fields
@@ -533,20 +539,19 @@ language: series.language, type, note, content_rating, is_default }`.
| Route | Behaviour |
|---|---|
| `GET /login`, `POST /login`, `POST /logout` | The gate (§4.11). Always reachable. |
| `GET /*` | Static assets of the page. |
| `GET /api/config` | `{ storage: boolean, gated: boolean }`, so the page knows which hosts to offer. |
| `GET /api/config` | `{ storage: boolean }`, so the page knows which hosts to offer. |
| `GET /api/fetch?url=` | Fetches `url` only if its host is in the allowlist (union of `Source.hosts` plus `web.archive.org`), with a browser User-Agent; streams the response back. Anything else: 403. |
| `POST /api/upload` | `{ url, host: "store" \| "catbox", userhash? }` (§4.6). `url` must pass the same allowlist. |
The API routes reject other origins (except the userscript, which is not bound
by CORS), cap response size at 20 MB, and are rate-limited per IP.
**Two deployments, one handler.** `app.ts` exports
**One handler; Bun now, Cloudflare later.** `app.ts` exports
`handle(req: Request, env: ServerEnv): Promise<Response>` using only
web-standard APIs (`fetch`, `Request`, `Response`, `crypto.subtle`):
| | Cloudflare (`worker.ts`) | Self-hosted (`bun.ts`) |
| | Cloudflare (`worker.ts`, deferred) | Self-hosted (`bun.ts`, **now**) |
|---|---|---|
| Run | `wrangler dev` / `wrangler deploy` | `bun run src/server/bun.ts` behind our reverse proxy |
| Static assets | Workers static assets, with `run_worker_first = true` so the gate also covers assets (confirm the key when scaffolding) | read from the built `dist/web/` |
@@ -554,9 +559,16 @@ web-standard APIs (`fetch`, `Request`, `Response`, `crypto.subtle`):
| Password | `wrangler secret put MBSEED_PASSWORD` | `MBSEED_PASSWORD` env var |
| Client IP for rate limits | `CF-Connecting-IP` | `server.requestIP(req)`, or `X-Forwarded-For` only when `MBSEED_TRUST_PROXY=1` |
Self-hosting also removes the Cloudflare-egress risk (§7).
Self-hosting also avoids the Cloudflare-egress risk. Our reverse proxy
terminates TLS; how `X-Forwarded-For` is trusted is set by `MBSEED_TRUST_PROXY`.
### 4.11 Access gate (`server/gate.ts`)
### 4.11 Access gate (`server/gate.ts`) — deferred
**Not built in the first version.** Decided 2026-10-14: the self-hosted site
starts open, and we add this if it gets abused or traffic grows too large.
Until then the server's protections are the host allowlist, the 20 MB cap and
the per-IP rate limit. The design below is kept for when it's needed; until
then there is no `/login`, and `/api/config` has no `gated` field.
An optional shared password locks the **whole** service: page, assets and API.
@@ -583,15 +595,16 @@ An optional shared password locks the **whole** service: page, assets and API.
- **Secrets.** The MangaBaka token stays in the user's browser (or userscript
storage) and is only sent to `api.mangabaka.org`. The server's only secrets
are `MBSEED_PASSWORD` and, when self-hosted, the `MBSEED_S3_*` credentials
(on Cloudflare, R2 is a binding). The catbox userhash is forwarded, never
are the `MBSEED_S3_*` credentials (and `MBSEED_PASSWORD` once the gate
exists). The catbox userhash is forwarded, never
stored.
- **Errors.** The UI shows a per-stage status; Wayback failure is a warning,
not a failure.
- **Cost.** On Cloudflare, free tiers cover expected use: a run is ~60–100 Worker
requests and ≤ ~6 MB stored for 30 days. Abuse of the proxy is the main cost
risk, hence the allowlist, the size cap, the per-IP rate limit and the
password gate.
- **Load.** A run is ~60–100 proxied requests and ≤ ~6 MB stored for 30 days.
Abuse of the open proxy is the main risk, hence the allowlist, the size cap
and the per-IP rate limit; the gate (§4.11) is the next step if needed.
(Earlier estimate for Cloudflare: $0 on free tiers; ~$5/month for a busy
public instance.)
## 6. Testing
@@ -611,8 +624,8 @@ An optional shared password locks the **whole** service: page, assets and API.
## 7. Open questions and risks
1. **Cloudflare egress.** Webtoons or Wayback may block or throttle Cloudflare
IPs. The userscript and a self-hosted server don't have this risk.
1. **Server egress.** Webtoons or Wayback may throttle our server's IP under
load. (Cloudflare IPs would be riskier; the userscript uses the user's IP.)
2. **When MangaBaka copies images** (submission or approval) and whether it
accepts our storage URLs, `litter.catbox.moe` / `files.catbox.moe` URLs, or
`swebtoon-phinf.pstatic.net` (`direct`). If it copies only on approval,
@@ -620,10 +633,11 @@ An optional shared password locks the **whole** service: page, assets and API.
3. **"Main cover" = `is_default: true`** is inferred from the API spec.
4. **Where users create a personal access token** in the MangaBaka UI.
5. **Whether the cover form accepts fields filled by a userscript.**
6. **Whether catbox accepts uploads from Cloudflare IPs** (catbox-via-server).
7. **Hosting details:** Cloudflare account and domains, or which of our own
servers and which S3-compatible bucket; whether the gate or the reverse
proxy's auth locks it.
6. **Whether catbox accepts uploads from our server** (catbox-via-server).
7. **Hosting details:** which of our servers and domain, and the storage
behind `server` — an S3-compatible bucket (MinIO, R2, B2…) as specified, or,
if simpler on our box, a local directory with a daily 30-day sweep (decide
in the plan; it would be a `Storage` implementation, nothing else changes).
## 8. Test data