Files

T

Lucas WintherandClaude Opus 5.5 d1b8350263 feat(web): old covers from the Wayback Machine start off

Co-Authored-By: Claude Opus 5.5 <[email protected]>

2026-10-15 11:28:27 +02:00

40 KiB

Raw Permalink Blame History

status

status
approved

001 — Webtoons covers

mangabaka-seeder gathers data about a series from an external site (a "source") and turns it into an edit request ("submission") for MangaBaka. It is modelled on yambs, the MusicBrainz seeder: sources turn into seeds, and an action decides what happens to them. The user always stays in control of what is submitted.

The first feature is covers: given a MangaBaka series link and a Webtoons series link, collect every cover the Webtoons series has used (current and historical) and request them added to the MangaBaka series, with the current poster as the default (main) cover for the edition's language.

This spec records the facts we verified about the external services, the module boundaries and the extension points. Revised 2026-10-14: the user gives both links directly, the product is a static web page first plus a userscript, the runtime is Bun, and images go to a temporary host with a 30-day expiry. Revised again the same day: catbox.moe and litterbox are also supported as image hosts, the server can be password-gated, and the same server code runs on Cloudflare Workers or under Bun on our own machine. Signed off by the user 2026-10-14, with two changes applied here: the server is self-hosted (Bun), and the password gate is deferred until abuse or traffic calls for it (§4.11 keeps the design). The Cloudflare entry is deferred too; the handler stays web-standard so it can be added later. Amended 2026-10-14 after testing against the real services (approved by the user): history granularity instead of a snapshot limit, a "tall" landing-art class, an honest User-Agent and 10 s pacing for Wayback, and one default cover per series (§2.2, §2.3, §4.2–§4.5).

1. Goals and non-goals

Goals

Two links in, a reviewable cover submission out. The user pastes a MangaBaka series link and a Webtoons series link. There is no automatic series matching.
Two front ends on one core:
1. a static web page (the primary product), served by our small server, which also proxies the sites browsers cannot read;
2. a userscript (Tampermonkey/Violentmonkey) on mangabaka.org, which needs no proxy because GM_xmlhttpRequest is not bound by CORS. A Bun CLI on the same core comes later.
Self-hosted server. The server is a single web-standard fetch(Request) → Response handler run by Bun.serve on our own machine, behind our reverse proxy (§4.10). Because it is web-standard, a Cloudflare Workers entry can be added later without touching it.
Choice of image host: our own storage (30-day expiry), litterbox (up to 72h), catbox.moe (permanent), or no rehosting (§4.6).
Find historical covers, not just the current one.
Work for everyone. With a MangaBaka personal access token, submit through the API. Without one, help the user enter the covers by hand.
New sources (Tapas, Tappytoon, Lezhin, Manta, …) are one new module under src/core/sources/, with no changes to the pipeline, the MangaBaka client or the front ends.
Never change MangaBaka data silently. API submissions always go through preview, explicit confirmation and save_mode: "review".
Never upload anything without explicit user action. The user chooses the content rating (default safe).

Non-goals (for now)

Editing series fields other than images, such as titles, links or tags. The submission layer is keyed by MangaBaka "resource" so this can come later.
Following a Webtoons series across language editions. Each title_no belongs to one language edition; run the tool once per edition link.
Running our own permanent image host. Our storage only needs to outlive the MangaBaka review, so it expires after 30 days; catbox.moe is the option for anyone who wants permanent links (§4.6).
Access control, for now. The site is open; the password gate (§4.11) is designed but deferred until abuse or traffic calls for it.
A Cloudflare Workers deployment, for now (§4.10).
User accounts, ever: if a gate is added, it is one shared password.

2. Verified external facts

Verified on 2026-10-14. Treat them as assumptions that the fixtures in test/fixtures/ pin down.

2.1 Webtoons

What	Where
Portrait poster (official series; 480×623 original)	`meta[property="og:image"]`
Square thumbnail	`meta[name="twitter:image"]`
Hero background (1920×321, official only)	`div.detail_bg` inline `style` → `background:url('…')`
Hero foreground (1200×240 PNG; on canvas series, the 1080×1080 square)	`.detail_header span.thmb img[src]`
Title	`meta[property="og:title"]` (fallback `h1.subj` / `h3.subj`)
Canonical URL	`link[rel="canonical"]`
`title_no`	`title_no` query parameter
Language	first path segment (`/en/`, `/fr/`, `/zh-hant/`, …)
Canvas vs official	`/canvas/` (or `/challenge/`) in the path; inline JS `isChallenge : true`
Mature	inline JS `ageGradeNotice : true`

Canvas series have no portrait poster. Their og:image is a 1080×1080 square.
Image hosts. webtoon-phinf.pstatic.net returns 403 without a Webtoons Referer; swebtoon-phinf.pstatic.net serves the same path with no Referer.
Original size. The ?type=… query selects a resized variant. Removing it returns the original. crop540_540 crops posters, so always strip the query.
Old hosts. Old snapshots reference swebtoon.phinf.naver.net, which becomes swebtoon-phinf.pstatic.net with the same path.
No limits so far. A browser User-Agent is enough. Age-gated titles still serve full HTML, and no cookies are required. (Tested from a residential IP, not from Cloudflare's; see §7.)
No usable JSON API. The app API needs signed requests. We scrape HTML.
CORS: neither the series HTML nor swebtoon-phinf images send Access-Control-Allow-Origin. A page can show the images in <img> but cannot read their bytes.

2.2 Historical covers (Wayback Machine)

Snapshot index. The CDX API at https://web.archive.org/cdx/search/cdx?url=<page>&matchType=prefix&filter=statuscode:200&collapse=timestamp:6&fl=timestamp,original returns at most one snapshot per month.
Raw pages. https://web.archive.org/web/<ts>id_/<original> returns the page as it was archived, without Wayback's toolbar.
History exists. Tower of God has about 2000 snapshots (2015–2026) that reference five distinct cover images. All the older ones tested are still served live from swebtoon-phinf.pstatic.net.
Fallback. If an image is no longer live, it is often still at https://web.archive.org/web/<ts>id_/<image-url>.
Not every image is a cover. Some historical og:images are landing-page art, so candidates are classified by shape and reviewed by the user.
CORS: no Access-Control-Allow-Origin seen on snapshots. CDX returned 503 during the check, so treat Wayback as unavailable from a plain page and flaky in general.
Bot detection (tested 2026-10-14). Requests sending a fake browser User-Agent from a non-browser client were answered 429 with the body "flagged … as suspected abusive bot traffic", even at one request per 10 s. With an honest User-Agent (mangabaka-seeder/<version> (+<repo URL>)), 2 CDX lookups and 24 snapshots at one request per 10 s ran with no 429 at all. So Wayback always gets the honest User-Agent; only Webtoons gets a browser one.
Mobile pages. m.webtoons.com has its own captures (35 months for Tower of God, against 121 for www.); they fill gaps in the www. history.
What the history holds (Tower of God, every 6 months, 2015–2026): 20 distinct images, all still live on swebtoon-phinf.pstatic.net. The 2015–2020 og:images are 510×510 character art, the 2020-10 – 2026-03 ones are 720×1230 landing-page art with an empty lower half, and only the current one (480×623) is a cover with the logo. Hence the "tall" class in §4.4.

2.3 MangaBaka

Endpoints. The site is https://mangabaka.org and the API is https://api.mangabaka.org (api.mangabaka.dev is deprecated). The OpenAPI spec is at https://mangabaka.org/api.json (~700 KB), with docs at https://mangabaka.org/data/api.
CORS is open. api.mangabaka.org answers with access-control-allow-origin: * and allows any header (including x-api-key) and POST, on both public reads and /v0/my/submissions/…. So the page and the userscript talk to MangaBaka directly, never through our server.
Public reads.
- GET /v1/series/{id} and GET /v1/series/{id}/links return the series and its links. Links are canonicalised, e.g. https://www.webtoons.com/-/-/-/list?title_no=95.
- GET /v1/series/{id}/images returns covers with id, type, language, index, note, content_rating and hashes.sha256, but not is_default. It is paginated (limit ≤ 50; Tower of God has 31 images, pagination.next links the next page), so the client follows next until it is null.
- Rate limits: about 180 requests per minute (search, which we no longer use, is about 30).
Image type values: volume (default), volume_back, chapter, season, banner, audiobook, other. There is no "main" type; the main cover is the one with is_default: true.
Submissions API. Auth is x-api-key: mb-… (a personal access token). All endpoints are v0 and marked internal.
- GET /v0/my/submissions/series-images/{id} returns the current image set and a version token.
- POST /v0/my/submissions/series-images/{id}/preview is a dry run that returns a diff.
- POST /v0/my/submissions/series-images/{id} saves, with body { version, user_note, save_mode, data: { images: [...] } }.
- images replaces the whole set, so it must contain the existing images.
- Exactly one image in the set must have is_default: true when the set is non-empty (OpenAPI series-images schema). There is one main cover per series, not per language.
- Snapshot: { data: { data: { images: [...] }, version } }. Each image has id, series_id, work_id, url, index, index_numeric, language, type, note, content_rating, is_default; the keys work_id, note and index_numeric are required even when null.
- Preview answers { data: { has_changes, changes: SubmissionDiff[] } }.
- A stale version gives 409.
- Users without the contributor role and with fewer than 25 approved submissions get 403.
Web form. Without API access, users use https://mangabaka.org/data/submissions/series/{id}/images. It cannot be pre-filled with query parameters, and its SvelteKit CSRF check blocks cross-site POSTs. (A userscript running on that page can still fill its fields; see §4.9.)
Image URLs. The spec describes the cover url as "Cover image URL. imgur is rejected (aggressive rate limiting). Acts as the cover identity on save." We don't know whether MangaBaka copies the image on submission or on approval, so the hosted URL must survive the review (§4.6).
content_rating: one of safe, suggestive, erotica, pornographic. The user chooses it (§4.7).
Test series: Tower of God is series 9999 (https://mangabaka.org/manhwa/9999/Tower-of-God; Webtoons title_no=95).

2.4 Image hosts

Host	Browser CORS	Expiry	Verdict
Our storage (a folder on our server)	n/a (served by our server)	30 days (daily sweep)	Default when configured.
litterbox.catbox.moe	yes (`access-control-allow-origin: *`)	`time` = `1h` / `12h` / `24h` / `72h` only	Supported; no account; 72h may be shorter than a review.
catbox.moe	no	permanent	Supported via our server (page) or `GM_xmlhttpRequest` (userscript).
0x0.st	unverified	configurable	Not used.

litterbox API: POST https://litterbox.catbox.moe/resources/internals/api.php, multipart reqtype=fileupload, time=<1h|12h|24h|72h>, fileToUpload; the response body is the file URL.
catbox API: POST https://catbox.moe/user/api.php, multipart reqtype=fileupload, fileToUpload, optional userhash (attaches the upload to a catbox account so it can be deleted later); the response body is the file URL. Its OPTIONS preflight returns 412 with no CORS headers.

3. Pipeline

 MangaBaka link ──▶ series id ──▶ GET /v1/series/{id}, /links, /images   (direct; CORS open)
                                         │  sanity check: a link has the same title_no
 Webtoons link ──▶ source.resolve ───────┤
                        │                ▼
          source.currentCovers + history (Wayback, via source.extractCovers)
                        │
                        ▼  CoverCandidate[]
          images.inspect  (bytes → dimensions, sha256, liveness)
                        │
                        ▼  InspectedCover[]
          covers.dedupe   (within candidates + against existing MangaBaka covers)
                        │
                        ▼
          seed.buildCoverSeed ──▶ CoverSeed (existing + new images, user's content rating)
                        │
                        ▼  front end: review UI → (explicit action) → rehost → submit / fill form

Every network call goes through an injected Transport (§4.2), so every stage runs unchanged in the server-backed page, the userscript, the CLI and tests.

4. Modules

src/
  core/                     pure TypeScript; no DOM, no Bun/Node APIs, no GM_*
    model.ts                shared types (SourceSeries, CoverCandidate, InspectedCover, CoverSeed)
    transport.ts            Transport interface; per-host rate limit, retries, timeouts
    pipeline.ts             runs the stages above; the only place they are composed
    links.ts                parse the two input links (MangaBaka series id, source URL)
    sources/
      source.ts             Source interface + helpers
      registry.ts           list of sources; findSource(url)
      webtoons/
        index.ts            Webtoons Source implementation
        parse.ts            pure HTML → { series, covers } extraction
        urls.ts             URL parsing, canonicalisation, image URL normalisation
    history/wayback.ts      generic Wayback history provider (CDX + archived pages)
    images.ts               probe dimensions from header bytes, sha256 (WebCrypto), classify
    covers.ts               dedupe, merge observations, pick default
    mangabaka/
      client.ts             typed API client (public reads + submissions)
      types.ts              API types (subset of the OpenAPI spec we use)
    hosting/
      host.ts               ImageHost interface + registry
      server.ts             upload via our server → its covers folder (30 days)
      litterbox.ts          litterbox.catbox.moe, 1h–72h (direct; CORS open)
      catbox.ts             catbox.moe, permanent (via server, or GM in the userscript)
      direct.ts             no rehosting: submit source URLs as-is
    seed/covers.ts          CoverSeed builder: existing + new images → submission body
  web/                      the static page (HTML + TS bundled by `bun build`) + login page
  server/
    app.ts                  the portable fetch(Request) handler: gate, assets, /api/*
    gate.ts                 (deferred) password check, signed session cookie, login rate limit
    storage.ts              Storage interface (put/get/sweep) + in-memory impl for tests
    storage-dir.ts          folder on disk (Bun file APIs; used only by bun.ts)
    worker.ts               (deferred) Cloudflare Workers entry (R2 binding, static assets)
    bun.ts                  Bun.serve entry for self-hosting (folder storage + daily sweep, assets from dist/)
  userscript/               entry + metadata banner, bundled by `bun build` into mbseed.user.js
  cli/                      later: Bun CLI on the same core
test/
  fixtures/                 saved HTML / CDX / JSON responses
  *.test.ts                 bun test

Dependency direction: web/, userscript/, server/ and cli/ may import core/; core/ imports none of them. server/ imports only the small part of core/ it needs (host allowlist, image sniffing). server/app.ts and server/gate.ts use only web-standard APIs, so the same code runs in both entries; worker.ts and bun.ts are thin adapters.

4.1 The `Source` interface (extension point)

export interface Source {
  /** Stable id, e.g. "webtoons". Used in notes and output. */
  readonly id: string;
  /** Human-readable name, e.g. "WEBTOON". */
  readonly name: string;

  /** True if this source handles the URL. */
  matches(url: URL): boolean;

  /** Fetches the series page and returns normalised series info. */
  resolve(url: URL, ctx: SourceContext): Promise<SourceSeries>;

  /** Covers currently shown by the source. */
  currentCovers(series: SourceSeries, ctx: SourceContext): Promise<CoverCandidate[]>;

  /**
   * Optional: pure extraction of cover candidates from a series page's HTML.
   * Sources that implement this get Wayback history for free.
   */
  extractCovers?(html: string, pageUrl: URL): CoverCandidate[];

  /** Optional source-specific history beyond Wayback (e.g. an API with old art). */
  historicalCovers?(series: SourceSeries, ctx: SourceContext): Promise<CoverCandidate[]>;

  /** Normalises an image URL to its largest, hotlinkable form. */
  normalizeImageUrl(url: string): string;

  /** True if a MangaBaka series link points at this source series. */
  isSameSeries(linkUrl: string, series: SourceSeries): boolean;

  /** Hosts this source fetches from; the server proxy allowlist is built from these. */
  readonly hosts: readonly string[];
}

export interface SourceSeries {
  source: string;          // Source.id
  externalId: string;      // e.g. Webtoons title_no "95"
  title: string;
  language: string;        // MangaBaka language code, e.g. "en", "zh-Hant"
  canonicalUrl: string;
  archiveUrls: string[];   // URL prefixes to look up in Wayback
  mature: boolean;         // source flags the series as mature/age-gated (§4.7)
  flags: Record<string, boolean>; // source-specific, e.g. { canvas: true }
}

export interface CoverCandidate {
  url: string;             // normalised
  role: "poster" | "square" | "banner" | "foreground" | "other";
  current: boolean;        // shown on the live page now
  seenAt?: string;         // archive timestamp (YYYYMMDD…) if historical
  archivedUrl?: string;    // Wayback fallback for the image itself
}

Adding a source (e.g. Tapas): create src/core/sources/tapas/ implementing Source (HTML extraction in a pure parse.ts with fixture tests), add it to registry.ts, and map its language codes to MangaBaka codes. Its hosts flow into the server allowlist automatically.

4.2 Transport (how each front end reaches the network)

export interface Transport {
  /** Fetch-like. Returns a standard Response. */
  fetch(url: string, init?: { method?: string; headers?: Record<string, string>;
                              body?: BodyInit; signal?: AbortSignal }): Promise<Response>;
}

Front end	MangaBaka API	Webtoons / Wayback / pstatic
Static page	`fetch` directly (CORS open)	`fetch("/api/fetch?url=…")` on our server (same origin; the session cookie rides along)
Userscript	`fetch` directly	`GM_xmlhttpRequest`, wrapped into a `Response`
CLI (later)	Bun `fetch`	Bun `fetch`
Tests	fake transport serving fixtures	fake transport serving fixtures

Shared behaviour lives in core/transport.ts and wraps whichever transport is injected:

limits each host to 1 request per second, and Wayback to 1 every 10 s;
times out after 20 s (uploads may ask for longer);
retries twice with backoff on 429/5xx, honouring Retry-After, but only for GET/HEAD: a retried POST could file a second edit request or upload twice;
sends Wayback the honest User-Agent (§2.2). (Browsers don't let a page set User-Agent; the server proxy and the userscript set it per host.)
never sends the MangaBaka token anywhere except api.mangabaka.org. In particular the token never passes through our server.

4.3 History

history/wayback.ts knows nothing about any one source. For each prefix in series.archiveUrls it:

queries CDX with collapse=timestamp:6, which lists every month that has a capture (one cheap request per prefix), and drops rows whose page is not the same series (matchType=prefix on title_no=95 also matches title_no=950);
splits the span from the first capture to now into buckets of the user's granularity — every 1, 3, 6 (default) or 12 months — and keeps one capture per bucket, preferring www. over m.;
fetches each chosen snapshot with the id_ modifier;
calls source.extractCovers(html, originalUrl);
normalises the image URLs and tags each candidate with seenAt and archivedUrl.

Tower of God at 6 months is 2 CDX lookups and 24 snapshots, about 4–5 minutes at one request per 10 s, so the UI shows progress and can cancel; a cancelled run keeps what it found.

Candidates that share a normalised URL are merged; the merged candidate records the first and last time it was seen. Wayback failures (timeouts, 5xx, rate limits) produce a warning, never a fatal error. A snapshot that still fails after its retries stops the history (Wayback is blocking or down; waiting on each remaining snapshot would take minutes apiece) and keeps the covers found so far. The user can turn history off.

4.4 Image inspection and classification

images.ts downloads each candidate through the transport (live URL first, then archivedUrl) and records width, height, sha256 (WebCrypto crypto.subtle.digest, available in browsers, Workers and Bun), contentType and live status. Dimensions come from the image header bytes, so no DOM is needed. Covers we've seen range from about 20 KB to 650 KB; downloads over 20 MB are refused. Candidates that cannot be downloaded at all are dropped with a warning.

Classification by aspect ratio (h / w):

Shape	Ratio	MangaBaka `type`	Selected by default
tall (landing-page art)	> 1.6	`other`	no — labelled "landing-page art"
portrait	1.15 – 1.6	`volume`	yes
square-ish	0.85 – 1.15	`other`	yes
wide	< 0.5	`banner`	yes
otherwise	–	`other`	yes

Real Webtoons posters are about 1.3 (480×623); landing-page art is 1.7 (720×1230) with an empty lower half (§2.2).

4.5 Series and dedupe (`links.ts`, `covers.ts`)

Inputs. links.ts accepts a MangaBaka series link or a bare id. Verified forms: https://mangabaka.org/9999 redirects (301) to the canonical https://mangabaka.org/manhwa/9999/Tower-of-God; /series/9999 is 404. So take the first all-digits path segment. Plus a Webtoons series URL (www. or m.).
Sanity check. If none of the MangaBaka series' links passes source.isSameSeries (same title_no), show a warning and require the user to confirm. It is a warning, not an error: the link may simply be missing on MangaBaka.
Duplicates. Images with the same sha256 are merged; the merged entry keeps the live URL and the widest seenAt range.
Already on MangaBaka. Candidates whose sha256 matches an existing MangaBaka image are dropped and shown as "already present".
Default selection. The suggested default is the current live poster, or the current live square if there is none (canvas). MangaBaka has one default per series (§2.3), so making a Webtoons edition's poster the default replaces the series' main cover for every language. The UI therefore shows a checkbox "Make this the series' main cover", on by default, with that warning. When on, that image gets is_default: true and every other image false; when the suggested image is already on MangaBaka, the existing image is the one marked. When off, existing defaults are left alone. The reviewer can still reject it.
Notes. Every new image gets a note, e.g. "WEBTOON poster (current)" or "WEBTOON poster, seen 2019-03 – 2021-01".

4.6 Image hosting

MangaBaka must be able to download each submitted image until a reviewer acts, which may take weeks. The user picks the host per run; the UI shows each host's expiry next to the button.

export interface ImageHost {
  readonly id: "server" | "litterbox" | "catbox" | "direct";
  /** Human-readable lifetime, shown in the UI ("30 days", "72 hours", "permanent"). */
  lifetime(opts: HostOptions): string;
  /** Rehosts one inspected image; returns a public URL. */
  upload(image: { sourceUrl: string; bytes: Uint8Array; sha256: string; contentType: string },
         ctx: HostContext): Promise<{ url: string }>;
}

Host	Page	Userscript	Lifetime
`server` (default when the server has storage)	`POST /api/upload { url, host: "store" }`	same	30 days (daily sweep)
`litterbox` (default otherwise)	multipart upload of the inspected bytes directly from the browser (CORS open), `time=72h` unless the user picks less	same, via `GM_xmlhttpRequest`	1h / 12h / 24h / 72h
`catbox`	`POST /api/upload { url, host: "catbox", userhash? }` — the server fetches the image and uploads it, because catbox has no CORS	directly via `GM_xmlhttpRequest`, no server needed	permanent
`direct`	no upload: the normalised source URL	same	as long as Webtoons keeps it

Server storage is, for now, a plain folder on our server (MBSEED_STORAGE_DIR). Files are named <sha256>.<ext> and served by our server at GET /covers/<sha256>.<ext>; the returned URL is <MBSEED_PUBLIC_URL>/covers/<sha256>.<ext>, so the server must be reachable from the internet for MangaBaka to fetch it.
- Writes go to a temp file in the same folder, then rename, so a reader never sees half a file.
- Expiry: bun.ts sweeps the folder at startup and every 24 hours, deleting files whose modification time is older than 30 days. A re-upload rewrites the file, which restarts its 30 days.
- Size cap. The folder's total size is capped by MBSEED_STORAGE_MAX_BYTES (default 1 GiB, about 170 worst-case runs per 30 days). bun.ts measures the folder at startup and keeps a running total in memory, updated on every write and sweep; uploads are serialised so two can't race past the cap. Re-uploading a file that already exists doesn't count again.
- When full, refuse; never evict. An upload that would exceed the cap gets 507 Insufficient Storage, and nothing is deleted early: older files may still be waiting for a MangaBaka reviewer. Space comes back as the daily sweep expires files. The page then offers litterbox or catbox instead.
- The per-image 20 MB cap still applies.
- GET /covers/* accepts only ^[0-9a-f]{64}\.(jpg|png|gif|webp)$ (no path traversal) and sets Content-Type from the extension.
- Kept behind the Storage interface so an S3-compatible bucket or an R2 binding can replace it later without touching app.ts. If the server has no storage configured, GET /api/config says so and the page hides the server host.
litterbox 72h warning. The UI warns that a review may take longer than 72 hours and the links will then be dead.
catbox userhash. Optional, entered by the user, kept in the browser (localStorage only if "remember" is ticked) or in GM_setValue. Through the page it passes through our server, which forwards it to catbox only and never logs or stores it.

Rules:

Upload only what we checked. For hosts reached through our server, the client sends the source URL, not bytes, so /api/upload can't be used to host arbitrary files; the server returns the sha256 of what it fetched and the client rejects a mismatch. Hosts reached directly get the inspected bytes.
Re-uploads are cheap. Server storage is keyed by hash, so rerunning overwrites the same object (and restarts its 30 days). For litterbox and catbox, the page keeps a per-session sha256 → url map and the userscript a persistent one (GM_setValue), so the same image isn't uploaded twice.
No upload without an explicit user action.
Failures are per image. A failed upload is reported and the image left out; if every upload fails, the action aborts.
Adding a host is one file in core/hosting/ plus one registry line.

4.7 Content rating

MangaBaka needs a content_rating per image. The tool can't judge it, so the user chooses it, default safe: one selector for all new images, plus a per-card override. If the source marks the series as mature (Webtoons ageGradeNotice : true), the UI shows a warning to consider a rating above safe. The default stays safe. Unattended runs (CLI --yes, later) must pass a rating explicitly.

4.8 Seed (`seed/covers.ts`)

buildCoverSeed(existing, newCovers, series, { contentRating }) returns:

interface CoverSeed {
  seriesId: number;
  userNote: string;  // "Add WEBTOON covers from <canonicalUrl>"
  images: SubmissionImage[]; // existing (unchanged except is_default) + new
  added: InspectedCover[];   // for display
  skipped: { cover: InspectedCover; reason: string }[];
}

Each new image becomes { id: null, series_id, work_id: null, url, index: null, index_numeric: null, language: series.language, type, note, content_rating, is_default }. withHostedUrls(seed, urlMap) swaps in the rehosted URLs after upload.

4.9 Front ends

Static page (src/web/, served by our server):

Two inputs: MangaBaka series link, Webtoons series link. Optional: "include history" (off since 2026-10-15, spec 002 §1), granularity (every 1 / 3 / 6 / 12 months, default 6).
Runs the pipeline and shows one card per new cover (thumbnail, dimensions, suggested type, language, note, content-rating selector, checkbox), plus "already present" and skipped covers.
Rehost selected uploads to the chosen host (§4.6) and replaces the shown URLs.
Without a token: "Copy all selected" and a link to https://mangabaka.org/data/submissions/series/{id}/images for manual entry.
With a token: the user pastes a personal access token (kept only in memory unless they tick "remember on this device", then localStorage). The page fetches the snapshot, shows the preview diff, and on confirmation rehosts and saves with save_mode: "review". 409: refetch, rebuild, re-preview, ask again (once). 403: explain the 25-approved-submissions rule and fall back to manual entry.

Userscript (src/userscript/, built to mbseed.user.js):

@match https://mangabaka.org/*. On a series page it adds a "Seed covers" panel; the MangaBaka series id comes from the page URL and the user pastes the Webtoons link.
Fetches Webtoons/Wayback/pstatic with GM_xmlhttpRequest (@connect webtoons.com, pstatic.net, web.archive.org, catbox.moe, and our server's host). Same review cards as the page.
Rehosts to litterbox or catbox directly, or to our server's storage (/api/upload; no storage keys in the script). With litterbox or catbox it works with no server at all.
Submits through the API when a token is set (stored with GM_setValue).
Without a token, on the cover form page it can fill the form fields for the selected covers. Whether the form accepts programmatically filled fields is unverified (§7).

4.10 Server (`src/server/`)

Route	Behaviour
`GET /*`	Static assets of the page.
`GET /api/config`	`{ storage: boolean, storageFull: boolean }`, so the page knows which hosts to offer (a full store is shown disabled, with the reason).
`GET /covers/<sha256>.<ext>`	Serves a stored cover (§4.6). Public: MangaBaka fetches these.
`GET /api/fetch?url=`	Fetches `url` only if its host is in the allowlist (union of `Source.hosts` plus `web.archive.org`), with a browser User-Agent for source hosts and the honest one for Wayback (§2.2); follows at most 5 redirects, each re-checked against the allowlist; returns the response. Anything else: 403.
`POST /api/upload`	`{ url, host: "store" \| "catbox", userhash? }` (§4.6). `url` must pass the same allowlist.

The API routes reject other origins (except the userscript, which is not bound by CORS), cap response size at 20 MB, and are rate-limited per IP.

One handler; Bun now, Cloudflare later. app.ts exports handle(req: Request, env: ServerEnv): Promise<Response> using only web-standard APIs (fetch, Request, Response, crypto.subtle):

	Cloudflare (`worker.ts`, deferred)	Self-hosted (`bun.ts`, now)
Run	`wrangler dev` / `wrangler deploy`	`bun run src/server/bun.ts` behind our reverse proxy
Static assets	Workers static assets, with `run_worker_first = true` so the gate also covers assets (confirm the key when scaffolding)	read from the built `dist/web/`
Storage	R2 binding	folder `MBSEED_STORAGE_DIR` (public base URL `MBSEED_PUBLIC_URL`, cap `MBSEED_STORAGE_MAX_BYTES`), or none
Password	`wrangler secret put MBSEED_PASSWORD`	`MBSEED_PASSWORD` env var
Client IP for rate limits	`CF-Connecting-IP`	`server.requestIP(req)`, or `X-Forwarded-For` only when `MBSEED_TRUST_PROXY=1`

Self-hosting also avoids the Cloudflare-egress risk. Our reverse proxy terminates TLS; how X-Forwarded-For is trusted is set by MBSEED_TRUST_PROXY.

4.11 Access gate (`server/gate.ts`) — deferred

Not built in the first version. Decided 2026-10-14: the self-hosted site starts open, and we add this if it gets abused or traffic grows too large. Until then the server's protections are the host allowlist, the 20 MB cap and the per-IP rate limit. The design below is kept for when it's needed; until then there is no /login, and /api/config has no gated field.

An optional shared password locks the whole service: page, assets and API.

Off by default. If MBSEED_PASSWORD is unset, the gate is open and /api/config reports gated: false. A self-hosted instance can instead rely on the reverse proxy's own auth (e.g. basic auth or SSO) and leave it unset.
Login. GET /login serves a minimal form. POST /login compares the password in constant time, then sets mbseed_session=<expiry>.<hmac> (HttpOnly; Secure; SameSite=Strict; Path=/, 30 days). The HMAC key is derived from MBSEED_PASSWORD, so changing the password logs everyone out. Login attempts are rate-limited per IP (5 per minute).
Every other route requires a valid, unexpired session cookie: browser page requests without one get a 302 to /login, and API requests get 401.
Userscript. It can't rely on the cookie, so API calls may instead send Authorization: Bearer <password> (constant-time compare). The userscript stores the password with GM_setValue. It needs this only for /api/upload with server storage or catbox-via-server; with litterbox or direct catbox it never calls our server.
No accounts. One password for the group; rotate it to revoke access.

5. Cross-cutting concerns

Secrets. The MangaBaka token stays in the user's browser (or userscript storage) and is only sent to api.mangabaka.org. The server has no secrets for now (MBSEED_PASSWORD once the gate exists). The catbox userhash is forwarded, never stored.
Errors. The UI shows a per-stage status; Wayback failure is a warning, not a failure.
Load. A run is ~60–100 proxied requests and ≤ ~6 MB stored for 30 days. Abuse of the open proxy is the main risk, hence the allowlist, the size cap and the per-IP rate limit; the gate (§4.11) is the next step if needed. (Earlier estimate for Cloudflare: $0 on free tiers; ~$5/month for a busy public instance.)

6. Testing

bun test against fixtures saved from real responses: Webtoons pages (official, canvas, mature, a 2015 Wayback snapshot), CDX output, MangaBaka series/links/images/snapshot/preview JSON.
Pure functions under test: link parsing, Webtoons parsing, URL normalisation, classification, dedupe, default selection, seed building, content-rating resolution, language mapping, the server allowlist, session cookie signing/expiry.
I/O (pipeline, mangabaka/client, history/wayback, every hosting/*, server/app.ts routes incl. the gate: no cookie → 302/401, bad password, expired cookie, Bearer header) with a fake transport and in-memory storage. No network in tests.
Manual smoke test: the page against Tower of God (MangaBaka 9999, Webtoons title_no=95), with "Rehost" and without submitting.

7. Open questions and risks

Server egress. Webtoons or Wayback may throttle our server's IP under load. (Cloudflare IPs would be riskier; the userscript uses the user's IP.)
When MangaBaka copies images (submission or approval) and whether it accepts our storage URLs, litter.catbox.moe / files.catbox.moe URLs, or swebtoon-phinf.pstatic.net (direct). If it copies only on approval, litterbox's 72h is likely too short.
"Main cover" = is_default: true is inferred from the API spec.
Where users create a personal access token in the MangaBaka UI.
Whether the cover form accepts fields filled by a userscript.
Whether catbox accepts uploads from our server (catbox-via-server).
Whether MangaBaka's hashes.sha256 is of the original bytes. Dedupe against existing covers relies on it; if MangaBaka re-encodes images, the "already present" check misses and the reviewer sees duplicates.
Hosting details: which of our servers and domain (MBSEED_PUBLIC_URL), where the covers folder lives (MBSEED_STORAGE_DIR), and whether the 1 GiB default cap suits that disk (~6 MB per run at most, kept 30 days).

8. Test data

Tower of God historical og:images:

2015  http://swebtoon.phinf.naver.net/20140710_28/1404979632969uKBA8_JPEG/01_EC8BA0EC9D98ED8391.jpg
2017  http://swebtoon.phinf.naver.net/20151006_141/1444125046528RTfRM_JPEG/_EB93BBEB80AF_EBA38CEABCA5_E29587EABCB1_EB9384EB84B0_ipa.jpg
      (twitter:image …/20151006_243/1444125042491PuNza_JPEG/…_mob.jpg)
2019  https://swebtoon-phinf.pstatic.net/20190318_291/1552868599909GoVLY_JPEG/10_EC8DB8EB84A4EC9DBC_ipad.jpg
      (twitter:image …/20190318_26/1552868584759vl9A1_JPEG/11_EC8DB8EB84A4EC9DBC_mobile.jpg)
2021  https://swebtoon-phinf.pstatic.net/20210127_106/1611705089085OYcUB_JPEG/1Tower-of-God-Mobile-Landing-Page.jpg
2023+ https://swebtoon-phinf.pstatic.net/20230324_114/1679612146968KhLLz_JPEG/6TowerOfGod_landingpage_mobile.jpg
live  https://swebtoon-phinf.pstatic.net/20250204_46/17386458444547o1b2_JPEG/95.jpg   (poster, 480×623)

The 2014, 2015 and 2019 images are still live on swebtoon-phinf.pstatic.net; the 2021 one is gone from Wayback's own copy but live on pstatic.

A 6-month sweep on 2026-10-14 found 20 distinct images (5 posters as og:image, 4 squares, 6 banners, 5 foregrounds), all live; see §2.2 for what they are. 2015 snapshots write background:url(…) without quotes and use the hosts swebtoon.phinf.naver.net and webtoon.phinf.naver.net; both map to swebtoon-phinf.pstatic.net with the same path. On the canvas example the og:image and twitter:image URLs differ but the bytes are identical (merged by sha256).

Other test inputs:

canvas: https://www.webtoons.com/en/canvas/crow-time/list?title_no=693372
mature: title_no 6065 (filename contains spaces and ); URL-encode it)
FR example: Violet Romance, 2Thumb_Poster_9782.jpg.

9. References

yambs (the model project, Go): /home/user/yambs on the maintainer's machine, with the same sources → seed → actions shape.
MangaBaka API docs: https://mangabaka.org/data/api; explorer: https://mangabaka.org/data/api/explorer; support: https://mangabaka.org/discord.
The maintainer is a regular MangaBaka contributor without API access (fewer than 25 approved submissions as of 2026-10-14), so the manual path must be first-class.

40 KiB Raw Permalink Blame History Unescape Escape