502 lines
23 KiB
Markdown
502 lines
23 KiB
Markdown
# Architecture
|
||
|
||
`mangabaka-seeder` is a command-line tool that gathers data about a series from
|
||
an external site (a "source") and turns it into an edit request ("submission")
|
||
for [MangaBaka](https://mangabaka.org). It is modeled on
|
||
[yambs](https://codeberg.org/derat/yambs), the MusicBrainz seeder: sources turn
|
||
into seeds, and an action decides what happens to them. The user always stays
|
||
in control of what is submitted.
|
||
|
||
The first feature is **covers**: given a Webtoons series URL, collect every
|
||
cover that series has used (current and historical) and request them added to
|
||
the matching MangaBaka series, with the current poster as the default (main)
|
||
cover for the edition's language.
|
||
|
||
This document is the design spec. It records the facts we verified about the
|
||
external services, the module boundaries, and the extension points.
|
||
|
||
---
|
||
|
||
## 1. Goals and non-goals
|
||
|
||
**Goals**
|
||
|
||
- One command: `mbseed <url>` produces a ready-to-review cover submission.
|
||
- Find historical covers, not just the current one.
|
||
- Work for everyone. With a MangaBaka API token, submit through the API. Without
|
||
one, fall back to a local review page that helps with manual entry.
|
||
- New sources (Tapas, Tappytoon, Lezhin, Manta, …) can be added as one new
|
||
module under `src/sources/`, with no changes to the pipeline, the MangaBaka
|
||
client or the actions.
|
||
- Never change MangaBaka data silently. API submissions always go through
|
||
preview, explicit confirmation and `save_mode: "review"`.
|
||
- Never upload anything without explicit user action. The user chooses the
|
||
content rating (default `safe`).
|
||
- Image hosts are pluggable too (catbox.moe by default, any S3-compatible
|
||
bucket, or no rehosting).
|
||
|
||
**Non-goals (for now)**
|
||
|
||
- Editing series fields other than images, such as titles, links or tags. The
|
||
submission layer is keyed by MangaBaka "resource" so this can come later.
|
||
- Following a Webtoons series across language editions. Each `title_no` belongs
|
||
to one language edition; run the tool once per edition URL.
|
||
- Hosting images on our own infrastructure by default. We rehost on a public
|
||
image host (catbox.moe), or on a user-configured S3-compatible bucket.
|
||
|
||
---
|
||
|
||
## 2. Verified external facts
|
||
|
||
These were verified on 2026-10-14. Treat them as assumptions that the fixtures
|
||
in `test/fixtures/` pin down.
|
||
|
||
### 2.1 Webtoons
|
||
|
||
| What | Where |
|
||
|---|---|
|
||
| Portrait poster (official series; 480×623 original) | `meta[property="og:image"]` |
|
||
| Square thumbnail | `meta[name="twitter:image"]` |
|
||
| Hero background (1920×321, official only) | `div.detail_bg` inline `style` → `background:url('…')` |
|
||
| Hero foreground (1200×240 PNG; on canvas series, the 1080×1080 square) | `.detail_header span.thmb img[src]` |
|
||
| Title | `meta[property="og:title"]` (fallback `h1.subj` / `h3.subj`) |
|
||
| Canonical URL | `link[rel="canonical"]` |
|
||
| `title_no` | `title_no` query parameter |
|
||
| Language | first path segment (`/en/`, `/fr/`, `/zh-hant/`, …) |
|
||
| Canvas vs official | `/canvas/` (or `/challenge/`) in the path; inline JS `isChallenge : true` |
|
||
|
||
- **Canvas series** have no portrait poster. Their `og:image` is a 1080×1080
|
||
square.
|
||
- **Image hosts.** `webtoon-phinf.pstatic.net` returns 403 without a Webtoons
|
||
`Referer`; `swebtoon-phinf.pstatic.net` serves the same path with no Referer.
|
||
- **Original size.** The `?type=…` query selects a resized variant. Removing it
|
||
returns the original. `crop540_540` crops posters, so always strip the query.
|
||
- **Old hosts.** Old snapshots reference `swebtoon.phinf.naver.net`, which
|
||
becomes `swebtoon-phinf.pstatic.net` with the same path.
|
||
- **No limits so far.** A browser User-Agent is enough. Age-gated titles still
|
||
serve full HTML, and no cookies are required.
|
||
- **No usable JSON API.** The app API needs signed requests. We scrape HTML.
|
||
|
||
### 2.2 Historical covers (Wayback Machine)
|
||
|
||
- **Snapshot index.** The CDX API at
|
||
`https://web.archive.org/cdx/search/cdx?url=<page>&matchType=prefix&filter=statuscode:200&collapse=timestamp:6&fl=timestamp,original`
|
||
returns at most one snapshot per month.
|
||
- **Raw pages.** `https://web.archive.org/web/<ts>id_/<original>` returns the
|
||
page as it was archived, without Wayback's toolbar.
|
||
- **History exists.** For Tower of God, snapshots from 2015 to 2025 reference
|
||
five distinct cover images. All the older ones tested are **still served live**
|
||
from `swebtoon-phinf.pstatic.net`.
|
||
- **Fallback.** If an image is no longer live, it is often still at
|
||
`https://web.archive.org/web/<ts>id_/<image-url>`.
|
||
- **Not every image is a cover.** Some historical `og:image`s are landing-page
|
||
art, so candidates are classified by shape and reviewed by the user.
|
||
|
||
### 2.3 MangaBaka
|
||
|
||
- **Endpoints.** The site is `https://mangabaka.org` and the API is
|
||
`https://api.mangabaka.org`. The OpenAPI spec is at
|
||
`https://mangabaka.org/api.json`, with docs at
|
||
`https://mangabaka.org/data/api`.
|
||
- **Public reads.**
|
||
- `GET /v2/series/match?q=<title>` and `GET /v2/series/search?q=…` find
|
||
series. They are limited to about 30 requests per minute.
|
||
- `GET /v1/series/{id}` and `GET /v1/series/{id}/links` return the series
|
||
and its links. Links are canonicalised, e.g.
|
||
`https://www.webtoons.com/-/-/-/list?title_no=95`.
|
||
- `GET /v1/series/{id}/images` returns covers with `type`, `language`,
|
||
`index`, `note`, `content_rating`, `is_default` and `hashes.sha256`.
|
||
- **Image `type` values:** `volume` (default), `volume_back`, `chapter`,
|
||
`season`, `banner`, `audiobook`, `other`. There is no "main" type; the main
|
||
cover is the one with `is_default: true`.
|
||
- **Submissions API.** Auth is `x-api-key: mb-…`. All endpoints are v0 and
|
||
marked internal.
|
||
- `GET /v0/my/submissions/series-images/{id}` returns the current image set
|
||
and a `version` token.
|
||
- `POST /v0/my/submissions/series-images/{id}/preview` is a dry run that
|
||
returns a diff.
|
||
- `POST /v0/my/submissions/series-images/{id}` saves, with body
|
||
`{ version, user_note, save_mode, data: { images: [...] } }`.
|
||
- `images` **replaces the whole set**, so it must contain the existing images.
|
||
- A stale `version` gives 409.
|
||
- Users without the contributor role and with fewer than 25 approved
|
||
submissions get 403.
|
||
- **Web form.** Without API access, users use
|
||
`https://mangabaka.org/data/submissions/series/{id}/images`. It cannot be
|
||
pre-filled with query parameters, and its SvelteKit CSRF check blocks
|
||
cross-site POSTs.
|
||
- **Image URLs.** The spec describes the cover `url` as "Cover image URL. imgur
|
||
is rejected (aggressive rate limiting). Acts as the cover identity on save."
|
||
- We don't know whether MangaBaka can fetch from `pstatic.net` (it is
|
||
hotlink-protected on some hosts).
|
||
- We don't know whether MangaBaka copies the image when the request is
|
||
submitted or when a reviewer approves it.
|
||
- So by default we **rehost** each new image on a host that keeps it
|
||
(see §4.6).
|
||
- **`content_rating`:** one of `safe`, `suggestive`, `erotica`,
|
||
`pornographic`. The user chooses it (§4.7).
|
||
|
||
---
|
||
|
||
## 3. Pipeline
|
||
|
||
```
|
||
┌──────────────┐ SourceSeries ┌──────────────────┐
|
||
URL ────▶ │ source.resolve│ ───────────────▶ │ mangabaka.resolve │──▶ MangaBaka series id
|
||
└──────────────┘ └──────────────────┘
|
||
│
|
||
▼
|
||
source.currentCovers + history (Wayback, via source.extractCovers)
|
||
│
|
||
▼ CoverCandidate[]
|
||
images.inspect (download, dimensions, sha256, liveness)
|
||
│
|
||
▼ InspectedCover[]
|
||
covers.dedupe (within candidates + against existing MangaBaka covers)
|
||
│
|
||
▼
|
||
seed.buildCoverSeed ──▶ CoverSeed (existing images + new images,
|
||
│ content_rating chosen by the user)
|
||
▼
|
||
action: print | serve | submit
|
||
│
|
||
▼ (only after explicit user action: serve button / submit confirm)
|
||
imageHost.upload ──▶ rehosted URLs replace source URLs in the seed
|
||
```
|
||
|
||
Each stage is a pure function or a small module with injected I/O (`fetch`),
|
||
so it can be tested with fixtures.
|
||
|
||
---
|
||
|
||
## 4. Modules
|
||
|
||
```
|
||
src/
|
||
cli.ts argument parsing (node:util parseArgs), wiring, exit codes
|
||
pipeline.ts runs the stages above; the only place they are composed
|
||
http.ts fetch wrapper: User-Agent, per-host rate limit, retries, timeouts
|
||
model.ts shared types (SourceSeries, CoverCandidate, InspectedCover, CoverSeed)
|
||
sources/
|
||
source.ts Source interface + helpers
|
||
registry.ts list of sources; findSource(url)
|
||
webtoons/
|
||
index.ts Webtoons Source implementation
|
||
parse.ts pure HTML → { series, covers } extraction
|
||
urls.ts URL parsing, canonicalisation, image URL normalisation
|
||
history/
|
||
wayback.ts generic Wayback history provider (CDX + archived pages)
|
||
images.ts download + probe dimensions + sha256 + classify by shape
|
||
covers.ts dedupe, merge observations, pick default
|
||
mangabaka/
|
||
client.ts typed API client (public reads + submissions)
|
||
resolve.ts source series → MangaBaka series id
|
||
types.ts API types (subset of the OpenAPI spec we use)
|
||
hosting/
|
||
host.ts ImageHost interface + upload cache (sha256 → url)
|
||
registry.ts list of hosts; selectHost(name, env)
|
||
catbox.ts catbox.moe (default)
|
||
s3.ts S3-compatible bucket (R2, B2, MinIO, AWS) via aws4fetch
|
||
direct.ts no rehosting: submit source URLs as-is
|
||
seed/
|
||
covers.ts CoverSeed builder: existing + new images → submission body
|
||
actions/
|
||
print.ts human-readable or --json output
|
||
serve.ts local review page (node:http, single HTML page, no framework)
|
||
submit.ts snapshot → preview → confirm → save (review mode)
|
||
test/
|
||
fixtures/ saved HTML / CDX / JSON responses
|
||
*.test.ts vitest
|
||
```
|
||
|
||
### 4.1 The `Source` interface (extension point)
|
||
|
||
```ts
|
||
export interface Source {
|
||
/** Stable id, e.g. "webtoons". Used in notes and CLI output. */
|
||
readonly id: string;
|
||
/** Human-readable name, e.g. "WEBTOON". */
|
||
readonly name: string;
|
||
|
||
/** True if this source handles the URL. */
|
||
matches(url: URL): boolean;
|
||
|
||
/** Fetches the series page and returns normalised series info. */
|
||
resolve(url: URL, ctx: SourceContext): Promise<SourceSeries>;
|
||
|
||
/** Covers currently shown by the source. */
|
||
currentCovers(series: SourceSeries, ctx: SourceContext): Promise<CoverCandidate[]>;
|
||
|
||
/**
|
||
* Optional: pure extraction of cover candidates from a series page's HTML.
|
||
* Sources that implement this get Wayback history for free. The history
|
||
* provider runs it on archived copies of `series.archiveUrls`.
|
||
*/
|
||
extractCovers?(html: string, pageUrl: URL): CoverCandidate[];
|
||
|
||
/** Optional source-specific history beyond Wayback (e.g. an API with old art). */
|
||
historicalCovers?(series: SourceSeries, ctx: SourceContext): Promise<CoverCandidate[]>;
|
||
|
||
/** Normalises an image URL to its largest, hotlinkable form. */
|
||
normalizeImageUrl(url: string): string;
|
||
|
||
/** True if a MangaBaka series link points at this source series. */
|
||
isSameSeries(linkUrl: string, series: SourceSeries): boolean;
|
||
}
|
||
|
||
export interface SourceSeries {
|
||
source: string; // Source.id
|
||
externalId: string; // e.g. Webtoons title_no "95"
|
||
title: string;
|
||
language: string; // MangaBaka language code, e.g. "en", "zh-Hant"
|
||
canonicalUrl: string;
|
||
archiveUrls: string[]; // URL prefixes to look up in Wayback
|
||
mature: boolean; // source flags the series as mature/age-gated (§4.7)
|
||
flags: Record<string, boolean>; // source-specific, e.g. { canvas: true }
|
||
}
|
||
|
||
export interface CoverCandidate {
|
||
url: string; // normalised
|
||
role: "poster" | "square" | "banner" | "foreground" | "other";
|
||
current: boolean; // shown on the live page now
|
||
seenAt?: string; // archive timestamp (YYYYMMDD…) if historical
|
||
archivedUrl?: string; // Wayback fallback for the image itself
|
||
}
|
||
```
|
||
|
||
**Adding a source** (e.g. Tapas) means:
|
||
|
||
1. Create `src/sources/tapas/` that implements `Source`. Put the HTML
|
||
extraction in a pure `parse.ts` and give it fixture tests.
|
||
2. Add it to the array in `src/sources/registry.ts`.
|
||
3. Map its language codes to MangaBaka codes.
|
||
|
||
The pipeline, Wayback history, image inspection, deduplication, MangaBaka
|
||
resolution and all actions work unchanged. A source with an API (Tappytoon, for
|
||
example) can skip `extractCovers` and implement `historicalCovers`, or neither.
|
||
|
||
### 4.2 History
|
||
|
||
`history/wayback.ts` knows nothing about any one source. For each prefix in
|
||
`series.archiveUrls` it:
|
||
|
||
1. queries CDX with `collapse=timestamp:6` (one snapshot per month, capped by
|
||
`--history-limit`, default 60);
|
||
2. fetches each snapshot with the `id_` modifier;
|
||
3. calls `source.extractCovers(html, originalUrl)`;
|
||
4. normalises the image URLs with `source.normalizeImageUrl` and tags each
|
||
candidate with `seenAt` and `archivedUrl`.
|
||
|
||
Candidates that share a normalised URL are merged, and the merged candidate
|
||
records the first and last time it was seen. Wayback failures (timeouts, 5xx,
|
||
rate limits) produce a warning and an empty history; they are never fatal.
|
||
`--no-history` skips this stage.
|
||
|
||
### 4.3 Image inspection and classification
|
||
|
||
`images.ts` downloads each candidate (live URL first, then `archivedUrl`) and
|
||
records its `width`, `height`, `sha256`, `contentType` and `live` status.
|
||
It keeps the bytes in memory for a later upload (§4.6). Covers we've seen
|
||
range from about 20 KB to 650 KB. A few dozen candidates fit easily in memory,
|
||
and downloads over 20 MB are refused as a safety cap. Dimensions
|
||
come from the image header, read with the `image-size` package. Candidates that
|
||
cannot be downloaded at all are dropped with a warning.
|
||
|
||
Classification by aspect ratio (h / w):
|
||
|
||
| Shape | Ratio | MangaBaka `type` |
|
||
|---|---|---|
|
||
| portrait | ≥ 1.15 | `volume` |
|
||
| square-ish | 0.85 – 1.15 | `other` |
|
||
| wide | < 0.5 | `banner` |
|
||
| otherwise | – | `other` |
|
||
|
||
### 4.4 Dedupe and default selection (`covers.ts`)
|
||
|
||
- **Duplicates.** Images with the same `sha256` are merged. The merged entry
|
||
keeps the live URL and the widest `seenAt` range.
|
||
- **Already on MangaBaka.** Candidates whose `sha256` matches an existing
|
||
MangaBaka image are dropped and reported as "already present".
|
||
- **Default selection.**
|
||
- The **default** is the current live poster.
|
||
- If there is none (canvas series), it is the current live square.
|
||
- The new default gets `is_default: true`, and every existing image gets
|
||
`is_default: false`. This is the "main cover" for the language edition.
|
||
- The reviewer on MangaBaka can still reject that change.
|
||
- **Notes.** Every new image gets a note, e.g.
|
||
`"WEBTOON poster (current)"` or `"WEBTOON poster, seen 2019-03 – 2021-01"`.
|
||
|
||
### 4.5 MangaBaka series resolution (`mangabaka/resolve.ts`)
|
||
|
||
1. If `--series <id>` is given, use it, and warn if no link of that series
|
||
passes `source.isSameSeries`.
|
||
2. Otherwise call `/v2/series/match?q=<title>`, then `/v2/series/search`. For
|
||
each result, fetch `/v1/series/{id}/links` and keep the series whose link
|
||
passes `source.isSameSeries`. For Webtoons that means the same `title_no`.
|
||
3. If no series or several series match, stop with exit code 2 and print the
|
||
candidates (id, title, URL) so the user can rerun with `--series`.
|
||
|
||
### 4.6 Image hosting (`hosting/`)
|
||
|
||
MangaBaka must be able to download every image we submit, and it may not do so
|
||
until a reviewer approves the request. So new images are **rehosted** on a host
|
||
that keeps them. This is a second extension point, shaped like sources:
|
||
|
||
```ts
|
||
export interface ImageHost {
|
||
readonly id: string; // "catbox" | "s3" | "direct" | …
|
||
/** Throws a descriptive error if required config/env is missing. */
|
||
checkConfig(env: NodeJS.ProcessEnv): void;
|
||
/** Uploads the bytes and returns a public, durable URL. */
|
||
upload(image: { bytes: Uint8Array; sha256: string; contentType: string;
|
||
filename: string }, ctx: HostContext): Promise<string>;
|
||
}
|
||
```
|
||
|
||
| Host | Selected by | Notes |
|
||
|---|---|---|
|
||
| `catbox` (default) | `--image-host catbox` | Uploads to `https://catbox.moe/user/api.php` (`reqtype=fileupload`, multipart `fileToUpload`). Free, no account, files are permanent. Set `CATBOX_USERHASH` to attach uploads to your catbox account so you can delete them later. |
|
||
| `s3` | `--image-host s3` | Any S3-compatible bucket (Cloudflare R2, Backblaze B2, MinIO, AWS). Signed `PUT` with `aws4fetch`. Key is `<source>/<externalId>/<sha256>.<ext>`. Configured by `MBSEED_S3_ENDPOINT`, `MBSEED_S3_BUCKET`, `MBSEED_S3_REGION` (default `auto`), `MBSEED_S3_ACCESS_KEY_ID`, `MBSEED_S3_SECRET_ACCESS_KEY` and `MBSEED_S3_PUBLIC_URL`. The public URL is `<PUBLIC_URL>/<key>`. |
|
||
| `direct` | `--image-host direct` | No upload: submits the normalised source URL. Use it if MangaBaka turns out to fetch `swebtoon-phinf.pstatic.net` fine. |
|
||
|
||
The upload rules:
|
||
|
||
- **Reuse earlier uploads.** Uploads are cached in
|
||
`$XDG_CACHE_HOME/mangabaka-seeder/uploads.json`, keyed by host id and
|
||
`sha256`, so rerunning never uploads the same image twice.
|
||
- **Upload the bytes we checked.** We send the bytes downloaded in §4.3, not a
|
||
URL for the host to fetch. That way the host gets exactly the image whose
|
||
hash we checked against MangaBaka.
|
||
- **No upload without an explicit user action.** Images go up only when the
|
||
user clicks *Rehost selected* in `serve`, or answers *yes* in `submit`.
|
||
`print` never uploads.
|
||
- **Failures are per image.** If an upload fails, that image is reported and
|
||
left out of the submission, and the others continue. If every upload fails,
|
||
the action aborts with exit code 1.
|
||
- **Adding a host** (e.g. a future "ours" server) is one file in `hosting/` plus
|
||
one line in `hosting/registry.ts`.
|
||
|
||
### 4.7 Content rating
|
||
|
||
MangaBaka needs a `content_rating` per image. The tool can't judge this
|
||
reliably, so **the user chooses it**, with `safe` as the default:
|
||
|
||
- **`--content-rating <rating>`** sets it for all new images.
|
||
- **`submit`** with no flag on a terminal asks
|
||
`Content rating for new covers [safe/suggestive/erotica/pornographic] (safe):`,
|
||
and Enter means `safe`.
|
||
- **`submit --yes`** requires `--content-rating` explicitly, so an unattended
|
||
run never assumes a rating; without it, exit code 2. On a non-interactive
|
||
stdin without `--yes`, the same rule applies.
|
||
- **`serve`** has a rating selector on each card, preset to the CLI value
|
||
(default `safe`). Selected ratings are shown next to each copied URL.
|
||
- **`print`** shows the rating it would use (`--content-rating` or `safe`).
|
||
- **Mature titles.** If the source marks the series as mature (e.g. Webtoons
|
||
`ageGradeNotice : true`), the prompt and the review page show a warning so
|
||
the user considers a rating above `safe`. The default is still `safe`.
|
||
|
||
### 4.8 Seed (`seed/covers.ts`)
|
||
|
||
`buildCoverSeed(existing, newCovers, series, { contentRating })` returns:
|
||
|
||
```ts
|
||
interface CoverSeed {
|
||
seriesId: number;
|
||
userNote: string; // "Add WEBTOON covers from <canonicalUrl>"
|
||
images: SubmissionImage[]; // existing (unchanged except is_default) + new
|
||
added: InspectedCover[]; // for display
|
||
skipped: { cover: InspectedCover; reason: string }[];
|
||
}
|
||
```
|
||
|
||
Each new image becomes:
|
||
`{ id: null, series_id, work_id: null, url, index: null, index_numeric: null,
|
||
language: series.language, type, note, content_rating, is_default }`.
|
||
`url` starts as the normalised source URL. `withHostedUrls(seed, urlMap)`
|
||
swaps in the rehosted URLs once the upload has happened.
|
||
|
||
### 4.9 Actions
|
||
|
||
| Action | Default when | Effect |
|
||
|---|---|---|
|
||
| `print` | `--action print` | Prints series, candidates, the planned seed, and skips. `--json` prints the seed as JSON. No uploads or writes. |
|
||
| `serve` | no `MANGABAKA_TOKEN` | Serves `http://localhost:8999` (see below). |
|
||
| `submit` | `MANGABAKA_TOKEN` set | Submits through the API (see below). |
|
||
|
||
**`serve`** shows a review page with one card per new cover. Each card has:
|
||
|
||
- a thumbnail, its dimensions, the suggested type, language and note
|
||
- a content-rating selector
|
||
- a checkbox.
|
||
|
||
The page also has:
|
||
|
||
- **"Rehost selected"**, which uploads the selected images to the configured
|
||
host and replaces the shown URLs with the hosted ones. With
|
||
`--image-host direct` it is hidden.
|
||
- **"Copy all selected"**, which copies the URLs.
|
||
- a link to `https://mangabaka.org/data/submissions/series/{id}/images`.
|
||
|
||
It is a single HTML string rendered server-side with inline JS and no
|
||
framework. It binds to localhost only, and its single POST endpoint
|
||
(`/rehost`) checks a random per-run token embedded in the page.
|
||
|
||
**`submit`**:
|
||
|
||
1. Pick the content rating (§4.7).
|
||
2. `GET` the snapshot (`version` plus current images) and rebuild the seed on
|
||
top of it.
|
||
3. `POST …/preview` with the source URLs and print the diff.
|
||
4. Ask "Upload N images to <host> and submit for review? [y/N]". `--yes` skips
|
||
this.
|
||
5. Rehost (§4.6), swap in the hosted URLs, and `POST` with
|
||
`save_mode: "review"`.
|
||
6. Handle the outcome:
|
||
- **409:** fetch the snapshot again, rebuild, preview again and ask again,
|
||
at most once. The upload cache prevents re-uploading.
|
||
- **403:** explain the 25-approved-submissions requirement and fall back to
|
||
`serve`.
|
||
|
||
---
|
||
|
||
## 5. Cross-cutting concerns
|
||
|
||
- **HTTP.** All network access goes through `http.ts`. It:
|
||
- sends a browser User-Agent;
|
||
- limits each host to 1 request per second, and Wayback to 1 every 1.5 s;
|
||
- times out after 20 s;
|
||
- retries twice with backoff on 429/5xx;
|
||
- never sends credentials to any host other than the one they belong to.
|
||
- **Logging.** Progress goes to stderr and results to stdout, so
|
||
`print --json` can be piped. `--verbose` logs every request.
|
||
- **Exit codes.**
|
||
- 0: success, including "nothing new to add"
|
||
- 1: unexpected error
|
||
- 2: bad usage, an unsupported URL, or an ambiguous or missing series match
|
||
- 3: the user declined to submit.
|
||
- **Config.** All configuration comes from environment variables:
|
||
- `MANGABAKA_TOKEN`
|
||
- `MANGABAKA_API` (default `https://api.mangabaka.org`, for testing)
|
||
- `MBSEED_IMAGE_HOST`, the default for `--image-host`
|
||
- `CATBOX_USERHASH`
|
||
- the `MBSEED_S3_*` variables (§4.6).
|
||
- **Secrets.** The MangaBaka token is sent only to the MangaBaka API, and the
|
||
S3 credentials only to the S3 endpoint.
|
||
|
||
---
|
||
|
||
## 6. Testing
|
||
|
||
- **Unit tests with vitest**, run against fixtures saved from real responses:
|
||
- Webtoons pages (official, canvas, an archived 2015 snapshot) and CDX output
|
||
- MangaBaka match, links, images, snapshot and preview JSON.
|
||
- **Pure functions under test:**
|
||
- parsing, URL normalisation and canonicalisation
|
||
- classification, dedupe and default selection
|
||
- seed building, content-rating resolution and the language mapping
|
||
- the upload cache.
|
||
- **I/O modules** (`pipeline`, `mangabaka/client`, `actions/submit`,
|
||
`history/wayback`, `hosting/catbox`, `hosting/s3`) are tested with an injected fake `fetch`.
|
||
- **No network in tests.** A manual smoke test is documented in the README:
|
||
`mbseed --action print https://www.webtoons.com/en/fantasy/tower-of-god/list?title_no=95`.
|