diff --git a/README.md b/README.md index d8ded5b..bc3d5df 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ MusicBrainz. > [the spec](docs/specs/001-webtoons-covers.md). Today's goal is one thing done well: **given a MangaBaka series link and a -WEBTOON or Tapas series link, find every cover that series has used, current +WEBTOON, Tapas or KakaoPage series link, find every cover that series has used, current and historical, and request them added to the MangaBaka series**. The current poster becomes the main (default) cover for the edition's language. @@ -82,11 +82,13 @@ least 25 approved submissions. | Source | Links | Current covers | Historical covers | |---|---|---|---| | WEBTOON (official & canvas) | `https://www.webtoons.com//…/list?title_no=N`, `m.webtoons.com` | poster, square, banner, foreground | Wayback Machine | +| KakaoPage (웹툰 & 웹소설) | `https://page.kakao.com/content/N` (or the old `…/home?seriesId=N`) | portrait cover (full size), landscape thumbnail | Wayback Machine | | Tapas (comics & novels) | `https://tapas.io/series/` (or `…/info`, the series number, or an `/episode/N` link) | portrait cover (full size), banner | Wayback Machine | -Tapas is English only. Mature Tapas series are shown only to signed-in adults: -the page can't read their current covers (it says so and still searches the -archive), but the userscript can, when you are signed in to Tapas in that browser. +Tapas is English only; KakaoPage is Korean. Mature Tapas series are shown only +to signed-in adults: their current covers can't be read (the tool says so and +still searches the archive). KakaoPage 15+ and 19+ titles are read normally and +show the content-rating warning. Each Webtoons language edition has its own `title_no`. Run the tool once per edition; the covers are tagged with that edition's language. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index e21b024..2f708dc 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -15,7 +15,7 @@ self-hosted Bun server (primary), a userscript, and later a Bun CLI. | `src/core/session.ts` | Review state shared by the page and the userscript: cards, main cover, rehosting (cards remember host, cache key and expiry), manual text, the download bundle, `restore` from history, token flow (preview → confirm → save for review). | | `src/core/zip.ts`, `download.ts` | Store-only ZIP writer (CRC-32, UTF-8 names); the download bundle: file names `NN_role_lang[_YYYY]_WxH.ext` and `covers.txt` (spec 002 §2). | | `src/core/saved.ts` | History model (spec 002 §3): `SavedSearch` (plain data, source by id), `toSaved`/`fromSaved` (drops expired links), `carryEdits` (search again), `pruneIds` (30 entries, 300 MB), `HistoryStore` interface and an in-memory one. | -| `src/core/sources/` | `webtoons/` and `tapas/` (spec 003): each `urls.ts` (links, image URL normalisation, same-series), `parse.ts` (pure HTML parsing, also used on Wayback snapshots), `index.ts` (the `Source`); `registry.ts` lists them and builds the proxy and upload allowlists. | +| `src/core/sources/` | `webtoons/`, `tapas/` (spec 003) and `kakaopage/` (spec 004; reads a JSON endpoint, not the page): each `urls.ts` (links, image URL normalisation, same-series), `parse.ts` (pure HTML parsing, also used on Wayback snapshots), `index.ts` (the `Source`); `registry.ts` lists them and builds the proxy and upload allowlists. | | `src/web/` | Built. The page ("Obi Shelf" design, see `DESIGN.md`): `template.ts` markup shared with the userscript panel, `app.ts` wiring (`mountApp`), `render.ts` shelf and timeline views, `view-model.ts` pure helpers, `transport.ts` (direct to MangaBaka/litterbox, everything else via `/api/fetch`), `styles.css` scoped for a shadow root, `save-file.ts` (Blob download, works in the panel's shadow root), `history-db.ts` (IndexedDB `HistoryStore`; null when storage is refused), `history-keeper.ts` (when the series on screen is saved: after a search, debounced edits, marks; flushes before a new search, an open and page hide; removal/clear stop tracking), `recent.ts` (History list and the start screen's recent shelf; the page only), `dark-reader.ts` (switches to the page's own dark theme and locks Dark Reader out when that extension is darkening the page). | | `src/server/` | Built. `app.ts`: web-standard handler (`createApp(env)`) — static assets, `/api/config`, `/api/fetch` allowlisted proxy (redirects re-checked, 20 MB cap), `/api/upload` (store or catbox; takes a source URL, never bytes), `/covers/*`. `storage.ts` (interface, in-memory) and `storage-dir.ts` (folder, size cap, 30-day sweep). `config.ts` (env, client IP), `rate-limit.ts`, `assets-path.ts`. `bun.ts`: thin `Bun.serve` entry. Gate and Workers entry deferred. | | `src/userscript/` | Built. `gm.ts`: `GM_xmlhttpRequest` transport (no proxy; honest User-Agent for Wayback) and GM storage. `main.ts`: a "Seed covers" launcher on MangaBaka series pages (follows client-side navigation) opening the page's app in a shadow-root panel; optional server URL in its settings. Built by `scripts/build-userscript.ts` into `dist/web/mbseed.user.js`, served at `/mbseed.user.js`. | @@ -61,6 +61,7 @@ requests) and smoke-tests it. Example deployment: `docker-compose.example.yml`. | Webtoons HTML / `swebtoon-phinf.pstatic.net` | server proxy or `GM_xmlhttpRequest` | Nothing works. | | Wayback (`web.archive.org`) | same | Warning; current covers only. | | Server storage (folder on our server) | server | `server` host hidden; litterbox/catbox/direct still work. | +| KakaoPage (`bff-page.kakao.com`, images on `page-images.kakaoentcdn.com`) | server proxy or `GM_xmlhttpRequest`, with a `page.kakao.com` Referer (`requestHeadersFor`) | That source fails. | | Shynet (`analytics.lucaswinther.info`) | `악역의 엔딩은 죽음뿐 - 웹툰 | 카카오페이지

kakaopage logo

검색 버튼
악역의 엔딩은 죽음뿐

악역의 엔딩은 죽음뿐

78.3만명
2.2만
금 연재 | 독점
수월,권겨을
첫편부터 4편 무료
7일마다 무료
show wait free detail
댓글 (22,085)
좋아요순
|
최신순
노아무열아젶에디레이번칼토이시드
·
2020.05.01
원작 소설 안본 사람들을 위해 말합니다. +악엔죽은 완전 C급입니다. +. +. +. +모든 사람들에게 배포가 시급하거든요. +안 본 사람 없게 해주세요.
악역의 엔딩은 죽음뿐 1화
4264
72
신고
권겨을
·
2020.05.01
안녕하세요! <악역의 엔딩은 죽음뿐> 원작자 권겨을입니다. +드디어 웹툰이 런칭 됐다니 실감이 잘 안납니다ㅠㅠ + +수월 작가님의 멋진 작화로 재탄생한, 게임 속 다섯 남주들 사이에서 살아남기 위해 고군분투하는 페페의 일대기! + +많은 사랑과 관심 부탁드립니다❤ +감사합니다!
악역의 엔딩은 죽음뿐 동영상 트레일러
3471
144
신고
아르
·
1시간전
봤는데 또 대여권 써버렸다ㅠㅜㅜ
악역의 엔딩은 죽음뿐 39화
좋아요
신고
다솜
·
1시간전
수갑을 찬 것은 끊었을 때의 추진력을 얻기 위함이었다
악역의 엔딩은 죽음뿐 24화
좋아요
신고
한편만더보자
·
1시간전
완결없는 대작은 손을 안대는 타입이라서.. 이틀동안 소설 정주행 끝내고 왔네요ㅋㅋㅋ +제 선택은~ 집착노예망사랑연하남입니닷✌
악역의 엔딩은 죽음뿐 프롤로그
좋아요
신고
(∂ω∂)
·
1시간전
남주낙찰
악역의 엔딩은 죽음뿐 22화
좋아요
신고
다음생엔여주로...
·
2시간전
이클리스는 왠지 친해지는게아닌 큰 댕댕이를 길들려야할것 같아....
악역의 엔딩은 죽음뿐 23화
1
신고
리아라
·
2시간전
오 봐주고 있었던 거네
악역의 엔딩은 죽음뿐 24화
좋아요
신고
벗어
·
4시간전
내가 활약할 날만 기다림
악역의 엔딩은 죽음뿐 25화
좋아요
신고
빨간망토차차❤️
·
4시간전
아니 식기 안보이냐? 눈 장식이네
악역의 엔딩은 죽음뿐 39화
좋아요
신고
웹툰덕후
·
5시간전
이클리스 때린 놈 나와!!!아주 그냥!!
악역의 엔딩은 죽음뿐 23화
1
신고
오늘밤너의겨털세가닥을가져가겠다
·
7시간전
아..주인님 소리 너무 좋아 귀여워 쓰담쓰담 해주고싶다 키우고싶다
악역의 엔딩은 죽음뿐 25화
3
신고
Fantana
·
7시간전
넌 나에게 목욕값을 줬어~~
악역의 엔딩은 죽음뿐 12화
1
신고
어공주와왕딸,그오빠는표절작이다
·
7시간전
애정하는 딸이 갑자기 사라져서 찾은거야? +그럴거면 처음부터 잘했어야지 +애가 본인한테 맞춰주니까 애지중지하지말고 +내가 꼬인것도 있지만 행동이 1초에 한번씩 바뀌는것같은 사람도 문제이겠지
악역의 엔딩은 죽음뿐 25화
2
신고
미안..이클리스가조아...
·
8시간전
아.. 잘못눌렀어..ㅜㅜ 봤던거 또 봤어..ㅜㅜ 다시 일주일 기다료야대ㅜㅜ 으엉~~
악역의 엔딩은 죽음뿐 10화
좋아요
신고
라즈베리
·
9시간전
작화너무이뻐요 ㅠ ㅠ
악역의 엔딩은 죽음뿐 12화
좋아요
신고
감정이입너무심한웹툰웹소설중독자
·
9시간전
아 진짜 역시 게임속의 오빠는 현실과 거리가 멀고도 멀구나
악역의 엔딩은 죽음뿐 21화
좋아요
신고
download link
(주)카카오페이지 사업자 정보
|
사업자 정보 확인

대표자 : 이진수 ᅵ 호스팅서비스사업자 : (주)카카오페이지

사업자 등록 번호 : 220-88-02594 ᅵ 통신판매업신고 : 2015-경기성남-0665

대표전화 : 1644-4755 ᅵ 이메일 : contact@kakaopage.com

주소 : 경기도 성남시 분당구 판교역로 221

ⓒ Kakao Page Corp.
ok mark

저작권 ok

\ No newline at end of file diff --git a/test/fixtures/wayback/kakaopage-54845189-2022.html b/test/fixtures/wayback/kakaopage-54845189-2022.html new file mode 100644 index 0000000..11b0a47 --- /dev/null +++ b/test/fixtures/wayback/kakaopage-54845189-2022.html @@ -0,0 +1,28 @@ +악역의 엔딩은 죽음뿐 - 웹툰 | 카카오페이지
카카오페이지 로고
홈웹툰웹소설책
검색
스테이지보관함내 정보
님다음
0
충전
아이콘
알림
아이콘
캐시 PASS VIP
컬러모드다크모드
로그아웃
로딩중
pc 앱 다운로드
플레이 스토어앱 스토어
\ No newline at end of file diff --git a/test/fixtures/wayback/kakaopage-54845189-2024.html b/test/fixtures/wayback/kakaopage-54845189-2024.html new file mode 100644 index 0000000..c681721 --- /dev/null +++ b/test/fixtures/wayback/kakaopage-54845189-2024.html @@ -0,0 +1,29 @@ +악역의 엔딩은 죽음뿐 - 웹툰 | 카카오페이지
추천탭, 총 5개 중 1번째웹툰탭, 총 5개 중 2번째웹소설탭, 총 5개 중 3번째책탭, 총 5개 중 4번째바로가기탭, 총 5개 중 5번째
로딩중
\ No newline at end of file diff --git a/test/fixtures/wayback/kakaopage-54845189-cdx-content.txt b/test/fixtures/wayback/kakaopage-54845189-cdx-content.txt new file mode 100644 index 0000000..9a022b3 --- /dev/null +++ b/test/fixtures/wayback/kakaopage-54845189-cdx-content.txt @@ -0,0 +1,48 @@ +20220930003210 https://page.kakao.com/content/54845189 +20221008033151 https://page.kakao.com/content/54845189 +20221101090844 https://page.kakao.com/content/54845189 +20221205234140 https://page.kakao.com/content/54845189 +20230324163551 https://page.kakao.com/content/54845189 +20230531233552 https://page.kakao.com/content/54845189 +20231001142338 https://page.kakao.com/content/54845189 +20240226032102 https://page.kakao.com/content/54845189 +20240318073658 https://page.kakao.com/content/54845189 +20240523032432 https://page.kakao.com/content/54845189 +20240601102321 https://page.kakao.com/content/54845189 +20240701122409 https://page.kakao.com/content/54845189 +20240801000811 https://page.kakao.com/content/54845189 +20240901010746 https://page.kakao.com/content/54845189 +20241001105828 https://page.kakao.com/content/54845189 +20241101010508 https://page.kakao.com/content/54845189 +20241203053728 https://page.kakao.com/content/54845189 +20250314051917 https://page.kakao.com/content/54845189 +20250406172928 https://page.kakao.com/content/54845189 +20250530014613 https://page.kakao.com/content/54845189 +20250714081524 https://page.kakao.com/content/54845189 +20250905000510 https://page.kakao.com/content/54845189 +20251007180706 https://page.kakao.com/content/54845189 +20251105010252 https://page.kakao.com/content/54845189 +20260115194553 https://page.kakao.com/content/54845189 +20260219143912 https://page.kakao.com/content/54845189 +20260510090433 https://page.kakao.com/content/54845189 +20260815043603 https://page.kakao.com/content/54845189 +20250116002021 https://page.kakao.com/content/54845189%0A +20260222073655 https://page.kakao.com/content/54845189/viewer/54846128 +20221112093759 https://page.kakao.com/content/54845189/viewer/54848086 +20260218104652 https://page.kakao.com/content/54845189/viewer/54848086 +20221112093801 https://page.kakao.com/content/54845189/viewer/54848086?utm_source=68413516352 +20260116045007 https://page.kakao.com/content/54845189/viewer/54872355 +20230316171512 https://page.kakao.com/content/54845189/viewer/60579636 +20230505142706 https://page.kakao.com/content/54845189/viewer/61824808 +20241220095541 https://page.kakao.com/content/54845189?utm_source=sh_clip&plpu=https%3A%2F%2Flink-page.kakao.com%2Fopen%2Fcontent%3Fseries_id%3D54845189%26utm_source%3Dsh_clip%26alfallback%3Dstore +20260908183105 https://page.kakao.com/content/54845189?orderby=desc +20221013131308 https://page.kakao.com/content/54845189?utm_source=sh_clip&pscme=kakaopage%3A%2F%2Fopen%2Fcontent%3Fseries_id%3D54845189%26utm_source%3Dsh_clip +20250813010346 https://page.kakao.com/content/54845189?tab_type=about +20260613024948 https://page.kakao.com/content/54845189?tab_type=about +20241204083755 https://page.kakao.com/content/54845189?tab_type=notice +20250813014455 https://page.kakao.com/content/54845189?tab_type=notice +20260317095133 https://page.kakao.com/content/54845189?tab_type=notice +20250812030043 https://page.kakao.com/content/54845189?tab_type=overview +20260119153727 https://page.kakao.com/content/54845189?tab_type=overview +20260224113711 https://page.kakao.com/content/54845189?tab_type=overview +20240522172309 https://page.kakao.com/content/54845189?tab_type=product diff --git a/test/fixtures/wayback/kakaopage-54845189-cdx-home.txt b/test/fixtures/wayback/kakaopage-54845189-cdx-home.txt new file mode 100644 index 0000000..1efdcd5 --- /dev/null +++ b/test/fixtures/wayback/kakaopage-54845189-cdx-home.txt @@ -0,0 +1,17 @@ +20200928100812 https://page.kakao.com/home?seriesId=54845189 +20201001065424 https://page.kakao.com/home?seriesId=54845189 +20201101082642 https://page.kakao.com/home?seriesId=54845189 +20210402034928 https://page.kakao.com/home?seriesId=54845189 +20210523164837 https://page.kakao.com/home?seriesId=54845189 +20211210081733 https://page.kakao.com/home?seriesId=54845189 +20220506064341 https://page.kakao.com/home?seriesId=54845189 +20220608173932 https://page.kakao.com/home?seriesId=54845189 +20220902101346 https://page.kakao.com/home?seriesId=54845189 +20210906090332 https://page.kakao.com/home?seriesId=54845189&utm_source=tw&utm_medium=thread&utm_campaign=09&utm_id=kakao +20210825154421 https://page.kakao.com/home?seriesId=54845189&utm_source=tw&utm_medium=thread&utm_campaign=endingdeath&utm_id=kakao +20210903094133 https://page.kakao.com/home?seriesId=54845189&utm_source=tw&utm_medium=thread&utm_campaign=endingdeath&utm_id=kakao +20210825185306 https://page.kakao.com/home?seriesId=54845189&utm_id=kakao +20220331090657 https://page.kakao.com/home?seriesId=54845189&utm_source=%E3%85%81%E3%84%B4%E3%84%B7%E3%84%B9 +20211201094036 https://page.kakao.com/home?seriesId=54845189&utm_source=11 +20220331122250 https://page.kakao.com/home?seriesId=54845189&utm_source=23%E3%85%87 +20220601111259 https://page.kakao.com/home?seriesId=54845189&utm_source=5968 diff --git a/test/gm.test.ts b/test/gm.test.ts index 41a439d..9dc35c7 100644 --- a/test/gm.test.ts +++ b/test/gm.test.ts @@ -26,6 +26,12 @@ describe("gmTransport", () => { expect(requests[0]!.headers!["user-agent"]).toMatch(/^Mozilla/); }); + test("KakaoPage's data endpoint gets its page as Referer", async () => { + const { gm, requests } = fakeGm((r) => r.onload({ status: 200, statusText: "OK", responseHeaders: "", response: new ArrayBuffer(0) })); + await gmTransport(gm).fetch("https://bff-page.kakao.com/api/gateway/api/v1/content/overview?series_id=1"); + expect(requests[0]!.headers!.referer).toBe("https://page.kakao.com/"); + }); + test("Wayback gets the honest User-Agent", async () => { const { gm, requests } = fakeGm((r) => r.onload({ status: 200, statusText: "OK", responseHeaders: "", response: new ArrayBuffer(0) })); await gmTransport(gm).fetch("https://web.archive.org/cdx/search/cdx?url=x"); @@ -65,7 +71,7 @@ test("banner: matches mangabaka.org, grants GM APIs, connects to every host it n expect(banner).toContain("// @version 0.1.0\n"); expect(banner).toContain("// @match https://mangabaka.org/*\n"); for (const grant of ["GM_xmlhttpRequest", "GM_getValue", "GM_setValue", "GM_deleteValue"]) expect(banner).toContain(`// @grant ${grant}\n`); - for (const host of ["api.mangabaka.org", "webtoons.com", "pstatic.net", "tapas.io", "d30womf5coomej.cloudfront.net", "web.archive.org", "catbox.moe", "mbseed.example.org"]) { + for (const host of ["api.mangabaka.org", "webtoons.com", "pstatic.net", "tapas.io", "d30womf5coomej.cloudfront.net", "kakao.com", "kakaoentcdn.com", "web.archive.org", "catbox.moe", "mbseed.example.org"]) { expect(banner).toContain(`// @connect ${host}\n`); } expect(banner).toEndWith("// ==/UserScript==\n\n"); diff --git a/test/kakaopage-parse.test.ts b/test/kakaopage-parse.test.ts new file mode 100644 index 0000000..2659f8a --- /dev/null +++ b/test/kakaopage-parse.test.ts @@ -0,0 +1,43 @@ +import { expect, test } from "bun:test"; +import { extractCovers, parseOverview } from "../src/core/sources/kakaopage/parse"; +import { fixtureJson, fixtureText } from "./helpers/fixtures"; + +const IMG = "https://page-images.kakaoentcdn.com/download/resource?kid="; + +test("overview of a comic: title, language, age grade, portrait and landscape", async () => { + const o = parseOverview(await fixtureJson("kakaopage/overview-54845189.json")); + expect(o).toMatchObject({ id: "54845189", title: "악역의 엔딩은 죽음뿐", language: "ko", ageGrade: 0, novel: false }); + expect(o.covers.map((c) => [c.role, c.url])).toEqual([ + ["poster", `${IMG}mW3j5/dJMcaacSrAK/WVoJJAYNlS9rayXUWi6AY0&filename=o1`], + ["banner", `${IMG}RmobN/hzCtfKUqaD/6XgE5RjbBGMOn6thodamr0&filename=o1`], + ]); +}); + +test("a novel, and a 19+ edition without a landscape image", async () => { + expect(parseOverview(await fixtureJson("kakaopage/overview-52654302-novel.json"))).toMatchObject({ id: "52654302", novel: true }); + const adult = parseOverview(await fixtureJson("kakaopage/overview-63659962-19.json")); + expect(adult).toMatchObject({ id: "63659962", ageGrade: 19 }); + expect(adult.covers.map((c) => c.role)).toEqual(["poster"]); +}); + +test("an answer without content is an error naming the reason", () => { + expect(() => parseOverview({ result: null, message: "상품 정보를 찾을 수 없습니다." })).toThrow(/상품 정보를 찾을 수 없습니다/); + expect(() => parseOverview("nonsense")).toThrow(/KakaoPage/); +}); + +test("2020 capture (old address): og:image plus the series' own thumbnails", async () => { + const covers = extractCovers(await fixtureText("wayback/kakaopage-54845189-2020.html"), new URL("https://page.kakao.com/home?seriesId=54845189")); + expect(covers.map((c) => [c.role, c.url])).toEqual([ + ["poster", `${IMG}dNtPVq/hyCvfq21v6/b4EGtZDhGNKvzdBjWXg290&filename=o1`], + ["banner", `${IMG}bx4cET/hyATqBNkj9/kUeJCNRNNpYDYRBtpK1Oh1&filename=o1`], + ]); +}); + +test("2022 and 2024 captures: the og:image only (other series on the page are ignored)", async () => { + const at = (y: number) => fixtureText(`wayback/kakaopage-54845189-${y}.html`); + const page = new URL("https://page.kakao.com/content/54845189"); + expect(extractCovers(await at(2022), page).map((c) => [c.role, c.url])).toEqual([["other", `${IMG}RmobN/hzCtfKUqaD/6XgE5RjbBGMOn6thodamr0&filename=o1`]]); + const y2024 = extractCovers(await at(2024), page); + expect(y2024.map((c) => [c.role, c.url])).toEqual([["other", `${IMG}dBnFgi/hzXap8fqJ4/Kzjixkeeek6kH1eOyRb490&filename=o1`]]); + expect(y2024[0]!.originalUrl).toBe("https://page-images.kakaoentcdn.com/download/resource?kid=dBnFgi/hzXap8fqJ4/Kzjixkeeek6kH1eOyRb490&filename=o1"); +}); diff --git a/test/kakaopage-source.test.ts b/test/kakaopage-source.test.ts new file mode 100644 index 0000000..c89c128 --- /dev/null +++ b/test/kakaopage-source.test.ts @@ -0,0 +1,62 @@ +import { expect, test } from "bun:test"; +import { waybackHistory } from "../src/core/history/wayback"; +import { allowedFetchHosts, allowedImageHosts, findSource } from "../src/core/sources/registry"; +import { SourceError } from "../src/core/sources/source"; +import { kakaopage } from "../src/core/sources/kakaopage"; +import { FakeTransport, json, text } from "./helpers/fake-transport"; +import { fixtureJson, fixtureText } from "./helpers/fixtures"; + +const API = "https://bff-page.kakao.com/api/gateway/api/v1/content/overview?series_id="; +const overview = (name: string) => async () => json(await fixtureJson(`kakaopage/${name}.json`)); + +test("registered; its hosts are allowlisted", () => { + expect(findSource(new URL("https://page.kakao.com/content/54845189"))?.id).toBe("kakaopage"); + for (const host of ["page.kakao.com", "bff-page.kakao.com", "page-images.kakaoentcdn.com", "dn-img-page.kakao.com"]) expect(allowedFetchHosts().has(host)).toBe(true); + for (const host of ["page-images.kakaoentcdn.com", "dn-img-page.kakao.com"]) expect(allowedImageHosts().has(host)).toBe(true); +}); + +test("resolve reads the overview; currentCovers reuses it", async () => { + const fake = new FakeTransport().on(`${API}54845189`, overview("overview-54845189")); + const series = await kakaopage.resolve(new URL("https://page.kakao.com/home?seriesId=54845189"), { transport: fake }); + expect(series).toEqual({ + source: "kakaopage", externalId: "54845189", title: "악역의 엔딩은 죽음뿐", language: "ko", + canonicalUrl: "https://page.kakao.com/content/54845189", + archiveUrls: ["page.kakao.com/content/54845189", "page.kakao.com/home?seriesId=54845189"], + mature: false, flags: { novel: false }, + }); + const covers = await kakaopage.currentCovers(series, { transport: fake }); + expect(fake.calls.length).toBe(1); + expect(covers.map((c) => [c.role, c.current])).toEqual([["poster", true], ["banner", true]]); +}); + +test("19+ is mature; a novel is flagged", async () => { + const fake = new FakeTransport().on(`${API}63659962`, overview("overview-63659962-19")).on(`${API}52654302`, overview("overview-52654302-novel")); + expect((await kakaopage.resolve(new URL("https://page.kakao.com/content/63659962"), { transport: fake })).mature).toBe(true); + expect((await kakaopage.resolve(new URL("https://page.kakao.com/content/52654302"), { transport: fake })).flags.novel).toBe(true); +}); + +test("an id KakaoPage doesn't know, and an HTTP error", async () => { + const fake = new FakeTransport().on(`${API}53998542`, overview("overview-not-found")).on(`${API}1`, () => text("blocked", 403)); + await expect(kakaopage.resolve(new URL("https://page.kakao.com/content/53998542"), { transport: fake })).rejects.toThrow(/상품 정보를 찾을 수 없습니다/); + await expect(kakaopage.resolve(new URL("https://page.kakao.com/content/1"), { transport: fake })).rejects.toBeInstanceOf(SourceError); +}); + +test("Wayback history through both addresses keeps only this id", async () => { + const fake = new FakeTransport().on(`${API}54845189`, overview("overview-54845189")) + .on((u) => u.startsWith("https://web.archive.org/cdx/") && u.includes("home"), async () => text(await fixtureText("wayback/kakaopage-54845189-cdx-home.txt"))) + .on((u) => u.startsWith("https://web.archive.org/cdx/"), async () => text(await fixtureText("wayback/kakaopage-54845189-cdx-content.txt"))) + .on((u) => /\/web\/2020\d+id_\//.test(u), async () => text(await fixtureText("wayback/kakaopage-54845189-2020.html"))) + .on((u) => /\/web\/202[12]\d+id_\//.test(u), async () => text(await fixtureText("wayback/kakaopage-54845189-2022.html"))) + .on((u) => /\/web\/20\d+id_\//.test(u), async () => text(await fixtureText("wayback/kakaopage-54845189-2024.html"))); + const series = await kakaopage.resolve(new URL("https://page.kakao.com/content/54845189"), { transport: fake }); + const found = await waybackHistory(kakaopage, series, { every: 12, transport: fake, warn: () => {} }); + const snapshots = fake.urls().filter((u) => u.includes("id_/")); + expect(snapshots.length).toBeGreaterThan(3); + expect(snapshots.every((u) => /seriesId=54845189(&|$)|content\/54845189(\/|$|\?)/.test(u))).toBe(true); + expect(new Set(found.map((c) => c.url))).toEqual(new Set([ + "https://page-images.kakaoentcdn.com/download/resource?kid=dNtPVq/hyCvfq21v6/b4EGtZDhGNKvzdBjWXg290&filename=o1", + "https://page-images.kakaoentcdn.com/download/resource?kid=bx4cET/hyATqBNkj9/kUeJCNRNNpYDYRBtpK1Oh1&filename=o1", + "https://page-images.kakaoentcdn.com/download/resource?kid=RmobN/hzCtfKUqaD/6XgE5RjbBGMOn6thodamr0&filename=o1", + "https://page-images.kakaoentcdn.com/download/resource?kid=dBnFgi/hzXap8fqJ4/Kzjixkeeek6kH1eOyRb490&filename=o1", + ])); +}); diff --git a/test/kakaopage-urls.test.ts b/test/kakaopage-urls.test.ts new file mode 100644 index 0000000..efebf46 --- /dev/null +++ b/test/kakaopage-urls.test.ts @@ -0,0 +1,49 @@ +import { describe, expect, test } from "bun:test"; +import { LinkError } from "../src/core/links"; +import { isKakaoPageUrl, isSameKakaoSeries, normalizeImageUrl, overviewUrl, parseKakaoPageUrl } from "../src/core/sources/kakaopage/urls"; + +const u = (s: string) => new URL(s); + +describe("links", () => { + test("both addresses, with or without the mobile host and extras", () => { + expect(parseKakaoPageUrl(u("https://page.kakao.com/content/54845189"))).toBe("54845189"); + expect(parseKakaoPageUrl(u("https://page.kakao.com/content/54845189/viewer/1?tab=episode"))).toBe("54845189"); + expect(parseKakaoPageUrl(u("https://m.page.kakao.com/home?seriesId=54845189&utm_source=tw"))).toBe("54845189"); + }); + + test("anything else is not a KakaoPage series", () => { + for (const s of ["https://page.kakao.com/", "https://page.kakao.com/content/abc", "https://page.kakao.com/home", "https://webtoon.kakao.com/content/x/2383"]) { + expect(isKakaoPageUrl(u(s))).toBe(false); + expect(() => parseKakaoPageUrl(u(s))).toThrow(LinkError); + } + }); + + test("the data endpoint", () => { + expect(overviewUrl("54845189")).toBe("https://bff-page.kakao.com/api/gateway/api/v1/content/overview?series_id=54845189"); + }); +}); + +describe("image URLs", () => { + test("the original on page-images.kakaoentcdn.com", () => { + const original = "https://page-images.kakaoentcdn.com/download/resource?kid=mW3j5/dJMcaacSrAK/WVoJJAYNlS9rayXUWi6AY0&filename=o1"; + expect(normalizeImageUrl("https://page-images.kakaoentcdn.com/download/resource?kid=mW3j5/dJMcaacSrAK/WVoJJAYNlS9rayXUWi6AY0&filename=o1/dims/resize/384")).toBe(original); + expect(normalizeImageUrl("//dn-img-page.kakao.com/download/resource?kid=mW3j5/dJMcaacSrAK/WVoJJAYNlS9rayXUWi6AY0&filename=th3", "https://page.kakao.com/content/1")).toBe(original); + expect(normalizeImageUrl("http://dn-img-page.kakao.com/download/resource?kid=mW3j5/dJMcaacSrAK/WVoJJAYNlS9rayXUWi6AY0&filename=th2".replace("&", "&"))).toBe(original); + }); + + test("a bare key becomes the original URL; other hosts are left alone", () => { + expect(normalizeImageUrl("bx4cET/hyATqBNkj9/kUeJCNRNNpYDYRBtpK1Oh1")) + .toBe("https://page-images.kakaoentcdn.com/download/resource?kid=bx4cET/hyATqBNkj9/kUeJCNRNNpYDYRBtpK1Oh1&filename=o1"); + expect(normalizeImageUrl("https://example.org/x.jpg?filename=th3")).toBe("https://example.org/x.jpg?filename=th3"); + }); +}); + +describe("same series", () => { + test("the exact id, in either address", () => { + expect(isSameKakaoSeries("https://page.kakao.com/content/54845189", "54845189")).toBe(true); + expect(isSameKakaoSeries("https://page.kakao.com/home?seriesId=54845189&utm_source=tw", "54845189")).toBe(true); + expect(isSameKakaoSeries("https://page.kakao.com/home?seriesId=548451890", "54845189")).toBe(false); + expect(isSameKakaoSeries("https://page.kakao.com/content/548451890", "54845189")).toBe(false); + expect(isSameKakaoSeries("nope", "54845189")).toBe(false); + }); +}); diff --git a/test/server-app.test.ts b/test/server-app.test.ts index 20c7377..0917ce3 100644 --- a/test/server-app.test.ts +++ b/test/server-app.test.ts @@ -55,6 +55,13 @@ describe("/api/fetch", () => { expect(new Headers(calls[1]!.init!.headers).get("user-agent")).toBe(TOOL_USER_AGENT); }); + test("KakaoPage's data endpoint is fetched with a page.kakao.com Referer", async () => { + const API = "https://bff-page.kakao.com/api/gateway/api/v1/content/overview?series_id=54845189"; + const { call, calls } = setup({}, { [API]: () => Response.json({ result: {} }) }); + expect((await call(fetchPath(API))).status).toBe(200); + expect(new Headers(calls[0]!.init!.headers).get("referer")).toBe("https://page.kakao.com/"); + }); + test.each(["https://evil.example/x", "http://169.254.169.254/latest", "ftp://www.webtoons.com/x", "https://www.webtoons.com:8443/x", "not a url"])( "%p is refused without being fetched", async (url) => { const { call, calls } = setup(); diff --git a/test/transport.test.ts b/test/transport.test.ts index b3e690f..c5b54e8 100644 --- a/test/transport.test.ts +++ b/test/transport.test.ts @@ -102,3 +102,12 @@ test("Wayback gets the honest User-Agent, everything else a browser one", () => expect(TOOL_USER_AGENT).toMatch(/^mangabaka-seeder\/\d+\.\d+\.\d+ \(\+https:\/\//); expect(userAgentFor("https://www.webtoons.com/en/x")).toMatch(/^Mozilla\/5\.0/); }); + +test("request headers per host: KakaoPage's data endpoint gets its page as Referer", async () => { + const { requestHeadersFor } = await import("../src/core/transport"); + expect(requestHeadersFor("https://bff-page.kakao.com/api/gateway/api/v1/content/overview?series_id=1")).toEqual({ + "user-agent": expect.stringMatching(/^Mozilla/), referer: "https://page.kakao.com/", + }); + expect(requestHeadersFor("https://www.webtoons.com/x")).toEqual({ "user-agent": expect.stringMatching(/^Mozilla/) }); + expect(requestHeadersFor("https://web.archive.org/cdx/search/cdx?url=x")).toEqual({ "user-agent": TOOL_USER_AGENT }); +}); diff --git a/test/webtoons-source.test.ts b/test/webtoons-source.test.ts index 6b8297a..ebb8bb6 100644 --- a/test/webtoons-source.test.ts +++ b/test/webtoons-source.test.ts @@ -68,8 +68,9 @@ test("registry finds the source and builds the proxy allowlist", () => { expect(findSource(new URL("https://www.webtoons.com/en/x/list?title_no=1"))).toBe(webtoons); expect(findSource(new URL("https://www.lezhin.com/en/comic/x"))).toBeUndefined(); expect([...allowedFetchHosts()].sort()).toEqual( - ["d30womf5coomej.cloudfront.net", "m.tapas.io", "m.webtoons.com", "swebtoon-phinf.pstatic.net", "tapas.io", "us-a.tapas.io", - "web.archive.org", "www.tapas.io", "www.webtoons.com"]); + ["bff-page.kakao.com", "d30womf5coomej.cloudfront.net", "dn-img-page.kakao.com", "m.page.kakao.com", "m.tapas.io", "m.webtoons.com", + "page-images.kakaoentcdn.com", "page.kakao.com", "swebtoon-phinf.pstatic.net", "tapas.io", "us-a.tapas.io", "web.archive.org", + "www.tapas.io", "www.webtoons.com"]); }); test("isSameSeries checks title_no", () => {