Lucas WintherandClaude Opus 5
b085087b05
The feed was generated from checked-in fixtures and only moved when somebody captured a page by hand. This fetches. bun run refresh caches each page raw under snapshots/ and rebuilds the feed from what it cached; build-feed prefers a snapshot and falls back to the fixture, so a clean checkout and the container build stay offline and reproducible. The workflow runs it twice a day and commits only when a page's bytes actually changed — a 304, an identical body or a rejected parse all leave the tree clean — then dispatches ci.yml, which already knows how to test, build and deploy. Scraping conduct is enforced in code rather than left to good intentions: one request per source per cycle, a six-hour floor checked per source, conditional requests, a User-Agent with a contact URL, and robots.txt honoured — failing closed, because a permission we could not read is not a permission we have. No retries; a retry is a second request. A body that yields zero events is rejected and the previous snapshot kept, so a redesigned wiki shows up as a stale timestamp rather than an emptied calendar. One source down is a warning; all of them down fails the run, so a cycle that learned nothing is never committed. Tested entirely offline against an injected fetch and clock — no request has ever been made to a live wiki from this code. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
129 lines
2.0 KiB
Plaintext
129 lines
2.0 KiB
Plaintext
# Logs
|
|
logs
|
|
*.log
|
|
npm-debug.log*
|
|
yarn-debug.log*
|
|
yarn-error.log*
|
|
lerna-debug.log*
|
|
|
|
# database
|
|
*.sqlite
|
|
|
|
# package-json
|
|
package-json.lock
|
|
|
|
# bun deploy file
|
|
node_modules.bun
|
|
|
|
# Diagnostic reports (https://nodejs.org/api/report.html)
|
|
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
|
|
|
|
# Runtime data
|
|
pids
|
|
*.pid
|
|
*.seed
|
|
*.pid.lock
|
|
|
|
# Directory for instrumented libs generated by jscoverage/JSCover
|
|
lib-cov
|
|
|
|
# Coverage directory used by tools like istanbul
|
|
coverage
|
|
*.lcov
|
|
|
|
# nyc test coverage
|
|
.nyc_output
|
|
|
|
# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
|
|
.grunt
|
|
|
|
# Bower dependency directory (https://bower.io/)
|
|
bower_components
|
|
|
|
# node-waf configuration
|
|
.lock-wscript
|
|
|
|
# Compiled binary addons (https://nodejs.org/api/addons.html)
|
|
build/Release
|
|
|
|
# Dependency directories
|
|
node_modules/
|
|
jspm_packages/
|
|
|
|
# TypeScript v1 declaration files
|
|
typings/
|
|
|
|
# TypeScript cache
|
|
*.tsbuildinfo
|
|
|
|
# Optional npm cache directory
|
|
.npm
|
|
|
|
# Optional eslint cache
|
|
.eslintcache
|
|
|
|
# Microbundle cache
|
|
.rpt2_cache/
|
|
.rts2_cache_cjs/
|
|
.rts2_cache_es/
|
|
.rts2_cache_umd/
|
|
|
|
# Optional REPL history
|
|
.node_repl_history
|
|
|
|
# Output of 'npm pack'
|
|
*.tgz
|
|
|
|
# Yarn Integrity file
|
|
.yarn-integrity
|
|
|
|
# dotenv environment variables file
|
|
.env
|
|
.env.test
|
|
|
|
# parcel-bundler cache (https://parceljs.org/)
|
|
.cache
|
|
|
|
# Next.js build output
|
|
.next
|
|
|
|
# Nuxt.js build / generate output
|
|
.nuxt
|
|
dist
|
|
|
|
# Gatsby files
|
|
.cache/
|
|
# Comment in the public line in if your project uses Gatsby and *not* Next.js
|
|
# https://nextjs.org/blog/next-9-1#public-directory-support
|
|
# public
|
|
|
|
# vuepress build output
|
|
.vuepress/dist
|
|
|
|
# Serverless directories
|
|
.serverless/
|
|
|
|
# FuseBox cache
|
|
.fusebox/
|
|
|
|
# DynamoDB Local files
|
|
.dynamodb/
|
|
|
|
# TernJS port file
|
|
.tern-port
|
|
|
|
# IDE
|
|
.idea/
|
|
|
|
# runtime data
|
|
data/
|
|
|
|
# build output
|
|
public/
|
|
|
|
# refresh bookkeeping (when we last checked a source).
|
|
# The snapshots themselves ARE tracked — they are what the feed is built from —
|
|
# but this file changes on every cycle, and committing it would mean a commit
|
|
# per run saying nothing changed.
|
|
snapshots/*.state.json
|