feat(core): jobs, scheduler, plugin registry, connections

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This commit is contained in:
Lucas Winther
2026-10-30 20:39:17 +01:00
co-authored by Claude Sonnet 5.5
parent 207c92bb4d
commit 166b8ecb5e
20 changed files with 744 additions and 10 deletions
+4 -2
View File
@@ -24,6 +24,8 @@ Small, specific, independently actionable. Remove only when actually done.
file, and compare steps and token use.
## Known issues
- [ ] Jobs: a job left `running` by a crash is never re-queued; add a stale-lock reaper using `locked_at`.
- [ ] Plugins: `ctx.log` is discarded; route it into `source_runs.log`.
- [ ] Restore: a new child work whose parent+position collides with an existing child of a re-matched parent fails
with `ImportError("conflict")` instead of being merged or repositioned.
- [ ] Restore: an unlink of identical attachment bytes between the pre-write and the commit could remove the file;
@@ -35,8 +37,8 @@ Small, specific, independently actionable. Remove only when actually done.
another connection still reports it; consider detaching that connection's `engagement_sources` row instead.
- [ ] Sync: pending `changed`/`conflict` events for a value the source has since moved past stay pending;
consider superseding them when a newer value for the same field arrives.
- [ ] Sync: two concurrent runs of the same connection can both insert a new `source_records` row for one key
(one fails on the unique key and is logged as a record error). The job worker should lock per connection.
- [ ] Sync: two concurrent runs of the same connection (outside the job queue, e.g. a manual run) can both insert a new
`source_records` row for one key. Queued syncs are serialised by `lockKey`.
- [ ] Sync: `ingestListens` does one lookup and one insert per listen; batch it for large backfills.
- [ ] Admin merge does not merge colliding children recursively; colliding children stay under the tombstoned parent.
- [ ] Matching: a shared parent resolved without external IDs or position (title/year reuse in