rclone/docs/plans/001-protondrive-tier1-workspace/task-18-review.md

Task 18 review (d211b45e6..28f1aaf8a)

Spec Compliance

• ✅ Spec compliant, with the handoff corrections applied. Every listed file has a hunk: init.d script, PORTS.md (28642, next free), config.yaml entry (fastlist: false), .gitea/workflows/ci.yml, TODO.md, PERMISSIONS.md. The plan's step 1 allowed ignoretests only for bisync/gitannex; the extra ignore of five tests is a deviation the implementer flagged and the brief's "fix backend failures until clean" could not satisfy. I judge it acceptable (see cache analysis) but it is a decision for the owner.
• ⚠️ Cannot verify: ci.yml on a real Gitea runner (label ubuntu-latest, uses: resolution of actions/* and golangci/golangci-lint-action, network for govulncheck). TODO.md records this, as required. .claude/settings.json not edited: correct, not counted.

Strengths

• Init script lifecycle is sound. I ran it in a scratch STATE_DIR (RCLONE_CONFIG=/notfound, nothing but loopback):
  • start in under 1s, six key=value lines plus _connect; a second start reuses the server (refcount 2); first stop keeps it, second stop kills it; no protonfake and nothing on 28642 afterwards.
  • build failure (fake go that exits 1): failed to start, rc=1, no process left.
  • port already taken: protonfake exits, script reports its log and failed to start, rc=1, no orphan.
  • shellcheck: only SC2034 (unused loop var i), warning level.
• Built binary instead of go run, so the pid is the server's. Output goes to a file and stderr to a log, stdin is /dev/null, so CombinedOutput in testserver.run does not hang on an inherited pipe.
• Repo root derived from the script location (dirname "$0"/../../..); works from any package directory.
• Cannot reach live remotes: the fake is a separate remote name TestProtonDriveFake: configured only through RCLONE_CONFIG_TESTPROTONDRIVEFAKE_* env (distinct prefix from ..._TESTPROTONDRIVE_...), endpoint is 127.0.0.1, CI sets RCLONE_CONFIG=/notfound. The live entry is untouched.
• DECISIONS.md entry for the shared session is accurate and records the revisit condition. TODO entries are specific and actionable. Stale "no Go on host" text removed from PERMISSIONS.md.

Cache diagnosis (the five ignored tests)

Diagnosis is right. nodeCache (backend/protondrive/cache.go:12-19) is per client/Fs (nodes.go:189). listChildren (nodes.go:753-757) and findChild (nodes.go:894-905) trust a cached complete listing, and a miss in it returns errNotFound with no server check. cache.add (cache.go:140-157) appends only to this cache's listing. So Fs B holding a complete listing of folder X never learns of a file Fs A creates in X. There is no TTL or any expiry anywhere in cache.go; only invalidate*/clear and the Move/DirMove cross-Fs invalidation.

Real user impact

• --compare-dest / --copy-dest (+--immutable) / checksum --download: only wrong when the reference Fs's folder is written through another Fs of the same account in the same process, after the reference Fs listed it. Normal use (reference dir is pre-existing and untouched) is fine. Overlapping rclone sync remote:a remote:a/sub style commands are the realistic hit. Low frequency.
• Mount + a second rclone command: this is a separate, larger staleness, not the one the tests hit. Different processes share nothing, and the mount's cache has no TTL and no events (M5 absent), so the mount never sees external writes until restart; VFS --dir-cache-time refreshes call List, which is served from the backend cache. The option help (protondrive.go:189-197) and docs/content/protondrive.md warn about it, and enable_caching defaults to true. The implementer's TODO entry does not mention it; the user docs (Task 20) should keep the mount warning.
• Wrong-answer severity: a stale miss makes --compare-dest re-copy a file (safe) but --copy-dest --immutable and Put-after-miss paths can attempt a create that the server rejects with 2500 (exists) rather than silently overwrite; I did not run this.

Fix options (recommend, not implement)

1. Cheap and spec-compatible, helps mounts but not the five tests: a listing TTL (e.g. 1 to 5 minutes, only for listings, checked in cache.children). Budget tests run in seconds, so TestBudgetUpload/TestBudgetMkdir/TestBudgetList (warm listing = 0 requests) are unaffected. Needs a clock injected into nodeCache for tests. It does not cover the five tests, which finish in milliseconds.
2. Re-list on miss, whatever flavour (always, age-limited, or only in read paths): breaks the budgets, because operations.Copy calls NewObject (a miss for new files) before every Put; any per-miss server check is a request per new file. A name-hash existence probe (check_available_hashes) is one request per miss: same budget break. An age gate ("only re-list if the listing is older than N ms") would be timing dependent and still fail tests that run within N.
3. A cache shared by the Fs of one account fixes the tests properly but needs registry state keyed by account; the spec's no-package-state rule forbids it. The only spec-compatible shape is something owned by a shared object the Fs already receive, and none exists. Owner decision, as the implementer says.
4. Test-side alternative to ignore: run those five against a second fake remote with enable_caching=false (they pass uncached per the report) so they still cover the backend's uncached path. Costs a second init.d script and port, so probably not worth it before M5.

Recommendation: keep ignore, keep the TODO, add the mount/no-TTL point to the TODO entry, and consider option 1 as a small separate task. Events (M5) are the right real fix.

Issues

Critical

None.

Important

None.

Minor

1. fstest/testserver/init.d/TestProtonDriveFake:55-61 stop (lines 63-76) sends only SIGTERM and after 10 s removes the pid file whether or not the process died, so a hung protonfake would be orphaned and untracked (port stays bound; the next start then fails with "address already in use"). Add kill -9 after the wait loop. Low likelihood (the fake exits on SIGTERM).
2. TestProtonDriveFake:57 unused loop variable i (shellcheck SC2034); use for _ in $(seq ...).
3. TestProtonDriveFake:24 build output goes to stderr, which testserver.run merges with stdout via CombinedOutput and then parses with ^key=value. A go build diagnostic that happened to look like a=b would be exported as an env var. Not realistic; redirecting build output to ${RUN_STATE}/build.log and printing it only on failure would remove the risk.
4. TestProtonDriveFake:24 rebuilds on every first start with a cold go build, and testserver.start has no timeout. On a fresh slow runner with a cold build cache this could take minutes but works. Acceptable.
5. .gitea/workflows/ci.yml:5-7 push: and pull_request: with no filter and no concurrency group: a push to a PR branch runs the job twice, and superseded pushes keep running. On a slow runner this contradicts "light". Add concurrency: {group: ci-${{ github.ref }}, cancel-in-progress: true}, or limit push to master.
6. ci.yml:44-48 fuzz loop: go test -list compiles the test binary, and each of the 5 -fuzz runs links again (about 5 compiles plus 100 s of fuzzing). Fine but not cheap; running all targets as one go test -fuzz is impossible, so this is the minimum. go-version-file: go.mod gives Go 1.26.0, while golangci-lint v2.15.0 is a prebuilt binary; if its build Go is older than the module's go 1.26.0 the lint step fails ("version lower than targeted"). Host lint passed with Go 1.27.4 only. Worth watching on first push.
7. ci.yml:26,51 go run ...govulncheck@latest is unpinned, so the job can break without a code change (brief mandated it). go build ./... builds all of rclone, the heaviest step on a slow runner (brief mandated it).
8. docs/TODO.md "Now" entry: the line Remaining: Task 19 (... is a very long line, unlike the surrounding wrapped text (cosmetic).
9. fstest/test_all/config.yaml: a bare -backends protondrive without -remotes runs both the fake and the live TestProtonDrive: entries. Pre-existing behaviour, but now that two entries exist, PERMISSIONS.md should say always pass -remotes TestProtonDriveFake: (the verified-commands row does).

Assessment

Task quality: Approved

Reasoning: The script's start/stop/refcount/failure paths work as run, nothing can reach the live remotes, and CI plus docs match the brief; remaining items are minor hardening. The five ignored tests are a genuine consequence of per-Fs caches with no expiry, and the implementer's refusal to add package state or a per-miss re-list is correct given the budget tests.