rclone/docs/plans/001-protondrive-tier1-workspace/final-fix-report.md

Final fix wave: implementer report

Branch protondrive-tier1, starting at 29ad26944. One commit per finding. The controller's ea8d09ce4 landed between mine and is not part of this work. Every test ran with RCLONE_CONFIG=/notfound. No dependency changes, nothing pushed, and no edits to the handoff, the workspace copy or spec section 13.

Commits

Commit	Finding
e95317b52	crypto C1: content key signature
db7afafc0	crypto C2: Argon2 caps on node and share keys
b3f5685e6	crypto I1: unverified hash key blocks writes
67e4cfeb7	crypto I2: no session key reuse for an unverified name
94eedc942	crypto I3: unsigned xattrs not used
674b538ca	transfer I-1: stable draft client ID when it can't be saved
788a711dc	transfer I-2: learnt block sizes make multi-thread reads linear
5dbf5b04c	auth I-1: error-code mapping (5001/5003 fatal, 2000 no-retry, 2028 retried)
b4a23a31e	auth I-2: x-pm-drive-requirements logged
b5da26169	tests I-1 and I-2: CI timeout, Go 1.27.x, govulncheck pinned
634a2423e	(found while working) tests honour RCLONE_CONFIG whichever tests run
30cfcac41	auth M-1 and M-2: refresh 5xx/429/408 retried
d3aee2af1	cheap minors: crypto M5 and M6, transfer M-3 and M-8, fake gofmt
bfa9b7122	test minors 1, 6, 8 and 9, tests minors 13 and 14 (docs), every other minor logged in TODO
ad3552e2e	ARCHITECTURE line for the block size cache
944b36005	fake test follows the renamed code constant (found by the final race run)

Per finding

crypto C1: unsigned or invalid content key signature (e95317b52)

• Failing test first: two new hostileContents cases in TestHostileServer ("unsigned content key the server made", "content key signed by another key"), and "ranged reads and updates of a file with content-key-unsigned / content-key-signature". Before the fix the read of an unsigned key returned content (security_test.go:204 An error is expected but got nil), the Update went through (:570), and the per-operation ERROR line was missing.
• Fake: the new corruptions content-key-unsigned and content-key-signature make a new content key and re-encrypt every block under it. The blocks and manifest are re-signed with the user's key, which is more than a real server can do, so only the content key signature can catch it.
• Change: decryptContentKey always verifies, so a missing signature is errSignatureMissing. contentKey(n, op) takes "reading" or "updating". Without the flag it refuses with errIntegrity and an op-specific hint. With the flag it logs an ERROR once per node and operation (reportOnce("content key "+op)). The pre-2023 all-addresses fallback now applies only to a signature that is present but invalid, as in the SDK. This also fixes transfer M-7 (Update logged "reading").
• Existing tests changed: TestContentKeyRoundTrip/NoSignature (now expects errSignatureMissing) and TestDownloadContentKey ("unsigned" is now refused, plus a new "unsigned before 2023" case).
• Deviation: none. There is no date exemption for unsigned keys, because the date comes from the server. Every golden fixture has a signed content key (TestGoldenFixtures passes). Recorded in DECISIONS (ruling 29) and on the docs page.

crypto C2: Argon2 parameters from the server (db7afafc0)

• Failing test first: TestNodeKeyS2KLimits. The patched keys (memory 2^18, passes 255, parallelism 255, and memory 2^31 at the library limit) got no errIntegrity. With a stub check, the memory case ran a 256 MiB KDF in 0.2 s.
• Change: decryptNodeKeys (used for node and share keys) calls unlockForeignKey. That function unarmors once, runs checkKeyS2K on every secret key and subkey packet before the KDF can run, then unlocks. Neither go-crypto nor gopenpgp exposes a key's S2K parameters, so s2kSpecifier finds them by hand. It parses the packet with go-crypto (cheap: no KDF), takes the public key's serialized body as the prefix, then walks the usage, count, cipher, AEAD and length octets the way go-crypto does.
• Caps: Argon2 may use at most 4 passes, parallelism 8 and 2^17 KiB (128 MiB). The defaults of gopenpgp and OpenPGP.js are 3, 4 and 64 MiB, and Proton's node keys use iterated-and-salted S2K anyway. Going over a cap is errIntegrity, and the flag can't override it.
• Iterated-salted: not capped further. go-crypto's one-byte count encoding already bounds it to 65011712 bytes hashed (4 times the default), and a lower cap could refuse genuine old keys.
• Fuzz: FuzzCheckKeyS2K ran 20 s locally (1.3M execs, no panic).
• Deviation: the account's own user keys are not checked. The ruling scopes this to foreign keys, and a genuine user key's parameters aren't known. This is logged in TODO. Address keys are safe, because their token's signature is checked before the unlock. DECISIONS entry added.

crypto I1: unverified hash key used for writes (b3f5685e6)

• Failing test first: "writing into a folder whose hash key the server made". The upload into the folder succeeded before the fix (:272).
• Change: node.hashKeyErr is set when the hash key signature fails and isn't date-exempt. targetFolder refuses without the flag, which covers create, upload, rename (within the folder), move into and copy into. With the flag it logs an ERROR once per folder. Reads are not affected. A comment notes that the date exemptions use the server's CreateTime (crypto M4). Docs page updated.
• Deviation: not folded into keys.unverified. Doing that would make every child unreadable, and the ruling covers writes only.

crypto I2: rename/move reusing an unverified name's session key (67e4cfeb7)

• Failing test first: "renaming or moving a node whose name the server encrypted". The new stored name's session key equalled the server's for both the rename and the move. The fake gained User.NameSessionKey and the name-unsigned corruption.
• Change: node.nameSigned records whether the name signature actually verified, taken before the pre-2021 exemption so that a back-dated name doesn't count. renameSessionKey returns nil, which means a fresh key, unless the name is signed. The same test checks that a verified name keeps its key.
• Deviation: a pre-2021 unsigned (exempt) name also gets a fresh key. This is the safe reading of "did not verify".

crypto I3: unsigned xattrs trusted (94eedc942)

• Failing test first: "unsigned extended attributes". The forged modtime (2099), the forged SHA-1 and the forged 1 TiB size were all used, and the folder's modtime too.
• Change: without the flag, when the xattr signature fails, n.X becomes blockSizesOnly(). The ERROR line says the modification time, size and SHA-1 are not used. With the flag the attributes are used, logged once per node by the existing verify report.
• Deviation: I did not use the plain "xattrs absent" fallback. That fallback reports the encrypted size, so every rclone copy of the file would fail its size check, which in practice makes the file unreadable. The ruling forbids that. Instead I keep only the recorded block sizes, which every read checks block by block, and take Size from their sum. ModTime falls back to the server's ModifyTime and there is no SHA-1. If there are no block sizes, Size is the encrypted size as before. The test checks that full and ranged reads still work. Docs page updated.

transfer I-1: draft_client_uid for env/connection-string remotes (674b538ca)

• Failing test first: TestDraftClientUIDStableWhenUnsaved, using a mapper whose Set does nothing. Two runs of the same remote got different IDs.
• Change: NewFs sets a random ID through m.Set, then reads it back with m.Get. If it didn't stick (env vars or a connection string), it uses client.derivedDraftClientUID(name) = hex(SHA-256("rclone draft client\0" + volumeID + "\0" + name))[:32].
• Deviation: I used the account's volume ID rather than the user ID. The account struct doesn't keep the user ID, and the volume ID is just as stable per account. Config-file remotes still get a random saved ID, so two machines with their own config files don't collide. The docs say that two machines running an env-configured remote of the same name count as one remote.

transfer I-2: quadratic multi-thread reads (788a711dc)

• Option chosen: a bounded per-client cache, not computing sizes from the listing. The packet framing (partial body lengths, the literal packet header and any compression) depends on the writer, so plaintext size can't be computed exactly from encrypted size.
• Failing test first: TestDownloadChunksUnrecordedBlockSizes, 32 blocks read as 8 sequential range opens. It fetched 145 blocks before the fix and 39 after. A range read afterwards fetches exactly its 3 blocks, and a block list changed since the sizes were learnt is errIntegrity.
• Change: blockSizeCache lives on client, so it is per Fs and not package state. It holds at most 16 revisions and 65536 blocks (about 2.5 MiB). The reader learns sizes in order in next(), as a contiguous prefix with each block's hash. openRead uses locate to start at the right block, and to stop at the last block when the prefix covers the range, which partly fixes transfer M-5.
• Correctness: every block is still hash-checked. If the start block's listed hash differs from the learnt hash, or a re-read block's hash or size differs, that is errIntegrity.
• Limit: chunks that start together still each read from block 1. The docs page, the TODO entry and the conformance checklist are updated.
• Existing tests changed: TestDownloadRanges (the unrecorded case now benefits from learnt sizes) and the always-3 nit in it.

auth I-1: error codes (5dbf5b04c)

• Codes checked in the SDK reference:
  • 5003 is "You need to update this app" (coreTypes.ts).
  • 5001 is from go-proton-api.
  • 2028 is "feature disabled, rate-limited or blocked because of abuse" (driveTypes.ts).
• Failing tests first: TestTranslateError (5001 fatal, 2000 no-retry), TestClientShouldRetry (2028 retried, and with Retry-After honoured), TestRegressionFeatureBlockedRetried, TestRegressionRenameResendNotFatal (the reviewer's lost-rename reproduction, which before the fix was fatal with "upgrade rclone"). TestRegressionAppVersionRejected now uses 5003.
• Change:
  • api: CodeFeatureBlocked=2028, CodeAppVersionMissing=5001 and CodeAppVersionBad=5003 (renamed; it used to mean 2028).
  • translateError: 5001/5003 are fatal with "upgrade rclone". 2000 is a NoRetryError that keeps Proton's message. 2028 falls through as a plain error.
  • shouldRetry: 2028 is retried, through RetryAfterError when a Retry-After header is present.
• Deviation: 2064 is left unmapped, because the user can set app_version wrong themselves. DECISIONS entry added (ruling 30).

auth I-2: x-pm-drive-requirements (b4a23a31e)

• Failing test first: TestDriveRequirementsLogged. The fake gained SetDriveRequirements.
• Change: session.do calls observeRequirements, which logs at debug, %q, truncated to 1 KiB, only when the joined value changes.

tests I-1 and I-2: CI (b5da26169)

• go test -race -timeout 40m, and timeout-minutes raised from 45 to 60 to make room for the fuzz step.
• go-version: '1.27.x'.
• govulncheck@v1.9.0, the latest release on proxy.golang.org at the time of writing (queried with go list -m -versions).
• No -p/-parallel limits: the race suite is one big package, so -p doesn't help, and limiting -parallel would make it slower.

Extra: tests read the real rclone.conf (634a2423e)

• While working I saw "Using config file from ~/.config/rclone/rclone.conf" in a captured test log. RCLONE_CONFIG is only applied by fstest.Initialise, which only TestIntegration runs, so any filtered run (-run X) read the real config holding the live TestProtonDrive: remote. Nothing contacted Proton, since the tests use env-configured fake remotes, but the file was read.
• TestMain now calls config.SetConfigPath(RCLONE_CONFIG) first; /notfound means memory-only. Earlier runs in this and previous sessions read that file.

Minors

Fixed:

• crypto M4: code comment, plus a TODO line.
• crypto M5: decryptBlock moved to the tests; the reencryptPassphrase alias removed.
• crypto M6: truncateName cuts on a rune boundary, with a test.
• transfer M-3: godoc rewrapped.
• transfer M-5: partly; the range end is known once sizes are learnt.
• transfer M-7: covered by C1.
• transfer M-8: the superseded message gives the state and revision.
• auth M-1 and M-2: refresh 5xx/429/408 retried, and 408 no longer kills the session; TestRefreshTransientErrorRetried failed first for 503 and for 408.
• tests 1: timing bounds raised to 3 s.
• tests 6: the 502-once test counts the uploads.
• tests 8: TestRefusedCommit pins the 408 exclusion.
• tests 9: "assumed" renamed and the always-3 bound fixed.
• tests 13: conformance checklist.
• tests 14: the docs explain errUploadUnconfirmed.

Logged in docs/TODO.md under "final review minors (2026-11-12)":

• crypto M1, M2, M3, M7 and M8.
• transfer M-1, M-2, M-4, M-6 and M-9.
• auth M-3 to M-11.
• tests 2, 3, 4, 5, 7, 10, 11, 12, 15, 16 and 17.
• tests 17 (.codex rules) is a permission-file change, which I left to the owner.

Not mine: tests 18 and 19 (handoff) and tests I-3 to I-5 (the controller's).

Verification (at ad3552e2e)

• go build ./...: OK.
• go vet ./backend/protondrive/...: OK.
• golangci-lint run ./backend/protondrive/...: 0 issues.
• Full race suite at ad3552e2e: RCLONE_CONFIG=/notfound go test -race -count=1 -timeout 40m ./backend/protondrive/...
  • backend/protondrive: ok, 332.6 s.
  • protonfake: ok.
  • protontest: failed TestFaultInjection/Skip. It still used the api.CodeAppVersionBad name for 2028, and my rename had missed it. Fixed in 944b36005; the protontest race run then passed (39.1 s), and lint was re-run with 0 issues.
• TestGoldenFixtures: PASS (also run alone with -v).

Suggested handoff ruling text

29. A file whose content key signature is missing or invalid is unverified, like an unverified node key (rulings 18, 22): reads, full and ranged, and updates are refused unless ignore_signature_errors is set, when an ERROR is logged once per file and operation. The pre-2023 fallback to the account's other address keys applies only to a signature that is present. Stricter than the SDK, which skips the check when the signature is absent. In the same wave: a folder whose hash key signature fails can't be written into without the flag; extended attributes whose signature fails aren't used without it (modtime from the server, no SHA-1, size from the recorded block sizes); a rename or move never reuses the session key of a name whose signature didn't verify; node and share keys with an Argon2 S2K beyond 4 passes, parallelism 8 or 128 MiB are an integrity error the flag can't override.

30. Error codes: 5001 and 5003 (missing or rejected app version) are a fatal error telling the user to upgrade rclone; 2000 (requirements not met, e.g. a stale OriginalHash after a resent rename) is a no-retry error keeping Proton's message; 2028 (feature disabled, rate-limited or blocked) is retried through the pacer, honouring Retry-After. This supersedes spec 6.6's 2000/2028 fatal mapping. x-pm-drive-requirements is logged at debug level when it changes.