protondrive: tighten tests and docs from the final review, log the rest

Timing bounds in parallel tests leave room for a slow race runner, the
block 502 test checks its fault fired, refusedCommit's 408 exclusion is
pinned, the user docs explain the "server doesn't show it yet" error and
the conformance checklist matches the current draft and trust rules.
Every other minor from the review is listed in docs/TODO.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas WintherandClaude Opus 5.5 committed 2026-11-12 14:48:17 +01:00
1 parent d3aee2af19
commit bfa9b7122b
6 files changed
+131 -7

No files matched your search

@@ -659,7 +659,7 @@ func TestDownloadEndlessBlockList(t *testing.T) {
var pages atomic.Int32
c := newTestClientTransport(t, srv, u, clientOptions{EnableCaching: true}, endlessBlockList(&pages))
n := seedFile(t, c, u, "f", testContent(4<<20+1))
for name, n := range map[string]*node{"claimed": n, "assumed": withoutBlockSizes(n)} {
for name, n := range map[string]*node{"claimed": n, "unrecorded": withoutBlockSizes(n)} {
t.Run(name, func(t *testing.T) {
pages.Store(0)
data, err := readRange(t, c, n, 0, -1)
+6 -2
View File
@@ -508,7 +508,9 @@ func TestRetryAfterDoesNotHoldOtherCalls(t *testing.T) {
start = time.Now()
_, _, err = c.about(context.Background())
require.NoError(t, err)
assert.Less(t, time.Since(start), 500*time.Millisecond)
// Far below the hold this guards against, with room for a slow
// runner under -race.
assert.Less(t, time.Since(start), 3*time.Second)
// loadAccount, the 429, its retry and the second call.
assert.Equal(t, 4, srv.Requests()["GET /core/v4/users"])
}
@@ -528,7 +530,9 @@ func TestRetryAfterCancel(t *testing.T) {
start = time.Now()
_, _, err = c.about(context.Background())
require.NoError(t, err)
assert.Less(t, time.Since(start), 500*time.Millisecond)
// Far below the hold this guards against, with room for a slow
// runner under -race.
assert.Less(t, time.Since(start), 3*time.Second)
}
func TestListPublicKeysRetried(t *testing.T) {
@@ -231,9 +231,30 @@ func TestUploadBlock502Once(t *testing.T) {
t.Parallel()
c, srv, u := newUploadClient(t, clientOptions{})
srv.Inject(protontest.Fault{Method: "POST", Pattern: storageUploadPattern, Status: http.StatusBadGateway, Times: 1})
srv.ResetRequests()
content := testContent(5 << 20)
uploadBytes(t, c, "f", content)
requireUploaded(t, u, map[string][]byte{"f": content})
assert.Equal(t, 3, srv.Requests()["POST "+storageUploadPattern], "two blocks and the resent one")
}
func TestRefusedCommit(t *testing.T) {
for _, test := range []struct {
name string
err error
want bool
}{
{"422 with a code", &api.Error{Status: 422, Code: api.CodeInvalidRequirements}, true},
{"408 with a code", &api.Error{Status: 408, Code: api.CodeInvalidRequirements}, false},
{"422 without a code", &api.Error{Status: 422}, false},
{"500 with a code", &api.Error{Status: 500, Code: api.CodeInvalidRequirements}, false},
{"200 with a code", &api.Error{Status: 200, Code: api.CodeInvalidRequirements}, false},
{"network error", io.ErrUnexpectedEOF, false},
} {
t.Run(test.name, func(t *testing.T) {
assert.Equal(t, test.want, refusedCommit(fmt.Errorf("commit: %w", test.err)))
})
}
}
// TestUploadBlock502Forever is issue 10224: a block which can't be
+96
View File
@@ -226,6 +226,102 @@ Small, specific, independently actionable. Remove only when actually done.
- [ ] Propose a CI check for this fork: `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`
and the `docs/` agent files exist, and `AGENTS.md` is at most 150 lines.
### final review minors (2026-11-12)
Minors from the whole-branch review (`final-review-*.md` in the plan
workspace) not fixed in its fix wave. Paths are under `backend/protondrive/`.
- [ ] A node fetched while it is being changed can overwrite the newer
cached node and stay cached: `put` after `fetchLinks` has no per-node
generation (`nodes.go` `fetchNode`, `cache.go` `put`).
- [ ] Unbounded per-client maps: `client.folderKeys` (kept whatever the
cache), the node cache, `client.reported`, `account.foreign`; `close()`
doesn't clear the private keys of cached file nodes.
- [ ] `listChildren` loops until ctx ends if the server keeps answering
`More` with the same `AnchorID` (`nodes.go`); stop when a page adds
nothing new.
- [ ] The name and hash key signature date exemptions
(`nameSignedSince`, `hashKeySignedSince`) and the content key fallback
read the server-supplied `CreateTime`, so a server can back-date a node
to exempt it (commented in `nodes.go`, as in the SDK).
- [ ] Earlier deferred crypto minors still present: the node passphrase
isn't checked as UTF-8 (`crypto.go` `decryptNodeKeys`); `publicKeys`
waiters get the fetching caller's ctx error (`account.go`);
`setChildrenLocked` overwrites a cached node without `put`'s
changed-parent rule (`cache.go`); an xattr `"Size": null` reads as 0
and skips the block-size fallback (`crypto.go` `parseXAttrs`); Move
detects an anonymous node by `SignatureEmail == ""` (`nodes.go`
`encryptForNewParent`).
- [ ] `publicKeyRing` rebuilds the own-address public ring on every
lookup, 2-3 times per decrypted node (`account.go`); build it once in
`loadAccount`.
- [ ] Two processes sharing one config (a mount and a cron job) can
delete each other's in-progress draft of the same new name, since the
draft client ID is per remote (`upload.go` `resolveConflict`).
- [ ] `pm-storage-token` is still sent if a storage request is
redirected to another host (`download.go` `downloadBlock`, `upload.go`
block upload); set `CheckRedirect` or refuse redirects.
- [ ] `errUploadMismatch` is a `NoRetryError`, so a source that changes
while read fails for the whole run, `--retries` passes included
(`protondrive.go` `translateError`); consider `NoLowLevelRetryError`.
- [ ] An own leftover draft revision on an active file is never deleted
and keeps taking quota (`upload.go` conflict path).
- [ ] A `createDraft` resend can report the client's own new draft as
one that "came back after it was deleted" (`upload.go` `createDraft`).
- [ ] Two `Fs` of an environment-configured remote in one process can't
see each other's token refresh, so the second fails with "session
expired" (`protondrive.go` `configStore`); document it.
- [ ] `rclone config` on a remote with a saved session asks "Already
signed in - sign in again?" with default no, even when the session is
dead and the user followed the reconnect hint (`protondrive.go`
`Config`, ruling 17).
- [ ] After a CAPTCHA in `rclone config`, the method question defaults
back to password sign-in (`protondrive.go` `Config`).
- [ ] A wrong key password on a saved session says "is the password
right?" without the reconnect hint (`account.go` `loadAccount`,
`protondrive.go` `connect`).
- [ ] Token refresh hasn't been seen working against the real server,
and its request lacks the `UID` and `RedirectURI` go-proton-api sends
(`api/types.go` `AuthRefreshRequest`); add it to the live checks.
- [ ] Session minors deferred since Tasks 2-4: an adopted stored token
whose access token is stale fails one call with a 401; the refresh uses
the first waiter's ctx; `serverTime` can step back; the half-signed-in
cleanup `DELETE` uses a possibly cancelled ctx; an empty SRP salt isn't
refused up front; `GET /core/v4/users` is fetched twice at sign-in
(`session.go`, `account.go`).
- [ ] Server-side Copy onto an existing name falls back to download and
re-upload (`protondrive.go` `Copy`).
- [ ] `Precision()` is 1 s while `ModTime` has milliseconds;
cross-account Move/Copy/DirMove aren't refused; the on-the-fly remote
hint names the bare backend; logging passes `nil`, not the `Fs`,
throughout (sweep before upstream).
- [ ] Goroutine-count leak checks remain (`download_internal_test.go`
`requireNoLeaks`, `nodes_internal_test.go`); don't add more.
- [ ] `TestListingRacingChange` can hang to the package timeout if it
fails before closing `release` (`nodes_internal_test.go`).
- [ ] `protonfake` `TestRun` leaks the `run` goroutine on a failed
require (`protontest/cmd/protonfake/main_test.go`).
- [ ] `TestClientBootstrapCreateVolumeRace` waits a fixed 2 s, and
`TestAcquireTokenCancel` sleeps 20 ms to let the wait start
(`nodes_internal_test.go`, `upload_internal_test.go`).
- [ ] The init.d `stop` has no SIGKILL escalation, and shellcheck reports
SC2034 (`fstest/testserver/init.d/TestProtonDriveFake`).
- [ ] CI runs on both `push` and `pull_request` with no `concurrency`
group, so twice per PR push (`.gitea/workflows/ci.yml`); check Gitea
supports `concurrency` first.
- [ ] The fake's read lag (`protontest` `SetReadLag`) never runs with the
contract suite; consider a second `fstests.Run` or a `protonfake
-read-lag` flag.
- [ ] The docs page describes CAPTCHA handling, 2FA prompting and
account-wide `about` that are proven only against the fake, while this
file says the docs avoid claiming them; soften one or the other.
- [ ] `rclone config create` and `config show` print the new remote's
`client_*` tokens and salted key password, which survives a logout;
add a docs line not to paste that output.
- [ ] `docs/PERMISSIONS.md` and `.claude/`/`.gemini/` allow
`golangci-lint run` but `.codex/rules/project.rules` doesn't; bring it
in step (owner, as it is a permission change).
## Someday
## Recurring
+4 -1
View File
@@ -308,7 +308,10 @@ as the same remote.
If the final step of an upload fails in a way which leaves it unknown
whether the file was saved, rclone reports "upload outcome unknown: it
may have succeeded" and doesn't retry that upload at once; a later run of
`rclone sync` or `rclone copy` checks the destination first.
`rclone sync` or `rclone copy` checks the destination first. The same
goes for "upload committed, but the server doesn't show it yet", which
means Proton accepted the upload but didn't list it within a few
seconds.
### Deleting files
+3 -3
View File
@@ -23,9 +23,9 @@ wins.
|---|---|---|---|---|
| `apiService` | `internal/apiService/apiService.ts`, `driveTypes.ts`, `coreTypes.ts`, `errorCodes.ts`, `errors.ts` | `session.go` (HTTP, auth headers, server time), `nodes.go` (`apiCall`, `shouldRetry`, pacers), `api/types.go` | `05c9115` (js/v0.23.0) | All HTTP goes through rclone's `fshttp`, with separate pacers for the API and storage hosts. `Retry-After` is honoured up to 60 s. API codes map to rclone errors in `translateError` (`protondrive.go`). `x-pm-appversion` is `external-drive-rclone@<semver>-<channel>`, always with a channel; `x-pm-drive-sdk-version` is never sent. No telemetry. |
| `shares` | `internal/shares/manager.ts`, `cryptoService.ts`, `apiService.ts` | `nodes.go` (`bootstrap`, `createVolume`), `crypto.go` | `05c9115` (js/v0.23.0) | Only the My Files share (`drive/v2/shares/my-files`) is used; no devices, photos or sharing. Root of trust: the share's creator and the root folder's signer must be one of the account's own addresses (compared after lowercasing ASCII letters only, verified with the account's own keys), else the whole drive is unverified. |
| `nodes` | `internal/nodes/nodesAccess.ts`, `nodesManagement.ts`, `cryptoService.ts`, `extendedAttributes.ts`, `validations.ts`, `nodeName.ts`, `mediaTypes.ts`, `cache.ts` | `nodes.go`, `cache.go`, `crypto.go` | `05c9115` (js/v0.23.0) | A key verified against an unverified folder key is itself unverified, all the way down, reads included; moving, copying, renaming or writing into such nodes needs `ignore_signature_errors`, which logs an ERROR per node and operation. Parent chains deeper than 2048, or cyclic, are an integrity error. Names that are not valid UTF-8 are refused on write. Proton Docs and Sheets are left out of listings. The node cache is in memory only and is not invalidated by events. The intended-recipient check on signatures is kept, which may be stricter than the SDK. |
| `upload` | `internal/upload/fileUploader.ts`, `streamUploader.ts`, `blockVerifier.ts`, `cryptoService.ts`, `apiService.ts`, `digests.ts`, `manager.ts`, `queue.ts` | `upload.go`, `crypto.go` | `05c9115` (js/v0.23.0) | No small-file path (`smallFileUploader.ts`, `files/small`; planned for M6). No thumbnails. A commit whose outcome can't be found out deletes nothing and returns `errUploadOutcomeUnknown`; a draft is deleted only on positive proof it never committed. Drafts are identified by a per-remote `draft_client_uid`, not the session UID. |
| `download` | `internal/download/fileDownloader.ts`, `seekableStream.ts`, `blockIndex.ts`, `cryptoService.ts`, `apiService.ts`, `queue.ts` | `download.go`, `crypto.go` | `05c9115` (js/v0.23.0) | No thumbnails (`thumbnailDownloader.ts`). A full read lists every block and verifies the manifest before releasing any byte; a range covering the whole file is read the same way. Other ranged reads check block hashes but not the manifest, as in the SDK. A file may list at most 2^19 blocks. A re-listed block whose hash changed is an integrity error. A ranged read of a file whose block sizes aren't recorded starts at the first block and discards the bytes before the range. |
| `nodes` | `internal/nodes/nodesAccess.ts`, `nodesManagement.ts`, `cryptoService.ts`, `extendedAttributes.ts`, `validations.ts`, `nodeName.ts`, `mediaTypes.ts`, `cache.ts` | `nodes.go`, `cache.go`, `crypto.go` | `05c9115` (js/v0.23.0) | A key verified against an unverified folder key is itself unverified, all the way down, reads included; moving, copying, renaming or writing into such nodes needs `ignore_signature_errors`, which logs an ERROR per node and operation. Parent chains deeper than 2048, or cyclic, are an integrity error. Names that are not valid UTF-8 are refused on write. Stricter than the SDK: a missing or invalid content key signature makes a file unverified; a folder whose hash key signature fails can't be written into without the flag; extended attributes whose signature fails aren't used without it; a rename or move never reuses the session key of a name whose signature failed; node and share keys with Argon2 beyond 4 passes, parallelism 8 or 128 MiB are refused. Proton Docs and Sheets are left out of listings. The node cache is in memory only and is not invalidated by events. The intended-recipient check on signatures is kept, which may be stricter than the SDK. |
| `upload` | `internal/upload/fileUploader.ts`, `streamUploader.ts`, `blockVerifier.ts`, `cryptoService.ts`, `apiService.ts`, `digests.ts`, `manager.ts`, `queue.ts` | `upload.go`, `crypto.go` | `05c9115` (js/v0.23.0) | No small-file path (`smallFileUploader.ts`, `files/small`; planned for M6). No thumbnails. A commit whose outcome can't be found out deletes nothing and returns `errUploadOutcomeUnknown`; a draft is deleted only when the server explicitly refused the commit (a 4xx with a Proton code, not 408) on a single attempt the pacer did not resend; no read-back decides deletion. Drafts are identified by a per-remote `draft_client_uid`, not the session UID; one which can't be saved is derived from the remote name and volume. |
| `download` | `internal/download/fileDownloader.ts`, `seekableStream.ts`, `blockIndex.ts`, `cryptoService.ts`, `apiService.ts`, `queue.ts` | `download.go`, `crypto.go` | `05c9115` (js/v0.23.0) | No thumbnails (`thumbnailDownloader.ts`). A full read lists every block and verifies the manifest before releasing any byte; a range covering the whole file is read the same way. Other ranged reads check block hashes but not the manifest, as in the SDK. A file may list at most 2^19 blocks. A re-listed block whose hash changed is an integrity error. A ranged read of a file whose block sizes aren't recorded starts at the furthest block whose size an earlier read of the same `Fs` learnt, else at the first block, and discards the bytes before the range. |
| `crypto` | `crypto/driveCrypto.ts`, `crypto/openPGPCrypto.ts` | `crypto.go` | `05c9115` (js/v0.23.0) | gopenpgp v3 in place of openpgp.js. Written format follows the SDK: v4 node key advertising AEAD, v6 PKESK content key, SEIPDv2 blocks; names, passphrases, hash keys and extended attributes without AEAD. |
| account / auth | `incubating/account/js/src/auth.ts`, `authWeb.ts`, `srp.ts`, `addresses.ts`, `accountApi.ts`, `sessionCredentials.ts`, `api-auth-types.ts`; `interface/account.ts` | `session.go` (SRP, session fork, refresh, logout), `account.go` (user and address keys, public keys) | `05c9115` (js/v0.23.0) | Browser sign-in is the session-fork flow with client ID `external-drive`. The key salt is chosen by the primary user key's ID. Several sessions sharing one config re-read it for up to 3 s after a rejected refresh to adopt another's new tokens; there is no cross-process lock. Key unlocking is serialised across `Fs` instances. |
| `events` | `internal/events/` | none | `05c9115` (js/v0.23.0) | Not implemented: no event polling and no `ChangeNotify` (M5). |