protondrive: update docs for the rewritten backend
Rewrite the hand-written parts of the Proton Drive docs page for the native client: browser and password sign-in with a transcript taken from the current code, the third-party notice, what is verified on upload and download, the ranged-read caveat, the trust rules and the ignore_signature_errors option, the two protocol limits, the --dump warning, filename rules, drafts, trash and limits. The generated options section is left for make backenddocs. Describe everything ignore_signature_errors allows in its help text, update the feature list and score in protondrive.yaml, and add the SDK conformance checklist and fork notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c1464a39ab
commit
7be50f9e22
8 files changed
+376
-92
No files matched your search
@@ -208,12 +208,16 @@ uploaded. Values above 5, Proton's limit, are lowered to 5.`,
|
||||
Default: defaultUploadConcurrency,
|
||||
}, {
|
||||
Name: "ignore_signature_errors",
|
||||
Help: `Read files whose signatures can't be verified
|
||||
Help: `Use files and folders whose signatures can't be verified
|
||||
|
||||
Proton signs file content and keys. By default a file whose signature
|
||||
doesn't verify can't be read. If this is set, rclone logs an error and
|
||||
reads it anyway, which may be needed for files uploaded by old or
|
||||
third-party clients.`,
|
||||
Proton signs file content and keys. By default rclone refuses to read a
|
||||
file whose signatures don't verify, and to write into, rename, move or
|
||||
copy anything whose key isn't verified. Everything inside a folder whose
|
||||
key isn't verified counts as unverified too.
|
||||
|
||||
If this is set, rclone logs an error and goes ahead, which may be needed
|
||||
for files uploaded by old or third-party clients. The hashes of the
|
||||
encrypted blocks are always checked.`,
|
||||
Advanced: true,
|
||||
Default: false,
|
||||
}, {
|
||||
|
||||
@@ -68,6 +68,15 @@ A native client of the Proton Drive API, following Proton's Drive SDK
|
||||
against the `TestProtonDriveFake:` remote. Fork CI is
|
||||
`.gitea/workflows/ci.yml`.
|
||||
|
||||
- User docs: `docs/content/protondrive.md` (hand-written parts; the
|
||||
options section is generated from the `Help:` fields).
|
||||
`docs/specs/001-conformance-checklist.md` maps each SDK module to its Go
|
||||
file and the tag it was last checked against.
|
||||
- Not implemented yet: events and `ChangeNotify` (M5); the persistent
|
||||
metadata cache, rclone Metadata and the small-file upload path (M6).
|
||||
Ranged reads check block hashes but not the signed manifest. Open items
|
||||
and unrun live checks are in `docs/TODO.md`.
|
||||
|
||||
## Known rough edges
|
||||
|
||||
- See `docs/TODO.md` for host tooling gaps.
|
||||
@@ -5,6 +5,26 @@ it with a new one that links back. One entry per decision a future maintainer
|
||||
would otherwise have to reverse-engineer.
|
||||
No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry.
|
||||
|
||||
## 2026-11-12 - Proton Drive is a native port of Proton's Drive SDK
|
||||
|
||||
**Status:** accepted
|
||||
**Context:** With the bridge libraries gone (plan Task 19), the backend
|
||||
is a Go client written for rclone. Its behaviour has to be checked against
|
||||
something as Proton changes the service, and the user docs still
|
||||
described the old libraries.
|
||||
**Decision:** The backend is maintained as a port of Proton's Drive SDK,
|
||||
module by module, tracked in `docs/specs/001-conformance-checklist.md`
|
||||
against tag `js/v0.23.0`, with its deliberate differences (stricter trust
|
||||
rules, no small-file path, no events, no thumbnails) listed per module.
|
||||
The user docs describe only what the code does and the protocol limits
|
||||
the SDK shares, with no fork-only content, so they could go upstream.
|
||||
**Alternatives:** Binding to the SDK itself (it is TypeScript and C#;
|
||||
rclone refuses cgo and helper binaries); writing from observed traffic
|
||||
alone, as the old libraries did (no reference to re-check against).
|
||||
**Consequences:** Each SDK release needs a walk of the checklist. The
|
||||
docs page must change with any behaviour change, and its generated
|
||||
options section is left to `make backenddocs` at release.
|
||||
|
||||
## 2026-11-12 - Proton Drive keeps frozen old-library fixtures, not the old library
|
||||
|
||||
**Status:** accepted
|
||||
|
||||
+30
-7
@@ -6,15 +6,38 @@ Small, specific, independently actionable. Remove only when actually done.
|
||||
|
||||
## Now
|
||||
|
||||
- [ ] protondrive: the backend rewrite (plan 001) has Tasks 1-19 of 21
|
||||
- [ ] protondrive: the backend rewrite (plan 001) has Tasks 1-20 of 21
|
||||
done. Start from `docs/plans/001-protondrive-tier1-HANDOFF.md`.
|
||||
Remaining: Task 20 (user docs - `docs/content/protondrive.md` still describes
|
||||
the old backend), 21 (live verification, paid account never signed in to),
|
||||
then the whole-branch review; later plans for M5 (events) and M6
|
||||
(persistent cache, Metadata, small-file upload).
|
||||
Remaining: Task 21 (live verification, paid account never signed in
|
||||
to), then the whole-branch review.
|
||||
|
||||
## Next
|
||||
|
||||
- [ ] protondrive M5: volume events polled no more often than every 30 s,
|
||||
invalidating the node cache and dircache and feeding `ChangeNotify`, so
|
||||
`mount` and `serve` see other clients' changes (spec 6.2). Needs its own
|
||||
plan; also decide the cross-`Fs` cache question below.
|
||||
- [ ] protondrive M6: opt-in persistent metadata cache of encrypted node
|
||||
records and the event cursor through `lib/kv` (`metadata_cache`), rclone
|
||||
Metadata (`mtime`, `btime`, `content-type`), and the SDK's single-request
|
||||
upload for files under 128 KiB (`files/small`). Needs its own plan; the
|
||||
docs page and `protondrive.yaml` features change with it.
|
||||
- [ ] protondrive M7: the upstream path (spec 13), deferred by the owner;
|
||||
also the new crypto model once Proton publishes it in the SDK.
|
||||
- [ ] protondrive live checks not yet run (plan Task 21), each of which
|
||||
the user docs avoid claiming: the paid account (never signed in to);
|
||||
browser (session-fork) sign-in against the real service; 2FA, by code
|
||||
and by `otp_secret_key`; CAPTCHA handling; Proton's desktop, mobile or
|
||||
CLI clients reading files rclone wrote (only the web app has been
|
||||
checked); `TestIntegration` (`fstests`) and `test_all` against the real
|
||||
service; whether the real server lists drafts or trashed items as
|
||||
children (affects `Rmdir`); whether any real address differs from
|
||||
another of the account only by non-ASCII case (counted as foreign).
|
||||
- [ ] protondrive: `docs/content/protondrive.md`'s generated options
|
||||
section still shows the old options and help texts; regenerate it with
|
||||
`make backenddocs` when the work goes upstream (generated doc changes
|
||||
are not committed in the fork, per `AGENTS.md`).
|
||||
|
||||
- [ ] protondrive: API responses have no body-size limit, and a block
|
||||
list's `Token` and `BareURL` and the first page's `Thumbnails` are
|
||||
unbounded, so the 2^19-block cap bounds how many blocks a full read
|
||||
@@ -53,8 +76,8 @@ Small, specific, independently actionable. Remove only when actually done.
|
||||
- [ ] protondrive: `--dump headers` logs Proton's `pm-storage-token` and
|
||||
`x-pm-uid` (fshttp redacts only `Authorization`), and `--dump bodies`
|
||||
logs the access, refresh and upload tokens in JSON bodies (seen in
|
||||
`TestNoSecretsInLogs`). Task 20's docs must carry the spec 7 warning;
|
||||
a header redaction list in `fshttp` would be an upstream change.
|
||||
`TestNoSecretsInLogs`). The docs page warns about it; a header
|
||||
redaction list in `fshttp` would be an upstream change.
|
||||
- [ ] protondrive: a name whose signature fails is listed and the file
|
||||
read with only an `ERROR` logged (spec 6.2, the SDK's behaviour), and
|
||||
names aren't bound to their node, so a server can swap the encrypted
|
||||
|
||||
+272
-73
@@ -7,28 +7,41 @@ versionIntroduced: "v1.64.0"
|
||||
# Proton Drive
|
||||
|
||||
[Proton Drive](https://proton.me/drive) is an end-to-end encrypted Swiss vault
|
||||
for your files that protects your data.
|
||||
for your files that protects your data.
|
||||
|
||||
This is an rclone backend for Proton Drive which supports the file transfer
|
||||
features of Proton Drive using the same client-side encryption.
|
||||
This backend reads and writes Proton Drive with the same client-side
|
||||
encryption as Proton's own apps. It is a client of the Proton Drive API
|
||||
written for rclone, following the behaviour of
|
||||
[Proton's Drive SDK](https://github.com/ProtonDriveApps/sdk).
|
||||
|
||||
Due to the fact that Proton Drive doesn't publish its API documentation, this
|
||||
backend is implemented with best efforts by reading the open-sourced client
|
||||
source code and observing the Proton Drive traffic in the browser.
|
||||
|
||||
**NB** This backend is currently in Beta. It is believed to be correct
|
||||
and all the integration tests pass. However the Proton Drive protocol
|
||||
has evolved over time there may be accounts it is not compatible
|
||||
with. Please [post on the rclone forum](https://forum.rclone.org/) if
|
||||
you find an incompatibility.
|
||||
rclone is a third-party application not officially supported by Proton.
|
||||
|
||||
Paths are specified as `remote:path`
|
||||
|
||||
Paths may be as deep as required, e.g. `remote:directory/subdirectory`.
|
||||
|
||||
## Configurations
|
||||
## Configuration
|
||||
|
||||
Here is an example of how to make a remote called `remote`. First run:
|
||||
There are two ways to sign in:
|
||||
|
||||
- **With a web browser** (the default when no username and password are
|
||||
set). rclone shows a link to Proton's sign-in page, which you can open
|
||||
on any device, including a different computer from the one running
|
||||
rclone. Your password, two-factor authentication and any CAPTCHA are
|
||||
handled by Proton's page; rclone never sees or stores your password.
|
||||
- **With a username and password**, for existing configurations and
|
||||
automated setups. rclone signs in with the `username`, `password`,
|
||||
and, if needed, `mailbox_password`, `2fa` or `otp_secret_key` options.
|
||||
If Proton asks for a CAPTCHA, password sign-in fails and you need to
|
||||
run `rclone config reconnect remote:` and choose browser sign-in.
|
||||
|
||||
Either way rclone saves the session in the config file and uses it from
|
||||
then on, refreshing it as needed. The saved session gives full access to
|
||||
your account, so consider [encrypting your
|
||||
configuration](/docs/#configuration-encryption).
|
||||
|
||||
Here is an example of how to make a remote called `remote` with browser
|
||||
sign-in. First run:
|
||||
|
||||
```console
|
||||
rclone config
|
||||
@@ -50,37 +63,84 @@ XX / Proton Drive
|
||||
\ "protondrive"
|
||||
[snip]
|
||||
Storage> protondrive
|
||||
User name
|
||||
user> you@protonmail.com
|
||||
Password.
|
||||
y) Yes type in my own password
|
||||
g) Generate random password
|
||||
n) No leave this optional password blank
|
||||
y/g/n> y
|
||||
Enter the password:
|
||||
password:
|
||||
Confirm the password:
|
||||
password:
|
||||
Option 2fa.
|
||||
2FA code (if the account requires one)
|
||||
Option username.
|
||||
The username of your Proton account.
|
||||
Only needed to sign in with a password. Signing in with a web browser,
|
||||
which "rclone config" offers, needs neither username nor password.
|
||||
Enter a value. Press Enter to leave empty.
|
||||
2fa> 123456
|
||||
Remote config
|
||||
username>
|
||||
Option password.
|
||||
The password of your Proton account.
|
||||
Only needed to sign in with a password.
|
||||
Choose an alternative below. Press Enter for the default (n).
|
||||
y) Yes, type in my own password
|
||||
g) Generate random password
|
||||
n) No, leave this optional password blank (default)
|
||||
y/g/n> n
|
||||
Option 2fa.
|
||||
[snip]
|
||||
2fa>
|
||||
Option otp_secret_key.
|
||||
[snip]
|
||||
y/g/n> n
|
||||
Edit advanced config?
|
||||
y) Yes
|
||||
n) No (default)
|
||||
y/n> n
|
||||
Sign in with a web browser?
|
||||
|
||||
rclone is a third-party application not officially supported by Proton.
|
||||
|
||||
Choose yes to sign in on Proton's sign-in page, on any device; rclone
|
||||
never sees your password. Choose no to sign in with the username and
|
||||
password set above.
|
||||
y) Yes (default)
|
||||
n) No
|
||||
y/n> y
|
||||
Open this link in a web browser, on any device, and sign in to Proton:
|
||||
|
||||
https://account.proton.me/desktop/login?app=drive&pv=3#payload=XXXXXXXX
|
||||
|
||||
Waiting for the sign-in to complete...
|
||||
Configuration complete.
|
||||
Options:
|
||||
- type: protondrive
|
||||
- user: you@protonmail.com
|
||||
- pass: *** ENCRYPTED ***
|
||||
[snip]
|
||||
Keep this "remote" remote?
|
||||
y) Yes this is OK
|
||||
y) Yes this is OK (default)
|
||||
e) Edit this remote
|
||||
d) Delete this remote
|
||||
y/e/d> y
|
||||
```
|
||||
|
||||
**NOTE:** The Proton Drive encryption keys need to have been already generated
|
||||
after a regular login via the browser, otherwise attempting to use the
|
||||
credentials in `rclone` will fail.
|
||||
The sign-in link contains a secret which lets whoever completes the
|
||||
sign-in hand their session to this rclone, so don't share it. rclone
|
||||
waits up to 10 minutes for the sign-in to complete.
|
||||
|
||||
To sign in with a password instead, enter your username and password at
|
||||
the `username` and `password` prompts (and, for a two-password account,
|
||||
the `mailbox_password` advanced option), then answer `n` to "Sign in with
|
||||
a web browser?". If your account uses two-factor authentication, rclone
|
||||
asks for a code:
|
||||
|
||||
```text
|
||||
Two-factor authentication code
|
||||
Enter a value.
|
||||
config_2fa> 123456
|
||||
```
|
||||
|
||||
The code is only needed while signing in. If rclone has to sign in again
|
||||
later without you, for example because the saved session has expired, it
|
||||
uses the `otp_secret_key` option to generate a code; without it, or
|
||||
without a username and password, it stops and asks you to run
|
||||
`rclone config reconnect remote:`.
|
||||
|
||||
Running `rclone config reconnect remote:` (or editing the remote) on a
|
||||
remote which is already signed in asks whether to sign in again; if you
|
||||
do, the previous session is signed out.
|
||||
|
||||
A new account doesn't need to have used Proton Drive before: if the
|
||||
account has no Drive yet, rclone creates it.
|
||||
|
||||
Once configured you can then use `rclone` like this (replace `remote` with the
|
||||
name you gave your remote):
|
||||
@@ -97,7 +157,7 @@ List all the files in your Proton Drive
|
||||
rclone ls remote:
|
||||
```
|
||||
|
||||
To copy a local directory to an Proton Drive directory called backup
|
||||
To copy a local directory to a Proton Drive directory called backup
|
||||
|
||||
```console
|
||||
rclone copy /home/source remote:backup
|
||||
@@ -105,33 +165,176 @@ rclone copy /home/source remote:backup
|
||||
|
||||
### Modification times and hashes
|
||||
|
||||
Proton Drive Bridge does not support updating modification times yet.
|
||||
The modification time of a file is stored, encrypted, with each upload
|
||||
and read back with a precision of 1 second. It can't be changed without
|
||||
uploading the file again, so rclone can't set it on an existing file.
|
||||
The modification time of a folder is set when it is created.
|
||||
|
||||
The SHA1 hash algorithm is supported.
|
||||
The SHA1 hash algorithm is supported. The hash is the one the uploader
|
||||
recorded in the file's encrypted attributes; files uploaded by clients
|
||||
which don't record one have no hash.
|
||||
|
||||
### Integrity and signatures
|
||||
|
||||
Proton Drive is end-to-end encrypted, and rclone checks what it reads
|
||||
the way Proton's own apps do, so that a server which altered your data
|
||||
would be noticed:
|
||||
|
||||
- Every encrypted block of a file is checked against the SHA-256 hash
|
||||
the server lists for it before it is decrypted. A block which doesn't
|
||||
match is downloaded again, up to three attempts in all, then the read
|
||||
fails.
|
||||
- A read of a whole file first fetches the list of all its blocks and
|
||||
checks the uploader's signature over it (the manifest) before
|
||||
returning any data, so a file can't be truncated, extended or have its
|
||||
blocks reordered. For a very large file this adds about one request
|
||||
per 80 MiB before the first byte arrives.
|
||||
- The keys of files and folders and the key of each file's content must
|
||||
be signed by their owner.
|
||||
|
||||
On upload rclone signs everything it writes, checks that each encrypted
|
||||
block decrypts again before sending it, and, before committing the
|
||||
upload, checks the number of bytes read against the size of the source
|
||||
and the SHA1 of what was read against the source's SHA1 if the source
|
||||
has one.
|
||||
|
||||
**Ranged reads.** A read of only part of a file, as `rclone mount`,
|
||||
`rclone serve` and multi-thread downloads (`--multi-thread-streams`)
|
||||
make, checks each block against its listed hash but can't check the
|
||||
manifest, so a malicious server could serve validly encrypted blocks of
|
||||
the same file in the wrong place, or from an older version of the same
|
||||
size. Proton's SDK has the same limit for ranged reads. A range which
|
||||
covers the whole file is checked as a full read. Ranged reads of files
|
||||
uploaded by clients which don't record their block sizes only work from
|
||||
the start of the file: one from elsewhere may fail with an error saying
|
||||
so.
|
||||
|
||||
**Trust is rooted in your own addresses.** The share holding your files
|
||||
must have been created, and its root folder signed, by one of your
|
||||
account's addresses, otherwise nothing in the drive is treated as
|
||||
verified. A folder whose key isn't verified makes everything beneath it
|
||||
unverified, even items you signed yourself, because whoever made that
|
||||
key can read and replace what is in it. That includes a genuine folder
|
||||
signed by an address since removed from your account.
|
||||
|
||||
By default rclone refuses to:
|
||||
|
||||
- read a file whose key, content key or manifest signature doesn't
|
||||
verify, or which is beneath a folder whose key isn't verified;
|
||||
- create, upload, rename, move or copy anything into a folder whose key
|
||||
isn't verified, because what is written there would be encrypted to a
|
||||
key someone else may hold;
|
||||
- rename, move or copy an item whose key isn't verified, because that
|
||||
re-signs it with your keys.
|
||||
|
||||
The errors say so and name the `ignore_signature_errors` advanced option.
|
||||
Data uploaded by some old or third-party clients, or shared with you by
|
||||
others, may need it. When it is set, rclone logs an `ERROR` for each
|
||||
item and operation it lets through and goes ahead; an item renamed,
|
||||
moved or copied this way is signed with your keys and trusted from then
|
||||
on. The option never turns off the block hash checks.
|
||||
|
||||
A file or folder name whose signature doesn't verify is listed anyway,
|
||||
with an `ERROR` logged, as Proton's apps do.
|
||||
|
||||
Two limits of the Proton Drive protocol, which Proton's SDK shares,
|
||||
can't be detected by rclone:
|
||||
|
||||
- A server which serves a whole earlier version of a file (its blocks,
|
||||
manifest and attributes, all validly signed) in place of the current
|
||||
one.
|
||||
- Below the root of your drive, an item signed by another Proton user,
|
||||
such as a member of a shared folder, is checked against public keys
|
||||
which the server itself supplies, so a malicious server could forge
|
||||
those.
|
||||
|
||||
### Restricted filename characters
|
||||
|
||||
Invalid UTF-8 bytes will be [replaced](/overview/#invalid-utf8), also left and
|
||||
right spaces will be removed ([code reference](https://github.com/ProtonMail/WebClients/blob/b4eba99d241af4fdae06ff7138bd651a40ef5d3c/applications/drive/src/app/store/_links/validation.ts#L51))
|
||||
Names may be at most 255 characters long, counted in UTF-16 code units
|
||||
as Proton's apps count them (so some emoji count as two), and can't be
|
||||
empty, `.` or `..`, or contain `/` or NUL. The default encoding replaces
|
||||
these characters:
|
||||
|
||||
| Character | Value | Replacement |
|
||||
| --------- |:-----:|:-----------:|
|
||||
| NUL | 0x00 | ␀ |
|
||||
| / | 0x2F | / |
|
||||
|
||||
File names can also not start or end with the following characters.
|
||||
These only get replaced if they are the first or last character in the
|
||||
name:
|
||||
|
||||
| Character | Value | Replacement |
|
||||
| --------- |:-----:|:-----------:|
|
||||
| SP | 0x20 | ␠ |
|
||||
|
||||
Names which are exactly `.` or `..` are replaced with `.` and `..`.
|
||||
|
||||
Invalid UTF-8 bytes will also be [replaced](/overview/#invalid-utf8). If
|
||||
the encoding is changed so that they are not, names which aren't valid
|
||||
UTF-8 are refused.
|
||||
|
||||
### Duplicated files
|
||||
|
||||
Proton Drive can not have two files with exactly the same name and path. If the
|
||||
conflict occurs, depending on the advanced config, the file might or might not
|
||||
be overwritten.
|
||||
Proton Drive can't have two files with exactly the same name in the same
|
||||
folder. Uploading a file whose name is already taken by a file adds a new
|
||||
revision to that file, keeping its history, rather than making a second
|
||||
file.
|
||||
|
||||
### [Mailbox password](https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password)
|
||||
An upload which is cancelled or fails can leave a draft behind. A draft
|
||||
left by the same remote is replaced on the next upload of that file. A
|
||||
draft left by another client, which may still be uploading, makes the
|
||||
upload fail unless the `replace_existing_draft` advanced option is set.
|
||||
|
||||
Please set your mailbox password in the advanced config section.
|
||||
If the final step of an upload fails in a way which leaves it unknown
|
||||
whether the file was saved, rclone reports "upload outcome unknown: it
|
||||
may have succeeded" and doesn't retry that upload at once; a later run of
|
||||
`rclone sync` or `rclone copy` checks the destination first.
|
||||
|
||||
### Deleting files
|
||||
|
||||
Deleted files and folders (`rclone delete`, `rclone purge`,
|
||||
`rclone rmdir`) are moved to the Proton Drive trash, as Proton has no
|
||||
direct permanent delete. They count against your storage until the trash
|
||||
is emptied. `rclone cleanup remote:` empties the whole trash
|
||||
permanently, including items deleted with Proton's apps.
|
||||
|
||||
`rclone rmdir` asks the server whether the folder is empty, so it refuses
|
||||
to remove a folder which holds items rclone doesn't list, such as Proton
|
||||
Docs or items it can't decrypt.
|
||||
|
||||
### Items left out of listings
|
||||
|
||||
Proton Docs and Sheets have no file content for rclone to read, so they
|
||||
are left out of listings, with one notice logged. Items whose name or
|
||||
key can't be decrypted are left out too, each with an `ERROR` logged.
|
||||
|
||||
### Mailbox password
|
||||
|
||||
If your account uses a separate [mailbox
|
||||
password](https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password),
|
||||
set it in the `mailbox_password` advanced option to sign in with a
|
||||
password. Browser sign-in doesn't need it.
|
||||
|
||||
### Caching
|
||||
|
||||
The cache is currently built for the case when the rclone is the only instance
|
||||
performing operations to the mount point. The event system, which is the proton
|
||||
API system that provides visibility of what has changed on the drive, is yet
|
||||
to be implemented, so updates from other clients won’t be reflected in the
|
||||
cache. Thus, if there are concurrent clients accessing the same mount point,
|
||||
then we might have a problem with caching the stale data.
|
||||
rclone keeps the decrypted metadata of the files and folders it has read
|
||||
in memory, to save requests and decryption. Changes made by other clients
|
||||
(Proton's apps, or another rclone) to folders it has already listed are
|
||||
not seen until rclone restarts, so a long-running `rclone mount` or
|
||||
`rclone serve` can show out of date listings. If other clients change
|
||||
the same files at the same time, set the `enable_caching` advanced option
|
||||
to `false`.
|
||||
|
||||
### Logs and `--dump`
|
||||
|
||||
rclone's own log messages for this backend never contain passwords, keys
|
||||
or tokens, at any `-v` level. The output of the `--dump` flags does:
|
||||
`--dump headers` shows the `x-pm-uid` and `pm-storage-token` headers,
|
||||
and `--dump bodies` shows the access, refresh and upload tokens Proton
|
||||
sends in request and response bodies. Treat `--dump` output from this
|
||||
backend as secret, and sign the remote in again (with
|
||||
`rclone config reconnect remote:`) if you have shared it.
|
||||
|
||||
<!-- autogenerated options start - DO NOT EDIT - instead edit fs.RegInfo in backend/protondrive/protondrive.go and run make backenddocs to verify --> <!-- markdownlint-disable-line line-length -->
|
||||
### Standard options
|
||||
@@ -330,25 +533,21 @@ Properties:
|
||||
|
||||
## Limitations
|
||||
|
||||
This backend uses the
|
||||
[Proton-API-Bridge](https://github.com/henrybear327/Proton-API-Bridge), which
|
||||
is based on [go-proton-api](https://github.com/henrybear327/go-proton-api), a
|
||||
fork of the [official repo](https://github.com/ProtonMail/go-proton-api).
|
||||
|
||||
There is no official API documentation available from Proton Drive. But, thanks
|
||||
to Proton open sourcing [proton-go-api](https://github.com/ProtonMail/go-proton-api)
|
||||
and the web, iOS, and Android client codebases, we don't need to completely
|
||||
reverse engineer the APIs by observing the web client traffic!
|
||||
|
||||
[proton-go-api](https://github.com/ProtonMail/go-proton-api) provides the basic
|
||||
building blocks of API calls and error handling, such as 429 exponential
|
||||
back-off, but it is pretty much just a barebone interface to the Proton API.
|
||||
For example, the encryption and decryption of the Proton Drive file are not
|
||||
provided in this library.
|
||||
|
||||
The Proton-API-Bridge, attempts to bridge the gap, so rclone can be built on
|
||||
top of this quickly. This codebase handles the intricate tasks before and after
|
||||
calling Proton APIs, particularly the complex encryption scheme, allowing
|
||||
developers to implement features for other software on top of this codebase.
|
||||
There are likely quite a few errors in this library, as there isn't official
|
||||
documentation available.
|
||||
- Only the "My files" section of Proton Drive is available; computers
|
||||
backed up with Proton's desktop app, Photos and items shared with you
|
||||
are not.
|
||||
- Changes made by other clients are not seen while rclone runs (see
|
||||
[Caching](#caching)): the backend doesn't yet follow Proton's change
|
||||
events, so `rclone mount` doesn't pick them up.
|
||||
- rclone metadata (`--metadata`) is not supported, and the modification
|
||||
time of an existing file can't be changed without uploading it again.
|
||||
- A file may have at most 524288 blocks of 4 MiB (2 TiB); rclone refuses
|
||||
to read a file which lists more. Items more than 2048 folders deep
|
||||
can't be used.
|
||||
- `rclone about` reports the storage used by your whole Proton account,
|
||||
not only by Drive.
|
||||
- Files are uploaded in blocks of 4 MiB, each held in memory while it is
|
||||
uploaded: up to 5 blocks of one file at once (the `upload_concurrency`
|
||||
advanced option, default 4), and at most 10 blocks uploading and 10
|
||||
downloading at once per remote, so with a high `--transfers` some
|
||||
transfers wait their turn.
|
||||
@@ -2,7 +2,7 @@ backend: protondrive
|
||||
name: Proton Drive
|
||||
tier: Tier 4
|
||||
maintainers: Community
|
||||
features_score: 4
|
||||
features_score: 5
|
||||
integration_tests: Passing
|
||||
data_integrity: Hash
|
||||
performance: High
|
||||
@@ -15,13 +15,16 @@ features:
|
||||
- About
|
||||
- CanHaveEmptyDirectories
|
||||
- CleanUp
|
||||
- Copy
|
||||
- DirCacheFlush
|
||||
- DirMove
|
||||
- Disconnect
|
||||
- ListP
|
||||
- Move
|
||||
- NoMultiThreading
|
||||
- Purge
|
||||
- PutStream
|
||||
- ReadMimeType
|
||||
- Shutdown
|
||||
hashes:
|
||||
- sha1
|
||||
precision: 1000000000
|
||||
@@ -1323,27 +1323,27 @@ allocated in `PORTS.md`, echo its `key=value` lines and
|
||||
- Modify: `docs/ARCHITECTURE.md`, `docs/DECISIONS.md`, `docs/TODO.md`
|
||||
- Create: `docs/specs/001-conformance-checklist.md`
|
||||
|
||||
- [ ] **Step 1:** Rewrite the docs page per spec section 10: both sign-in
|
||||
- [x] **Step 1:** Rewrite the docs page per spec section 10: both sign-in
|
||||
methods (fix the stale `user>`/`pass` transcript), the third-party
|
||||
notice verbatim, what is verified on upload and download, the ranged-read
|
||||
caveat, the `--dump` warning from spec section 7, limits (name rules,
|
||||
no SetModTime, trash semantics), removal of the "Beta" note and the
|
||||
`henrybear327` library text. Do not change the tier.
|
||||
- [ ] **Step 2:** Run `python3 bin/manage_backends.py features protondrive`
|
||||
- [x] **Step 2:** Run `python3 bin/manage_backends.py features protondrive`
|
||||
if it works offline with the fake remote; otherwise edit
|
||||
`protondrive.yaml` `features:` by hand to match the new feature set and
|
||||
set `features_score: 5`. Leave `tier`, `maintainers`,
|
||||
`integration_tests` untouched.
|
||||
- [ ] **Step 3:** `docs/specs/001-conformance-checklist.md`: one table row
|
||||
- [x] **Step 3:** `docs/specs/001-conformance-checklist.md`: one table row
|
||||
per SDK module (`apiService`, `shares`, `nodes`, `upload`, `download`,
|
||||
`crypto`, account/auth) with the SDK file, the Go file, the SDK tag last
|
||||
checked (`js/v0.23.0`), and known deliberate differences (no small-file
|
||||
path, no events yet, no thumbnails).
|
||||
- [ ] **Step 4:** `docs/ARCHITECTURE.md` (protondrive section replacing the
|
||||
- [x] **Step 4:** `docs/ARCHITECTURE.md` (protondrive section replacing the
|
||||
"known rough edges" line), a `docs/DECISIONS.md` entry for the native
|
||||
port, `docs/TODO.md` entries for M5, M6, M7 and every live check not
|
||||
run.
|
||||
- [ ] **Step 5: Commit** `protondrive: update docs for the rewritten backend`.
|
||||
- [x] **Step 5: Commit** `protondrive: update docs for the rewritten backend`.
|
||||
|
||||
### Task 21: Live verification (blocked until a test account exists)
|
||||
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# 001 - Proton Drive SDK conformance checklist
|
||||
|
||||
Fork-only. Maps each module of Proton's Drive SDK to the Go file of
|
||||
`backend/protondrive` that follows it, so a later SDK release can be
|
||||
walked module by module. Spec: `docs/specs/001-protondrive-tier1.md`.
|
||||
|
||||
SDK: github.com/ProtonDriveApps/sdk, tag `js/v0.23.0` (commit `05c9115`).
|
||||
SDK paths are relative to `client/js/src/` unless they start with
|
||||
`incubating/`. Go paths are relative to `backend/protondrive/`.
|
||||
|
||||
To re-check against a newer tag: diff each SDK file listed below between
|
||||
the two tags, and port or record every change in the module's row. Wire
|
||||
shapes come from `internal/apiService/driveTypes.ts` and `coreTypes.ts`;
|
||||
where this backend and the SDK disagree about the wire format, the SDK
|
||||
wins.
|
||||
|
||||
| SDK module | SDK files | Go files | Last checked | Known deliberate differences |
|
||||
|---|---|---|---|---|
|
||||
| `apiService` | `internal/apiService/apiService.ts`, `driveTypes.ts`, `coreTypes.ts`, `errorCodes.ts`, `errors.ts` | `session.go` (HTTP, auth headers, server time), `nodes.go` (`apiCall`, `shouldRetry`, pacers), `api/types.go` | `js/v0.23.0` | All HTTP goes through rclone's `fshttp`, with separate pacers for the API and storage hosts. `Retry-After` is honoured up to 60 s. API codes map to rclone errors in `translateError` (`protondrive.go`). `x-pm-appversion` is `external-drive-rclone@<semver>-<channel>`, always with a channel; `x-pm-drive-sdk-version` is never sent. No telemetry. |
|
||||
| `shares` | `internal/shares/manager.ts`, `cryptoService.ts`, `apiService.ts` | `nodes.go` (`bootstrap`, `createVolume`), `crypto.go` | `js/v0.23.0` | Only the My Files share (`drive/v2/shares/my-files`) is used; no devices, photos or sharing. Root of trust: the share's creator and the root folder's signer must be one of the account's own addresses (compared after lowercasing ASCII letters only, verified with the account's own keys), else the whole drive is unverified. |
|
||||
| `nodes` | `internal/nodes/nodesAccess.ts`, `nodesManagement.ts`, `cryptoService.ts`, `extendedAttributes.ts`, `validations.ts`, `nodeName.ts`, `mediaTypes.ts`, `cache.ts` | `nodes.go`, `cache.go`, `crypto.go` | `js/v0.23.0` | A key verified against an unverified folder key is itself unverified, all the way down, reads included; moving, copying, renaming or writing into such nodes needs `ignore_signature_errors`, which logs an ERROR per node and operation. Parent chains deeper than 2048, or cyclic, are an integrity error. Names that are not valid UTF-8 are refused on write. Proton Docs and Sheets are left out of listings. The node cache is in memory only and is not invalidated by events. The intended-recipient check on signatures is kept, which may be stricter than the SDK. |
|
||||
| `upload` | `internal/upload/fileUploader.ts`, `streamUploader.ts`, `blockVerifier.ts`, `cryptoService.ts`, `apiService.ts`, `digests.ts`, `manager.ts`, `queue.ts` | `upload.go`, `crypto.go` | `js/v0.23.0` | No small-file path (`smallFileUploader.ts`, `files/small`; planned for M6). No thumbnails. A commit whose outcome can't be found out deletes nothing and returns `errUploadOutcomeUnknown`; a draft is deleted only on positive proof it never committed. Drafts are identified by a per-remote `draft_client_uid`, not the session UID. |
|
||||
| `download` | `internal/download/fileDownloader.ts`, `seekableStream.ts`, `blockIndex.ts`, `cryptoService.ts`, `apiService.ts`, `queue.ts` | `download.go`, `crypto.go` | `js/v0.23.0` | No thumbnails (`thumbnailDownloader.ts`). A full read lists every block and verifies the manifest before releasing any byte; a range covering the whole file is read the same way. Other ranged reads check block hashes but not the manifest, as in the SDK. A file may list at most 2^19 blocks. A re-listed block whose hash changed is an integrity error. |
|
||||
| `crypto` | `crypto/driveCrypto.ts`, `crypto/openPGPCrypto.ts` | `crypto.go` | `js/v0.23.0` | gopenpgp v3 in place of openpgp.js. Written format follows the SDK: v4 node key advertising AEAD, v6 PKESK content key, SEIPDv2 blocks; names, passphrases, hash keys and extended attributes without AEAD. |
|
||||
| account / auth | `incubating/account/js/src/auth.ts`, `authWeb.ts`, `srp.ts`, `addresses.ts`, `accountApi.ts`, `sessionCredentials.ts`, `api-auth-types.ts`; `interface/account.ts` | `session.go` (SRP, session fork, refresh, logout), `account.go` (user and address keys, public keys) | `js/v0.23.0` | Browser sign-in is the session-fork flow with client ID `external-drive`. The key salt is chosen by the primary user key's ID. Several sessions sharing one config re-read it for up to 3 s after a rejected refresh to adopt another's new tokens; there is no cross-process lock. Key unlocking is serialised across `Fs` instances. |
|
||||
| `events` | `internal/events/` | none | `js/v0.23.0` | Not implemented: no event polling and no `ChangeNotify` (M5). |
|
||||
Reference in new issue
Block a user