Skip to content

fix(backup): support older rclone on restore and add a backup --list preflight#478

Merged
mrrobot47 merged 1 commit into
EasyEngine:developfrom
mrrobot47:fix/rclone-version-gate-and-preflight
Jun 12, 2026
Merged

fix(backup): support older rclone on restore and add a backup --list preflight#478
mrrobot47 merged 1 commit into
EasyEngine:developfrom
mrrobot47:fix/rclone-version-gate-and-preflight

Conversation

@mrrobot47

Copy link
Copy Markdown
Member

Summary

Two robustness follow-ups to the rclone memory-budgeting work merged in #477, both addressing issues reported from the field:

  1. Restore is broken on older rclone (regression from fix: cap rclone backup/restore memory to prevent OOM kills #477).
  2. ee site backup --list shows a raw sh: 1: rclone: not found when rclone is missing, instead of the friendly install message that already exists for other paths.

1 — Gate multi-thread download flags by rclone version

#477's rclone_download() unconditionally passes --multi-thread-write-buffer-size and --multi-thread-chunk-size. These flags are recent, and rclone hard-fails on unknown flags (Fatal error: unknown flag, exit code 2) — it does not ignore them — so any older rclone aborts the copy and restore/rollback fails entirely. Distro-packaged rclone is frequently older than these versions.

Confirmed introduction versions (rclone changelog):

Flag Introduced Released
--multi-thread-streams, --multi-thread-cutoff v1.48.0 Jun 2019
--multi-thread-write-buffer-size v1.63.0 Jun 2023
--multi-thread-chunk-size v1.64.0 Sep 2023

The two newer flags are now gated behind the detected rclone version (get_rclone_version()), with thresholds in RCLONE_MIN_VERSION_MT_* constants:

  • < 1.63 → only --multi-thread-streams (+ --transfers/--buffer-size), all available since v1.48.
  • >= 1.63 → adds --multi-thread-write-buffer-size.
  • >= 1.64 → adds --multi-thread-chunk-size.
  • version undeterminable → newer flags skipped (safe default).

The memory budget already assumes these flags' defaults, so gating them out changes only which knobs reach rclone, not the computed footprint — the OOM protection from #477 is unaffected on every rclone version.

2 — Friendly preflight for ee site backup --list

The rclone-presence and backend-config checks (with their friendly "rclone is not installed… install via https://rclone.org/downloads/" messages) lived inside pre_backup_restore_checks(). The --list branch returns before that runs, so a missing rclone surfaced as a raw sh: 1: rclone: not found.

Extracted those checks into check_rclone_available() and call it on the --list path (and from pre_backup_restore_checks() for the backup/restore paths), so --list now gets the same clear guidance.

Testing

  • php -l src/helper/Site_Backup_Restore.php passes.
  • Verified the gated command across rclone versions (1.53/1.60.1/1.62.2 → streams only; 1.63 → + write-buffer; 1.64/1.70 → + both; unknown → streams only) — well-formed, no stray flags or double spaces.
  • Verified ee site backup --list now routes through check_rclone_available() before any rclone invocation.

Notes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves robustness of EasyEngine’s rclone-backed site backup/restore flow by (1) avoiding restore failures on older rclone versions that hard-fail on unknown flags, and (2) ensuring ee site backup --list runs the same friendly rclone/config preflight as backup/restore paths.

Changes:

  • Gate newer rclone multi-thread download flags (--multi-thread-write-buffer-size, --multi-thread-chunk-size) behind detected rclone version.
  • Add get_rclone_version() and minimum-version constants to drive the gating behavior (safe default when version cannot be determined).
  • Extract rclone presence/backend checks into check_rclone_available() and invoke it on the --list path and from pre_backup_restore_checks().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Two robustness follow-ups to the rclone memory-budgeting work merged in EasyEngine#477:

1) Restore on older rclone (regression). rclone_download() unconditionally passed --multi-thread-write-buffer-size (added in rclone v1.63.0) and --multi-thread-chunk-size (added in v1.64.0). Older rclone hard-fails on unknown flags (Fatal error: unknown flag, exit code 2), so any rclone < 1.63 -- common in distro packages -- broke restore/rollback. Both flags are now gated behind the detected rclone version (get_rclone_version()), with thresholds in RCLONE_MIN_VERSION_MT_* constants. The memory budget already assumes their defaults, so gating them out changes only which knobs reach rclone, not the computed footprint; --transfers/--buffer-size/--multi-thread-streams (all >= v1.48, long-standing) are always passed. When the version cannot be determined, the newer flags are skipped (safe).

2) Friendly preflight on read paths. The rclone-presence and backend-config checks lived inside pre_backup_restore_checks(), which the 'backup --list' branch and the 'restore --id' branch (via verify_backup_id() -> rclone lsf) both reach before, so a missing rclone surfaced as a raw 'sh: 1: rclone: not found' (--list) or a misleading 'Invalid backup ID' (restore --id). Extracted the checks into check_rclone_available() and call it on both read paths (and from pre_backup_restore_checks() for the main backup/restore flows).
@mrrobot47 mrrobot47 force-pushed the fix/rclone-version-gate-and-preflight branch from 51fac0a to 2465122 Compare June 12, 2026 05:30
@mrrobot47 mrrobot47 requested a review from Copilot June 12, 2026 05:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/helper/Site_Backup_Restore.php:893

  • The rclone presence check runs rclone --version, which can still emit a raw shell error like sh: 1: rclone: not found (and can also print the version on success). Since the goal here is a clean, friendly preflight, prefer checking availability via command -v rclone (silenced) before any rclone invocation.
		$command     = 'rclone --version';
		$return_code = EE::exec( $command );

		if ( ! $return_code ) {

@mrrobot47 mrrobot47 merged commit 38f8aaa into EasyEngine:develop Jun 12, 2026
1 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants