Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

* **role:duplicity**: Add Debian and Ubuntu support (proven on Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04 and Ubuntu 26.04). The role now installs the `gnupg` package itself, so backups also work on minimal installs that ship without `gpg`.
* **role:python_venv**: Add an optional per-venv `pip_constraints` key that pins transitive dependencies through a pip constraints file, without having to list them as direct packages.
* **role:opensearch**: Add `opensearch__path_repo` to register file system paths as `path.repo` in `opensearch.yml`, required for file system based snapshot repositories.
* **role:nextcloud**: Add `nextcloud__jobs_timeout_start_sec` to configure the start-up timeout of the `nextcloud-jobs.service`. Defaults to `10m`. Raise it on instances where background jobs regularly need longer than 10 minutes.
* **role:icinga2_agent, role:icinga2_master**: Deploy a systemd drop-in override ensuring the Icinga 2 service starts after SSSD on hosts where SSSD is installed.
Expand All @@ -39,6 +41,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

* **role:duplicity**: Validate the role variables at start, and align the task tags with the LFOps vocabulary. The `duplicity:script` tag is gone (the `duba` script now deploys under `duplicity:configure`), and the new `duplicity:dump` tag manages the backup schedule.
* **role:schedule_reboot**: Hosts without an explicit reboot window no longer all reboot at exactly 04:00. Each host is now assigned a deterministic minute within the 04:00-04:59 window (staggered by hostname), so a fleet does not reboot in lockstep. Pin `schedule_reboot__reboot_time__group_var` (or the `__host_var`) to a fixed time to keep a specific window.
* **role:monitoring_plugins**: A source install now deploys the plugins into a self-contained Python virtual environment and provisions a suitable Python by itself, so it works on RHEL 8 (where the system Python 3.6 is too old) without any manual Python setup. The Linuxfabrik library (an independent project) is deployed newest straight from GitHub and the third-party dependencies are installed unpinned, so a source install always tracks the newest code for the selected `monitoring_plugins__version`. The dependencies the former source install placed into the home directories of root and the icinga user (`pip --user`) are cleaned up on the next run.
* **role:icinga2_agent**: The `icinga2_agent:update` tag now refreshes the apt cache before the upgrade on Debian-family hosts, so it reliably installs the latest package (e.g. to roll out security updates) instead of running against a stale cache. RHEL-family hosts are unaffected, since dnf refreshes its metadata on its own.
Expand All @@ -49,6 +52,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

* **role:duplicity**: Swift backups now work out of the box on Python 3.10 and newer (all supported RHEL, Debian and Ubuntu releases). The role pins a modern `oslo.*` stack in the venv, which fixes the `collections.Mapping` crash (the previously required manual workaround is gone) and drops the deprecated, source-only `netifaces` dependency. As a result the role no longer installs a C compiler (`gcc`) or development headers on backup hosts: the build toolchain is gone from production machines, which is a significant reduction of the attack surface.
* **role:keycloak**: Keycloak is no longer restarted a second time right after it was started on a fresh install, and a configuration change no longer starts the service on hosts that pin `keycloak__state` to `stopped`.
* **role:nextcloud**: Set the `text workspace_available` app config key as `boolean` instead of `string`. Newer Nextcloud enforces the config lexicon - the text app declares the key as `ValueType::BOOL`
* **role:php**: The `php:update` tag now refreshes the apt cache before the upgrade on Debian-family hosts, so it reliably installs the latest packages (e.g. to roll out security updates) instead of running against a stale cache. RHEL-family hosts are unaffected, since dnf refreshes its metadata on its own.
Expand Down
2 changes: 1 addition & 1 deletion COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Which Ansible role is proven to run on which OS?
| dnf_makecache | | | x | x | x | | | | |
| dnf_versionlock | | | x | x | | | | | Fedora 40 |
| docker | | | x | (x) | (x) | | | | |
| duplicity | | | x | x | x | | | | Fedora 35 |
| duplicity | x | x | x | x | x | x | x | x | Fedora 35 |
| elastic_agent | (x) | (x) | (x) | x | (x) | (x) | x | (x) | |
| elastic_agent_fleet_server | (x) | (x) | (x) | x | (x) | (x) | x | (x) | |
| elasticsearch | (x) | (x) | x | x | (x) | (x) | x | (x) | |
Expand Down
12 changes: 4 additions & 8 deletions roles/duplicity/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Note that this role does not support running with `--check`, as it first creates

Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/README.md) that installs this role runs these for you. Optional ones can be disabled via the playbook's skip variables.

* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed.
* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed. On Debian and Ubuntu, no additional repositories are required.
* `duplicity`, `python-swiftclient` and `python-keystoneclient` must be installed into a Python 3 virtual environment in `/opt/python-venv/duplicity` (role: [linuxfabrik.lfops.python_venv](https://github.com/Linuxfabrik/lfops/tree/main/roles/python_venv)).

**Attention**
Expand All @@ -43,12 +43,12 @@ Manual steps:

`duplicity:configure`

* Deploys the configuration for duplicity.
* Deploys the configuration for duplicity, including the `duba` script.
* Triggers: none.

`duplicity:script`
`duplicity:dump`

* Just deploys the `duba` script.
* Manages the daily backup schedule (deploys and enables the `duba` systemd timer).
* Triggers: none.

`duplicity:state`
Expand Down Expand Up @@ -276,10 +276,6 @@ duplicity__timer_enabled: true

* Make sure your `duplicity__gpg_encrypt_master_key_block` is correct and has an empty line after the `-----BEGIN PGP PUBLIC KEY BLOCK-----`.

**`duplicity` fails with `AttributeError: module 'collections' has no attribute 'Mapping'` in `oslo_config/cfg.py`**

* Manually install `'oslo.config>=9'`, e.g. `/opt/python-venv/duplicity/bin/pip install 'oslo.config>=9'`.

## License

[The Unlicense](https://unlicense.org/)
Expand Down
2 changes: 1 addition & 1 deletion roles/duplicity/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,4 @@ duplicity__loglevel: 'notice'

duplicity__swift_authurl: 'https://swiss-backup02.infomaniak.com/identity/v3'
duplicity__swift_authversion: '3'
duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] }}'
duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] | d("") }}'
143 changes: 143 additions & 0 deletions roles/duplicity/meta/argument_specs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# argument_specs validates required variables and types automatically at role entry.
# use this for simple "is defined" / type checks. for complex validations
# (value ranges, cross-variable logic), use ansible.builtin.assert in the tasks.
argument_specs:
main:
options:

duplicity__backup_backend:
type: 'str'
required: false
default: 'swift'
choices:
- 'sftp'
- 'swift'
description: 'The backup backend being used.'

duplicity__backup_dest:
type: 'str'
required: false
description: 'The backup destination, combined with the backup source path to form the target URL for duplicity.'

duplicity__backup_dest_container:
type: 'str'
required: false
description: 'The Swift container, used to separate backups on the destination.'

duplicity__backup_full_if_older_than:
type: 'str'
required: false
default: '30D'
description: 'After how long a full backup instead of an incremental one is done.'

duplicity__backup_retention_time:
type: 'str'
required: false
default: '30D'
description: 'The retention time of the backups.'

duplicity__backup_sources__dependent_var:
type: 'list'
elements: 'dict'
required: false
default: []
description: 'Directories to back up. Dependent-role injection.'

duplicity__backup_sources__group_var:
type: 'list'
elements: 'dict'
required: false
default: []
description: 'Directories to back up. Group-level override.'

duplicity__backup_sources__host_var:
type: 'list'
elements: 'dict'
required: false
default: []
description: 'Directories to back up. Host-level override.'

duplicity__excludes:
type: 'list'
elements: 'str'
required: false
default:
- '**/*.git*'
- '**/*.svn*'
- '**/*.temp'
- '**/*.tmp'
- '**/.cache'
- '**/cache'
- '**/log'
description: 'Global exclude shell patterns for duplicity.'

duplicity__gpg_encrypt_master_key:
type: 'str'
required: true
description: 'The long key ID of the master GPG key.'

duplicity__gpg_encrypt_master_key_block:
type: 'str'
required: true
description: 'The ASCII-armored public master GPG key, used in addition to the local key to encrypt the backups.'

duplicity__loglevel:
type: 'str'
required: false
default: 'notice'
choices:
- 'debug'
- 'error'
- 'info'
- 'notice'
- 'warning'
description: 'The duplicity log level.'

duplicity__logrotate:
type: 'int'
required: false
description: 'Number of days log files are kept before being rotated out.'

duplicity__on_calendar:
type: 'str'
required: false
description: 'The OnCalendar definition for the daily systemd timer.'

duplicity__on_calendar_hour:
type: 'str'
required: false
default: '23'
description: 'Shorthand to set the hour of duplicity__on_calendar.'

duplicity__sftp_password:
type: 'str'
required: false
description: 'Password for the SSH user used by the SFTP connection. Required when duplicity__backup_backend is sftp.'

duplicity__swift_authurl:
type: 'str'
required: false
default: 'https://swiss-backup02.infomaniak.com/identity/v3'
description: 'The authentication URL for Swift.'

duplicity__swift_authversion:
type: 'str'
required: false
default: '3'
description: 'The authentication version for Swift.'

duplicity__swift_login:
type: 'dict'
required: false
description: 'The Swift username and password (subkeys username and password). Required when duplicity__backup_backend is swift.'

duplicity__swift_tenantname:
type: 'str'
required: false
description: 'The Swift tenant name.'

duplicity__timer_enabled:
type: 'bool'
required: false
default: true
description: 'The state of the daily systemd timer.'
45 changes: 28 additions & 17 deletions roles/duplicity/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,17 @@
name: 'shared'
tasks_from: 'platform-variables.yml'

tags:
- 'always'


- block:

- name: 'Install required packages'
ansible.builtin.package:
name: '{{ __duplicity__required_packages }}'
state: 'present'

- name: 'mkdir /etc/duba'
ansible.builtin.file:
path: '/etc/duba'
Expand All @@ -19,7 +30,7 @@

- block:

- name: 'Combined Paths:'
- name: 'Combined Paths'
ansible.builtin.debug:
var: 'duplicity__backup_sources__combined_var'

Expand All @@ -43,11 +54,6 @@
register: 'duplicity__gpg_import_result'
changed_when: '"not changed" not in duplicity__gpg_import_result.stderr'

- name: 'gpg --import /tmp/public-master-key'
ansible.builtin.command: 'gpg --import /tmp/public-master-key'
register: 'duplicity__gpg_import_result'
changed_when: '"not changed" not in duplicity__gpg_import_result.stderr'

- name: 'rm -f /tmp/public-master-key'
ansible.builtin.file:
path: '/tmp/public-master-key'
Expand All @@ -66,6 +72,8 @@
backup: true
src: 'etc/duba/duba.json.j2'
dest: '/etc/duba/duba.json'
owner: 'root'
group: 'root'
mode: 0o600 # file contains secrets

- name: 'Deploy /etc/systemd/system/duba.service'
Expand All @@ -77,22 +85,22 @@
group: 'root'
mode: 0o644

- name: 'Deploy /etc/systemd/system/duba.timer'
- name: 'Deploy /usr/local/bin/duba'
ansible.builtin.template:
backup: true
src: 'etc/systemd/system/duba.timer.j2'
dest: '/etc/systemd/system/duba.timer'
src: 'usr/local/bin/duba.j2'
dest: '/usr/local/bin/duba'
owner: 'root'
group: 'root'
mode: 0o644
register: 'duplicity__systemd_duba_timer_result'
mode: 0o755

- name: 'Deploy /etc/logrotate.d/duplicity'
ansible.builtin.template:
backup: true
src: 'etc/logrotate.d/duplicity.j2'
dest: '/etc/logrotate.d/duplicity'
owner: 'root'
group: 'root'
mode: 0o644

tags:
Expand All @@ -102,18 +110,20 @@

- block:

- name: 'Deploy /usr/local/bin/duba'
- name: 'Deploy /etc/systemd/system/duba.timer'
ansible.builtin.template:
backup: true
src: 'usr/local/bin/duba.j2'
dest: '/usr/local/bin/duba'
src: 'etc/systemd/system/duba.timer.j2'
dest: '/etc/systemd/system/duba.timer'
owner: 'root'
mode: 0o755
group: 'root'
mode: 0o644
register: 'duplicity__systemd_duba_timer_result'

tags:
- 'duplicity'
- 'duplicity:configure'
- 'duplicity:script'
- 'duplicity:dump'


- block:
Expand All @@ -122,9 +132,10 @@
ansible.builtin.systemd:
name: 'duba.timer'
state: '{{ duplicity__timer_enabled | bool | ternary("started", "stopped") }}'
enabled: '{{ duplicity__timer_enabled }}'
enabled: '{{ duplicity__timer_enabled | bool }}'
daemon_reload: '{{ duplicity__systemd_duba_timer_result | d({}) is changed }}'

tags:
- 'duplicity'
- 'duplicity:dump'
- 'duplicity:state'
3 changes: 3 additions & 0 deletions roles/duplicity/vars/Debian.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# gpg is required for key generation, key import and duplicity's encryption at backup time.
__duplicity__required_packages:
- 'gnupg'
3 changes: 3 additions & 0 deletions roles/duplicity/vars/RedHat.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# gpg is required for key generation, key import and duplicity's encryption at backup time.
__duplicity__required_packages:
- 'gnupg2'
3 changes: 3 additions & 0 deletions roles/duplicity/vars/Ubuntu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# gpg is required for key generation, key import and duplicity's encryption at backup time.
__duplicity__required_packages:
- 'gnupg'
Loading
Loading