Skip to content

Add management cluster upgrade CI step for openshift-dpf#81757

Draft
omertuc wants to merge 1 commit into
openshift:mainfrom
omertuc:mgmtup
Draft

Add management cluster upgrade CI step for openshift-dpf#81757
omertuc wants to merge 1 commit into
openshift:mainfrom
omertuc:mgmtup

Conversation

@omertuc

@omertuc omertuc commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

See commit message

Related rh-ecosystem-edge/openshift-dpf#237

Summary by CodeRabbit

Adds an optional OpenShift DPF CI job that upgrades the management cluster from the previous z-stream to the next one. The new step securely connects to the hypervisor, locates the latest deployed cluster, and runs make upgrade-management with configurable target version support. It is registered with required credentials, ownership, resource limits, and a two-hour timeout.

CI step for rh-ecosystem-edge/openshift-dpf#237
which adds `make upgrade-management` to the openshift-dpf repo.

The deploy-cluster step must deploy a cluster that is not on the latest
z-stream, since the upgrade script defaults to upgrading to z+1 of the
live cluster version.

Assisted-By: Claude <noreply@anthropic.com>
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 10, 2026
@openshift-ci

openshift-ci Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci

openshift-ci Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: omertuc
Once this PR has been reviewed and has the lgtm label, please assign tsorya for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Adds a DPF hypervisor management-cluster upgrade step that connects over SSH, locates the last deployed cluster, runs make upgrade-management, and registers the step as a two-hour OpenShift DPF CI test job.

Changes

DPF management upgrade

Layer / File(s) Summary
Step definition and remote execution
ci-operator/step-registry/dpf/hypervisor/upgrade-management/*
Defines ownership, metadata, credentials, runtime settings, and an SSH script that locates the deployed DPF directory and runs make upgrade-management.
CI job registration
ci-operator/config/rh-ecosystem-edge/openshift-dpf/...
Adds the upgrade-management test with reporting configuration, intranet capability, and a two-hour timeout.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CIJob
  participant UpgradeStep
  participant Hypervisor
  CIJob->>UpgradeStep: execute dpf-hypervisor-upgrade-management
  UpgradeStep->>Hypervisor: establish SSH connection
  UpgradeStep->>Hypervisor: locate last deployed DPF directory
  UpgradeStep->>Hypervisor: run make upgrade-management with KUBECONFIG
Loading

Possibly related PRs

  • openshift/release#81564: Adds a related DPF hypervisor SSH step using the same bastion host and deployment-directory lookup script.

Suggested reviewers: szigmon, wabouhamad


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error The new script logs the internal REMOTE_HOST IP and cats /tmp/last-openshift-dpf-dir.sh, exposing internal host/path data in CI logs. Remove or redact internal host/IP from echo/verbose SSH logs, and stop printing the downloaded last-openshift-dpf-dir.sh; log only sanitized status.
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning The new upgrade-management step hardcodes IPv4 10.6.135.45 in both the script and ref YAML, which violates the IPv6-safe criteria. Replace the IPv4 default with an IPv6-safe hostname/configurable value, and verify the step in an IPv6 disconnected CI job.
✅ Passed checks (13 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the new management cluster upgrade CI step added for openshift-dpf.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No Ginkgo test titles were added; the PR only changes ci config, OWNERS, and shell/YAML step-registry files.
Test Structure And Quality ✅ Passed No Ginkgo test code was touched; the PR only adds ci-operator config and step-registry assets, so the check is not applicable.
Microshift Test Compatibility ✅ Passed No Ginkgo e2e tests were added or modified; the PR only changes CI/job config and step-registry scripts.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The PR adds CI/job YAML and shell steps only; no new Ginkgo e2e tests or SNO-sensitive node-topology assumptions were introduced.
Topology-Aware Scheduling Compatibility ✅ Passed PASS: The PR only adds CI step-registry config/scripts; no workload manifests/controllers or topology-sensitive scheduling logic (affinity, nodeSelector, spread, PDBs) were introduced.
Ote Binary Stdout Contract ✅ Passed PR only adds ci-operator config and a shell step script; no OTE binary process-level stdout writes were introduced.
No-Weak-Crypto ✅ Passed PASS: The added scripts/config contain no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB usage, custom crypto, or secret comparisons.
Container-Privileges ✅ Passed Inspected the new ci-operator and step-registry YAMLs; they add refs, creds, resources, and timeouts, with no privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@omertuc: no rehearsable tests are affected by this change

Note: If this PR includes changes to step registry files (ci-operator/step-registry/) and you expected jobs to be found, try rebasing your PR onto the base branch. This helps pj-rehearse accurately detect changes when the base branch has moved forward.
The following jobs are not rehearsable without the network-access-rehearsals-ok, and approved labels present on this PR. This is due to the restrict_network_access field being set to false. The network-access-rehearsals-ok label can be added by any openshift org member other than the PR's author by commenting: /pj-rehearse network-access-allowed:

Test name
pull-ci-rh-ecosystem-edge-openshift-dpf-main-upgrade-management

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jul 10, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh (2)

19-19: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Pass SSH options as an array and quote configured values.

The unquoted ${SSH_OPTS} and ${REMOTE_HOST} expansions are fragile and can split or glob arguments. Use an array such as SSH_OPTS=(...), invoke "${SSH_OPTS[@]}", and quote "root@${REMOTE_HOST}" for both ssh and scp.

Also applies to: 23-23, 28-28, 43-43

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`
at line 19, Replace the scalar SSH_OPTS with an array of options and invoke it
as "${SSH_OPTS[@]}". Quote the remote target as "root@${REMOTE_HOST}" in every
ssh and scp invocation, including the commands at the referenced locations, to
prevent word splitting and glob expansion.

Source: Linters/SAST tools


4-6: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Verify reuse before carrying forward the duplicated SSH setup.

This new step explicitly duplicates boilerplate from sanity-existing, network-tests, and deploy-cluster. Confirm /step-finder was used and either reuse an existing helper or track the shared extraction as a follow-up; otherwise fixes to SSH and temporary-file handling will continue to drift across steps.

As per coding guidelines: always use /step-finder before creating new step-registry components.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`
around lines 4 - 6, The new step duplicates SSH and cluster-location setup
without verifying whether an existing helper can be reused. Use /step-finder to
locate shared step-registry utilities, reuse an appropriate helper if available,
and otherwise document the shared extraction as a follow-up while preserving
consistent SSH and temporary-file handling across
dpf-hypervisor-sanity-existing, network-tests, deploy-cluster, and the new
upgrade-management commands.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`:
- Line 16: The SSH configuration disables host authentication before remotely
fetching and sourcing content. Update SSH_OPTS to use a trusted mounted
known_hosts file with strict host-key verification, and replace the local source
of last-openshift-dpf-dir.sh with parsing of a single validated path after
securely retrieving it.
- Around line 12-14: Replace the predictable /tmp/id_rsa and
/tmp/last-openshift-dpf-dir.sh paths in the upgrade-management script with files
under a newly created mktemp -d directory. Register an EXIT trap to remove the
directory, and update all references in the credential decoding, chmod,
download, and script execution commands to use the private temporary paths.

---

Nitpick comments:
In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`:
- Line 19: Replace the scalar SSH_OPTS with an array of options and invoke it as
"${SSH_OPTS[@]}". Quote the remote target as "root@${REMOTE_HOST}" in every ssh
and scp invocation, including the commands at the referenced locations, to
prevent word splitting and glob expansion.
- Around line 4-6: The new step duplicates SSH and cluster-location setup
without verifying whether an existing helper can be reused. Use /step-finder to
locate shared step-registry utilities, reuse an appropriate helper if available,
and otherwise document the shared extraction as a follow-up while preserving
consistent SSH and temporary-file handling across
dpf-hypervisor-sanity-existing, network-tests, deploy-cluster, and the new
upgrade-management commands.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5584fa09-9783-49c8-bc01-81e712295d31

📥 Commits

Reviewing files that changed from the base of the PR and between b7b2f4e and 61497f2.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/rh-ecosystem-edge/openshift-dpf/rh-ecosystem-edge-openshift-dpf-main-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (5)
  • ci-operator/config/rh-ecosystem-edge/openshift-dpf/rh-ecosystem-edge-openshift-dpf-main.yaml
  • ci-operator/step-registry/dpf/hypervisor/upgrade-management/OWNERS
  • ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh
  • ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-ref.metadata.json
  • ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-ref.yaml

Comment on lines +12 to +14
cat /var/run/dpf-ci/private-key | base64 -d >/tmp/id_rsa
echo "" >>/tmp/id_rsa
chmod 600 /tmp/id_rsa

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Use a private temporary directory for credentials and downloaded scripts.

The predictable /tmp/id_rsa and /tmp/last-openshift-dpf-dir.sh paths can be pre-created as symlinks by another process in the pod, redirecting key writes or the downloaded script. Create a mktemp -d directory, use it for both files, and remove it with an EXIT trap.

Also applies to: 28-37

🧰 Tools
🪛 ast-grep (0.44.1)

[warning] 12-12: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/id_rsa
Note: [CWE-377] Insecure Temporary File.

(predictable-tmp-file-bash)


[warning] 13-13: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/id_rsa
Note: [CWE-377] Insecure Temporary File.

(predictable-tmp-file-bash)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`
around lines 12 - 14, Replace the predictable /tmp/id_rsa and
/tmp/last-openshift-dpf-dir.sh paths in the upgrade-management script with files
under a newly created mktemp -d directory. Register an EXIT trap to remove the
directory, and update all references in the credential decoding, chmod,
download, and script execution commands to use the private temporary paths.

Source: Linters/SAST tools

echo "" >>/tmp/id_rsa
chmod 600 /tmp/id_rsa

SSH_OPTS="-i /tmp/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o ConnectTimeout=30 -o ServerAliveInterval=10 -o ServerAliveCountMax=3 -o BatchMode=yes"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Pin the hypervisor host key before sourcing remote content.

StrictHostKeyChecking=no with UserKnownHostsFile=/dev/null disables server authentication. Because the script fetches and sources last-openshift-dpf-dir.sh, a wrong host or MITM can execute arbitrary code in the CI pod. Mount a trusted known_hosts/fingerprint and enable strict verification; the local source should also be removed or replaced with parsing of a single validated path.

Also applies to: 35-37

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/dpf/hypervisor/upgrade-management/dpf-hypervisor-upgrade-management-commands.sh`
at line 16, The SSH configuration disables host authentication before remotely
fetching and sourcing content. Update SSH_OPTS to use a trusted mounted
known_hosts file with strict host-key verification, and replace the local source
of last-openshift-dpf-dir.sh with parsing of a single validated path after
securely retrieving it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant