rabbitmq: resolve internal_interface through Ansible - #2578
Conversation
72a4cdb to
2941d94
Compare
2941d94 to
f1875b4
Compare
f1875b4 to
5b3a4a1
Compare
5b3a4a1 to
6db241e
Compare
6db241e to
45650ab
Compare
There was a problem hiding this comment.
Hey - I've found 1 security issue, and 1 other issue
Security issues:
- Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'. (link)
Fixed security issues:
- Command injection from untrusted input passed to OS command execution (link)
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="osism/utils/inventory.py" line_range="172" />
<code_context>
+ )
+
+ try:
+ with open(value_path) as fp:
+ return fp.read()
+ except OSError as exc:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Open the result file with an explicit encoding to avoid locale-dependent behavior.
Using the platform default encoding is brittle for non-ASCII content and non‑UTF‑8 locales. Please specify `encoding="utf-8"` here and ensure the producing side also writes UTF‑8 so the encoding is consistent end to end.
Suggested implementation:
```python
try:
with open(value_path, encoding="utf-8") as fp:
return fp.read()
except OSError as exc:
raise HostContextResolutionError("ansible wrote no value") from exc
```
To fully implement your suggestion end-to-end, you should also verify that the Ansible task or plugin producing `value_path` writes the file using UTF-8 encoding (e.g., ensure templates or modules use UTF-8 and that any explicit file writes specify `encoding: utf-8` or equivalent). Those changes will be in the Ansible side rather than in `osism/utils/inventory.py`.
</issue_to_address>
### Comment 2
<location path="osism/utils/inventory.py" line_range="150-156" />
<code_context>
result = subprocess.run(
command,
capture_output=True,
text=True,
timeout=timeout,
env=env,
)
</code_context>
<issue_to_address>
**security (python.lang.security.audit.dangerous-subprocess-use-audit):** Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'.
*Source: opengrep*
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| ) | ||
|
|
||
| try: | ||
| with open(value_path) as fp: |
There was a problem hiding this comment.
suggestion (bug_risk): Open the result file with an explicit encoding to avoid locale-dependent behavior.
Using the platform default encoding is brittle for non-ASCII content and non‑UTF‑8 locales. Please specify encoding="utf-8" here and ensure the producing side also writes UTF‑8 so the encoding is consistent end to end.
Suggested implementation:
try:
with open(value_path, encoding="utf-8") as fp:
return fp.read()
except OSError as exc:
raise HostContextResolutionError("ansible wrote no value") from excTo fully implement your suggestion end-to-end, you should also verify that the Ansible task or plugin producing value_path writes the file using UTF-8 encoding (e.g., ensure templates or modules use UTF-8 and that any explicit file writes specify encoding: utf-8 or equivalent). Those changes will be in the Ansible side rather than in osism/utils/inventory.py.
Add resolve_in_host_context(), which templates a Jinja2 expression in a
host's Ansible variable context and returns the result. Nothing calls it
yet; the callers follow.
"ansible-inventory --host" returns variables as defined, so anything
Jinja-valued comes back as the raw "{{ ... }}". osism/defaults
all/README.md ("Consuming these values from code") tells external tooling
how such a value has to be resolved instead: through templating in the
host context, never by re-implementing the expression in the consumer.
This helper is that operation, in one place, so callers do not each grow
their own approximation of Jinja.
It runs "ansible <host> -m copy" with the expression as the content and a
file in a temporary directory as the destination, then reads the file:
- "-c local" keeps the module on the controller. No connection is made to
the host, so a node that is down or unreachable still resolves, and
ANSIBLE_GATHERING=explicit states that no fact gathering happens.
- The value is read back from a file rather than parsed out of Ansible's
output. The package runs under more than one ansible-core (2.19.11 in
its own image, 2.18.x in the osism-ansible, kolla-ansible and
ceph-ansible images), and their callbacks differ in wording and in
which stream they use. "--tree" would give structured output but is
deprecated for removal in ansible-core 2.23.
- Module arguments are passed as JSON, so a value containing spaces
survives.
- Facts are supplied as extra vars rather than through a fact-cache
plugin. The redis cache plugin lives in community.general, which is not
installed here, and a jsonfile cache is honoured by an ad-hoc lookup on
2.18 but not on 2.19. Extra vars behave identically on both, and the
precedence is right for facts, which already outrank host_vars.
- Failure is decided by the return code alone. The undefined-variable
sentinel differs between the two versions ("VARIABLE IS NOT DEFINED!"
against "<<error1-'x' is undefined>>"), so matching on it would work on
one and not the other. On failure the error text Ansible produced is
passed through unparsed, because it names the variable that could not be
resolved.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
get_rabbitmq_node_addresses() read internal_interface with
"ansible-inventory --host", which returns variables as defined, so a
Jinja-valued internal_interface came back as the raw "{{ ... }}". It then
resolved that by hand: a regex captured the contents of the first
"{{ ... }}" and the dotted path was walked through the ansible facts.
That covers exactly one shape -- a single expression whose dotted path is
rooted in facts, i.e. the testbed's
internal_interface: "{{ ansible_local.testbed_network_devices.management }}"
and nothing else:
- An internal_interface pointing at an inventory variable fails, because
the walk only ever looks in the facts:
"Could not resolve template '{{ some_var }}' from facts for <host>".
- A mixed literal and template such as "vlan{{ vlan_id }}" is passed
through verbatim, because re.match requires "{{" at offset 0, and the
lookup then asks for a fact named ansible_vlan{{ vlan_id }}.
- The regex is unanchored, so "{{ base }}.100" matches only the leading
expression and silently discards the ".100" tail, resolving to the
wrong interface -- a wrong answer rather than an error.
- Filters, hostvars lookups and defaults are not evaluated at all.
Use resolve_in_host_context() instead, so Ansible does the templating in
the host's own variable context. This is what osism/defaults
all/README.md ("Consuming these values from code") prescribes for
external consumers, and it makes the supported set "whatever Jinja2
supports" rather than a list of anticipated shapes. The whole resolver
goes away, along with the subsequent walk from interface name to
ansible_<name> to ipv4.address, since the expression covers all of it.
The resolved value is validated as an IPv4 address before use. The return
code already reports a templating failure, but a value that comes back
looking nothing like an address must not be passed on as one either.
The integration case for an internal_interface that points at an inventory
variable was marked xfail(strict) when it was added, because the resolver
could not resolve it. It passes now, so the marker goes: with strict set,
leaving it would fail the suite on the unexpected pass. That is the
demonstration this change needed -- the shapes that already worked are
still covered by the same tests, and the one that did not now passes
against real Ansible rather than against a mock.
The tests for the deleted resolver go with it: Jinja2 traversal, the
non-string and non-dict cases, the interface-name to fact-key mapping and
the ipv4 extraction all tested behaviour that is now Ansible's. What
replaces them asserts the contract that remains -- that the expression and
the cached facts are handed over unchanged, that a resolution failure
surfaces Ansible's own message, and that a non-address result is refused.
Verified against ansible-core 2.18.9 and 2.19.11 with the reported
variable shape ("{{ vlan_var }}" where vlan_var is itself
"vlan{{ id }}"), a dotted interface name, a dashed one, and the
fact-derived shape the old resolver supported, plus a missing fact.
One behaviour change worth noting for review: a missing internal_interface
is now reported through Ansible's undefined-variable message rather than
a dedicated one.
osism status rabbitmq shares the helper and is fixed with it.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
45650ab to
386e976
Compare
|
@sourcery-ai dismiss |
Fixes the reported bug. Two commits: the reusable helper, then the switch-over.
The bug
get_rabbitmq_node_addresses()readinternal_interfacewithansible-inventory --host, which returns variables as defined, so a Jinja-valuedinternal_interfacecame back as the raw{{ ... }}. It then resolved that by hand: a regex captured the contents of the first{{ ... }}and the dotted path was walked through the ansible facts.That covers exactly one shape — a single expression whose dotted path is rooted in facts, i.e. the testbed's
internal_interface: "{{ ansible_local.testbed_network_devices.management }}"— and nothing else:internal_interfacepointing at an inventory variable fails, because the walk only looks in the facts:Could not resolve template '{{ some_var }}' from facts for <host>. This is what the reporter hit.vlan{{ vlan_id }}is passed through verbatim, becausere.matchrequires{{at offset 0; the lookup then asks for a fact namedansible_vlan{{ vlan_id }}.{{ base }}.100matches only the leading expression and silently discards the.100tail, resolving to the wrong interface — a wrong answer rather than an error.hostvarslookups anddefault()are not evaluated at all.The fix
Delegate the lookup to Ansible, which is what
osism/defaultsall/README.md("Consuming these values from code") prescribes for external consumers of these variables. The supported set becomes "whatever Jinja2 supports" instead of a list of anticipated shapes. The resolver goes away, along with the subsequent walk from interface name toansible_<name>toipv4.address, since the expression covers all of it.resolve_in_host_context()has Ansible template the expression andcopythe result into a file, run with-c localso the module executes on the controller: no connection is made, a node that is down still resolves, and the value is read back byte-exact rather than parsed out of human-readable output.The package runs under more than one ansible-core — 2.19.11 in its own image, 2.18.x in the osism-ansible, kolla-ansible and ceph-ansible images — so the lookup avoids every surface where those differ:
redisplugin lives incommunity.general, which is not installed here, and ajsonfilecache is honoured by an ad-hoc lookup on 2.18 but not on 2.19.--treeis deprecated for removal in 2.23.VARIABLE IS NOT DEFINED!on 2.18 and<<error1-'x' is undefined>>on 2.19.The resolved value is validated as an IPv4 address before use, so a non-address string cannot be passed on as one.
Coverage
The
xfail(strict=True)case added in the previous PR passes now, so its marker goes — withstrictset, leaving it would fail the suite on the unexpected pass. That transition is the demonstration: the shapes that already worked are still covered by the same tests, and the one that did not now passes against real Ansible.The two remaining unit tests that described only the deleted resolver's internals (a facts-walk yielding a non-string, and hitting a non-dict) go with the code they covered — Ansible has no such failure modes, so there is nothing to express at the integration level.
Verification. Against ansible-core 2.18.9 and 2.19.11: the reported shape, a dotted interface name, a dashed one, the fact-derived shape the old resolver supported, and a missing fact.
One behaviour change for review: a missing
internal_interfaceis now reported through Ansible's undefined-variable message rather than a dedicated one.osism status rabbitmqshares the helper and is fixed with it.Fixes:
🤖 Generated with Claude Code