-
Notifications
You must be signed in to change notification settings - Fork 4
Add synthetic base fixtures and the SONiC E2E job #2566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| --- | ||
| # Node preparation for the SONiC config-generation E2E golden test. | ||
| # | ||
| # The test's NetBox fixture is a docker compose stack (tests/e2e/compose.yaml), | ||
| # so the node only needs Docker: ensure-docker installs docker-compose-plugin | ||
| # along with docker-ce. No kind, kubectl or helm. netbox-manager's own E2E job | ||
| # still provisions NetBox on kind, so there is nothing to keep in sync here. | ||
| - name: Prepare the SONiC E2E node | ||
| hosts: all | ||
|
|
||
| pre_tasks: | ||
| # This CI node is IPv6-only and learns its address and default route via | ||
| # SLAAC / Router Advertisements. If Docker enables | ||
| # net.ipv6.conf.all.forwarding=1, the kernel stops honouring RAs at the | ||
| # default accept_ra=1, so the SLAAC default route expires and the node | ||
| # drops off the network a few minutes into the run. accept_ra=2 keeps RAs | ||
| # honoured even while forwarding is on, preserving the default route. | ||
| # Compose's default bridge network is IPv4-only, so Docker probably will | ||
| # not turn on IPv6 forwarding here -- but this guard is cheap insurance | ||
| # against a severe failure mode (an unreachable CI node) and is kept | ||
| # deliberately; removing it should be its own separate experiment, not a | ||
| # side effect of some other change. Set it here -- before Docker enables | ||
| # forwarding -- so the route never lapses. See | ||
| # https://docs.docker.com/engine/daemon/ipv6/ and | ||
| # https://forums.docker.com/t/docker-removes-host-ipv6-default-route/83238 | ||
| - name: Keep accepting IPv6 RAs after Docker enables forwarding (preserve default route) | ||
| become: true | ||
| ansible.builtin.copy: | ||
| dest: /etc/sysctl.d/99-sonic-e2e-accept-ra.conf | ||
| owner: root | ||
| group: root | ||
| mode: "0644" | ||
| content: | | ||
| net.ipv6.conf.all.accept_ra = 2 | ||
| net.ipv6.conf.default.accept_ra = 2 | ||
| {% if ansible_default_ipv6.interface is defined %} | ||
| net.ipv6.conf.{{ ansible_default_ipv6.interface }}.accept_ra = 2 | ||
| {% endif %} | ||
|
|
||
| - name: Apply the accept_ra sysctl settings now | ||
| become: true | ||
| ansible.builtin.command: | ||
| cmd: sysctl -p /etc/sysctl.d/99-sonic-e2e-accept-ra.conf | ||
| changed_when: true | ||
| # A node without an ansible_default_ipv6.interface fact (or otherwise | ||
| # missing these IPv6 keys) fails sysctl -p; this is best-effort | ||
| # insurance for the common IPv6-only case, not a hard requirement, so | ||
| # do not fail pre-run over it. | ||
| failed_when: false | ||
|
|
||
| roles: | ||
| - ensure-pip | ||
| - ensure-pipenv | ||
| - ensure-docker | ||
|
|
||
| tasks: | ||
| - name: Ensure the Docker service is running | ||
| become: true | ||
| ansible.builtin.service: | ||
| name: docker | ||
| state: started | ||
| enabled: true | ||
|
|
||
| # openssl is used by tests/e2e/sonic_golden_test.sh directly. curl is used | ||
| # by the NetBox container's healthcheck rather than by the script itself, | ||
| # but ensure-docker needs it anyway. python3-venv provides the venv module | ||
| # the script uses for the seeding venv. | ||
| - name: Install required packages | ||
| become: true | ||
| ansible.builtin.apt: | ||
| name: | ||
| - curl | ||
| - openssl | ||
| - python3-venv |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| - name: Run the SONiC config-generation E2E golden test | ||
| hosts: all | ||
|
|
||
| vars: | ||
| python_venv_dir: /tmp/venv | ||
|
|
||
| tasks: | ||
| - name: Install dependencies | ||
| ansible.builtin.shell: | ||
| executable: /bin/bash | ||
| chdir: "{{ zuul.project.src_dir }}" | ||
| cmd: | | ||
| set -e | ||
| set -o pipefail | ||
| set -x | ||
|
|
||
| {{ python_venv_dir }}/bin/pipenv install --dev --deploy | ||
| {{ python_venv_dir }}/bin/pipenv run pip install . | ||
|
|
||
| - name: Run the E2E golden test | ||
| ansible.builtin.shell: | ||
| executable: /bin/bash | ||
| chdir: "{{ zuul.project.src_dir }}" | ||
| cmd: | | ||
| set -e | ||
| set -o pipefail | ||
| set -x | ||
|
|
||
| # The script invokes bare `pipenv`; the netbox-manager checkout comes | ||
| # from Zuul's required-projects, so a Depends-On change to it is | ||
| # tested against the changed code and seed data. | ||
| export PATH="{{ python_venv_dir }}/bin:${PATH}" | ||
| export NETBOX_MANAGER_DIR="{{ ansible_user_dir }}/{{ zuul.projects['github.com/osism/netbox-manager'].src_dir }}" | ||
|
|
||
| # Serial by default: concurrent seeding deadlocks on dcim_device FK | ||
| # row locks (see tests/e2e/sonic_golden_test.sh, Phase 2). Set | ||
| # `sonic_e2e_seed_parallel: 4` in the job vars to opt back in. | ||
| export SEED_PARALLEL="{{ sonic_e2e_seed_parallel | default(1) }}" | ||
| tests/e2e/sonic_golden_test.sh |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """config_db table coverage report for the SONiC E2E golden set. | ||
|
|
||
| This is a reporting tool, not a gate: it is not invoked from | ||
| sonic_golden_test.sh, and the golden comparison (tests/e2e/compare.py) | ||
| remains the only check that actually fails a run. It exists so the "N of M | ||
| config_db tables covered" claim made when the golden set is extended can be | ||
| re-derived by anyone, at any time, instead of only having existed as an | ||
| ad-hoc one-off script run once during development. | ||
|
|
||
| It works in two independent steps: | ||
|
|
||
| 1. Derive the set of tables the generator can emit by grepping | ||
| osism/tasks/conductor/sonic/ (excluding the generated schema package, | ||
| _generated/, which is data rather than emission logic) for direct | ||
| ``config["TABLE"]``/``cfg["TABLE"]`` assignments -- including a nested | ||
| item assignment such as ``config["ACL_TABLE"]["SSH_ONLY"] = ...`` or a | ||
| ``.update(...)`` call, but not a mere read such as | ||
| ``"x" in config["VERSIONS"]``. This is a static, syntactic approximation | ||
| of "tables the generator can populate", not a guarantee every branch | ||
| that reaches it is exercised. | ||
| 2. Collect the set of tables that are non-empty in at least one file under | ||
| tests/e2e/golden/*.json. | ||
|
|
||
| The report prints both counts and, if any emitted table is never non-empty | ||
| across the golden set, lists them and exits 1. | ||
| """ | ||
|
|
||
| import re | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| REPO_ROOT = Path(__file__).resolve().parents[2] | ||
| GENERATOR_DIR = REPO_ROOT / "osism" / "tasks" / "conductor" / "sonic" | ||
| GOLDEN_DIR = REPO_ROOT / "tests" / "e2e" / "golden" | ||
|
|
||
| # Matches config["TABLE"] or cfg["TABLE"], optionally followed by one or | ||
| # more ["key"]/[expr] accessors, then either an assignment ("=" but not | ||
| # "==") or a .update(...) call -- i.e. the table is a write target, not | ||
| # merely read. | ||
| _TABLE_ASSIGNMENT_RE = re.compile( | ||
| r'(?:config|cfg)\["([A-Z][A-Z0-9_]*)"\](?:\[[^\]]*\])*\s*(?:=(?!=)|\.update\()' | ||
| ) | ||
|
|
||
|
|
||
| def emitted_tables(generator_dir=GENERATOR_DIR): | ||
| """Tables the generator can emit, derived from source assignments.""" | ||
| tables = set() | ||
| for path in sorted(Path(generator_dir).rglob("*.py")): | ||
| if "_generated" in path.parts: | ||
| continue | ||
| for line in path.read_text().splitlines(): | ||
| if line.lstrip().startswith("#"): | ||
| continue | ||
| for match in _TABLE_ASSIGNMENT_RE.finditer(line): | ||
| tables.add(match.group(1)) | ||
| return tables | ||
|
|
||
|
|
||
| def covered_tables(golden_dir=GOLDEN_DIR): | ||
| """Tables that are non-empty in at least one golden file.""" | ||
| import json | ||
|
|
||
| tables = set() | ||
| for path in sorted(Path(golden_dir).glob("*.json")): | ||
| config = json.loads(path.read_text()) | ||
| for table, value in config.items(): | ||
| if value: | ||
| tables.add(table) | ||
| return tables | ||
|
|
||
|
|
||
| def main(argv=None): | ||
| emitted = emitted_tables() | ||
| covered = covered_tables() | ||
| missing = sorted(emitted - covered) | ||
|
|
||
| print(f"generator emits {len(emitted)} tables; {len(covered)} non-empty") | ||
| if missing: | ||
| print("not covered by any golden file:") | ||
| for table in missing: | ||
| print(f" {table}") | ||
| return 1 | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.