nist_sync: export NIST 800-53 Rev 5 controls to Gemara format #14797
nist_sync: export NIST 800-53 Rev 5 controls to Gemara format #14797ggbecker wants to merge 12 commits into
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
Implements OPENSCAP-6801. Adds tooling to export ComplianceAsCode NIST 800-53 Rev 5 control files (rhel8/9/10) to three Gemara artifact types: GuidanceCatalog (utils/nist_sync/gemara/guidance.py) Abstract NIST 800-53 standard layer built from the OSCAL catalog at utils/nist_sync/data/nist_800_53_rev5_catalog.json. Includes official control statement prose (with parameter substitution) as objectives, OSCAL guidance prose as rationale, and baseline applicability (low, moderate, high) from the three baseline JSON files. ControlCatalog (utils/nist_sync/gemara/catalog.py) Per-product: maps 1,196 NIST controls to XCCDF rule IDs as assessment requirements. NIST families become groups; baselines become applicability- groups. CaC control status maps to Gemara lifecycle states. MappingDocument (utils/nist_sync/gemara/mapping.py) Per-product: bidirectional traceability between NIST control IDs and ComplianceAsCode rule IDs with relationship type, strength, and confidence level derived from the CaC control status. All three artifact types pass CUE schema validation against the Gemara 1.1.0 schema (cue vet). Supporting files: gemara/status_map.py - CaC status → Gemara state/relationship/confidence gemara/schema.py - schema constants and structural validators export_to_gemara.py - CLI: --products, --validate, --gemara-schema, --no-guidance test_gemara_export.py - 87-test suite: structure, cross-references, accuracy generate_complyctl_bundle.py - OCI bundle generator for complyctl testing test_complyctl_e2e.sh - E2E test in UBI9 container against RHEL9 data stream .github/workflows/export-to-gemara.yml - CI on NIST control/script changes
…ibility Three root causes were found by reading the complytime-providers source code and inspecting provider gRPC logs: 1. Assessment-plan id must equal requirement-id go-gemara v0.0.1 populates AssessmentConfiguration.RequirementID from the plan 'id' field, not 'requirement-id'. The 'ap-' prefix we added made the id (e.g. 'ap-accounts_tmout') not match any data stream rule. Fix: use the short CaC rule name as both id and requirement-id. 2. requirement-id must be the short CaC rule name (no XCCDF prefix) filterValidRules() strips 'xccdf_org.ssgproject.content_rule_' from data stream rule IDs before comparing. Passing full XCCDF IDs caused no match. 3. 'datastream' target variable bypasses OS auto-detection On UBI9, ID_LIKE=fedora causes the provider to pick ssg-fedora-ds.xml before ssg-rhel9-ds.xml (alphabetical filesystem walk). Setting the 'datastream' variable in complytime.yaml targets the correct product data stream directly without overriding /etc/os-release. Also updates test_complyctl_e2e.sh to reflect the correct setup: no /etc/os-release override needed, uses 'datastream' variable and 'cis' as the XCCDF tailoring base profile (covers all 22 NIST 800-53 moderate rules). Verified end-to-end in a UBI9 container against ssg-rhel9-ds.xml: complyctl get → bundle synced complyctl generate → tailoring.xml created (22 requirements) complyctl scan → 3 passed, 1 failed (configure_custom_crypto_policy_cis) Results: arf.xml + evaluation-log written to /results/
|
For some reason, the test script passes everything except the first check Indeed, no guidance_catalog.yaml was generated when I run export_to_gemara.py. Is that expected? |
The guidance_catalog.yaml requires the OSCAL catalog (nist_800_53_rev5_catalog.json), which is not part of the project, run download_oscal.py first: python3 utils/nist_sync/download_oscal.py I've updated the output to point this out |
The OSCAL catalog (nist_800_53_rev5_catalog.json) is gitignored because it is 10 MB. When a reviewer runs export_to_gemara.py without downloading it first, guidance_catalog.yaml is silently not produced; the test script then reports [FAIL] with a misleading "run export_to_gemara.py first" message even though they already did. - export_to_gemara.py: print an explicit [SKIP] block with instructions to run download_oscal.py when the OSCAL catalog is not available - test_gemara_export.py: treat missing guidance_catalog.yaml as [SKIP] (not [FAIL]), consistent with how missing per-product files are handled
…icies Audit against https://github.com/complytime/complytime-policies revealed several schema conformance issues. Fixed: - Bump gemara-version from 1.1.0 to 1.2.0 (current version used in complytime-policies CI cue vet validation) - author.type: "Human" → "Software" in both ControlCatalog and MappingDocument builders (export is machine-generated) - map_state() fallback: "experimental" is not a valid #Lifecycle value; use "Draft" instead (valid: Active|Draft|Deprecated|Retired) - MappingDocument target-reference.entry-type: "Control" → "AssessmentRequirement" (CaC rules are assessment requirements, not Gemara controls) - test_gemara_export.py: fix valid_rels set — remove non-existent "partially-implements", add missing "implemented-by", "supports", "supported-by", "relates-to" to match the real schema - catalog.py: align ControlCatalog assessment-requirements with reference format from complytime-policies — use bare rule names as requirement IDs (was compound "control--rule"), add state:Active on each requirement, product-scoped applicability groups (e.g. rhel9-low/moderate/high), text pattern "Rule 'X' MUST be verified"
ggbecker
force-pushed
the
gemara
branch
2 times, most recently
from
da5e524 to
da29b90
Compare
Review: Gemara Artifact Correctness & Ecosystem IntegrationI reviewed this PR with a focus on three areas: (1) whether the generated Gemara artifacts conform to the CUE schema, (2) whether they work end-to-end with complyctl and the OpenSCAP provider, and (3) how the bundling and publishing approach fits with the existing complytime ecosystem. I cross-referenced the code against four upstream repositories: complyctl, complytime-providers, complytime-policies, and gemaraproj/gemara. What works wellThe CaC-to-Gemara conversion logic is solid. A few highlights:
Actionable items1. Publish through complytime-policies instead of a custom pipelineThis is the main architectural suggestion. The PR currently has no production registry — OCI artifacts only exist in local test registries ( Suggested approach: Sync the generated YAML files to complytime-policies (via an automated PR workflow from this repo's CI). complytime-policies handles validation, bundling, signing, and publishing to The org-infra reusable workflows ( 2. Consider splitting the PRThe core value — A possible split:
3. Merge Policy generation into
|
|
@marcusburghardt I have addressed your feedback and the new workflow would create PRs such: ggbecker/complytime-policies#1 (for the real complytime repo then we would need to create credentials and tie them up here in this repo) If you can review this again. Thanks |
Adds Vagrantfile (generic/rhel9 box via libvirt), populate_inventory.sh, and Ansible playbooks (setup.yml, scan.yml, scan_baseline.yml) that orchestrate the full complyctl workflow on a real RHEL9 VM. Updates TESTING.md: removes the two-approach table, recommends the Vagrant workflow as the primary testing path, and removes container-specific content.
Documents the full pipeline from CaC control files through SCAP data stream build, Gemara artifact generation, OCI bundle push, and complyctl scan, including real YAML excerpts for each output file and a file flow diagram. Committed as a standalone commit so it can be dropped cleanly if the walkthrough is moved to a more permanent location later.
Shared module used by both the local Vagrant bundle path and the complytime-policies PR path. Avoids duplicating extract_rules_from_catalog() and generate_policy() across scripts. The generate_policy() function gains optional catalog_url/guidance_url parameters: when provided, mapping-references include file:// relative URLs and imports.guidance is populated (complytime-policies mode). When omitted, behavior matches the existing local-push path.
Structural validation for Policy dicts: checks metadata.type, required top-level fields, non-empty assessment-plans with id/requirement-id per plan, and that imports.catalogs reference-ids are present in mapping-references. Used by generate_policies_pr.py before writing files.
Remove the inline extract_rules_from_catalog() and generate_policy() implementations (~135 lines) and import them from gemara.policy instead. The local Vagrant push path (--push, --baseline) is unchanged.
Four new test functions covering:
- extract_rules_from_catalog() output structure
- generate_policy() with and without guidance layer
- baseline filtering via applicability-groups
- validate_policy() error detection for missing required fields
Also fixes a pre-existing bug in test_accuracy_vs_source(): the old code
used compound IDs (ac-2.5--rule_name) with .split('--',1)[1], but the
current schema stores bare rule names as assessment-requirement IDs.
…ation Script that stages Gemara artifacts into the complytime-policies directory layout (bundles/, governance/catalogs|guidance|policies/) so they can be submitted as a PR to complytime/complytime-policies, where the existing gemara-publish-action CI handles OCI publishing, cosign signing, and Quay promotion. The Policy uses file:// relative URLs in mapping-references so the go-gemara assembler can resolve catalog and guidance dependencies without network access during bundle construction. Usage: python3 utils/nist_sync/generate_policies_pr.py --products rhel9 python3 utils/nist_sync/generate_policies_pr.py --products rhel8,rhel9,rhel10 --policies-repo ~/complytime-policies --create-pr
Triggers on push to master when NIST 800-53 control files or Gemara export
scripts change (same paths as export-to-gemara.yml), weekly on Wednesday,
and workflow_dispatch.
On each run:
1. Exports Gemara artifacts (export_to_gemara.py)
2. Stages them into complytime-policies layout (generate_policies_pr.py)
3. Clones complytime/complytime-policies using POLICIES_REPO_TOKEN
4. Reuses an existing open PR branch if one is found, otherwise creates
a new branch named update/nist-800-53-from-content-<sha>
5. Commits and pushes only when files changed
6. Opens a PR on complytime-policies (skipped if PR already exists)
Requires secret POLICIES_REPO_TOKEN: a PAT or GitHub App installation
token with Contents + Pull requests write access on complytime-policies.
3 participants
Description:
Adds
utils/nist_sync/export_to_gemara.pyto export ComplianceAsCode NIST 800-53 Rev 5 control files (rhel8/9/10) to three Gemara artifact types per product: GuidanceCatalog (abstract NIST standard text from OSCAL), ControlCatalog (NIST controls → XCCDF rule IDs), and MappingDocument (NIST↔rule traceability). All artifacts pass CUE schema validation against Gemara 1.1.0.Adds
generate_complyctl_bundle.pyto package the output as a complyctl-compatible OCI bundle andtest_complyctl_e2e.shto runcomplyctl get/generate/scanagainst RHEL9 content in a UBI9 container.Adds
.github/workflows/export-to-gemara.ymlto run the export and CUE validation on CI whenever NIST control files or export tooling change.Rationale:
Review Hints: