Skip to content

compile and platform xml to yaml converters #878

Open
mlee03 wants to merge 22 commits into
NOAA-GFDL:mainfrom
mlee03:xml-converters
Open

compile and platform xml to yaml converters #878
mlee03 wants to merge 22 commits into
NOAA-GFDL:mainfrom
mlee03:xml-converters

Conversation

@mlee03
Copy link
Copy Markdown
Contributor

@mlee03 mlee03 commented Apr 22, 2026

Describe your changes

This PR

  1. introduces yamltools/converters/compile/convert.py and test.py scripts to convert compile xmls to almost-perfect-and-ready-to-use compile yamls.
  2. .md to run an agent to convert the compile xml to yaml [COMING SOON]
  3. .md to run an agent that updates already converted yaml to the latest schema. [COMING SOON]

Create brand new platform.yaml

Issue ticket number and link (if applicable)

Fixes #858

Checklist before requesting a review

  • I ran my code
  • I tried to make my code readable
  • I tried to comment my code
  • I wrote a new test, if applicable
  • I wrote new instructions/documentation, if applicable
  • I ran pytest and inspected it's output
  • I ran pylint and attempted to implement some of it's feedback
  • No print statements; all user-facing info uses logging module

Note: If you are a code maintainer updating the tag or releasing a new fre-cli version, please use the release_procedure.md template. To quickly use this template, open a new pull request, choose your branch, and add ?template=release_procedure.md to the end of the url.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

❌ Patch coverage is 95.20958% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.04%. Comparing base (ea3f744) to head (0bafaaa).
⚠️ Report is 21 commits behind head on main.

Files with missing lines Patch % Lines
fre/yamltools/converters/compile/converter.py 84.84% 15 Missing ⚠️
fre/yamltools/converters/compile/test_converter.py 99.57% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #878      +/-   ##
==========================================
+ Coverage   84.35%   85.04%   +0.69%     
==========================================
  Files          71       73       +2     
  Lines        4997     5337     +340     
==========================================
+ Hits         4215     4539     +324     
- Misses        782      798      +16     
Flag Coverage Δ
unittests 85.04% <95.20%> (+0.69%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
fre/yamltools/converters/compile/test_converter.py 99.57% <99.57%> (ø)
fre/yamltools/converters/compile/converter.py 84.84% <84.84%> (ø)

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea3f744...0bafaaa. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mlee03 mlee03 marked this pull request as ready for review May 20, 2026 14:06
@mlee03 mlee03 requested a review from Copilot May 20, 2026 15:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new XML→YAML “compile” converter and accompanying pytest coverage, plus updates project dependencies.

Changes:

  • Added compile-converter.py to parse compile XML into a YAML-friendly structure.
  • Added a large pytest suite covering component/experiment parsing and file conversion behavior.
  • Added beautifulsoup4 to pyproject.toml dependencies.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File Description
pyproject.toml Adds a new Python dependency (beautifulsoup4).
fre/yamltools/converters/compile-converter.py Introduces XML parsing + YAML output conversion CLI/script.
fre/yamltools/converters/test.py Adds pytest coverage for the converter behavior.
fre/yamltools/converters/init.py Marks package directory (empty init).

Comment thread fre/yamltools/converters/compile/converter.py Outdated
Comment thread fre/yamltools/converters/compile/converter.py
Comment thread fre/yamltools/converters/compile/converter.py Outdated
if experiment_name:
break
with open(yaml_path, 'w', encoding='utf-8') as f:
yaml.dump(yamldict, f, sort_keys=False)
Comment on lines +2 to +7

import xml.etree.ElementTree as ET
import argparse
import yaml

def parse_component(component: ET.Element) -> dict[str, any]:
import importlib.util

# Import functions from compile-converter (hyphenated module name)
spec = importlib.util.spec_from_file_location("compile_converter", "compile-converter.py")
Comment thread fre/yamltools/converters/compile/test_converter.py
Comment thread pyproject.toml Outdated
@mlee03 mlee03 requested a review from Copilot May 20, 2026 15:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Comment on lines +2 to +7

import xml.etree.ElementTree as ET
import argparse
import yaml

def parse_component(component: ET.Element) -> dict[str, any]:
# Remove None values
return {k: v for k, v in d.items() if v is not None}

def parse_experiment(experiment: ET.Element) -> Dict[str, Any]:
Comment thread fre/yamltools/converters/compile/converter.py
Comment thread fre/yamltools/converters/compile/converter.py Outdated
if compile_elem is not None:
elem = compile_elem.find(flag)
if elem is not None and elem.text:
return elem.text.strip()
Comment on lines +5 to +16
import importlib.util

# Import functions from compile-converter (hyphenated module name)
spec = importlib.util.spec_from_file_location("compile_converter", "compile-converter.py")
compile_converter = importlib.util.module_from_spec(spec)
spec.loader.exec_module(compile_converter)

parse_component = compile_converter.parse_component
parse_experiment = compile_converter.parse_experiment
xml_to_yaml = compile_converter.xml_to_yaml


'src': components if components else [],
}

def xml_to_yaml(xml_path: str, yaml_path: str, experiment_name: str = None):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Can you add a test in test_converter.py that tests an xml with multiple ?

Comment thread fre/yamltools/converters/compile/agent.md Outdated
Comment thread fre/yamltools/converters/compile/agent.md Outdated
Comment thread pyproject.toml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xml to yaml converter

2 participants