From cb7be27be3bd00126fc5df0109326958b7bf2f5a Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Fri, 20 Mar 2026 12:02:04 -0400 Subject: [PATCH 01/20] refactor the fre make readme and add quickstart --- docs/usage/guides/fre_make_guide.rst | 64 +-------------------- fre/make/README.md | 83 +++++++++++++++++++--------- 2 files changed, 60 insertions(+), 87 deletions(-) diff --git a/docs/usage/guides/fre_make_guide.rst b/docs/usage/guides/fre_make_guide.rst index abae4d272..c5823c04e 100644 --- a/docs/usage/guides/fre_make_guide.rst +++ b/docs/usage/guides/fre_make_guide.rst @@ -1,5 +1,7 @@ Guide ---------- +To follow along with the guide, refer to the quickstart instructions `here `_. + 1. Bare-metal Build: .. code-block:: @@ -53,65 +55,3 @@ To run all of fre make subtools: # Run all of fremake: create and checkout script, makefile, dockerfile, container # creation script, and model container fre make all -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute - -Quickstart ----------- -The quickstart instructions can be used with the null model example located in the fre-cli repository: https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example - -1. Bare-metal Build: - -.. code-block:: - - # Create and run checkout script: checkout script will check out source code as - defined in the compile.yaml - fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute - - # Create Makefile - fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod - - # Create and run the compile script to generate a model executable - fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute - -2. Bare-metal Build Multi-target: - -.. code-block:: - - # Create and run checkout script: checkout script will check out source code as - defined in the compile.yaml - fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute - - # Create Makefile - fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod -t debug - - # Create and run a compile script for each target specified; generates model executables - fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute - -3. Container Build: - -In order for the container to build successfully, the parallel checkout feature is disabled. - -.. code-block:: - - # Create checkout script - fre make checkout-script -y null_model.yaml -p hpcme.2023 -t prod - - # Create Makefile - fre make makefile -y null_model.yaml -p hpcme.2023 -t prod - - # Create the Dockerfile and container build script: the container build script (createContainer.sh) - uses the Dockerfile to build a model container - fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute - -4. Run all of fremake: - -`all` kicks off the compilation automatically - -.. code-block:: - - # Bare-metal: create and run checkout script, create makefile, create and RUN compile script to - generate a model executable - fre make all -y null_model.yaml -p ncrc5.intel23 -t prod --execute - - # Container: create checkout script, makefile, create dockerfile, and create and RUN the container - build script to generate a model container - fre make all -y null_model.yaml -p hpcme.2023 -t prod --execute diff --git a/fre/make/README.md b/fre/make/README.md index 82efe85c6..7fc925cb0 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -7,51 +7,84 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, - container creation - parallel checkouts for bare-metal build** -** **Note: Users will not be able to create containers without access to podman** +**Note: Users will not be able to create containers without access to podman** -## Guide +## Quickstart -### **Bare-metal Build:** +The quickstart instructions can be used with the null model example located in the fre-cli repository: https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example -Use fre make subtools to checkout code, create a Makefile, and compile code: +Users can clone the fre-cli repository and invoke the commands below from the root of the repository. +### Bare-metal Build: ```bash -# Create and run checkout script -fre make checkout-script -y [model yaml file] -p [platform] -t [target] --execute +# Create and run checkout script: checkout script will check out source code as defined in the compile.yaml +fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute # Create Makefile -fre make makefile -y [model yaml file] -p [platform] -t [target] +fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod -# Create and run the compile script -fre make compile-script -y [model yaml file] -p [platform] -t [target] --execute +# Create and run the compile script to generate a model executable +fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute ``` - -Or use `fre make all` to do the job of all 3 subtools in one step: +### Bare-metal Build (Multi-target): ```bash -# Run fre make checkout-script, fre make makefile, and fre make compile-script in order -fre make all -y [model yaml file] -p [platform] -t [target] [other options...] -``` +# Create and run checkout script: checkout script will check out source code as defined in the compile.yaml +fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute + +# Create Makefile +fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod -t debug -### **Container Build:** -***To reiterate, users will not be able to create containers unless they have podman access on gaea.*** +# Create and run a compile script for each target specified; generates model executables +fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute +``` -Use fre make subtools to checkout code, create a Makefile, and build a container: +### Container Build: +In order for the container to build successfully, the parallel checkout feature is disabled. ```bash -# Create and run checkout script -fre make checkout-script -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute +# Create checkout script +fre make checkout-script -y null_model.yaml -p hpcme.2023 -t prod # Create Makefile -fre make makefile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] +fre make makefile -y null_model.yaml -p hpcme.2023 -t prod -# Create and run the Dockerfile -fre make dockerfile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute +# Create the Dockerfile and container build script: the container build script (createContainer.sh) uses the Dockerfile to build a model container +fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute ``` -Use `fre make all` to do the job of all 3 subtools in one step: +### Run all of fremake: + +`all` kicks off the compilation automatically ```bash -# Run fre make checkout-script, fre make makefile, and fre make dockerfile in order -fre make all -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute +# Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable +fre make all -y null_model.yaml -p ncrc5.intel23 -t prod --execute + +# Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container +fre make all -y null_model.yaml -p hpcme.2023 -t prod --execute ``` + +## Subtools +- `fre make checkout-script [options]` + - Purpose: + - Options: + - `-y, --yamlfile [model yaml] (str; required)` + - `-p, --platform` + - `-t, --target` + - `-gj, --gitjobs` + - `-npc, --no-parallel-checkout` + - `--execute` + - `--force-checkout` +- `fre make makefile [options]` + - Purpose: + - Options: +- `fre make compile-script [options]` + - Purpose: + - Options: +- `fre make dockerfile [options]` + - Purpose: + - Options: +- `fre make all [options]` + - Purpose: + - Options: From c85602fe25d826a4122bc5b714dc66b57738e34c Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Fri, 20 Mar 2026 12:14:33 -0400 Subject: [PATCH 02/20] update paths in quickstart --- fre/make/README.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/fre/make/README.md b/fre/make/README.md index 7fc925cb0..15329cb2b 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -18,25 +18,25 @@ Users can clone the fre-cli repository and invoke the commands below from the ro ```bash # Create and run checkout script: checkout script will check out source code as defined in the compile.yaml -fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute +fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute # Create Makefile -fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod +fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod # Create and run the compile script to generate a model executable -fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute +fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute ``` ### Bare-metal Build (Multi-target): ```bash # Create and run checkout script: checkout script will check out source code as defined in the compile.yaml -fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute +fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute # Create Makefile -fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod -t debug +fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug # Create and run a compile script for each target specified; generates model executables -fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute +fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute ``` ### Container Build: @@ -44,13 +44,13 @@ In order for the container to build successfully, the parallel checkout feature ```bash # Create checkout script -fre make checkout-script -y null_model.yaml -p hpcme.2023 -t prod +fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod # Create Makefile -fre make makefile -y null_model.yaml -p hpcme.2023 -t prod +fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod # Create the Dockerfile and container build script: the container build script (createContainer.sh) uses the Dockerfile to build a model container -fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute +fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute ``` ### Run all of fremake: @@ -59,10 +59,10 @@ fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute ```bash # Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable -fre make all -y null_model.yaml -p ncrc5.intel23 -t prod --execute +fre make all -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute # Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container -fre make all -y null_model.yaml -p hpcme.2023 -t prod --execute +fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute ``` ## Subtools From af16acf753e50bd70382bfec3fd1c2e640c257c7 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 31 Mar 2026 12:42:25 -0400 Subject: [PATCH 03/20] fill in subtools --- fre/make/README.md | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/fre/make/README.md b/fre/make/README.md index 15329cb2b..17083c630 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -67,24 +67,51 @@ fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t pro ## Subtools - `fre make checkout-script [options]` - - Purpose: + - Purpose: Create and run a checkout script. - Options: - - `-y, --yamlfile [model yaml] (str; required)` - - `-p, --platform` - - `-t, --target` + - `-y, --yamlfile [model yaml] (required)` + - `-p, --platform [platform] (required)` + - `-t, --target [target] (required)` - `-gj, --gitjobs` - `-npc, --no-parallel-checkout` - `--execute` - `--force-checkout` - `fre make makefile [options]` - - Purpose: + - Purpose: Create a Makefile. - Options: + - `-y, --yamlfile [model yaml] (required)` + - `-p, --platform [platform] (required)` + - `-t, --target [target] (required)` - `fre make compile-script [options]` - - Purpose: + - Purpose: Create and run a compile script to generate a model executable. - Options: + - `-y, --yamlfile [model yaml] (required)` + - `-p, --platform [platform] (required)` + - `-t, --target [target] (required)` + - `-n --nparallel` + - `-mj --makejobs` + - `-e, --execute` + - `-v, --verbose` - `fre make dockerfile [options]` - - Purpose: + - Purpose: Create and run a Dockerfile to generate a model container. - Options: + - `-y, --yamlfile [model yaml] (required)` + - `-p, --platform [platform] (required)` + - `-t, --target [target] (required)` + - `-nft, --no-format-transfer` + - `-e, --execute` - `fre make all [options]` - Purpose: + - For a bare-metal build: Create a checkout script, Makefile, and compile script to generate a model executable + - For a container build: Create a checkout script, Makefile, and Dockerfile to generate a model container. - Options: + - `-y, --yamlfile [model yaml] (required)` + - `-p, --platform [platform] (required)` + - `-t, --target [target] (required)` + - `-n --nparallel` + - `-mj --makejobs` + - `gj, --gitjobs` + - `-npc, --no-parallel-checkout` + - `-nft, --no-format-transfer` + - `-e, --execute` + - `-v, --verbose` From 4b410d379dc4628933da8cada30df601433dfd8a Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 09:52:21 -0400 Subject: [PATCH 04/20] address some comments --- docs/usage/guides/fre_make_guide.rst | 1 - fre/make/README.md | 29 ++++++++++++++++------------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/usage/guides/fre_make_guide.rst b/docs/usage/guides/fre_make_guide.rst index c5823c04e..e3e5118a2 100644 --- a/docs/usage/guides/fre_make_guide.rst +++ b/docs/usage/guides/fre_make_guide.rst @@ -1,6 +1,5 @@ Guide ---------- -To follow along with the guide, refer to the quickstart instructions `here `_. 1. Bare-metal Build: diff --git a/fre/make/README.md b/fre/make/README.md index 17083c630..91d582759 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -3,39 +3,40 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, * Fre make Supports: - multiple targets; use `-t` flag to define each target + - multiple platforms; use `-p` flag to define each platform - bare-metal build - container creation - - parallel checkouts for bare-metal build** + - parallel checkouts for bare-metal build **Note: Users will not be able to create containers without access to podman** ## Quickstart -The quickstart instructions can be used with the null model example located in the fre-cli repository: https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example +The quickstart instructions builds the null model using yaml files located in the fre-cli repository [here](https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example) Users can clone the fre-cli repository and invoke the commands below from the root of the repository. -### Bare-metal Build: +### Bare-metal Build (Single target) ```bash -# Create and run checkout script: checkout script will check out source code as defined in the compile.yaml +# Create and run checkout script fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute -# Create Makefile +# Create the Makefile fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -# Create and run the compile script to generate a model executable +# Create and run the compile script fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute ``` -### Bare-metal Build (Multi-target): +### Bare-metal Build (Multiple targets): ```bash -# Create and run checkout script: checkout script will check out source code as defined in the compile.yaml +# Create and run checkout script fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute -# Create Makefile +# Create the Makefile fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug -# Create and run a compile script for each target specified; generates model executables +# Create and run a compile script for each target specified fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute ``` @@ -46,10 +47,10 @@ In order for the container to build successfully, the parallel checkout feature # Create checkout script fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod -# Create Makefile +# Create the Makefile fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod -# Create the Dockerfile and container build script: the container build script (createContainer.sh) uses the Dockerfile to build a model container +# Create the Dockerfile and container build script fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute ``` @@ -76,12 +77,14 @@ fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t pro - `-npc, --no-parallel-checkout` - `--execute` - `--force-checkout` + - `fre make makefile [options]` - Purpose: Create a Makefile. - Options: - `-y, --yamlfile [model yaml] (required)` - `-p, --platform [platform] (required)` - `-t, --target [target] (required)` + - `fre make compile-script [options]` - Purpose: Create and run a compile script to generate a model executable. - Options: @@ -92,6 +95,7 @@ fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t pro - `-mj --makejobs` - `-e, --execute` - `-v, --verbose` + - `fre make dockerfile [options]` - Purpose: Create and run a Dockerfile to generate a model container. - Options: @@ -100,6 +104,7 @@ fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t pro - `-t, --target [target] (required)` - `-nft, --no-format-transfer` - `-e, --execute` + - `fre make all [options]` - Purpose: - For a bare-metal build: Create a checkout script, Makefile, and compile script to generate a model executable From bed9d85598af33b8c7ec14d5508e1cdc64d1a653 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 09:57:36 -0400 Subject: [PATCH 05/20] update readme --- fre/make/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/fre/make/README.md b/fre/make/README.md index 91d582759..f325173ab 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -16,6 +16,7 @@ The quickstart instructions builds the null model using yaml files located in th Users can clone the fre-cli repository and invoke the commands below from the root of the repository. ### Bare-metal Build (Single target) +For the bare-metal build, the parallel checkout feature is the default behavior. ```bash # Create and run checkout script From 594504d564f5769069cc039b91f8ecf245c6ac1f Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 10:22:22 -0400 Subject: [PATCH 06/20] update readme --- fre/make/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fre/make/README.md b/fre/make/README.md index f325173ab..1e12a7113 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -55,7 +55,7 @@ fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 - fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute ``` -### Run all of fremake: +### All-in-one fre make subtool: `all` kicks off the compilation automatically From b33cbd14bd188bd52dab1f452424591fa89cd7d9 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 10:39:57 -0400 Subject: [PATCH 07/20] update note --- fre/make/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fre/make/README.md b/fre/make/README.md index 1e12a7113..7beac9a67 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -8,7 +8,8 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, - container creation - parallel checkouts for bare-metal build -**Note: Users will not be able to create containers without access to podman** +**Note: The container engine used to create the container (such as podman or docker) is specified in the `platforms.yaml` with the `containerBuild` key. +Please ensure the container engine is acccesible before running fre make.** ## Quickstart From 7813ee171a1f8acddea0494faf0dd740df6f207c Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 12:04:18 -0400 Subject: [PATCH 08/20] replace guide with quickstart --- docs/usage.rst | 2 +- docs/usage/fre_make_quickstart.rst | 60 ++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 docs/usage/fre_make_quickstart.rst diff --git a/docs/usage.rst b/docs/usage.rst index 84a25b1fb..f8133772e 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -30,7 +30,7 @@ Post-processing Yamls Build FMS model =============== .. include:: usage/build_fms_model.rst -.. include:: usage/guides/fre_make_guide.rst +.. include:: usage/fre_make_quickstart.rst Run FMS model ============= diff --git a/docs/usage/fre_make_quickstart.rst b/docs/usage/fre_make_quickstart.rst new file mode 100644 index 000000000..cf7572442 --- /dev/null +++ b/docs/usage/fre_make_quickstart.rst @@ -0,0 +1,60 @@ +Quickstart +----------------- +The quickstart instructions builds the null model using yaml files located in the fre-cli repository [here](https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example) + +Users can clone the fre-cli repository and invoke the commands below from the root of the repository. + +1. All-in-one fre make subtool: + +`all` kicks off the compilation automatically + +.. code-block:: + + # Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable + fre make all -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + + # Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container + fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute + +2. Bare-metal Build (Single target) + +For the bare-metal build, the parallel checkout feature is the default behavior. + +.. code-block:: + + # Create and run checkout script + fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + + # Create the Makefile + fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod + + # Create and run the compile script + fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + +3. Bare-metal Build (Multiple targets): + +.. code-block:: + + # Create and run checkout script + fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute + + # Create the Makefile + fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug + + # Create and run a compile script for each target specified + fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute + +4. Container Build: + +In order for the container to build successfully, the parallel checkout feature is disabled. + +.. code-block:: + + # Create checkout script + fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod + + # Create the Makefile + fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod + + # Create the Dockerfile and container build script + fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute From 010ab3412b3809e2a8a09c15d03b153427c2e80c Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 12:24:00 -0400 Subject: [PATCH 09/20] fix link --- docs/usage/fre_make_quickstart.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/usage/fre_make_quickstart.rst b/docs/usage/fre_make_quickstart.rst index cf7572442..0d08f682c 100644 --- a/docs/usage/fre_make_quickstart.rst +++ b/docs/usage/fre_make_quickstart.rst @@ -1,6 +1,6 @@ Quickstart ----------------- -The quickstart instructions builds the null model using yaml files located in the fre-cli repository [here](https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example) +The quickstart instructions builds the null model using yaml files located in the fre-cli repository `here `__. Users can clone the fre-cli repository and invoke the commands below from the root of the repository. From 2aa357d97e34eb7d1c70faf2ae78a8d323c648e0 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 12:25:20 -0400 Subject: [PATCH 10/20] remove guide --- docs/usage/guides/fre_make_guide.rst | 56 ---------------------------- 1 file changed, 56 deletions(-) delete mode 100644 docs/usage/guides/fre_make_guide.rst diff --git a/docs/usage/guides/fre_make_guide.rst b/docs/usage/guides/fre_make_guide.rst deleted file mode 100644 index e3e5118a2..000000000 --- a/docs/usage/guides/fre_make_guide.rst +++ /dev/null @@ -1,56 +0,0 @@ -Guide ----------- - -1. Bare-metal Build: - -.. code-block:: - - # Create checkout script - fre make checkout-script -y [model yaml file] -p [platform] -t [target] - - # Or create and RUN checkout script - fre make checkout-script -y [model yaml file] -p [platform] -t [target] --execute - - # Create Makefile - fre make makefile -y [model yaml file] -p [platform] -t [target] - - # Create the compile script - fre make compile-script -y [model yaml file] -p [platform] -t [target] - - # Or create and RUN the compile script - fre make compile-script -y [model yaml file] -p [platform] -t [target] --execute - -Users can also run all fre make commands in one subtool: - -.. code-block:: - - # Run all of fremake: creates checkout script, makefile, compile script, and model executable - fre make all -y [model yaml file] -p [platform] -t [target] [other options...] --execute - -2. Container Build: - -For the container build, parallel checkouts are not supported. In addition the platform must be a container platform. - -Gaea users will not be able to create containers unless they have requested and been given podman access. - -.. code-block:: - - # Create checkout script - fre make checkout-script -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] - - # Create Makefile - fre make makefile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] - - # Create a Dockerfile - fre make dockerfile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] - - # Or create and RUN the Dockerfile - fre make dockerfile -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute - -To run all of fre make subtools: - -.. code-block:: - - # Run all of fremake: create and checkout script, makefile, dockerfile, container - # creation script, and model container - fre make all -y [model yaml file] -p [CONTAINER PLATFORM] -t [target] --execute From 73eaea939c36e98675d2a1039fff61c73a07d6a7 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 7 Apr 2026 12:27:22 -0400 Subject: [PATCH 11/20] quickstart moved into rst file --- fre/make/README.md | 60 +--------------------------------------------- 1 file changed, 1 insertion(+), 59 deletions(-) diff --git a/fre/make/README.md b/fre/make/README.md index 7beac9a67..673e2b9af 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -8,65 +8,7 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, - container creation - parallel checkouts for bare-metal build -**Note: The container engine used to create the container (such as podman or docker) is specified in the `platforms.yaml` with the `containerBuild` key. -Please ensure the container engine is acccesible before running fre make.** - -## Quickstart - -The quickstart instructions builds the null model using yaml files located in the fre-cli repository [here](https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/make/tests/null_example) - -Users can clone the fre-cli repository and invoke the commands below from the root of the repository. -### Bare-metal Build (Single target) -For the bare-metal build, the parallel checkout feature is the default behavior. - -```bash -# Create and run checkout script -fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute - -# Create the Makefile -fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod - -# Create and run the compile script -fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute -``` -### Bare-metal Build (Multiple targets): - -```bash -# Create and run checkout script -fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute - -# Create the Makefile -fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug - -# Create and run a compile script for each target specified -fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute -``` - -### Container Build: -In order for the container to build successfully, the parallel checkout feature is disabled. - -```bash -# Create checkout script -fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod - -# Create the Makefile -fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod - -# Create the Dockerfile and container build script -fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute -``` - -### All-in-one fre make subtool: - -`all` kicks off the compilation automatically - -```bash -# Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable -fre make all -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute - -# Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container -fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute -``` +**Note: The container engine used to create the container (such as podman or docker) is specified in the `platforms.yaml` with the `containerBuild` key. Please ensure the container engine is acccesible before running fre make.** ## Subtools - `fre make checkout-script [options]` From 1cbb8d2b12953b78119b1714ad9ce948388056a5 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Wed, 8 Apr 2026 11:57:28 -0400 Subject: [PATCH 12/20] update doc --- docs/usage/build_fms_model.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/usage/build_fms_model.rst b/docs/usage/build_fms_model.rst index 74fe7bae4..9be5d4fc9 100644 --- a/docs/usage/build_fms_model.rst +++ b/docs/usage/build_fms_model.rst @@ -7,6 +7,7 @@ Capabilities ----------------- Fremake Canopy Supports: - multiple target use; ``-t`` flag to define each target (for multiple platform-target combinations) + - multiple platform use; ``-p`` flag to define each platform (for multiple platform-target combinations) - bare-metal build - container creation - parallel checkouts for bare-metal build From 3cc619f38a47f8842d36641e55950fcef04058a7 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Mon, 13 Apr 2026 11:52:38 -0400 Subject: [PATCH 13/20] Update fremake quickstart --- docs/usage/build_fms_model.rst | 4 ++-- docs/usage/fre_make_quickstart.rst | 32 ++++++++++++++++++------------ 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/docs/usage/build_fms_model.rst b/docs/usage/build_fms_model.rst index 9be5d4fc9..eb5b12281 100644 --- a/docs/usage/build_fms_model.rst +++ b/docs/usage/build_fms_model.rst @@ -15,7 +15,7 @@ Fremake Canopy Supports: - one yaml format - additional library support if needed -**Note: Users will not be able to create containers without access to podman. To get access, submit a helpdesk ticket.** +**Note: Users will not be able to create containers without access to podman (on GAEA). To get access, submit a helpdesk ticket.** Required configuration files: @@ -23,7 +23,7 @@ Required configuration files: - Compile Yaml - Platforms yaml -These yamls are combined and further parsed through the ``fre make`` tools (see the "Guide" section for the step by step process). +These yamls are combined and further parsed through the ``fre make`` tools. The final combined yaml includes the name of the compile experiment, the platform and target passed in the command line subtool, as well as compile and platform yaml information. The platform that was passed corresponds to the one defined in the platforms YAML file. This file details essential configuration info such as setting up the runtime environment, listing what compiler to use, and providing which container applications to use. These configurations vary based on the specific site where the user is building the model executable or container. Additionally the platform and target passed are used to fill in the build directory in which the compile script is created and run. diff --git a/docs/usage/fre_make_quickstart.rst b/docs/usage/fre_make_quickstart.rst index 0d08f682c..4c80e1ad5 100644 --- a/docs/usage/fre_make_quickstart.rst +++ b/docs/usage/fre_make_quickstart.rst @@ -1,8 +1,14 @@ Quickstart ----------------- -The quickstart instructions builds the null model using yaml files located in the fre-cli repository `here `__. +The quickstart instructions build the null model by loading the FRE module and using yaml files located in the fre-cli repository. -Users can clone the fre-cli repository and invoke the commands below from the root of the repository. +To access the null_model configuration, clone the fre-cli repository: + +.. code-block:: + + git clone --recursive https://github.com/NOAA-GFDL/fre-cli.git + + cd fre/make/tests/null_example 1. All-in-one fre make subtool: @@ -11,10 +17,10 @@ Users can clone the fre-cli repository and invoke the commands below from the ro .. code-block:: # Bare-metal: create and run checkout script, create makefile, create and RUN compile script to generate a model executable - fre make all -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + fre make all -y null_model.yaml -p ncrc5.intel23 -t prod --execute # Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container - fre make all -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute + fre make all -y null_model.yaml -p hpcme.2023 -t prod --execute 2. Bare-metal Build (Single target) @@ -23,26 +29,26 @@ For the bare-metal build, the parallel checkout feature is the default behavior. .. code-block:: # Create and run checkout script - fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute # Create the Makefile - fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod + fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod # Create and run the compile script - fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod --execute + fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod --execute 3. Bare-metal Build (Multiple targets): .. code-block:: # Create and run checkout script - fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute + fre make checkout-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute # Create the Makefile - fre make makefile -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug + fre make makefile -y null_model.yaml -p ncrc5.intel23 -t prod -t debug # Create and run a compile script for each target specified - fre make compile-script -y fre/make/tests/null_example/null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute + fre make compile-script -y null_model.yaml -p ncrc5.intel23 -t prod -t debug --execute 4. Container Build: @@ -51,10 +57,10 @@ In order for the container to build successfully, the parallel checkout feature .. code-block:: # Create checkout script - fre make checkout-script -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod + fre make checkout-script -y null_model.yaml -p hpcme.2023 -t prod # Create the Makefile - fre make makefile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod + fre make makefile -y null_model.yaml -p hpcme.2023 -t prod # Create the Dockerfile and container build script - fre make dockerfile -y fre/make/tests/null_example/null_model.yaml -p hpcme.2023 -t prod --execute + fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute From d5387e93741b2b4e49c4ab8fe3df28e0114f12cd Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Mon, 13 Apr 2026 12:19:23 -0400 Subject: [PATCH 14/20] update toolage and README --- docs/tools/make.rst | 14 ++++++++++---- fre/make/README.md | 8 ++++++++ 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/docs/tools/make.rst b/docs/tools/make.rst index c9aa496dc..6c31cff4a 100644 --- a/docs/tools/make.rst +++ b/docs/tools/make.rst @@ -7,9 +7,10 @@ - ``-y, --yamlfile [experiment yaml] (required)`` - ``-p, --platform [platform] (required)`` - ``-t, --target [target] (required)`` - - ``-j, --njobs [number of jobs to run simultaneously]`` + - ``-gj, --gitjobs [number of submodules to clone in parallel]`` - ``-npc, --no-parallel-checkout (for container build)`` - ``-e, --execute`` + - ``force-checkout`` ``makefile`` ------------- @@ -30,7 +31,7 @@ - ``-y, --yamlfile [experiment yaml] (required)`` - ``-p, --platform [platform] (required)`` - ``-t, --target [target] (required)`` - - ``-j, --njobs [number of jobs to run simultaneously]`` + - ``-mj, --makejobs [number of recipes from the Makefile to run in parallel]`` - ``-n, --parallel [number of concurrent module compiles]`` - ``-e, --execute`` @@ -44,6 +45,8 @@ - ``-y, --yamlfile [experiment yaml] (required)`` - ``-p, --platform [platform] (required)`` - ``-t, --target [target] (required)`` + - ``-nft, --no-format-transfer`` + - ``-e, --execute`` ``all`` ------- @@ -54,6 +57,9 @@ - ``-y, --yamlfile [experiment yaml] (required)`` - ``-p, --platform [platform] (required)`` - ``-t, --target [target] (required)`` - - ``-npc, --no-parallel-checkout (for container build)`` - - ``-j, --njobs [number of jobs to run simultaneously]`` - ``-n, --parallel [number of concurrent module compiles]`` + - ``-mj --makejobs [number of recipes from the Makefile to run in parallel]`` + - ``-gj, --gitjobs [number of submodules to clone in parallel]`` + - ``-npc, --no-paralel-checkout`` + - ``-nft, --no-format-transfer`` + - ``-e, --execute`` diff --git a/fre/make/README.md b/fre/make/README.md index 673e2b9af..de20c6ae6 100644 --- a/fre/make/README.md +++ b/fre/make/README.md @@ -10,6 +10,14 @@ Through the fre-cli, `fre make` can be used to create and run a checkout script, **Note: The container engine used to create the container (such as podman or docker) is specified in the `platforms.yaml` with the `containerBuild` key. Please ensure the container engine is acccesible before running fre make.** +## Getting Started + +The quickstart instructions [here](https://noaa-gfdl.readthedocs.io/projects/fre-cli/en/latest/usage.html#quickstart), will build the null model using YAML configurations located in the fre-cli repository. These configurations are combined to create a resolved dictionary that will then be parsed for information to: + +1. Create and run a checkout script (using source code for the `FMS`, `ice_param`, and `coupler` components defined in the `compile.yaml`) +2. Create a Makefile +3. Create and run either a compile.sh (for bare-metal builds) or a Dockerfile and createContainer.sh (for container builds) + ## Subtools - `fre make checkout-script [options]` - Purpose: Create and run a checkout script. From 735754830587e0bdaddc894a8cd60d2d911c6e4c Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Wed, 6 May 2026 15:28:55 -0400 Subject: [PATCH 15/20] add better error message with options for how to troubleshoot --- fre/make/create_checkout_script.py | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/fre/make/create_checkout_script.py b/fre/make/create_checkout_script.py index fc2412f38..82b905b82 100644 --- a/fre/make/create_checkout_script.py +++ b/fre/make/create_checkout_script.py @@ -186,17 +186,19 @@ def checkout_create(yamlfile: str, platform: tuple, target: tuple, elif Path(checkout_sh_path).exists() and not force_checkout: fre_logger.info("Checkout script PREVIOUSLY created here: %s", checkout_sh_path) - fre_logger.warning("If editing source code after creating and running the checkout script for the " - "bare-metal build, continue to follow each fre make subtool individually ('makefile', " - "'compile-script' or 'dockerfile') to avoid conflicting 'existing checkout script' " - "errors (advise against using fre make all)") if execute: try: subprocess.run(args=[checkout_sh_path], check=True) except Exception as exc: - raise OSError(f"\nError executing checkout script: {checkout_sh_path}.", - "\nSRC DIR might exist already. Try removing test folder: " - f"{platform_info['modelRoot']} or specifying --force-checkout\n") from exc + raise OSError(f"Error executing checkout script: {checkout_sh_path}.", + "\n\n!!SRC DIR might exist already!!\n" + "1. If not editing the checked out source code directly, try either:\n" + f" - removing the output folder ({platform_info['modelRoot']})\n" + " - specifying --force-checkout\n" + "2. If editing checked out source code directly for the bare-metal build:\n" + " - users should now follow up with running each fre make subtool individually\n" + " ('makefile', 'compile-script' or 'dockerfile') to avoid conflicting 'existing\n" + " checkout script' errors seen with 'fre make all'\n") from exc else: return From f5d25699dcde2811c9b7cfb2500b82a6f56ffa80 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 26 May 2026 09:33:26 -0400 Subject: [PATCH 16/20] update platforms and quickstart --- docs/usage/fre_make_quickstart.rst | 8 ++--- fre/make/tests/null_example/platforms.yaml | 35 +++++++++++++++------- fre/make/tests/test_create_checkout.py | 2 +- fre/make/tests/test_create_dockerfile.py | 2 +- fre/make/tests/test_create_makefile.py | 2 +- fre/make/tests/test_run_fremake.py | 2 +- 6 files changed, 33 insertions(+), 18 deletions(-) diff --git a/docs/usage/fre_make_quickstart.rst b/docs/usage/fre_make_quickstart.rst index 4c80e1ad5..8bf328caa 100644 --- a/docs/usage/fre_make_quickstart.rst +++ b/docs/usage/fre_make_quickstart.rst @@ -20,7 +20,7 @@ To access the null_model configuration, clone the fre-cli repository: fre make all -y null_model.yaml -p ncrc5.intel23 -t prod --execute # Container: create checkout script, makefile, create dockerfile, and create and RUN the container build script to generate a model container - fre make all -y null_model.yaml -p hpcme.2023 -t prod --execute + fre make all -y null_model.yaml -p hpcme.intel25 -t prod --execute 2. Bare-metal Build (Single target) @@ -57,10 +57,10 @@ In order for the container to build successfully, the parallel checkout feature .. code-block:: # Create checkout script - fre make checkout-script -y null_model.yaml -p hpcme.2023 -t prod + fre make checkout-script -y null_model.yaml -p hpcme.intel25 -t prod # Create the Makefile - fre make makefile -y null_model.yaml -p hpcme.2023 -t prod + fre make makefile -y null_model.yaml -p hpcme.intel25 -t prod # Create the Dockerfile and container build script - fre make dockerfile -y null_model.yaml -p hpcme.2023 -t prod --execute + fre make dockerfile -y null_model.yaml -p hpcme.intel25 -t prod --execute diff --git a/fre/make/tests/null_example/platforms.yaml b/fre/make/tests/null_example/platforms.yaml index 4511aabc5..c4159b71d 100644 --- a/fre/make/tests/null_example/platforms.yaml +++ b/fre/make/tests/null_example/platforms.yaml @@ -7,17 +7,19 @@ platforms: "module load fre/bronx-23", "module load cray-hdf5/1.12.2.11", "module load cray-netcdf/4.9.0.11"] - mkTemplate: !join ["/ncrc/home2/fms/local/opt/fre-commands/bronx-20/site/ncrc5/", *INTEL, ".mk"] + mkTemplate: "/home/Dana.Singh/fre/singh/fre-make-updates/fre/mkmf/templates/ncrc5-intel-classic.mk" #!join ["/ncrc/home2/fms/local/opt/fre-commands/bronx-20/site/ncrc5/", *INTEL, ".mk"] modelRoot: ${HOME}/fremake_canopy/test - - name: hpcme.2023 + + ## one-stage build + - hpcme.intel25 compiler: intel - RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] + RUNenv: "" modelRoot: /apps container: True containerBuild: "podman" - containerRun: "apptainer" - containerBase: "docker.io/ecpe4s/noaa-intel-prototype:2023.09.25" - mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" + containerRun: "apptainer" + containerBase: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3" + mkTemplate: "/apps/mkmf/templates/hpcme-intel2025.03-oneapi.mk" volume: *VOLUME - name: ci.gnu compiler: gnu @@ -40,14 +42,27 @@ platforms: containerBase: "ghcr.io/rem1776/hpc-mini:alpine3gcc14.2" mkTemplate: "linux-ubuntu-xenial-gnu.mk" containerOutputLocation: ${TEST_BUILD_DIR}/fremake_canopy/test +# - name: con.twostep +# compiler: intel +# RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] +# modelRoot: /apps +# container: True +# containerBuild: "podman" +# containerRun: "apptainer" +# containerBase: "ecpe4s/noaa-intel-prototype:2023.09.25" +# mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" +# container2step: True +# container2base: "ecpe4s/noaa-intel-prototype:2023.09.25" + + ## hpcme.intel25 - name: con.twostep compiler: intel - RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] + RUNenv: "" modelRoot: /apps container: True containerBuild: "podman" containerRun: "apptainer" - containerBase: "ecpe4s/noaa-intel-prototype:2023.09.25" - mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" + containerBase: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3" + mkTemplate: "/apps/mkmf/templates/hpcme-intel2025.03-oneapi.mk" container2step: True - container2base: "ecpe4s/noaa-intel-prototype:2023.09.25" + container2base: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3rte" diff --git a/fre/make/tests/test_create_checkout.py b/fre/make/tests/test_create_checkout.py index ac8b0c97b..3d4ca1044 100644 --- a/fre/make/tests/test_create_checkout.py +++ b/fre/make/tests/test_create_checkout.py @@ -13,7 +13,7 @@ # Set platform and target PLATFORM = ["ci.gnu"] -CONTAINER_PLATFORM = ["hpcme.2023"] +CONTAINER_PLATFORM = ["hpcme.intel25"] TARGET = ["debug"] # Set output directory diff --git a/fre/make/tests/test_create_dockerfile.py b/fre/make/tests/test_create_dockerfile.py index 4436dfac1..b7b1d1dd8 100644 --- a/fre/make/tests/test_create_dockerfile.py +++ b/fre/make/tests/test_create_dockerfile.py @@ -16,7 +16,7 @@ YAMLDIR = "fre/make/tests/null_example" YAMLFILE = "null_model.yaml" YAMLPATH = f"{YAMLDIR}/{YAMLFILE}" -PLATFORM = ["hpcme.2023"] +PLATFORM = ["hpcme.intel25"] TARGET = ["debug"] BADOPT = ["foo"] EXPERIMENT = "null_model_full" diff --git a/fre/make/tests/test_create_makefile.py b/fre/make/tests/test_create_makefile.py index fc2f24a9f..0e75ab9db 100644 --- a/fre/make/tests/test_create_makefile.py +++ b/fre/make/tests/test_create_makefile.py @@ -11,7 +11,7 @@ NM_EXAMPLE = Path("null_example") YAMLFILE = "null_model.yaml" BM_PLATFORM = ["ci.gnu"] -CONTAINER_PLATFORM = ["hpcme.2023"] +CONTAINER_PLATFORM = ["hpcme.intel25"] CONTAINER_PLAT2 = ["con.twostep"] TARGET = ["debug"] EXPERIMENT = "null_model_full" diff --git a/fre/make/tests/test_run_fremake.py b/fre/make/tests/test_run_fremake.py index b61a41662..517b8188b 100644 --- a/fre/make/tests/test_run_fremake.py +++ b/fre/make/tests/test_run_fremake.py @@ -20,7 +20,7 @@ YAMLFILE = "null_model.yaml" YAMLPATH = f"{YAMLDIR}/{YAMLFILE}" PLATFORM = [ "ci.gnu" ] -CONTAINER_PLATFORM = ["hpcme.2023"] +CONTAINER_PLATFORM = ["hpcme.intel25"] CONTAINER_PLAT2 = ["con.twostep"] TARGET = ["debug"] BADOPT = ["foo"] From dc0d8e7ec7e1d54065c59f13bf68256e4e79cc6b Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 26 May 2026 10:00:32 -0400 Subject: [PATCH 17/20] update platform name --- fre/tests/test_fre_make_cli.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fre/tests/test_fre_make_cli.py b/fre/tests/test_fre_make_cli.py index bebb4d50b..ef6111319 100644 --- a/fre/tests/test_fre_make_cli.py +++ b/fre/tests/test_fre_make_cli.py @@ -108,13 +108,13 @@ def test_cli_fre_make_create_checkout_baremetal_npc(): os.access(Path(f"{OUT_PATH}/fremake_canopy/test/null_model_full/src/checkout.sh"), os.X_OK)]) def test_cli_fre_make_create_checkout_container(): - ''' fre make checkout-script -y null_model.yaml -p hpcme.2023 -t debug''' + ''' fre make checkout-script -y null_model.yaml -p hpcme.intel25 -t debug''' OUT_PATH=f"{OUT_PATH_BASE}/fremake_out_container" # Set paths and click options yamlfile = Path("fre/make/tests/null_example/") - platform = "hpcme.2023" + platform = "hpcme.intel25" target = "debug" # Delete existing checkout.sh if it exists From 2c62ebdbe71aac64f8e6abf23c09ce2087ca17bd Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 26 May 2026 10:32:21 -0400 Subject: [PATCH 18/20] revert platform changes (to be included in another PR) --- docs/usage/fre_make_quickstart.rst | 2 +- fre/make/tests/null_example/platforms.yaml | 51 ++++++++++++++-------- fre/make/tests/test_create_checkout.py | 2 +- fre/make/tests/test_create_dockerfile.py | 2 +- fre/make/tests/test_create_makefile.py | 2 +- fre/make/tests/test_run_fremake.py | 2 +- fre/tests/test_fre_make_cli.py | 4 +- 7 files changed, 40 insertions(+), 25 deletions(-) diff --git a/docs/usage/fre_make_quickstart.rst b/docs/usage/fre_make_quickstart.rst index 8bf328caa..ae50c649b 100644 --- a/docs/usage/fre_make_quickstart.rst +++ b/docs/usage/fre_make_quickstart.rst @@ -1,6 +1,6 @@ Quickstart ----------------- -The quickstart instructions build the null model by loading the FRE module and using yaml files located in the fre-cli repository. +The quickstart instructions build the null model by loading the FRE module on Gaea C5, and using yaml files located in the fre-cli repository. To access the null_model configuration, clone the fre-cli repository: diff --git a/fre/make/tests/null_example/platforms.yaml b/fre/make/tests/null_example/platforms.yaml index c4159b71d..34fd333f8 100644 --- a/fre/make/tests/null_example/platforms.yaml +++ b/fre/make/tests/null_example/platforms.yaml @@ -11,15 +11,15 @@ platforms: modelRoot: ${HOME}/fremake_canopy/test ## one-stage build - - hpcme.intel25 + - name: hpcme.2023 compiler: intel - RUNenv: "" + RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] modelRoot: /apps container: True containerBuild: "podman" - containerRun: "apptainer" - containerBase: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3" - mkTemplate: "/apps/mkmf/templates/hpcme-intel2025.03-oneapi.mk" + containerRun: "apptainer" + containerBase: "docker.io/ecpe4s/noaa-intel-prototype:2023.09.25" + mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" volume: *VOLUME - name: ci.gnu compiler: gnu @@ -42,20 +42,22 @@ platforms: containerBase: "ghcr.io/rem1776/hpc-mini:alpine3gcc14.2" mkTemplate: "linux-ubuntu-xenial-gnu.mk" containerOutputLocation: ${TEST_BUILD_DIR}/fremake_canopy/test -# - name: con.twostep -# compiler: intel -# RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] -# modelRoot: /apps -# container: True -# containerBuild: "podman" -# containerRun: "apptainer" -# containerBase: "ecpe4s/noaa-intel-prototype:2023.09.25" -# mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" -# container2step: True -# container2base: "ecpe4s/noaa-intel-prototype:2023.09.25" - - ## hpcme.intel25 - name: con.twostep + compiler: intel + RUNenv: [". /spack/share/spack/setup-env.sh", "spack load libyaml", "spack load netcdf-fortran@4.5.4", "spack load hdf5@1.14.0"] + modelRoot: /apps + container: True + containerBuild: "podman" + containerRun: "apptainer" + containerBase: "ecpe4s/noaa-intel-prototype:2023.09.25" + mkTemplate: "/apps/mkmf/templates/hpcme-intel21.mk" + container2step: True + container2base: "ecpe4s/noaa-intel-prototype:2023.09.25" + +### TO-DO: Update fre make tests in another PR +# to replace hpcme.2023: +# Also used for fre make quick start + - name: hpcme.intel25 compiler: intel RUNenv: "" modelRoot: /apps @@ -66,3 +68,16 @@ platforms: mkTemplate: "/apps/mkmf/templates/hpcme-intel2025.03-oneapi.mk" container2step: True container2base: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3rte" + volume: *VOLUME +# hpcme.intel25 info to replace con.twostep +# - name: con.twostep +# compiler: intel +# RUNenv: "" +# modelRoot: /apps +# container: True +# containerBuild: "podman" +# containerRun: "apptainer" +# containerBase: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3" +# mkTemplate: "/apps/mkmf/templates/hpcme-intel2025.03-oneapi.mk" +# container2step: True +# container2base: "gitlab.gfdl.noaa.gov:5050/fre/hpc-me/base-ubuntu24.04-intel:2025.3rte" diff --git a/fre/make/tests/test_create_checkout.py b/fre/make/tests/test_create_checkout.py index 3d4ca1044..ac8b0c97b 100644 --- a/fre/make/tests/test_create_checkout.py +++ b/fre/make/tests/test_create_checkout.py @@ -13,7 +13,7 @@ # Set platform and target PLATFORM = ["ci.gnu"] -CONTAINER_PLATFORM = ["hpcme.intel25"] +CONTAINER_PLATFORM = ["hpcme.2023"] TARGET = ["debug"] # Set output directory diff --git a/fre/make/tests/test_create_dockerfile.py b/fre/make/tests/test_create_dockerfile.py index b7b1d1dd8..4436dfac1 100644 --- a/fre/make/tests/test_create_dockerfile.py +++ b/fre/make/tests/test_create_dockerfile.py @@ -16,7 +16,7 @@ YAMLDIR = "fre/make/tests/null_example" YAMLFILE = "null_model.yaml" YAMLPATH = f"{YAMLDIR}/{YAMLFILE}" -PLATFORM = ["hpcme.intel25"] +PLATFORM = ["hpcme.2023"] TARGET = ["debug"] BADOPT = ["foo"] EXPERIMENT = "null_model_full" diff --git a/fre/make/tests/test_create_makefile.py b/fre/make/tests/test_create_makefile.py index 0e75ab9db..fc2f24a9f 100644 --- a/fre/make/tests/test_create_makefile.py +++ b/fre/make/tests/test_create_makefile.py @@ -11,7 +11,7 @@ NM_EXAMPLE = Path("null_example") YAMLFILE = "null_model.yaml" BM_PLATFORM = ["ci.gnu"] -CONTAINER_PLATFORM = ["hpcme.intel25"] +CONTAINER_PLATFORM = ["hpcme.2023"] CONTAINER_PLAT2 = ["con.twostep"] TARGET = ["debug"] EXPERIMENT = "null_model_full" diff --git a/fre/make/tests/test_run_fremake.py b/fre/make/tests/test_run_fremake.py index 517b8188b..b61a41662 100644 --- a/fre/make/tests/test_run_fremake.py +++ b/fre/make/tests/test_run_fremake.py @@ -20,7 +20,7 @@ YAMLFILE = "null_model.yaml" YAMLPATH = f"{YAMLDIR}/{YAMLFILE}" PLATFORM = [ "ci.gnu" ] -CONTAINER_PLATFORM = ["hpcme.intel25"] +CONTAINER_PLATFORM = ["hpcme.2023"] CONTAINER_PLAT2 = ["con.twostep"] TARGET = ["debug"] BADOPT = ["foo"] diff --git a/fre/tests/test_fre_make_cli.py b/fre/tests/test_fre_make_cli.py index ef6111319..bebb4d50b 100644 --- a/fre/tests/test_fre_make_cli.py +++ b/fre/tests/test_fre_make_cli.py @@ -108,13 +108,13 @@ def test_cli_fre_make_create_checkout_baremetal_npc(): os.access(Path(f"{OUT_PATH}/fremake_canopy/test/null_model_full/src/checkout.sh"), os.X_OK)]) def test_cli_fre_make_create_checkout_container(): - ''' fre make checkout-script -y null_model.yaml -p hpcme.intel25 -t debug''' + ''' fre make checkout-script -y null_model.yaml -p hpcme.2023 -t debug''' OUT_PATH=f"{OUT_PATH_BASE}/fremake_out_container" # Set paths and click options yamlfile = Path("fre/make/tests/null_example/") - platform = "hpcme.intel25" + platform = "hpcme.2023" target = "debug" # Delete existing checkout.sh if it exists From f45814018344dddd17e3991b143df9804d1d5e0c Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 26 May 2026 11:38:51 -0400 Subject: [PATCH 19/20] whoops - update platforms.yaml from testing path --- fre/make/tests/null_example/platforms.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fre/make/tests/null_example/platforms.yaml b/fre/make/tests/null_example/platforms.yaml index 34fd333f8..0c8b5e89d 100644 --- a/fre/make/tests/null_example/platforms.yaml +++ b/fre/make/tests/null_example/platforms.yaml @@ -7,7 +7,7 @@ platforms: "module load fre/bronx-23", "module load cray-hdf5/1.12.2.11", "module load cray-netcdf/4.9.0.11"] - mkTemplate: "/home/Dana.Singh/fre/singh/fre-make-updates/fre/mkmf/templates/ncrc5-intel-classic.mk" #!join ["/ncrc/home2/fms/local/opt/fre-commands/bronx-20/site/ncrc5/", *INTEL, ".mk"] + mkTemplate: !join ["/ncrc/home2/fms/local/opt/fre-commands/bronx-20/site/ncrc5/", *INTEL, ".mk"] modelRoot: ${HOME}/fremake_canopy/test ## one-stage build From 81f279d00da2c81e18061f7243e9c8e1c705e238 Mon Sep 17 00:00:00 2001 From: Dana Singh Date: Tue, 26 May 2026 14:16:42 -0400 Subject: [PATCH 20/20] add little bit to error msg --- fre/make/create_checkout_script.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fre/make/create_checkout_script.py b/fre/make/create_checkout_script.py index 82b905b82..a24add164 100644 --- a/fre/make/create_checkout_script.py +++ b/fre/make/create_checkout_script.py @@ -198,7 +198,7 @@ def checkout_create(yamlfile: str, platform: tuple, target: tuple, "2. If editing checked out source code directly for the bare-metal build:\n" " - users should now follow up with running each fre make subtool individually\n" " ('makefile', 'compile-script' or 'dockerfile') to avoid conflicting 'existing\n" - " checkout script' errors seen with 'fre make all'\n") from exc + " checkout script/SRC DIR` errors seen with 'fre make all'\n") from exc else: return