Skip to content

75 notebooks #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dalonsoa merged 7 commits into from
Jul 29, 2026
Merged

75 notebooks #84

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
71d441e
update capacity plot
dalonsoa Jul 16, 2026
6cf4d68
Move legend in prices plot
dalonsoa Jul 16, 2026
e524b00
Simplify code following review
dalonsoa Jul 17, 2026
3d5ff46
Update capacity notebook
dalonsoa Jul 17, 2026
008bd04
Update pre-commit-hooks to remove empy cells
dalonsoa Jul 17, 2026
b230c86
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 17, 2026
46a7f08
Do not test on release.
dalonsoa Jul 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ on:
workflow_call:

jobs:
test_release:
name: Latest MUSE2 release
uses: ./.github/workflows/test-with-muse2.yml
with:
muse2_source: release
secrets: inherit
# test_release:
# name: Latest MUSE2 release
# uses: ./.github/workflows/test-with-muse2.yml
# with:
# muse2_source: release
# secrets: inherit
Comment on lines +12 to +17

test_main:
name: Current MUSE2 main branch
Expand Down
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ repos:
rev: 0.9.1
hooks:
- id: nbstripout
args: [--drop-empty-cells]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v2.1.0
hooks:
Expand Down
84 changes: 27 additions & 57 deletions notebooks/capacity.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@
"This notebook contains example data processing using the output of an example\n",
"model.\n",
"\n",
"Output files are mostly in CSV format. The format of output files is documented [here][output-format].\n",
"\n",
"We begin by loading the `model.toml` input file to get the list of milestone years.\n",
"Output files are mostly in CSV format. The format of output files is documented [in MUSE2 documentation][output-format].\n",
"\n",
"[output-format]: https://energysystemsmodellinglab.github.io/MUSE2/file_formats/output_files.html"
]
Expand All @@ -24,20 +22,11 @@
"metadata": {},
"outputs": [],
"source": [
"import tomllib\n",
"\n",
"import pandas as pd\n",
"\n",
"from muse2_data_analysis.helpers import get_example_input_dir, get_example_output_dir\n",
"\n",
"INPUT_DIR = get_example_input_dir()\n",
"OUTPUT_DIR = get_example_output_dir()\n",
"from muse2_data_analysis.helpers import get_example_output_dir\n",
"\n",
"with (INPUT_DIR / \"model.toml\").open(\"rb\") as f:\n",
" model = tomllib.load(f)\n",
"\n",
"# We need to know the milestone years for processing the assets file\n",
"years = model[\"milestone_years\"]"
"OUTPUT_DIR = get_example_output_dir()"
]
},
{
Expand All @@ -48,14 +37,13 @@
"## Load and process output data\n",
"\n",
"We next load the output data. In this case, we want to calculate how much capacity was invested in\n",
"different processes for different agents. This information can be found in the `assets.csv` output\n",
"file.\n",
"different processes for different agents. This information can be found in the `asset_capacities.csv`\n",
"output file.\n",
"\n",
"The `assets.csv` file contains information about different assets, including when they were\n",
"commissioned and decommissioned as well as their capacity. To calculate the overall capacity for a\n",
"given agent and process type, we have to process this data. Note that different assets owned by the\n",
"same agent may have the same process ID if the agent has reinvested in the same process type in a\n",
"different year."
"We also need some metadata, like the agent that manages the asset and the process the asset corresponds\n",
"to. That time-independent information is contained in the `assets.csv` file. To calculate the overall\n",
"capacity for a given agent and process type as a function of milestone year, we have to combine these\n",
"two files and process the data."
]
},
{
Expand All @@ -65,28 +53,15 @@
"metadata": {},
"outputs": [],
"source": [
"# The assets.csv file contains info about which assets were invested in and when\n",
"assets = pd.read_csv(OUTPUT_DIR / \"assets.csv\")\n",
"\n",
"# Assets with no decommission_year are effectively decommissioned after time horizon\n",
"assets[\"decommission_year\"] = assets[\"decommission_year\"].fillna(years[-1] + 1)\n",
"\n",
"# Calculate capacity for each type of process for each agent\n",
"capacity = pd.DataFrame()\n",
"for year in years:\n",
" active = assets[\n",
" (year >= assets[\"commission_year\"]) & (year < assets[\"decommission_year\"])\n",
" ]\n",
"\n",
" # This only works because each agent is responsible for one and only one commodity\n",
" cap_sum = active.groupby([\"agent_id\", \"process_id\"])[\"capacity\"].sum().reset_index()\n",
"\n",
" df = pd.DataFrame(cap_sum)\n",
" df[\"year\"] = year\n",
"\n",
" capacity = pd.concat([capacity, df])\n",
"\n",
"capacity"
"asset_capacities = pd.read_csv(OUTPUT_DIR / \"asset_capacities.csv\")\n",
"\n",
"merged = asset_capacities.merge(assets, on=[\"asset_id\", \"group_id\"])\n",
"agg_capacities = merged.groupby([\"milestone_year\", \"agent_id\", \"process_id\"])[\n",
" \"capacity\"\n",
"].sum()\n",
"agg_capacities_wide = agg_capacities.unstack([\"agent_id\", \"process_id\"], fill_value=0)\n",
Comment on lines +59 to +63

@tsmbland tsmbland Jul 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does work though 🤷

@dalonsoa dalonsoa Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might work in this case because there's no group_id in any of the rows. Maybe if there were, things would fail.

@tsmbland tsmbland Jul 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it with the simple_divisible model (which has rows with group_id populated) and it worked, so I think it's fine

"agg_capacities_wide"
]
},
{
Expand All @@ -96,10 +71,7 @@
"source": [
"## Plot results\n",
"\n",
"Finally, we plot the results.\n",
"\n",
"Note that each of the agents has invested in only one process type; otherwise there would be\n",
"multiple bars per plot here."
"Finally, we plot the results."
]
},
{
Expand All @@ -111,22 +83,20 @@
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"agents = capacity[\"agent_id\"].unique()\n",
"_, axes = plt.subplots(1, len(agents))\n",
"agents = agg_capacities_wide.columns.get_level_values(\"agent_id\").unique()\n",
"fig, axes = plt.subplots(1, len(agents), figsize=(4 * len(agents), 4))\n",
"for ax, agent in zip(axes, agents):\n",
" capacity[capacity[\"agent_id\"] == agent].pivot(\n",
" index=\"year\", columns=\"process_id\", values=\"capacity\"\n",
" ).plot(kind=\"bar\", ax=ax)\n",
" ax.set_title(agent)\n",
" ax.set_xlabel(\"Year\")\n",
" ax.set_ylabel(\"Capacity\")\n",
" ax.legend(title=\"Process\")"
" agg_capacities_wide[agent].plot(\n",
" kind=\"bar\", stacked=True, ax=ax, title=agent, xlabel=\"Year\", ylabel=\"Capacity\"\n",
" )\n",
Comment on lines +86 to +91

@tsmbland tsmbland Jul 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not too concerned about this. If there was only one agent then you'd plot in a different way anyway

Comment on lines +89 to +91
"\n",
"plt.tight_layout()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "muse2-data-analysis",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -140,7 +110,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.14.2"
"version": "3.14.6"
}
},
"nbformat": 4,
Expand Down
8 changes: 5 additions & 3 deletions notebooks/prices.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,15 @@
"\n",
"ax.set_xlabel(\"Milestone year\")\n",
"ax.set_ylabel(\"Price\")\n",
"ax.legend(title=\"Time slice\");"
"ax.legend(\n",
" title=\"Time slice\", bbox_to_anchor=(1.05, 1), loc=\"upper left\", borderaxespad=0.0\n",
");"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "muse2-data-analysis",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -70,7 +72,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.14.2"
"version": "3.14.6"
}
},
"nbformat": 4,
Expand Down
Loading