From fe13d348d624f66d84edf187f6e75bebba10a5e0 Mon Sep 17 00:00:00 2001 From: nick-gorman Date: Mon, 25 May 2026 14:24:04 +1000 Subject: [PATCH] Fix #74: double-encode # in archive URLs to match nemweb's on-disk names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Post-2024-07 PUBLIC_ARCHIVE# monthly MMS files are stored on nemweb.com.au with literal `%23` characters in their filenames (not `#`). NEMOSIS's `download_unzip_csv` was sending URLs with single `%23` encoding — nemweb URL-decodes `%23` to `#`, looks for a `#`-named file, finds none, and returns HTTP 400. The result: dynamic_data_compiler fails with NoDataToReturn for any post-2024-07 PUBLIC_ARCHIVE# table (DISPATCHPRICE, DISPATCHLOAD, etc.) on a cold cache. To match the real filename on disk the URL needs `%2523` so nemweb decodes it once to `%23`. Verified directly against the live server: single `%23` returns 400; `%2523` returns 200 with the real zip body. Three changes here: 1. `src/nemosis/downloader.py::download_unzip_csv` — change the `url.replace("#", "%23")` step to `url.replace("#", "%2523")`. This is the only place NEMOSIS percent-encodes URLs at fetch time; pre-2024-08 PUBLIC_DVD_* filenames don't contain `#` so the replace is a no-op for them. 2. `tests/fixtures/build.py` — apply the same fix in `http_get` (which the build pipeline uses to fetch fixtures from real AEMO), and write the resulting fixtures to disk under their `%23`-form name in `mms_fixture_path` so they match nemweb's actual filename layout. 3. Rename the existing 48 PUBLIC_ARCHIVE# fixture zips in `tests/fixtures/data/.../MMSDM/2024_*/` and `MMSDM/2025_*/` from `…#…zip` to `…%23…zip` via `git mv`. Required because the offline test suite stands up a `http.server` over those files and serves them at the URL NEMOSIS requests — once NEMOSIS sends `%2523`, the server decodes it to `%23` and needs to find a `%23`-named file on disk. Why the offline tests didn't catch this before: the fixture filenames used literal `#`, which disagreed with how real nemweb stores the same files. Python's `http.server` URL-decoded `%23` → `#` and happily served them, so the encoding mismatch was masked. After this change the fixture filenames mirror the real on-disk layout and the offline suite would now flag a regression in the URL encoding. All 222 offline tests pass. Live verification: $ curl -I '…/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202412010000.zip' HTTP/1.1 400 Bad Request $ curl -I '…/PUBLIC_ARCHIVE%2523DISPATCHPRICE%2523FILE01%2523202412010000.zip' HTTP/1.1 200 OK (2,140,354 bytes) Fixes #74. Co-Authored-By: Claude Opus 4.7 (1M context) --- src/nemosis/downloader.py | 14 ++++++++++++-- tests/fixtures/build.py | 16 ++++++++++++++-- ...23BIDDAYOFFER_D%23FILE01%23202408010000.zip} | Bin ...23BIDPEROFFER_D%23FILE01%23202408010000.zip} | Bin ...PATCHCONSTRAINT%23FILE01%23202408010000.zip} | Bin ...TERCONNECTORRES%23FILE01%23202408010000.zip} | Bin ...%23DISPATCHLOAD%23FILE01%23202408010000.zip} | Bin ...23DISPATCHPRICE%23FILE01%23202408010000.zip} | Bin ...SPATCHREGIONSUM%23FILE01%23202408010000.zip} | Bin ...ATCH_UNIT_SCADA%23FILE01%23202408010000.zip} | Bin ...23MNSP_DAYOFFER%23FILE01%23202408010000.zip} | Bin ...OFTOP_PV_ACTUAL%23FILE01%23202408010000.zip} | Bin ...INGINTERCONNECT%23FILE01%23202408010000.zip} | Bin ...%23TRADINGPRICE%23FILE01%23202408010000.zip} | Bin ...23BIDDAYOFFER_D%23FILE01%23202409010000.zip} | Bin ...23BIDPEROFFER_D%23FILE01%23202409010000.zip} | Bin ...PATCHCONSTRAINT%23FILE01%23202409010000.zip} | Bin ...TERCONNECTORRES%23FILE01%23202409010000.zip} | Bin ...%23DISPATCHLOAD%23FILE01%23202409010000.zip} | Bin ...23DISPATCHPRICE%23FILE01%23202409010000.zip} | Bin ...SPATCHREGIONSUM%23FILE01%23202409010000.zip} | Bin ...ATCH_UNIT_SCADA%23FILE01%23202409010000.zip} | Bin ...23MNSP_DAYOFFER%23FILE01%23202409010000.zip} | Bin ...OFTOP_PV_ACTUAL%23FILE01%23202409010000.zip} | Bin ...INGINTERCONNECT%23FILE01%23202409010000.zip} | Bin ...%23TRADINGPRICE%23FILE01%23202409010000.zip} | Bin ...23BIDDAYOFFER_D%23FILE01%23202412010000.zip} | Bin ...23BIDPEROFFER_D%23FILE01%23202412010000.zip} | Bin ...PATCHCONSTRAINT%23FILE01%23202412010000.zip} | Bin ...TERCONNECTORRES%23FILE01%23202412010000.zip} | Bin ...%23DISPATCHLOAD%23FILE01%23202412010000.zip} | Bin ...23DISPATCHPRICE%23FILE01%23202412010000.zip} | Bin ...SPATCHREGIONSUM%23FILE01%23202412010000.zip} | Bin ...ATCH_UNIT_SCADA%23FILE01%23202412010000.zip} | Bin ...23MNSP_DAYOFFER%23FILE01%23202412010000.zip} | Bin ...OFTOP_PV_ACTUAL%23FILE01%23202412010000.zip} | Bin ...INGINTERCONNECT%23FILE01%23202412010000.zip} | Bin ...%23TRADINGPRICE%23FILE01%23202412010000.zip} | Bin ...23BIDDAYOFFER_D%23FILE01%23202501010000.zip} | Bin ...23BIDPEROFFER_D%23FILE01%23202501010000.zip} | Bin ...PATCHCONSTRAINT%23FILE01%23202501010000.zip} | Bin ...TERCONNECTORRES%23FILE01%23202501010000.zip} | Bin ...%23DISPATCHLOAD%23FILE01%23202501010000.zip} | Bin ...23DISPATCHPRICE%23FILE01%23202501010000.zip} | Bin ...SPATCHREGIONSUM%23FILE01%23202501010000.zip} | Bin ...ATCH_UNIT_SCADA%23FILE01%23202501010000.zip} | Bin ...23MNSP_DAYOFFER%23FILE01%23202501010000.zip} | Bin ...OFTOP_PV_ACTUAL%23FILE01%23202501010000.zip} | Bin ...INGINTERCONNECT%23FILE01%23202501010000.zip} | Bin ...%23TRADINGPRICE%23FILE01%23202501010000.zip} | Bin 50 files changed, 26 insertions(+), 4 deletions(-) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202408010000.zip => PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202408010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202409010000.zip => PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202409010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202412010000.zip => PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202412010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202501010000.zip} (100%) rename tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/{PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202501010000.zip => PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202501010000.zip} (100%) diff --git a/src/nemosis/downloader.py b/src/nemosis/downloader.py index e333b95..5986fe6 100644 --- a/src/nemosis/downloader.py +++ b/src/nemosis/downloader.py @@ -352,7 +352,13 @@ def download_to_dir(url, down_load_to, force_redo=False): Streams the response so large files don't have to fit in memory. """ - url = url.replace('#', '%23') + # Post-2024-07 AEMO archive files are stored on nemweb with literal + # `%23` in their on-disk filenames (not `#`). To match, the URL must + # contain `%2523` so nemweb decodes it once to `%23` and finds the + # file. A single `%23` would decode to `#` and 400. Pre-Aug-2024 + # PUBLIC_DVD_* filenames don't contain `#`, so the replace is a + # no-op for the older path. See issue #74. + url = url.replace('#', '%2523') filename = url.split('/')[-1].split('?')[0] path = os.path.join(down_load_to, filename) downloaded = download_to_path(url, path, force_redo=force_redo) @@ -370,7 +376,11 @@ def download_to_path(url, path_and_name, force_redo=False): mid-stream, the partial output file is removed before the exception propagates. """ - url = url.replace('#', '%23') + # See `download_to_dir` for why this is `%2523` and not `%23`. + # Repeated here because `download_to_path` is also called directly + # (e.g. from `download_csv`); the replace is idempotent (`%2523` + # contains no `#`) so double-encoding via `download_to_dir` is safe. + url = url.replace('#', '%2523') if os.path.isfile(path_and_name) and not force_redo: return False diff --git a/tests/fixtures/build.py b/tests/fixtures/build.py index 16a916b..84c6ae3 100644 --- a/tests/fixtures/build.py +++ b/tests/fixtures/build.py @@ -54,7 +54,12 @@ def http_get(url: str) -> requests.Response: log.info("GET %s", url) - r = requests.get(url.replace("#", "%23"), headers=USR_AGENT, timeout=180) + # Post-2024-07 PUBLIC_ARCHIVE# files are stored on nemweb with literal + # `%23` in their on-disk filenames. The HTTP URL must contain `%2523` + # so nemweb decodes it once to `%23` and finds the file. A single + # `%23` would decode to `#` and 400. Mirrors the fix in + # nemosis.downloader.download_unzip_csv (issue #74). + r = requests.get(url.replace("#", "%2523"), headers=USR_AGENT, timeout=180) r.raise_for_status() return r @@ -140,13 +145,20 @@ def mms_filename(table: str, era_date: date, chunk: int) -> str: def mms_fixture_path(table: str, era_date: date, chunk: int) -> Path: year, month = era_date.year, f"{era_date.month:02d}" + # Post-2024-07 archives are stored on nemweb with literal `%23` in + # the filename (not `#`). Mirror that on disk so the offline mock + # server serves files under the same name the real server uses — + # otherwise NEMOSIS's now-`%2523`-encoded URLs won't match. Pre-Aug-2024 + # PUBLIC_DVD_* filenames don't contain `#`, so the replace is a no-op + # there. See issue #74. + on_disk_name = mms_filename(table, era_date, chunk).replace("#", "%23") return ( FIXTURE_ROOT / "Data_Archive/Wholesale_Electricity/MMSDM" / str(year) / f"MMSDM_{year}_{month}" / "MMSDM_Historical_Data_SQLLoader/DATA" - / mms_filename(table, era_date, chunk) + / on_disk_name ) diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202408010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202408010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202408010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_08/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202408010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202409010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202409010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202409010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_09/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202409010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202412010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202412010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202412010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2024/MMSDM_2024_12/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202412010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDDAYOFFER_D#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDDAYOFFER_D%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#BIDPEROFFER_D#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23BIDPEROFFER_D%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHCONSTRAINT#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHCONSTRAINT%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHINTERCONNECTORRES#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHINTERCONNECTORRES%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHLOAD#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHLOAD%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHPRICE#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHPRICE%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCHREGIONSUM#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCHREGIONSUM%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#DISPATCH_UNIT_SCADA#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23DISPATCH_UNIT_SCADA%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#MNSP_DAYOFFER#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23MNSP_DAYOFFER%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#ROOFTOP_PV_ACTUAL#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23ROOFTOP_PV_ACTUAL%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGINTERCONNECT#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGINTERCONNECT%23FILE01%23202501010000.zip diff --git a/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202501010000.zip b/tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202501010000.zip similarity index 100% rename from tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE#TRADINGPRICE#FILE01#202501010000.zip rename to tests/fixtures/data/Data_Archive/Wholesale_Electricity/MMSDM/2025/MMSDM_2025_01/MMSDM_Historical_Data_SQLLoader/DATA/PUBLIC_ARCHIVE%23TRADINGPRICE%23FILE01%23202501010000.zip