Skip to content

[Hubs] Fix ingestion OOM errors by eliminating strcat() in emptiness checks #2151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 7 commits into
base: dev
Choose a base branch
from
+6 −3
Open

[Hubs] Fix ingestion OOM errors by eliminating strcat() in emptiness checks #2151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d70eeda
Initial plan
Copilot May 18, 2026
7b94d47
Fix ingestion memory errors by replacing strcat() in empty checks
Copilot May 18, 2026
24e794a
Fix isnotempty(strcat()) pattern in finops-hub-14.0 and latest ARM te…
Copilot May 26, 2026
15a8bc1
Address PR feedback: revert deploy JSON edits and update changelog
Copilot May 26, 2026
a62f59f
Polish changelog entry wording per review feedback
Copilot May 26, 2026
4830100
Merge branch 'dev' into copilot/fix-ingestion-memory-errors
flanakin May 26, 2026
28d925d
Move ingestion memory fix changelog note to v15
Copilot May 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs-mslearn/toolkit/changelog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: FinOps toolkit changelog
description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more.
author: MSBrett
ms.author: brettwil
ms.date: 05/19/2026
ms.date: 05/26/2026
ms.topic: reference
ms.service: finops
ms.subservice: finops-toolkit
Expand Down Expand Up @@ -49,6 +49,9 @@ _Released June 2026_

- **Changed**
- Added a callout to the `config_RunBackfillJob` backfill option clarifying that it isn't supported on Microsoft Customer Agreement (MCA) billing accounts or billing profiles ([#2113](https://github.com/microsoft/finops-toolkit/issues/2113)).
- **Fixed**
- Fixed Data Factory ingestion memory pressure during emptiness filtering.
- Replaced `isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))` with separate `isnotempty()` checks in FinOps hub ingestion scripts to avoid temporary string allocation.

<!-- prettier-ignore-start -->
> [!div class="nextstepaction"]
Expand Down
2 changes: 1 addition & 1 deletion ...mplates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_0.kql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,7 @@ Costs_transform_v1_0()
and (ListUnitPrice == 0 or ContractedUnitPrice == 0)
and x_EffectiveUnitPrice != 0
and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')
and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))
and isnotempty(x_SkuMeterId) and isnotempty(x_SkuOfferId)
| as allCosts
| where tmp_MissingPrices
| extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))
Expand Down
2 changes: 1 addition & 1 deletion ...mplates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql
View file
Open in desktop
Comment thread
flanakin marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ Costs_transform_v1_2()
and (isempty(ListUnitPrice) or isempty(ContractedUnitPrice) or ListUnitPrice == 0 or ContractedUnitPrice == 0)
and x_EffectiveUnitPrice != 0
and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')
and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))
and isnotempty(x_SkuMeterId) and isnotempty(x_SkuOfferId)
| as allCosts
| where tmp_MissingPrices
| extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))
Expand Down
Loading