Skip to content

Scope orphaned-event cleanup to recent ingest window to eliminate 8-hour ES CPU spikes#2279

Open
niemyjski with Copilot wants to merge 4 commits into
mainfrom
copilot/investigate-cluster-cpu-spikes
Open

Scope orphaned-event cleanup to recent ingest window to eliminate 8-hour ES CPU spikes#2279
niemyjski with Copilot wants to merge 4 commits into
mainfrom
copilot/investigate-cluster-cpu-spikes

Conversation

Copilot AI commented May 31, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Limits orphaned-event stack, project, and organization scans to events created within the last three days.
  • Applies the same cutoff to cardinality queries, partitioned terms queries, and delete-by-query operations.
  • Captures one cutoff per job run so all three cleanup phases use an atomic boundary.
  • Uses the current Elastic.Clients.Elasticsearch and repository architecture from main.
  • Adds regression coverage for stack, project, and organization orphans at and immediately before the inclusive cutoff.

Why

The cleanup job runs every eight hours. Scanning all event history caused unnecessary Elasticsearch CPU spikes even though actionable orphan cleanup is limited to the practical recent-ingest window.

Verification

  • dotnet build tests/Exceptionless.Tests/Exceptionless.Tests.csproj --no-restore --nologo --verbosity:minimal
  • Exceptionless.Tests --filter-class Exceptionless.Tests.Jobs.CleanupDataJobTests: 28 passed
  • Exact lookback-boundary regression: passed
  • Local Aspire dogfood with healthy Elasticsearch 8.19.15, Redis, API, and Jobs resources

Compatibility

No public API, WebSocket, configuration, or frontend behavior changes.

@CLAassistant

CLAassistant commented May 31, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ niemyjski
❌ Copilot
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI linked an issue Jun 1, 2026 that may be closed by this pull request
Copilot AI and others added 2 commits June 1, 2026 00:22
Co-authored-by: niemyjski <1020579+niemyjski@users.noreply.github.com>
Co-authored-by: niemyjski <1020579+niemyjski@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate cluster CPU spikes during orphan data cleanup Scope orphaned-event cleanup to recent ingest window to eliminate 8-hour ES CPU spikes Jun 1, 2026
Copilot AI requested a review from niemyjski June 1, 2026 00:28
…luster-cpu-spikes

# Conflicts:
#	src/Exceptionless.Core/Jobs/CleanupOrphanedDataJob.cs
#	tests/Exceptionless.Tests/Jobs/CleanupDataJobTests.cs
@niemyjski niemyjski marked this pull request as ready for review July 10, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate cluster CPU spikes

3 participants