Skip to content

fix(dataproc): add robust error handling for quota exhaustion#4351

Open
angelcaamal wants to merge 3 commits into
mainfrom
fix/dataproc-test-robustness
Open

fix(dataproc): add robust error handling for quota exhaustion#4351
angelcaamal wants to merge 3 commits into
mainfrom
fix/dataproc-test-robustness

Conversation

@angelcaamal

@angelcaamal angelcaamal commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Description

Improve test suite robustness against quota and infrastructure errors

Key Changes

  • Tests now identify and skip failures caused by DISKS_TOTAL_GB quota exhaustion

Fixes Internal: b/525842927

Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.

Checklist

  • I have followed guidelines from CONTRIBUTING.MD and Samples Style Guide
  • Tests pass: npm test (see Testing)
  • Lint pass: npm run lint (see Style)
  • Required CI tests pass (see CI testing)
  • These samples need a new API enabled in testing projects to pass (let us know which ones)
  • These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
  • This pull request is from a branch created directly off of GoogleCloudPlatform/nodejs-docs-samples. Not a fork.
  • This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
  • This sample adds a new sample directory, and I created GitHub Actions workflow for this sample
  • This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
  • Please merge this PR for me once it is approved

Note: Any check with (dev), (experimental), or (legacy) can be ignored and should not block your PR from merging (see CI testing).

@angelcaamal angelcaamal added the api: dataproc Issues related to the Managed Service for Apache Spark API. label Jun 19, 2026
@product-auto-label product-auto-label Bot added the samples Issues that are directly related to samples. label Jun 19, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces error handling in Dataproc system tests to gracefully skip execution when quota limits, resource exhaustion, or disk limits are encountered. The reviewer pointed out that skipping cluster creation will cause the corresponding after cleanup hooks to fail with NOT_FOUND errors when attempting to delete the non-existent cluster. It is recommended to wrap the cleanup logic in try-catch blocks to safely ignore these errors.

Comment thread dataproc/system-test/createCluster.test.js
Comment thread dataproc/system-test/submitJob.test.js
@angelcaamal angelcaamal marked this pull request as ready for review June 19, 2026 20:39
@angelcaamal angelcaamal requested review from a team as code owners June 19, 2026 20:39

@amcolin amcolin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: dataproc Issues related to the Managed Service for Apache Spark API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants