Fix flaky BigQuery file loads by safely handling concurrent mkdirs#38426
Fix flaky BigQuery file loads by safely handling concurrent mkdirs#38426shunping wants to merge 1 commit intoapache:masterfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a race condition in the BigQuery file loading process where multiple concurrent workers attempt to create the same destination directory simultaneously. By catching and validating the Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the directory creation logic in bigquery_file_loads.py to handle potential race conditions between concurrent workers. It wraps the mkdirs call in a try-except block to ignore IOError if the directory was successfully created by another worker. No review comments were provided for this change, and I have no additional feedback.
|
Assigning reviewers: R: @damccorm for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
1 participant
The test
TestBigQueryFileLoads.test_reshuffle_before_load_0is flaky (https://github.com/apache/beam/actions/runs/25584850292/job/75111278535?pr=38423) with the following traceback:When writing files for BigQuery loads, multiple workers or bundles processing records for the same destination can run concurrently. They all share the same file_prefix and construct the exact same destination directory path.
When two workers concurrently execute the code at https://github.com/shunping/beam/blob/a8e7ffab3716459518aeea2e81fe7181c6885179/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py#L136:
this could lead to a race condition when both workers see the directory non-existent and try to create it. As a result, only one worker will be able to create the directory successfully, while the other will raise a "File exists" error.
In this PR, we fix it by ignoring if a directory exists in the concurrent worker scenario.