Skip to content

Send heartbeat when downloading spooled segments#604

Open
azawlocki-sbdt wants to merge 4 commits into
trinodb:masterfrom
azawlocki-sbdt:azawlocki/query-heartbeat-for-spooling
Open

Send heartbeat when downloading spooled segments#604
azawlocki-sbdt wants to merge 4 commits into
trinodb:masterfrom
azawlocki-sbdt:azawlocki/query-heartbeat-for-spooling

Conversation

@azawlocki-sbdt
Copy link
Copy Markdown

@azawlocki-sbdt azawlocki-sbdt commented May 18, 2026

Description

Resolves #554

Trino can abandon a query if the client is silent for too long. When results are delivered via the spooling protocol, downloading a segment from S3/MinIO blocks the client and may cause query timeout if the download takes too long (the timeout is set in Trino configuration, current default is 5 min).

This PR implements a heartbeat mechanism: a background thread is sending periodic HEAD requests to the coordinator's nextUri while a spooled segment is being downloaded. The heartbeat is started and stopped around each SpooledSegment.data fetch inside SegmentIterator._load_next_segment(). Heartbeat thread is not started if spooling is not used.

Heartbeat requests are emitted in 30s intervals by default. Custom interval can be set using the new heartbeat_interval param for Connection. Setting it to None disables heartbeat.

Unit tests are added in the new module tests/unit/test_client_spooling.py.
An integration test is added in tests/integration/test_dbapi_integration.py.

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

* Fix some things. ({issue}`issuenumber`)

@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented May 18, 2026

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@azawlocki-sbdt azawlocki-sbdt changed the title Senf heartbeat when downloading spooled segments Send heartbeats when downloading spooled segments May 18, 2026
@azawlocki-sbdt azawlocki-sbdt marked this pull request as draft May 18, 2026 17:01
@azawlocki-sbdt azawlocki-sbdt force-pushed the azawlocki/query-heartbeat-for-spooling branch from 8c23aae to 018afff Compare May 18, 2026 17:03
@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented May 18, 2026

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented May 22, 2026

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

1 similar comment
@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented May 22, 2026

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@azawlocki-sbdt azawlocki-sbdt changed the title Send heartbeats when downloading spooled segments Send heartbeat when downloading spooled segments May 22, 2026
@azawlocki-sbdt azawlocki-sbdt force-pushed the azawlocki/query-heartbeat-for-spooling branch from 64e228b to 38cf8a3 Compare May 25, 2026 08:43
@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented May 25, 2026

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@azawlocki-sbdt azawlocki-sbdt marked this pull request as ready for review May 25, 2026 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Send query heartbeat while downloading results

1 participant