Skip to content

wait_result aborts on a transient result-backend error instead of retrying the poll #655

Description

@mdrachuk

Taskiq version

0.12.4 (also present on master as of today), taskiq-redis 1.2.3, redis-py 8.0.1

Python version

Python 3.14

OS

Linux (Docker, python:3.14)

What happened?

AsyncTaskiqTask.wait_result treats the failure of a single is_ready() poll as fatal. With a
network-backed result backend, one blip — a Redis read timeout, a failover, a brief connection
reset — ends the wait, even though the timeout budget has barely been touched and the task itself
is unaffected.

In taskiq/task.py:

async def is_ready(self) -> bool:
    try:
        return await self.result_backend.is_result_ready(self.task_id)
    except Exception as exc:
        raise ResultIsReadyError from exc

async def wait_result(self, check_interval=0.2, timeout=-1.0, with_logs=False):
    start_time = time()
    while not await self.is_ready():          # <-- any backend error propagates out of the loop
        if 0 < timeout < time() - start_time:
            raise TaskiqResultTimeoutError(timeout=timeout)
        await asyncio.sleep(check_interval)
    return await self.get_result(with_logs=with_logs)

is_ready() deliberately converts every backend exception into ResultIsReadyError, but
wait_result never catches it, so the loop unwinds on the first bad poll. get_result() has the
same shape: a blip in the window between "ready" and the read raises ResultGetError out of
wait_result.

Why this matters: a failed poll carries no information about the task. The worker may be
mid-run, or may have finished and written its result microseconds later. Callers that log or
persist the outcome have to record a failure for a task that likely succeeded. Since timeout
already bounds the wait, ending it early on a transient error gives up budget that was explicitly
granted.

This bites hardest for long-running tasks, where the polling window is wide and the chance of
catching one bad poll approaches certainty. We hit it with a ~750s result timeout on agent tasks
that routinely run for minutes.

Proposed fix

Either would solve it; the second is more conservative:

  1. Treat backend errors as "not ready" until the timeout expires. Retry the poll, and on expiry
    raise TaskiqResultTimeoutError with the last backend error as __cause__. Arguably what
    timeout already promises, but it does change behavior for anyone relying on fast failure.
  2. Opt in via a parameter, e.g. wait_result(..., retry_on_backend_error: bool = False),
    leaving today's behavior as the default.

In both cases, backing the check_interval off while the backend is erroring would avoid hammering
a struggling instance.

Our workaround is a local loop over is_ready()/get_result() that catches ResultBackendError,
backs off, and keeps the original deadline. It works, but it duplicates library logic to get retry
behavior that only the library can place cleanly.

Happy to open a PR for whichever direction you prefer.

Relevant log output

redis.exceptions.TimeoutError: Timeout reading from redis:6379

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.14/site-packages/taskiq/task.py", line 45, in is_ready
    return await self.result_backend.is_result_ready(self.task_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.14/site-packages/taskiq_redis/redis_backend.py", line 128, in is_result_ready
    return bool(await redis.exists(self._task_name(task_id)))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [redis-py connection frames elided]
  File "/app/.venv/lib/python3.14/site-packages/taskiq/task.py", line 47, in is_ready
    raise ResultIsReadyError from exc
taskiq.exceptions.ResultIsReadyError: Cannot find out if the task is ready

The caller frame, for context — a wait with most of its 750s budget still unspent:

  File "/app/src/workers/ticker/ticker.py", line 132, in _record_result
    task_result = await task.wait_result(timeout=self._result_timeout_seconds)
  File "/app/.venv/lib/python3.14/site-packages/taskiq/task.py", line 100, in wait_result
    while not await self.is_ready():

Broker initialization code

ticks_broker = (
    RedisStreamBroker(
        redis_uri,
        queue_name="taskiq:ticks",
        consumer_group_name="taskiq:ticks",
        idle_timeout=_NEVER_REDELIVER_MS,
        maxlen=100_000,
    )
    .with_result_backend(RedisAsyncResultBackend(redis_uri, result_ex_time=900))
)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions