Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions serverless/workers/concurrent-handler.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,25 @@ def update_request_rate(request_history):
# request_rate = redis_client.get('recent_request_rate')
```

## Automatic job stopping

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added per-job stop capability section based on PR #510 which introduces JobsToStop in worker_state.py and stop_job() in rp_scale.py. The PR's inline docs in docs/serverless/worker.md confirm handlers need no changes and async handlers can catch asyncio.CancelledError.

Source: runpod/runpod-python#510

When a worker processes multiple jobs concurrently, individual jobs can be stopped without affecting sibling jobs. If a request expires, times out, or is cancelled, Runpod signals the worker to stop only that job while other in-progress jobs continue running.

This behavior is automatic and requires no handler changes. If your async handler needs to release resources when stopped, catch `asyncio.CancelledError`:

```python
async def process_request(job):
resource = acquire_resource()
try:
await do_work(job)
return result
except asyncio.CancelledError:
release_resource(resource)
raise
```

Re-raise `CancelledError` after cleanup so the worker correctly marks the job as stopped.

## Next steps

Now that you've created a concurrent handler, you're ready to:
Expand Down
13 changes: 13 additions & 0 deletions serverless/workers/handler-functions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,19 @@ Always test your async code thoroughly to properly handle asynchronous exception

</Tip>

When a job is stopped due to timeout, expiration, or cancellation, async handlers receive an `asyncio.CancelledError`. Catch this exception to release resources, then re-raise it:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added asyncio.CancelledError handling guidance based on PR #510. The handle_job() method in rp_scale.py catches CancelledError and logs the stop, confirming this is the expected exception type for job cancellation.

Source: runpod/runpod-python#510


```python handler.py
async def async_handler(job):
resource = acquire_resource()
try:
await process(job)
return result
except asyncio.CancelledError:
release_resource(resource)
raise
```

### Concurrent handlers

Concurrent handlers process multiple requests simultaneously with a single worker. Use these for small, rapid operations that don't fully utlize the worker's GPU.
Expand Down
Loading