diff --git a/serverless/workers/concurrent-handler.mdx b/serverless/workers/concurrent-handler.mdx index a1d0ac6b..3c6859f7 100644 --- a/serverless/workers/concurrent-handler.mdx +++ b/serverless/workers/concurrent-handler.mdx @@ -223,6 +223,25 @@ def update_request_rate(request_history): # request_rate = redis_client.get('recent_request_rate') ``` +## Automatic job stopping + +When a worker processes multiple jobs concurrently, individual jobs can be stopped without affecting sibling jobs. If a request expires, times out, or is cancelled, Runpod signals the worker to stop only that job while other in-progress jobs continue running. + +This behavior is automatic and requires no handler changes. If your async handler needs to release resources when stopped, catch `asyncio.CancelledError`: + +```python +async def process_request(job): + resource = acquire_resource() + try: + await do_work(job) + return result + except asyncio.CancelledError: + release_resource(resource) + raise +``` + +Re-raise `CancelledError` after cleanup so the worker correctly marks the job as stopped. + ## Next steps Now that you've created a concurrent handler, you're ready to: diff --git a/serverless/workers/handler-functions.mdx b/serverless/workers/handler-functions.mdx index 5cc9c581..b14c6336 100644 --- a/serverless/workers/handler-functions.mdx +++ b/serverless/workers/handler-functions.mdx @@ -153,6 +153,19 @@ Always test your async code thoroughly to properly handle asynchronous exception +When a job is stopped due to timeout, expiration, or cancellation, async handlers receive an `asyncio.CancelledError`. Catch this exception to release resources, then re-raise it: + +```python handler.py +async def async_handler(job): + resource = acquire_resource() + try: + await process(job) + return result + except asyncio.CancelledError: + release_resource(resource) + raise +``` + ### Concurrent handlers Concurrent handlers process multiple requests simultaneously with a single worker. Use these for small, rapid operations that don't fully utlize the worker's GPU.