Skip to content

Shut down executors in hello_activity_multiprocess - #335

Open
barmoshe wants to merge 2 commits into
temporalio:mainfrom
barmoshe:fix-executor-shutdown
Open

Shut down executors in hello_activity_multiprocess#335
barmoshe wants to merge 2 commits into
temporalio:mainfrom
barmoshe:fix-executor-shutdown

Conversation

@barmoshe

Copy link
Copy Markdown

What was changed

hello_activity_multiprocess.py constructed a ProcessPoolExecutor and a multiprocessing.Manager() inline in the Worker(...) call and never shut either down, so the sample was demonstrating a resource leak. Both are now entered as context managers outside the worker, so they outlive it and are shut down once it is done with them.

Two details worth calling out:

  • The manager is entered first so it is torn down last. It owns the heartbeat queue, and _MultiprocessingSharedStateManager._heartbeat_processor can still be blocked on self._heartbeat_queue.get(True, 0.5) for up to its poll timeout after the worker has exited. Shutting the manager down before the pool leaves that read on a dead connection.
  • The queue poller executor is now created explicitly. SharedStateManager.create_from_multiprocessing builds its own ThreadPoolExecutor(1) when one is not passed, and neither the shared state manager nor the worker ever shuts it down, so without this the sample would still be leaving an executor unshut. Passing our own lets it be shut down with the others.

That second point is the one piece here that is arguably sdk-python's lifetime problem rather than the sample's. If you would rather the SDK owned that executor, say so and I will drop it and keep this PR to the pool and the manager.

Not included

The same inline-executor pattern (activity_executor=ThreadPoolExecutor(5)) is in about a dozen other samples, including hello_activity.py, hello_cron.py, hello_exception.py and hello_mtls.py. I kept this PR to the file the issue names. Happy to do the rest as a follow-up if you want them.

Issue #50 also names hello_activity_threaded.py, which was removed in #173. Separately, the root README.md still links that deleted file; happy to send a one-line fix for it.

Testing

  • Ran uv run hello/hello_activity_multiprocess.py against temporal server start-dev four times. The activity runs in a separate process, the result is correct, and the process exits in about four seconds with no shutdown traceback, confirming the added executor does not hang on shutdown.
  • uv run poe lint clean.

Checklist

  1. Closes [Feature Request] Properly shutdown executors when done using them #50

barmoshe and others added 2 commits July 30, 2026 13:46
The sample passed a freshly constructed ProcessPoolExecutor and
multiprocessing.Manager straight into the Worker and never shut either down,
so the sample was teaching a resource leak. Both are now entered as context
managers, outside the worker so they outlive it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two follow-ups on the previous commit:

- The manager is now entered first so it is torn down last. It owns the
  heartbeat queue, and the SDK's heartbeat poller can still be blocked reading
  that queue for up to its 0.5s poll timeout after the worker exits.
- create_from_multiprocessing builds its own ThreadPoolExecutor when no queue
  poller executor is passed, and nothing ever shuts that one down. Pass an
  explicit one so it is shut down with the rest.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@barmoshe
barmoshe requested a review from a team as a code owner July 30, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Properly shutdown executors when done using them

1 participant