There are a few instances within task functions where print statements are used to warn of unexpected behavior. These prints are written to the standard out of the dask worker command call in the batch script, resulting in all workers writing to that stream.
I think its valuable to separate the print streams so that any messages can be correlated with the worker performing a specific task. It'd be nice to setup a logging instance for each worker at the beginning of the dask workflow that logs task info, performance, any error/warning messages, etc.
There are a few instances within task functions where print statements are used to warn of unexpected behavior. These prints are written to the standard out of the
dask workercommand call in the batch script, resulting in all workers writing to that stream.I think its valuable to separate the print streams so that any messages can be correlated with the worker performing a specific task. It'd be nice to setup a logging instance for each worker at the beginning of the dask workflow that logs task info, performance, any error/warning messages, etc.