Alternative Weight-Loading Approach

[EricHuangCCU]

Hello, I am a dLlama user.
First, I would like to thank you for your contribution to this project, which has given me many new ideas about running LLMs on edge devices.

While using dLlama, I observed that in some cases, multi-node inference takes longer than single-node inference. The main reason is that, before multi-node inference starts, the root node needs to transfer the weights to the workers via Ethernet, which increases the weight-loading time.

To improve this issue, I modified the implementation so that workers can also specify the model path using the --model parameter at runtime. In this way, each worker can load the weights directly from its local disk instead of receiving them from the root node.

My test results show that this approach can significantly reduce the weight-loading time. Therefore, I would like to share this feature with the project for your reference.

Thank you very much.