Feature Summary
Router mode to allow "hot swap" of models
Detailed Description
** I am not an expert in this realm (self hosted AI). I am just starting out. Please consider me a "greenhorn". At the very least, I have been successfull in building llama.cpp and stable-diffusion.cpp from source (due to the age of my test hardware). **
llama.cpp has "router mode" that allows to load/unload models without restarting the server. llama-server's built in webgui makes this load/unload process even easier. With llama.cpp's router mode it also has a --models-dir and --models-preset flag that allows faster spin up and testing by just launching the llama-server with those flags.
I know stable-diffusion.cpp does not currently have a built-in webgui and this is where this request comes in.
For fast spin up/testing it would be great if stable-diffusion.cpp had a "router mode" similar to llama.cpp and a simple built-in webgui similar to llama.cpp's llama-server webgui.
Alternatives you considered
I have tried as much of the webgui options as possible and I find the example of the llama.cpp "router mode" and built-in webgui (simple webgui) are beneficial and at least a good minimum for fast testing before settling on a framework.
Additional context
Due to the age of my testing hardware, some of the webgui projects won't install or work right despite being able to compile stable-diffusion.cpp successfully from source to work with my hardware. When I compile stable-diffusion.cpp from source, it works on my hardware. Some of the webgui projects introduce extra requirements that are more dependant on specific environments that target newer hardware. The prices of gpus is becoming more "unobtanium" for a hobbyist like me and "renting" gpus is not a route I want to take; especially if I can still build from source and have it work on my hardware.
Feature Summary
Router mode to allow "hot swap" of models
Detailed Description
** I am not an expert in this realm (self hosted AI). I am just starting out. Please consider me a "greenhorn". At the very least, I have been successfull in building llama.cpp and stable-diffusion.cpp from source (due to the age of my test hardware). **
llama.cpp has "router mode" that allows to load/unload models without restarting the server. llama-server's built in webgui makes this load/unload process even easier. With llama.cpp's router mode it also has a --models-dir and --models-preset flag that allows faster spin up and testing by just launching the llama-server with those flags.
I know stable-diffusion.cpp does not currently have a built-in webgui and this is where this request comes in.
For fast spin up/testing it would be great if stable-diffusion.cpp had a "router mode" similar to llama.cpp and a simple built-in webgui similar to llama.cpp's llama-server webgui.
Alternatives you considered
I have tried as much of the webgui options as possible and I find the example of the llama.cpp "router mode" and built-in webgui (simple webgui) are beneficial and at least a good minimum for fast testing before settling on a framework.
Additional context
Due to the age of my testing hardware, some of the webgui projects won't install or work right despite being able to compile stable-diffusion.cpp successfully from source to work with my hardware. When I compile stable-diffusion.cpp from source, it works on my hardware. Some of the webgui projects introduce extra requirements that are more dependant on specific environments that target newer hardware. The prices of gpus is becoming more "unobtanium" for a hobbyist like me and "renting" gpus is not a route I want to take; especially if I can still build from source and have it work on my hardware.