I think they mentioned on their discord that it’s in the pipeline

I really wish there was an update feature built into OWUI!

Do you know if there is openai API compatibility? More specifically I’d like to get home assistant to interact with the LLM via the custom OpenAI hacs addon

I have two 3090 Turbo GPUs and it seems like oobabooga doesn’t split the load between the two cards when I try to run TheBloke/dolphin-2.7-mixtral-8x7b-AWQ.

Does anyone know how to make text generation webui use both cards? Do I need an nvlink between the two cards?

I’m sure that’s not that big of a deal to some people. For example, I’m mainly using LLMs for use in my home assistant instance