What a great setup, you can nearly duplicate all the stuff that requires the cloud on your local machine. While open-webui really likes using docker, I have to say that the overhead is pretty big particularly when all that is really going on is the need to isolate python environments. So instead of that I’ve been using the miracles of pipx install to accomplish all that and not have to lose all that memory to a docker container.
This is particularly relevant on Macs where you. have to allocate memory to docker, so if you have a big machine, you can’t share the storage. So here is a quick guide as of the beginning of December on how to make it all work.
Ollama
Ollama is a great way to pull models and is installed as simply as brew install ollama which is great. Then it is just ollama serve and you have yourself on your local host an end point that can get to any OpenAI compatible cloud provider or to local machines
Open-Webui
This used to be pretty hard, but now you can just do a pipx install open-webui --python 3.11 and you get an environment with open-webui using its supported version of python. It is completely isolated and works like a bundled Mac app, you just need to make sure to include ~/.local/bin in your path and you are ready to go
Ngrok
OK, I had not heard of this until it is mentioned with open-webui, but you basically login to Ngrok.com and then brew install ngrok and then when you get your token, just doing a ngrok http --url=_your domain they give you_.com 8080 will remote your local installation up there with an SSL authentication.
So here are the things you need to do thanks to the miracles of one password is just fetch the token which should in the login entry ‘ngrok’ and in the field ‘auth token’. We use --reveal so the password is right there. You can nicely edit the config file as well with ngrok config edit and even add it to Chezmoi as well. Then it is a matter of using the free oauth authenticator against the nice static endpoint they give you:
brew instal ngrok
open https://ngrok.com
# sign in and you get an authentication token
# put it in 1Password along with your static domain
ngrok config add-authtoken "$(op item get "ngrok" --field "auth token" --reveal)"
# now you are ready to go
open-webui serve &
# now connect ngrok to your localhost:8080 to a url
# and make sure only certain domains can access it
ngrok http --url "$(op item get "ngrok" --field "static domain")" --oauth google --oauth-allow-domain tongfamily.com Figuring out if it all works
Well all ils well if you check the local ngrok interfaces at http://localhost:4040 and the open-webui at localhost:8080
Leave a Reply
Only people in my network can comment.