So there are a zillion of these, including ones that do what yours does. But that being said, I'm going to always upvote every single one of them because this is better and more important that what the big companies are doing across the board.
Fwiw, I'm liking the koboldcpp "ecosystem", including koboldlite; basically it's letting me try a little of everything, though there may be something of a "jack of all trades master of none" thing going on. But it has these nice little presets for, e.g. llms, voice, music, images, a "chatgpt"-esque interface, etc.
Thank you! Would love to hear which ones you've used. Working with local files without anything going to the cloud was the main thing I wanted. I tried Jan, OpenWork, Chatbox and Lobe, and couldn't get that working well in any of them. If there's one I missed it would be great to check it out!!
Have you tried Open WebUI? It doesn't determine which model works best for your hardware, but it otherwise has a very large feature set with a large community, and runs offline by default. Geared more for technical users, however.
Haven't tried it yet, but I like that you shared your process in such an honest way, and kept it no frills, as in no LLM wall-of-text readme in the repo. Wish we could see that more often these days.
It's still pretty tricky to do this as a solo dev to keep feature parity in my experience (fun fact, even Claude's own Desktop app is made in Electron).
That being said, majority of the heavy LLM lifting happens natively with llama.cpp :)
I'm actually building something similar and thank you very much for sharing you gave me a few inspirations. I want mine to integrate seemlesly with an animated Ai companion (it's harder than it sounds haha). Anyways great stuff! Gave it a star.
Looks great. One thing I am missing is the ability to change the installation directory for the models. For space reasons, I want them to be on a different partition.
Linux. I saw you support HOB_USER_DATA, which is great. But I want my config to remain in .config/Hob and only move the models (or whatever else needs lots of space) to another disk/directory.
Edit: Actually I think HOB_USER_DATA is enough for my use case.
So there are a zillion of these, including ones that do what yours does. But that being said, I'm going to always upvote every single one of them because this is better and more important that what the big companies are doing across the board.
Fwiw, I'm liking the koboldcpp "ecosystem", including koboldlite; basically it's letting me try a little of everything, though there may be something of a "jack of all trades master of none" thing going on. But it has these nice little presets for, e.g. llms, voice, music, images, a "chatgpt"-esque interface, etc.
Thank you! Would love to hear which ones you've used. Working with local files without anything going to the cloud was the main thing I wanted. I tried Jan, OpenWork, Chatbox and Lobe, and couldn't get that working well in any of them. If there's one I missed it would be great to check it out!!
Have you tried Open WebUI? It doesn't determine which model works best for your hardware, but it otherwise has a very large feature set with a large community, and runs offline by default. Geared more for technical users, however.
I am using Hermes and I did not try charts, but the rest it can do with ease. And I use DeepSeek flash API as a driver most of the time.
In PasClaw, I specifically built it to do local embedding and other local only functionality. Whether it works well or not yet is another story.
Definitely check out Osaurus for macOS and Windows. Heavily privacy focused with e2ee iOS and iPadOS tethers.
Orca works great.
Haven't tried it yet, but I like that you shared your process in such an honest way, and kept it no frills, as in no LLM wall-of-text readme in the repo. Wish we could see that more often these days.
thank you! Haha I always make it a point these days do handwrite these things. I think everyone is tired of overly verbose perfectly written AI text.
Why does it have to be JS? With all this AI, why can't we ship native apps.
It's still pretty tricky to do this as a solo dev to keep feature parity in my experience (fun fact, even Claude's own Desktop app is made in Electron).
That being said, majority of the heavy LLM lifting happens natively with llama.cpp :)
I'm actually building something similar and thank you very much for sharing you gave me a few inspirations. I want mine to integrate seemlesly with an animated Ai companion (it's harder than it sounds haha). Anyways great stuff! Gave it a star.
Thank you. Yes please fork that's exactly why I put it out on MIT license :)
> Automatically finds the best open-weight model for your hardware on first install
Do you have examples of what hardware runs what models?
Looks great. One thing I am missing is the ability to change the installation directory for the models. For space reasons, I want them to be on a different partition.
Good call. Are you on PC/Mac/Linux?
Linux. I saw you support HOB_USER_DATA, which is great. But I want my config to remain in .config/Hob and only move the models (or whatever else needs lots of space) to another disk/directory.
Edit: Actually I think HOB_USER_DATA is enough for my use case.
Did you happen to try bionic?
https://lmstudio.ai/
This doesn't seem to be open source? They link a GitHub account, but I don't see the repo for this tool.
Correct, they’re closed source and you can’t opt out of analytics
Can we configure it to use local models but in LAN? Like Ollama or Strata.
So basically allow you to point the LLM to a hosted model on your local network if I understand correctly?