15 points | by saikatsg an hour ago
7 comments
This seems to be the demo:
https://chat.webllm.ai/
I am getting:
WebGPUNotAvailableError: WebGPU is not supported in your current environment, but it is necessary to run the WebLLM engine.
You can enable WebGPU support in Google Chrome by turning on hardware acceleration and activating the WebGPU flag. It Works.
Project is de facto dead, used it for many years and had to rip it out 6 months ago, don't waste your time.
What did you switch to?
We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime
And ONNX is what Transformers.js uses as well, at least for the moment.
Transformers.js supports a lot of models since v4
This seems to be the demo:
https://chat.webllm.ai/
I am getting:
On both, FireFox and Chromium on Linux.You can enable WebGPU support in Google Chrome by turning on hardware acceleration and activating the WebGPU flag. It Works.
Project is de facto dead, used it for many years and had to rip it out 6 months ago, don't waste your time.
What did you switch to?
We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime
And ONNX is what Transformers.js uses as well, at least for the moment.
Transformers.js supports a lot of models since v4