You may want to look into different models that are more accurate and maybe an AEC layer to remove background noise. Or at the very least a RNN de-noiser on the mic channel. Also, you may want to stream audio to the model instead of holding it all in memory and transcribe at the very end as that can potentially allow you to take the app much further than it is now.
I see Claude implemented a very crude upsampling/downsampling algorithm, which is what LLMs usually do when prompted to handle such a problem. But I would suggest restraining the model from implementing DSP processing on their own and instead use battle tested libraries. You can use rubato's FFT Resampler.
Audio processing is genuinely a hard engineering problem, LLMs usually don't get it right. If you decide to get deep into it, the knowledge you'll get is very rewarding.
I built FnScribe because I'm pretty privacy concious and wanted a wispr flow-like app that kept everything local and on device. Currently works for Mac (sillicon and intel).
It's dead simple. Hold the fn key, speak and release. I use a quantized Wisper small.en model for transcription. It inserts the text into the active application. There's also a hands-free model for longer dictation. Audio transcription is kept in memory. There's no account or transcription history. Clipboard contents are restored after it inserts it. GPLv3, Mac-only, English only..still in alpha. Hope you enjoy it! Would love some feedback.
https://voxtype.io/
https://tryvoiceink.com/
There's so many of these, at this point I've seen 10 clones make the front page each time as if there never existed local only options before.
Also whisper is pretty outdated vs parakeet
You may want to look into different models that are more accurate and maybe an AEC layer to remove background noise. Or at the very least a RNN de-noiser on the mic channel. Also, you may want to stream audio to the model instead of holding it all in memory and transcribe at the very end as that can potentially allow you to take the app much further than it is now.
I see Claude implemented a very crude upsampling/downsampling algorithm, which is what LLMs usually do when prompted to handle such a problem. But I would suggest restraining the model from implementing DSP processing on their own and instead use battle tested libraries. You can use rubato's FFT Resampler.
Audio processing is genuinely a hard engineering problem, LLMs usually don't get it right. If you decide to get deep into it, the knowledge you'll get is very rewarding.
I see that it changes the menu bar icon to indicate status. How is this communicated when windows are full screen and the menu bar isn’t visible?
I built FnScribe because I'm pretty privacy concious and wanted a wispr flow-like app that kept everything local and on device. Currently works for Mac (sillicon and intel).
It's dead simple. Hold the fn key, speak and release. I use a quantized Wisper small.en model for transcription. It inserts the text into the active application. There's also a hands-free model for longer dictation. Audio transcription is kept in memory. There's no account or transcription history. Clipboard contents are restored after it inserts it. GPLv3, Mac-only, English only..still in alpha. Hope you enjoy it! Would love some feedback.
nice one.