I mean, you have "AI" which means just about anything in marketing speak, "Agentic" is kind of becoming similar, hopefully they don't goof that one too badly, would be nice to know what you are trying to sell me. Used to be "Cloud" meant storage not just hosting (I guess it still does).
Then there's "Smart" in front of Car, Phone, TV, and so on... Meaning different things.
I do think "Open Weight" should be more commonly used. There's definitely communities that spring up that build the training infrastructure and inference infrastructure around open models on the other hand.
I took a look into local options for ASR and diarization some months ago, I missed that VibeVoice now has this feature.
My conclusions back then (which only came from a shallow research on the topic and 0 real experience mind you) was that Whisper + Pyannote was the "stable" approach.
Have the VibeVoice, Voxtral, Qwen or the Nemo solutions caught up in segmentation and speaker recognition?
Look at the "News" section in the readme - The original TTS model is gone from this repo (you can still find it other places), but the SST/ASR, long form TTS, and streaming TTS models are newer.
Note that this just covers the Speech-to-Text/Speech-Recognition aspect (a-la whisper), there's also models for long-form Text-To-Speech and steaming Text-To-Speech.
I'd be willing to bet it will be "Word of the Year" for 2026. Merriam-Webster had 'slop' for 2025, and 'polarization' for 2024. Is there a prediction market for this?
Local? No idea. Cloud? Eleven Labs, probably. But it's described as "cloning" not "training". Not sure what the distinction is or why it matters if the end result is you can to generate any TTS that sounds like you. There might very well be an important one, I just don't know it.
Seems quite heavy for a STT model, Parakeet and Whisper are much smaller and perform great for quick dictation and transcription of longer files. I guess that's due to additional accuracy and speaker diarisation?
The TTS example clip in the repo of 'spontaneous singing' is creepy as fuck
I think we should stop calling this type of models open source. They are indeed "open weight." The training code is proprietary and never revealed.
https://github.com/microsoft/VibeVoice/issues/102
Indeed. We now live in a world where freeware is named open source. We are very sorry, Stallman.
> we should stop calling this type of model open source. They are indeed "open weight”
This ship has sailed. It’s now in the same category as hacker/cracker and the pronunciation of GIF.
I think you mean GIF.
It's the same as GIS, you wouldn't say jizz now would you?
I absolutely do, every single time it comes up.
The developer of the format declared the pronunciation 30+ years ago. It has always been jif.
i am absolutely going to from now on
How do you pronounce giraffe?
How do you pronounce gift?
I take it that you haven’t met the Arcgees people…
And "hallucination" which should have been "delusion".
Way early on (spring 2023) people tried to stop it, but no luck.
I mean, you have "AI" which means just about anything in marketing speak, "Agentic" is kind of becoming similar, hopefully they don't goof that one too badly, would be nice to know what you are trying to sell me. Used to be "Cloud" meant storage not just hosting (I guess it still does).
Then there's "Smart" in front of Car, Phone, TV, and so on... Meaning different things.
I do think "Open Weight" should be more commonly used. There's definitely communities that spring up that build the training infrastructure and inference infrastructure around open models on the other hand.
Openwashing is the new greenwashing, which, coincidently, seems to have gone out of fashion a few hundred datacentres ago.
it was replaced with abundancewashing
This is not a new model. Also, it hallucinates a lot. Also, it's very heavy and slow in inference. It's also bad in multilingual.
Edit: I'm talking purely about speech to text (STT). Not sure about the other things this can do.
Yeah, I don't get why it is suddenly getting so much attention today, it is all over twitter too
well duh, they updated the news section
https://github.com/microsoft/VibeVoice/commit/e73d1e17c3754f...
which is microsoft for "we removed two dead links". AI innovation knows no limits!
I think this was all covered when they said it was released by Microsoft?
The nuance is lost on LLM agentic dominant partakers.
looks like this offers ASR support in GGUF https://github.com/CrispStrobe/CrispASR -- haven't tested
I the past month or so, I added 2 models to my app Whisper Memos (https://whispermemos.com):
- Cohere Transcribe (self hosted)
- Grok Speech To Text (they provide an API, only $0.10/hr!)
They are both excellent. I'm not sure about this one. Would you like to see it in a consumer speech to text app?
Does Cohere work with longer transcripts? Do you have to do some magic to merge recordings over 35 seconds long?
I've had good experiences with the Mistral Voxtral models (I've used the API, but some of the model-variants are open weight)
Have you tried qwen?
Any non-Musk alternatives that are comparable in quality and cost?
Voxtral competes on price ($0.003/min) and quality. Speechmatics has best in class accuracy but is a bit more expensive ($0.004/min)
Our default is still OpenAI Whisper. Grok is just a choice for users who might prefer it.
I took a look into local options for ASR and diarization some months ago, I missed that VibeVoice now has this feature.
My conclusions back then (which only came from a shallow research on the topic and 0 real experience mind you) was that Whisper + Pyannote was the "stable" approach.
Have the VibeVoice, Voxtral, Qwen or the Nemo solutions caught up in segmentation and speaker recognition?
Interesting to see "vibe" enshrined by the likes of Microsoft as an AI product word.
Especially when "vibe coded" can have a negative connotation meaning quickly put together without understanding.
I’m just surprised they put the name of the e-waste slop company in their product
Which makes it even more weird they get offended when people use Mircoslop. They are the ones leaning into the marketing
Isn't this project the one Microsoft published but then soon after pulled it for security/safety reasons? What has changed since then?
It’s confusing (at least for me) because the project covers a number of things including what you are mentioning.
Look at the "News" section in the readme - The original TTS model is gone from this repo (you can still find it other places), but the SST/ASR, long form TTS, and streaming TTS models are newer.
Great post last night from Simon: https://simonwillison.net/2026/Apr/27/vibevoice/
Note that this just covers the Speech-to-Text/Speech-Recognition aspect (a-la whisper), there's also models for long-form Text-To-Speech and steaming Text-To-Speech.
“VibeVoice can only handle up to an hour of audio”
Why?
Holy moly, a Microsoft AI product that isn't named Copilot!
Missed opportunity to call it Vopilot
So we've really just settled on Vibe as the verb for AI then?
I'd be willing to bet it will be "Word of the Year" for 2026. Merriam-Webster had 'slop' for 2025, and 'polarization' for 2024. Is there a prediction market for this?
it'll probably be something we're not even talking about yet - we still have 7 months in which to make the world even worse
Why use precise technical language when you can just vibe with your AI system?
You have selected Microsoft Sam as the computer's default voice.
My friends and I had fun in the computer lab with Microsoft Sam, inputting long strings of characters to create funny sound effects. Sususususususu.
What’s the current state of the art, for each of training locally and in the cloud, for learning my voice?
open weights i would say S2: https://github.com/rodrigomatta/s2.cpp
Locally maybe https://voicebox.sh/
Elevenlabs in the cloud.
Local? No idea. Cloud? Eleven Labs, probably. But it's described as "cloning" not "training". Not sure what the distinction is or why it matters if the end result is you can to generate any TTS that sounds like you. There might very well be an important one, I just don't know it.
Interesting story about this repo/product/author by cybersecurity researcher Kevin Beaumont: https://cyberplace.social/@GossiTheDog/116454846703138243
Maybe Microsoft’s real strength was never making the best model, it was knowing you don’t need to, as long as you own the platform everyone builds on.
For me its giving me very poor results
Seems quite heavy for a STT model, Parakeet and Whisper are much smaller and perform great for quick dictation and transcription of longer files. I guess that's due to additional accuracy and speaker diarisation?
The TTS example clip in the repo of 'spontaneous singing' is creepy as fuck