FYI I find that Whisper is far from the best transcription models out there as well. I did a bunch of testing and I found that Assembly AI's transcription models are significant better than Whisper and even ElevenLabs. I guess I now sound like an ad.
When a whisper-based agent thinks you’ve stopped talking, it will begin its own reply. If you’re younger, you’re more likely to get interrupted by a bot that’s prematurely decided that you’re done speaking.
I built this expecting the opposite. A lot of voice AI is being pointed at elderly callers right now, and the assumed risk is that the model mishears them, so I went to measure how bad it is.
It isn't bad. On 2,760 Common Voice clips, with age brackets matched on accent, gender and speaker so age is the only thing varying, Whisper large-v3 gets 4.67% WER on speakers in their seventies against 6.53% for twenty-somethings. Substitutions, deletions and insertions all fall with age. Deletions in particular don't rise, which is what you'd see if quiet speech were being dropped.
Since Whisper's decoder is a language model and could plausibly be repairing older speakers' word choices rather than hearing them better, I re-ran the same clips through wav2vec2, which is pure CTC with no decoder and no LM: 10.30% for the sixties against 14.23% for the twenties. Larger effect, same direction. So it's acoustic, not a decoder artifact.
Where age does hurt is turn-taking. Voice agents end your turn after a fixed stretch of silence, typically 500-800ms. At 700ms, 8.0% of twenties utterances contain an internal pause long enough to be misread as end-of-turn, versus 19.7% for the sixties. Older speakers take about twice as many pauses inside a single utterance. WER is structurally blind to this: the words that arrive are transcribed correctly while the speaker gets cut off.
A maintainer corrected me on that and he was right. Mark Backman at Daily/Pipecat pointed out that production stacks don't endpoint on a bare VAD threshold anymore. Pipecat's default is smart-turn, a semantic model that listens to the waveform. So I measured smart-turn v3 on the same clips: the gap halves to +5.9pp and stops excluding zero. The fixed-threshold result describes a real and common configuration, not every configuration, and the README says so above the fold.
Things that mattered methodologically, mostly because they nearly produced wrong answers:
- One Common Voice contributor holds 9,792 clips in a single shard, and about seven people account for half the 60+ audio. Uncapped, a "bracket" is a description of one person. Capped at 25 clips each; all intervals bootstrap over speakers, not clips.
- The corpus's younger contributors skew non-native, so age and accent are entangled. Brackets are matched on the (accent, gender) pair. I also spent a while believing accent explained the whole result, based on a 40-clip pilot; at full sample it moves WER by at most 0.44pp. The wrong claim is corrected in the README rather than deleted.
- My first VAD was manufacturing the headline. A relative-energy threshold called breathy trailing-off speech "silence", and breathiness correlates with age, so the detector's error was correlated with the variable under study. WebRTC VAD disagreed on 36% of the eighties clips. Both are now recorded.
- fp16 on MPS is verified against fp32/CPU rather than assumed, because a quantised Whisper KV cache can take large-v3 from 1.91% WER to 100%.
There's a fourth result in there that's really about a different industry. Several startups now sell daily phone check-ins for older adults claiming to detect cognitive decline from "voice biomarkers". Validating that needs gated clinical corpora, but the within-speaker noise floor doesn't. Pause features vary about 99% within the same speaker, frequently within one sitting. Getting a 10% shift in total pause time above that noise takes roughly 19 days of daily calls per reading, so a "six-week trend" is two or three noisy measurements. Speech rate needs under one call. Pause metrics are the most-cited biomarker and the least usable for per-person drift.
Limits, stated because they're real: Common Voice's older speakers are volunteers who chose to record themselves, so this is healthy aging, not clinical. It's read speech, not conversation. And the disordered-speech follow-up is blocked on corpus licensing — TORGO and UASpeech need signed agreements and their HuggingFace mirrors look like unlicensed redistribution, so I didn't use them. The one openly-licensed option turned out to be fixed 3-second podcast excerpts where 74% start mid-speech, which makes the pause metric meaningless on it. That failure is written up in METHOD.md instead of being quietly dropped.
Just a word of advice, it could go a long way to have a human write the README rather than an LLM.
Right now, it’s a little hard to follow what the actual findings are, why they are what they are, and why that’s important.
FYI I find that Whisper is far from the best transcription models out there as well. I did a bunch of testing and I found that Assembly AI's transcription models are significant better than Whisper and even ElevenLabs. I guess I now sound like an ad.
> And where a stack endpoints on a fixed silence threshold, those same speakers get talked over two to two and a half times as often
Is the reader expected to understand this sequence of words? Because I'm struggling a bit.
When a whisper-based agent thinks you’ve stopped talking, it will begin its own reply. If you’re younger, you’re more likely to get interrupted by a bot that’s prematurely decided that you’re done speaking.
So many billions of dollars spent on data centres, and they could have saved a lot of it by just employing you to do the writeups instead.
The README for this project is AI slop
Validating that needs gated clinical corpora
I built this expecting the opposite. A lot of voice AI is being pointed at elderly callers right now, and the assumed risk is that the model mishears them, so I went to measure how bad it is.
It isn't bad. On 2,760 Common Voice clips, with age brackets matched on accent, gender and speaker so age is the only thing varying, Whisper large-v3 gets 4.67% WER on speakers in their seventies against 6.53% for twenty-somethings. Substitutions, deletions and insertions all fall with age. Deletions in particular don't rise, which is what you'd see if quiet speech were being dropped.
Since Whisper's decoder is a language model and could plausibly be repairing older speakers' word choices rather than hearing them better, I re-ran the same clips through wav2vec2, which is pure CTC with no decoder and no LM: 10.30% for the sixties against 14.23% for the twenties. Larger effect, same direction. So it's acoustic, not a decoder artifact.
Where age does hurt is turn-taking. Voice agents end your turn after a fixed stretch of silence, typically 500-800ms. At 700ms, 8.0% of twenties utterances contain an internal pause long enough to be misread as end-of-turn, versus 19.7% for the sixties. Older speakers take about twice as many pauses inside a single utterance. WER is structurally blind to this: the words that arrive are transcribed correctly while the speaker gets cut off.
A maintainer corrected me on that and he was right. Mark Backman at Daily/Pipecat pointed out that production stacks don't endpoint on a bare VAD threshold anymore. Pipecat's default is smart-turn, a semantic model that listens to the waveform. So I measured smart-turn v3 on the same clips: the gap halves to +5.9pp and stops excluding zero. The fixed-threshold result describes a real and common configuration, not every configuration, and the README says so above the fold.
Things that mattered methodologically, mostly because they nearly produced wrong answers:
- One Common Voice contributor holds 9,792 clips in a single shard, and about seven people account for half the 60+ audio. Uncapped, a "bracket" is a description of one person. Capped at 25 clips each; all intervals bootstrap over speakers, not clips.
- The corpus's younger contributors skew non-native, so age and accent are entangled. Brackets are matched on the (accent, gender) pair. I also spent a while believing accent explained the whole result, based on a 40-clip pilot; at full sample it moves WER by at most 0.44pp. The wrong claim is corrected in the README rather than deleted.
- My first VAD was manufacturing the headline. A relative-energy threshold called breathy trailing-off speech "silence", and breathiness correlates with age, so the detector's error was correlated with the variable under study. WebRTC VAD disagreed on 36% of the eighties clips. Both are now recorded.
- fp16 on MPS is verified against fp32/CPU rather than assumed, because a quantised Whisper KV cache can take large-v3 from 1.91% WER to 100%.
There's a fourth result in there that's really about a different industry. Several startups now sell daily phone check-ins for older adults claiming to detect cognitive decline from "voice biomarkers". Validating that needs gated clinical corpora, but the within-speaker noise floor doesn't. Pause features vary about 99% within the same speaker, frequently within one sitting. Getting a 10% shift in total pause time above that noise takes roughly 19 days of daily calls per reading, so a "six-week trend" is two or three noisy measurements. Speech rate needs under one call. Pause metrics are the most-cited biomarker and the least usable for per-person drift.
Limits, stated because they're real: Common Voice's older speakers are volunteers who chose to record themselves, so this is healthy aging, not clinical. It's read speech, not conversation. And the disordered-speech follow-up is blocked on corpus licensing — TORGO and UASpeech need signed agreements and their HuggingFace mirrors look like unlicensed redistribution, so I didn't use them. The one openly-licensed option turned out to be fixed 3-second podcast excerpts where 74% start mid-speech, which makes the pause metric meaningless on it. That failure is written up in METHOD.md instead of being quietly dropped.
Everything runs locally, no API key, no spend.
[dead]