Semi-related: has the rate of published exploits picked up as if late, or is it simply the fact that there’s hype around ai as security tool (offense or defense) so it’s simply in the news more often?
Feels like there’s something new every other day - linux, windows, mobile, various commonplace tools used by everybody, the list goes on
There are reports from people who manage security bugs in OSS that there has been a big uptick in reports: initially low quality ones that were mostly bogus, but now many more legitimate ones as well.
A bit of both (it finds new things and news is hyped/blown up), and a third factor is that more people are trying to find things. The authors might have been able to do this already, like you still need to have a decent understanding to get useful work out of it and verify the results, but the shiny new toy factor makes people spend more hours on it that they'd have spent doing something else otherwise
I've reported a few very serious issues to vendors of widely used tools in recent weeks, and it's been even more difficult than usual to get them to be acknowledged - the teams that respond are reportedly swamped.
This is pure guesswork, I am not a security researcher, but my guess would be that AI is increasing the amount of low quality exploitable attack surface available, while simultaneously providing security researchers with an accelerant for their work. Which is to say, its great if you use it well and really bad if you use it poorly.
"This is notably fast given that this is the first time that an Android driver bug I reported was patched within 90 days of the vendor first learning about the vulnerability."
This makes me feel better about Google, but also makes me kind of frightened of the rest of Android. I wonder what Apple's response time is?
I've reported security bugs to Apple before. Was a couple years back but I remember it taking around 6 months to patch (there was a couple back and forth for me to get a more reliable POC). Maybe 2 months from when I submitted a POC with 100% reproducibility
This is a great bug report! I am not a kernel expert by any means even though I have read some about it... 10+ years ago. And I was able to follow along and see what was going on.
It does make me scared for what other dangers lurk since this was a really bad one and it was so little work to find.
Also of note: so many security issues lately have been done using AI. This report makes me think two things:
1. Expertise is still immensely valuable, the more niche, the more valuable.
2. There are lots of niches still where AI doesn't dominate...
hm. surprised there aren't idioms like copy_(to|from)_user for these kinds of kernel to userspace mappings for custom device nodes that ensure bounds are supplied...
There have been some V4L2 enhancements to support hardware video decoding pending a merge for a long time, they do seem to be in the mainline kernel now, I guess people didn't want to wait that long.
Hmmm... I'd like someone to double check my thinking here. I posted this exact prompt for gpt 5.5 xhigh:
```
does this look right to you? don't do any searches or check memory, just think through first principles
static int vpu_mmap(struct file fp, struct vm_area_struct vm) { unsigned long pfn; struct vpu_core core = container_of(fp->f_inode->i_cdev, struct vpu_core, cdev); vm_flags_set(vm, VM_IO | VM_DONTEXPAND | VM_DONTDUMP); / This is a CSRs mapping, use pgprot_device */ vm->vm_page_prot = pgprot_device(vm->vm_page_prot); pfn = core->paddr >> PAGE_SHIFT; return remap_pfn_range(vm, vm->vm_start, pfn, vm->vm_end-vm->vm_start, vm->vm_page_prot) ? -EAGAIN : 0; }
```
And it correctly identified the issue at hand, without web searches. I'd love to try something more comprehensive, e.g. shoving whole chunks of the codebase into the prompt instead of just the specific function, but it seems the latent ability to catch security exploits is there.
So then.... I wonder how this got out in the first place. I know I'm using a toy example but would love to learn more!
As an anecdote, I provided fragnesia.c and the subsequent proposed patch to fix the issue and while it was not able to discover an entirely new vulnerability, I think it was able to find 2 new ways of exploiting the same underlying bug.
This is quite impressive considering I’m just a dumbass with a Claude subscription.
It's the usual problem of having no consequences for the person who wrote catastrophic code like this and the company who released it. If the person who wrote this were to be imprisoned for the rest of their life, for instance, or if the company were to be fined $1 million per user put at risk (which would probably mean a $1-10 trillion fine for Google -enough to trigger bankruptcy), then things would be very different
I most definitely don't agree with him, and I find the idea absolutely repugnant. Devils advocate though, I would be much more careful with the code that I write if regulators passed some dogshit law like that.
Well, one scenario would be that everybody who writes code would do so for money.
Take my friend who is a property lawyer. The firm she works for buys her insurance, because it would be insane to operate without insurance, but the only available insurance is personal insurance, it insures a specific person to do property law. So, although her day job is helping that $100Bn farm equipment company buy a $10M new factory from a $100Bn construction firm, at the weekend she is covered by that same insurance when she represents her friend buying a $500k cottage. AIUI this is a completely normal arrangement.
If that was the situation for programming, the company is going to buy your $100M exploit insurance because they need a programmer, but it's personal insurance so you could work on your Game jam game using the same insurance, and it'd be crazy to just "Go commando" if you don't have employment and thus insurance, in case somehow your "Galaga but also Blue Prince and somehow a visual novel" Game jam entry causes a $10M damages payment.
I understand that, though I wouldn't stop. I'd just go much slower and radically change my methodology. Failures in other engineering domains come with massive legal consequences, and they have for a very long time. In mesopotamia if a house collapsed and killed someone inside, the builder was put to death. People still built houses in the hundreds of thousands.
It really just introduces a legal burden to prove competence and work in good-faith, and nets immense power to throw out ridiculous deadlines. Your managers are legally responsible too, and if they push beyond what's reasonable you have just cause to bring them to court in a way that you currently don't. To re-emphasize, I don't think this is a better world, but it's not unlivable.
Sure, but home builders today very rarely get put to death, and it takes a particular kind of intentional fuck-up to have a plumber, or a drywaller, or electrician placed in prison.
If I was personally liable for damages, and there was an insurance program or some sort - similar to how doctors & dentists practice - sure, I'd probably still write code, very carefully. But if there was a decent change of me spending the rest of my life in prison because something I wrote on a Friday at 4pm under some amount of stress? No thanks. I can re-train as a plumber, and stand knee-deep in shit all day.
> If the person who wrote this were to be imprisoned for the rest of their life [...] then things would be very different
Yes, they certainly would. You wouldn't have smartphones, for instance.
I can't tell if this is satirical or not. But there are so many takes like this recently (hold the website liable for user content, hold the corporate developer liable for zero days in a project they happened to touch) that would all result in the same outcome (no more product at all) that I can't help but wonder if there's some luddite psy-op trying desperately to bring us back to a pre-Internet era in any way they can...
I read about Pixel 9 Dolby Decoder bug, and it is based on integer overflow. It was a mistake to allow "+" operator to overflow, and this must be fixed in new languages like Rust, but it is not.
In Rust the decision about whether to pay for overflow checks or just wrap (because all modern hardware will just wrap if you don't check and that's cheaper) is a choice you can make when compiling software, by default you get checks except in release builds but you can choose checks everywhere, even in release builds or no checks even in debug.
By definition in Rust it's incorrect to overflow the non-overflowing integer types, and so if you intend say wrapping you should use the explicit wrapping operations such as wrapping_add or the Wrapping<T> types in which the default operators do wrap - but if you turn off checks then it's still safe to be wrong, just as if you'd call the wrapping operations by hand instead of using the non-wrapping operations.
That Dolby overflow code looks awkward enough that I can't imagine writing it in Rust even if the checking was off - but I wasn't there. However the reason it's on Project Zero is that it resulted in a bounds miss, and that Rust would have prevented anyway.
I've been using this as a touchstone for whether or not we are actually going to take security seriously for a long time.
We've moved slightly closer to this, but in a world where we're still arguing over memory safety being necessary we've probably still got a ways to go before we notice that addition silently overflowing is a top-10 security issue. It's the silent top-10 security issue, I guess.
It isn't because no ISA implements add like that, so there's always performance on the table if you check every time, and people would probably endlessly moan about how Rust is 20% slower than C on this add-heavy microbenchmark.
That said you can enable overflow checks in Rust's release mode. It's literally two lines:
[profile.release]
overflow-checks = true
I wonder if it would make sense for ISAs to have trapping versions of add and subtract. RISC-V's justification for not doing that is that it's only a couple more instructions to check afterwards. It would be interesting to see the performance difference of `overflow-check = true` on high performance RISC-V chips once they are available.
It does seem like "What if we offer checked integer arithmetic operations?" is a cheaper experiment than CHERI's "What if we mechanically reify extent based provenance"?"
Randomizing the kernel location is of marginal utility at best. There are so many info leaks that KASLR ends up being only a small speed bump on the way to exploitation.
Project Zero has to report bugs to Android through the front door, and deal with Android VRP severity classification? I always assumed they could just walk over to the Android office and advocate for their bugs, face to face.
Semi-related: has the rate of published exploits picked up as if late, or is it simply the fact that there’s hype around ai as security tool (offense or defense) so it’s simply in the news more often?
Feels like there’s something new every other day - linux, windows, mobile, various commonplace tools used by everybody, the list goes on
There are reports from people who manage security bugs in OSS that there has been a big uptick in reports: initially low quality ones that were mostly bogus, but now many more legitimate ones as well.
A bit of both (it finds new things and news is hyped/blown up), and a third factor is that more people are trying to find things. The authors might have been able to do this already, like you still need to have a decent understanding to get useful work out of it and verify the results, but the shiny new toy factor makes people spend more hours on it that they'd have spent doing something else otherwise
https://lwn.net/Articles/1065620/
I've reported a few very serious issues to vendors of widely used tools in recent weeks, and it's been even more difficult than usual to get them to be acknowledged - the teams that respond are reportedly swamped.
This is pure guesswork, I am not a security researcher, but my guess would be that AI is increasing the amount of low quality exploitable attack surface available, while simultaneously providing security researchers with an accelerant for their work. Which is to say, its great if you use it well and really bad if you use it poorly.
Not low quality if it works!
The low quality refers to the features with security holes. So no, it didn't work (in this hypothetical).
I think AI helped researchers navigate better in the codebase, not necessarily the AI is succeeding in exploiting.
"This is notably fast given that this is the first time that an Android driver bug I reported was patched within 90 days of the vendor first learning about the vulnerability."
This makes me feel better about Google, but also makes me kind of frightened of the rest of Android. I wonder what Apple's response time is?
I've reported security bugs to Apple before. Was a couple years back but I remember it taking around 6 months to patch (there was a couple back and forth for me to get a more reliable POC). Maybe 2 months from when I submitted a POC with 100% reproducibility
At least in the past there has been instances where Apple sat on security bugs for years until they were fixed, one example: https://jonbottarini.com/2021/12/09/dont-reply-a-clever-phis...
I've heard they cleaned up their program recently to respond much quicker nowadays
This is a great bug report! I am not a kernel expert by any means even though I have read some about it... 10+ years ago. And I was able to follow along and see what was going on.
It does make me scared for what other dangers lurk since this was a really bad one and it was so little work to find.
Also of note: so many security issues lately have been done using AI. This report makes me think two things:
1. Expertise is still immensely valuable, the more niche, the more valuable.
2. There are lots of niches still where AI doesn't dominate...
hm. surprised there aren't idioms like copy_(to|from)_user for these kinds of kernel to userspace mappings for custom device nodes that ensure bounds are supplied...
There have been some V4L2 enhancements to support hardware video decoding pending a merge for a long time, they do seem to be in the mainline kernel now, I guess people didn't want to wait that long.
Hmmm... I'd like someone to double check my thinking here. I posted this exact prompt for gpt 5.5 xhigh:
```
does this look right to you? don't do any searches or check memory, just think through first principles
static int vpu_mmap(struct file fp, struct vm_area_struct vm) { unsigned long pfn; struct vpu_core core = container_of(fp->f_inode->i_cdev, struct vpu_core, cdev); vm_flags_set(vm, VM_IO | VM_DONTEXPAND | VM_DONTDUMP); / This is a CSRs mapping, use pgprot_device */ vm->vm_page_prot = pgprot_device(vm->vm_page_prot); pfn = core->paddr >> PAGE_SHIFT; return remap_pfn_range(vm, vm->vm_start, pfn, vm->vm_end-vm->vm_start, vm->vm_page_prot) ? -EAGAIN : 0; }
```
And it correctly identified the issue at hand, without web searches. I'd love to try something more comprehensive, e.g. shoving whole chunks of the codebase into the prompt instead of just the specific function, but it seems the latent ability to catch security exploits is there.
So then.... I wonder how this got out in the first place. I know I'm using a toy example but would love to learn more!
As an anecdote, I provided fragnesia.c and the subsequent proposed patch to fix the issue and while it was not able to discover an entirely new vulnerability, I think it was able to find 2 new ways of exploiting the same underlying bug.
This is quite impressive considering I’m just a dumbass with a Claude subscription.
How do you know it didn't search the web?
It's the usual problem of having no consequences for the person who wrote catastrophic code like this and the company who released it. If the person who wrote this were to be imprisoned for the rest of their life, for instance, or if the company were to be fined $1 million per user put at risk (which would probably mean a $1-10 trillion fine for Google -enough to trigger bankruptcy), then things would be very different
If this rule were implemented, would you be walking free right now? Think it over.
I most definitely don't agree with him, and I find the idea absolutely repugnant. Devils advocate though, I would be much more careful with the code that I write if regulators passed some dogshit law like that.
I would stop writing code for money.
Well, one scenario would be that everybody who writes code would do so for money.
Take my friend who is a property lawyer. The firm she works for buys her insurance, because it would be insane to operate without insurance, but the only available insurance is personal insurance, it insures a specific person to do property law. So, although her day job is helping that $100Bn farm equipment company buy a $10M new factory from a $100Bn construction firm, at the weekend she is covered by that same insurance when she represents her friend buying a $500k cottage. AIUI this is a completely normal arrangement.
If that was the situation for programming, the company is going to buy your $100M exploit insurance because they need a programmer, but it's personal insurance so you could work on your Game jam game using the same insurance, and it'd be crazy to just "Go commando" if you don't have employment and thus insurance, in case somehow your "Galaga but also Blue Prince and somehow a visual novel" Game jam entry causes a $10M damages payment.
I understand that, though I wouldn't stop. I'd just go much slower and radically change my methodology. Failures in other engineering domains come with massive legal consequences, and they have for a very long time. In mesopotamia if a house collapsed and killed someone inside, the builder was put to death. People still built houses in the hundreds of thousands.
It really just introduces a legal burden to prove competence and work in good-faith, and nets immense power to throw out ridiculous deadlines. Your managers are legally responsible too, and if they push beyond what's reasonable you have just cause to bring them to court in a way that you currently don't. To re-emphasize, I don't think this is a better world, but it's not unlivable.
Sure, but home builders today very rarely get put to death, and it takes a particular kind of intentional fuck-up to have a plumber, or a drywaller, or electrician placed in prison.
If I was personally liable for damages, and there was an insurance program or some sort - similar to how doctors & dentists practice - sure, I'd probably still write code, very carefully. But if there was a decent change of me spending the rest of my life in prison because something I wrote on a Friday at 4pm under some amount of stress? No thanks. I can re-train as a plumber, and stand knee-deep in shit all day.
> If the person who wrote this were to be imprisoned for the rest of their life [...] then things would be very different
Yes, they certainly would. You wouldn't have smartphones, for instance.
I can't tell if this is satirical or not. But there are so many takes like this recently (hold the website liable for user content, hold the corporate developer liable for zero days in a project they happened to touch) that would all result in the same outcome (no more product at all) that I can't help but wonder if there's some luddite psy-op trying desperately to bring us back to a pre-Internet era in any way they can...
Yes...no one would write any code.
I read about Pixel 9 Dolby Decoder bug, and it is based on integer overflow. It was a mistake to allow "+" operator to overflow, and this must be fixed in new languages like Rust, but it is not.
In Rust the decision about whether to pay for overflow checks or just wrap (because all modern hardware will just wrap if you don't check and that's cheaper) is a choice you can make when compiling software, by default you get checks except in release builds but you can choose checks everywhere, even in release builds or no checks even in debug.
By definition in Rust it's incorrect to overflow the non-overflowing integer types, and so if you intend say wrapping you should use the explicit wrapping operations such as wrapping_add or the Wrapping<T> types in which the default operators do wrap - but if you turn off checks then it's still safe to be wrong, just as if you'd call the wrapping operations by hand instead of using the non-wrapping operations.
That Dolby overflow code looks awkward enough that I can't imagine writing it in Rust even if the checking was off - but I wasn't there. However the reason it's on Project Zero is that it resulted in a bounds miss, and that Rust would have prevented anyway.
__builtin_add_overflow Exists and it’s basically free on most CPUs out there.
I've been using this as a touchstone for whether or not we are actually going to take security seriously for a long time.
We've moved slightly closer to this, but in a world where we're still arguing over memory safety being necessary we've probably still got a ways to go before we notice that addition silently overflowing is a top-10 security issue. It's the silent top-10 security issue, I guess.
Isn't it often combined with poor bounds checks to be exploitable? It's not as if rust or VM based languages don't help a lot with this
It isn't because no ISA implements add like that, so there's always performance on the table if you check every time, and people would probably endlessly moan about how Rust is 20% slower than C on this add-heavy microbenchmark.
That said you can enable overflow checks in Rust's release mode. It's literally two lines:
I wonder if it would make sense for ISAs to have trapping versions of add and subtract. RISC-V's justification for not doing that is that it's only a couple more instructions to check afterwards. It would be interesting to see the performance difference of `overflow-check = true` on high performance RISC-V chips once they are available.It does seem like "What if we offer checked integer arithmetic operations?" is a cheaper experiment than CHERI's "What if we mechanically reify extent based provenance"?"
fascinating how GrapheneOS achieves high security level on the same hardware where Google failed to even randomize android's kernel location
Randomizing the kernel location is of marginal utility at best. There are so many info leaks that KASLR ends up being only a small speed bump on the way to exploitation.
Here's a cool project that inventories all your KASLR info leaks: https://github.com/bcoles/kasld
It's easy to be secure if you just remove features. There's obvious tension here.
KASLR isn't an effective mitigation against anything, and to me this is part of GrapheneOS's catalog of superficial but meaningless claims.
google has lost its focus with pixel phones
Project Zero has to report bugs to Android through the front door, and deal with Android VRP severity classification? I always assumed they could just walk over to the Android office and advocate for their bugs, face to face.