Good to know, I'm on Linux, switching our dev/stg/prod servers over to it partly because we had all this workaround mechanics in place so that "apt update" updating docker packages wouldn't restart services (we typically don't rotate machines out of the load for just an apt update). Podman + quadlets conversion was not terribly hard, and has eliminated this issue.
I don't really know of any distro that doesn't do that. All of Docker Inc. default installs and all of distros I know of don't automatically add you to the docker group. docker.com instructions has the infamous "linux post-install instructions" that explain and walk you though it.
The tragedy is of course that when security and usability collide, 80/20 rule will apply where 80% of people will pick usability over security. I have worked with many with the title >= "Senior Engineers" who saw that page, read the explanation, and still had no idea what the ramifications of their changes were. "Yeah sure it said any user in the docker group will be able to get root on the host, but aren't containers isolated?"
That’s the workflow feature I badly want: for it to create a side list of things like that. Currently it either accumulates slop or goes on side quests far too easily.
This might be as easy as a directive to populate a .md file.
> This might be as easy as a directive to populate a .md file.
It probably is. But do you really think anyone is gonna bother with the multiple daily (or hourly for green field projects) `+8,234/-3,734` PRs everyone is submitting?
The joke I was referring to is the common
// ksmith (3/23/1997): This is a temporary hack for now. Find a better way to do this asap.
This is one of the main reasons people like Podman. Docker has this "feature" but as far as I remember, it needed some obscure configuration. I guess they don't add it as default as it will break many current setups.
I feel like everyone pointing out "known Docker vulnerability" is missing the point: the presence of a security hole should not be seen as permission to exploit.
Another security hole would be storing your passwords in a plaintext file on the desktop. Stupid? Yes. But I still would not want my agent to assume permission to access email when it's being blocked by 2FA.
Even in "bypass permissions" mode I expect it to pause and clarify and not behave as a paperclip maximizer.
User namespaces significantly rise the risk of exploits and many setups disable them. One may argue that Docker should have used them when they were available, but that would break too many useful setups involving privileged containers.
This was of course dependent on yolo mode, but automatic approval has also been pulling stunts like this. A recent example is data that was purposely kept away from Codex in a folder far far away. When it found a single reference it just went for the data when having an issue. Lesson learned, keep essential data and Codex separated on different machines. Codex remote ssh actually helps here.
Fwiw separate machines for the agents is awesome in general anyway.
I have agent frontends running on a low power server where every session is in tmux. So i can just resume from my home pc and pickup where i left off without reestablishing context. I do have to manually feed it data it can access bit that’s also a feature. Also let’s me shutdown the home pc if it’s some long running task since the server is much more power efficient.
Or, learn your local OS' permission system, have it in a directory right next to your banking credentials (or something even more outrageous) and nothing could go wrong even if you tried to.
I got annoyed when codex by default running in needing confirmation mode, read my .ssh, because it was debugging some slow render (hosting service) command, the fact that CLI did not stop it or require my permission to not only read anything outside the project (and not accidentally), but this specific location, seems like a malice to me. I checked and on defaults it has no problems or preference for asking for any approval regarding reading outside the project dir.
And yes, of course you want to run it in a safe way, and if reading that revealed any secrets that would be on me, it didn't but I still think it's not cool at all.
You should not be using docker with LLMs. You should be using VMs, which have a much, much smaller attack surface than Docker, and significantly more reasonable defaults.
The "attack vector" people try to protect themselves is "agent edited wrong file", not "LLM blew 0day on escaping sandboxing", containers are more than enough for what stupid stuff agents sometimes try, no need to go for a full-blown VM. Even UNIX permissions would be enough, but I think that's lost knowledge at this point.
Obviously if you setup a bi-directional share/link between what you are trying to contain and your host, you're not quite containing it at all... Don't do that! :)
Using the least amount of security features is a huge amateur mistake.
Best practice is to use 2 redundant layers of security, such that if one fails, there is still another one.
Using just the minimum amount of security technically possible is almost by definition hubris.
An example would be that you never point a gun at someone you don't want to shoot, regardless if there's bullets in the gun. If someone tells you, "you don't need to control where you point the gun, you just need to keep the gun unloaded and you can point it in jest to whoever you want, you can even pull the trigger technically", you know you have a reckless fool, regardless of whether they are technically right.
> Using the least amount of security features is a huge amateur mistake.
Not understand your threat I'd say would be a even bigger amateur mistake, you're not trying to protect yourself against some forever 3rd party attacker here, you're trying to prevent a agent rewriting the wrong file on your disk, that's basically it.
Give it the least amount of permissions, don't bi-directionally sync stuff, pass things in, then take them out again, literally the agent couldn't and wouldn't try to break through 2 layers of security in order to get your banking details or whatever.
all unreliable tools are attackers. Even if you're using well-aligned LLMs like Opus, you should assume that any input you give it -- including all dependencies from npm, etc. -- are at risk of compromise, which could result in attempted exfiltration of data or system takeover. You can be absolutely sure that there are thousands of well-motivated hacker groups, both national and private, looking for ways in.
Because it effectively makes no difference to my security posture. My user account also has sudo access (it requests TouchID but I also wouldn't die on the hill if someone said they have no password sudo access), and realistically everything of value on this machine exists in my home directory. Being able to escalate to root really doesn't give an attacker very much that they don't already have if they've got access to my user account.
And containers were initially and primarily about convenience not security. They were a way to quickly launch a preconfigured environment to respond to demand or to eliminate the need to manualy configure dev and test environments and avoid the "works on my machine" phenomenon.
People will more often than not, take the path of least resistance. Even if you tell them it's dangerous they will not care. People run this stuff on their primary workstation, unconfined, with permissions disabled because they don't want be bothered with accepting permission requests. This is all well and good until it decides to drop your production database or delete your home directory. Most of them don't even learn their lesson after that even.
Becuase a lot of devs don't know this stuff. There's a reason security engineers (as in SWEs who specialize in securing specific attack surfaces) remain in hot demand.
I realize this is supposed to be a post about how scary the security vulnerabilities these agents will find are.
But personally I love when agents do things like this and appreciate the help. Last thing in the world I want is for them to nerf the models.
This has been a known Docker "feature" since the beginning, nothing new here. This pattern is used to configure host machines by some tools.
Every time I try to install Docker there's a warning that being in the "docker" group is equivalent to having root access.
You should probably know about this workaround by now.
I recently switched over to podman and it's been great!
Podman on Windows - never been able to fully get rid of it and it throws errors on boot after uninstall. Was a fan, am now not.
Good to know, I'm on Linux, switching our dev/stg/prod servers over to it partly because we had all this workaround mechanics in place so that "apt update" updating docker packages wouldn't restart services (we typically don't rotate machines out of the load for just an apt update). Podman + quadlets conversion was not terribly hard, and has eliminated this issue.
I think that's distro-specific. Some set it up with more secure defaults (unix socket with permissions), others less (TCP socket).
I don't really know of any distro that doesn't do that. All of Docker Inc. default installs and all of distros I know of don't automatically add you to the docker group. docker.com instructions has the infamous "linux post-install instructions" that explain and walk you though it.
The tragedy is of course that when security and usability collide, 80/20 rule will apply where 80% of people will pick usability over security. I have worked with many with the title >= "Senior Engineers" who saw that page, read the explanation, and still had no idea what the ramifications of their changes were. "Yeah sure it said any user in the docker group will be able to get root on the host, but aren't containers isolated?"
No, docker access means root. You can use "rootless" mode, in this case it means root in a user namespace (that is not the "host" user namespace).
It would be cooler if the llm said something like:
> I noticed the machine doesn't have copy-fail patched, here is a quick workaround for not having root access for now.
> // TODO: find a better way to do this in the future.
That’s the workflow feature I badly want: for it to create a side list of things like that. Currently it either accumulates slop or goes on side quests far too easily.
This might be as easy as a directive to populate a .md file.
> This might be as easy as a directive to populate a .md file.
It probably is. But do you really think anyone is gonna bother with the multiple daily (or hourly for green field projects) `+8,234/-3,734` PRs everyone is submitting?
The joke I was referring to is the common
This is one of the main reasons people like Podman. Docker has this "feature" but as far as I remember, it needed some obscure configuration. I guess they don't add it as default as it will break many current setups.
That and podman lets you configure away from docker.io.
I feel like everyone pointing out "known Docker vulnerability" is missing the point: the presence of a security hole should not be seen as permission to exploit.
Another security hole would be storing your passwords in a plaintext file on the desktop. Stupid? Yes. But I still would not want my agent to assume permission to access email when it's being blocked by 2FA.
Even in "bypass permissions" mode I expect it to pause and clarify and not behave as a paperclip maximizer.
This is why you need either a rootless container setup or user namespaces to remap the container user to irrelevant host users. https://docs.docker.com/engine/security/userns-remap/
Weak that this isn't the default.
User namespaces significantly rise the risk of exploits and many setups disable them. One may argue that Docker should have used them when they were available, but that would break too many useful setups involving privileged containers.
Run coding agents in a docker container with limited permissions. FWIW, I run it with
Or put it in a microvm using eg smolmachines.
I've never used smolmachines but I'm curious; why this over a container?
Containers are not security boundaries. Vulnerabilities in containers are much more common than in VMs.
This was of course dependent on yolo mode, but automatic approval has also been pulling stunts like this. A recent example is data that was purposely kept away from Codex in a folder far far away. When it found a single reference it just went for the data when having an issue. Lesson learned, keep essential data and Codex separated on different machines. Codex remote ssh actually helps here.
What in heaven's name is a "folder far far away"?
(It sounds like you put it on an SSD on an extension cord and moved it to the kitchen or something.)
Fwiw separate machines for the agents is awesome in general anyway.
I have agent frontends running on a low power server where every session is in tmux. So i can just resume from my home pc and pickup where i left off without reestablishing context. I do have to manually feed it data it can access bit that’s also a feature. Also let’s me shutdown the home pc if it’s some long running task since the server is much more power efficient.
Or, learn your local OS' permission system, have it in a directory right next to your banking credentials (or something even more outrageous) and nothing could go wrong even if you tried to.
This very thread was an example where it unintentionally got root access though.
Because of how Docker works, not because of how Unix permissions work.
sudo can work non-interactively via settings in sudoers and sudoers.d . I am not sure about run0, but I would bet it has something similar.
Using docker for such a task seems to me overly over-engineered. Or maybe I need more context there.
I got annoyed when codex by default running in needing confirmation mode, read my .ssh, because it was debugging some slow render (hosting service) command, the fact that CLI did not stop it or require my permission to not only read anything outside the project (and not accidentally), but this specific location, seems like a malice to me. I checked and on defaults it has no problems or preference for asking for any approval regarding reading outside the project dir.
And yes, of course you want to run it in a safe way, and if reading that revealed any secrets that would be on me, it didn't but I still think it's not cool at all.
clever girl...
Hold onto your butts.
You should not be using docker with LLMs. You should be using VMs, which have a much, much smaller attack surface than Docker, and significantly more reasonable defaults.
The "attack vector" people try to protect themselves is "agent edited wrong file", not "LLM blew 0day on escaping sandboxing", containers are more than enough for what stupid stuff agents sometimes try, no need to go for a full-blown VM. Even UNIX permissions would be enough, but I think that's lost knowledge at this point.
Not if the host's version of .git is accessible inside the container via a bind mount.
Obviously if you setup a bi-directional share/link between what you are trying to contain and your host, you're not quite containing it at all... Don't do that! :)
Using the least amount of security features is a huge amateur mistake.
Best practice is to use 2 redundant layers of security, such that if one fails, there is still another one.
Using just the minimum amount of security technically possible is almost by definition hubris.
An example would be that you never point a gun at someone you don't want to shoot, regardless if there's bullets in the gun. If someone tells you, "you don't need to control where you point the gun, you just need to keep the gun unloaded and you can point it in jest to whoever you want, you can even pull the trigger technically", you know you have a reckless fool, regardless of whether they are technically right.
> Using the least amount of security features is a huge amateur mistake.
Not understand your threat I'd say would be a even bigger amateur mistake, you're not trying to protect yourself against some forever 3rd party attacker here, you're trying to prevent a agent rewriting the wrong file on your disk, that's basically it.
Give it the least amount of permissions, don't bi-directionally sync stuff, pass things in, then take them out again, literally the agent couldn't and wouldn't try to break through 2 layers of security in order to get your banking details or whatever.
This is true but it's not really a security scenario. The LLM isn't an attacker it's just an unreliable tool.
all unreliable tools are attackers. Even if you're using well-aligned LLMs like Opus, you should assume that any input you give it -- including all dependencies from npm, etc. -- are at risk of compromise, which could result in attempted exfiltration of data or system takeover. You can be absolutely sure that there are thousands of well-motivated hacker groups, both national and private, looking for ways in.
This is a classic attack path that was already captured by plenty of EDRs/XDRs/CWPPs a couple years ago.
Right, why is their login user in the docker group? Mine sure isn’t.
Because it effectively makes no difference to my security posture. My user account also has sudo access (it requests TouchID but I also wouldn't die on the hill if someone said they have no password sudo access), and realistically everything of value on this machine exists in my home directory. Being able to escalate to root really doesn't give an attacker very much that they don't already have if they've got access to my user account.
Rather, why do people still run agents as their own user. IMO, agent sessions should at least be containerised with just necessary code mounted.
Safety and simplicity are concepts that often won't get along very well with eachother.
And containers were initially and primarily about convenience not security. They were a way to quickly launch a preconfigured environment to respond to demand or to eliminate the need to manualy configure dev and test environments and avoid the "works on my machine" phenomenon.
People will more often than not, take the path of least resistance. Even if you tell them it's dangerous they will not care. People run this stuff on their primary workstation, unconfined, with permissions disabled because they don't want be bothered with accepting permission requests. This is all well and good until it decides to drop your production database or delete your home directory. Most of them don't even learn their lesson after that even.
Convenience. Want to run `docker run ...` without password, want IDEs and agents to be able to run containers...
Use podman then, or rootless docker if you can make it work
For most CRUD apps running in docker its enough to just tell the "agent" to use podman.
Becuase a lot of devs don't know this stuff. There's a reason security engineers (as in SWEs who specialize in securing specific attack surfaces) remain in hot demand.
this is the new GTD