Important to clarify that this was not the Grok agent deciding to read the files.
I don't think the LLM had anything to do with this decision at all. It looks like the Grok tool starts a session by deterministically kicking off a full upload of the user's current repository (and maybe their directory if not version tracked? Not clear if this user had previously run "git init" in their home directory) to Grok's servers.
One possible "innocent"
explanation could be that xAI then run vector embedding on every file to help later provide the right context. I don't think thats a worthwhile tradeoff here, especially since other popular coding agents get by just using grep/ripgrep run locally.
Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.
So many of the replies are saying that they should've restricted access using .md files and whatnot. Is really any guarantee that they even follow those? It seems like even if you ask pretty please don't touch those files, there's a chance they will. So many people have just willingly installed spyware on their computers and big tech calls this the next big thing.
I will keep banging this drum until people listen:
Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.
The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.
It's wild that we've known for decades to use ACLs to make sure people don't have access to files we don't want them to have access to, but somehow a computer pretending to be a person doesn't get that same treatment.
ACLs, nothing, we've known about in-band signalling since forever and still this whole segment of the industry seems to either not know about it, or forgets about it at a cadence so regular it may as well not know about it.
System-level ACLs; mandatory or discretionary access control; secure-by-default application and network configurations are all for naught if you take an LLM, run it with all the privileges you'd have an accountable, judgemental operator, and then tell it to act based on arbitrary untrusted input which might include prompt injection attacks, something which cannot generally be sanitized.
Well-defined, well-enforced security policies can mitigate disasters, but many in the wild right now just don't account for this kind of threat model.
It's just purposeful blindness - I worked for a company building out tooling that insisted markdown based security was good enough and showed it off for anyone at the company to attack because they were so sure.
It took me less than 5 minutes to completely disable... nobody cared, they just kept going - check the box and move on.
Software "engineering" in particular has always been more than 50% cargo culting. Good engineering practices never matter when the alternative is just going through the motions of whatever rituals are in vogue.
We laugh, and rightly so, at the time sharing systems of the 70s and 80s that didn't use passwords. I bet allowing a whole other virtual person to run outside a VM, with access to an actual file system instead of a version-controlled branch, will be seen as a worst practice in ten years.
ACLs aren't even the issue here, first you need an entity separate from "the current user" to grant that access to.
That's what jails, sandboxes or capability-based systems bring to the table. But you have to use them, most of those AI tools and their IDE integrations don't.
Once you have those you can think about which access to grant that entity.
What do you find hard about it? Is it a conceptual thing or the tools themselves? I'm very far from a sysadmin, but it's just such a ground level using-linux type thing in my brain, I don't think I ever thought I had a choice about learning it!
I think ACL is only a part of the solution. If he runs the agent with his account, the ACL would not really help. But I do admit, I might have an outdated understanding of ACLs.
One cannot pound that particular drum enough. "Guardrails" in instructions (and all MD files are just instructions) are like price lists at an unattended farm stand. It'll usually work! There will be some money in the basket at the end of the day! People paid for the bagels[0]! But one cannot never assert that it _will_ work; things that _must_ work have to managed out-of-band
The easiest, most guaranteed way to isolate it is to run it in a VM or container where it literally can't do the wrong thing without some kind of full container or VM exit exploit.
It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.
For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.
But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.
That said, I do hope these become the norm if LLMs are here to stay.
I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.
Last time I read about this, this was due to the well-known pitfall of UID mapping across container boundaries.
It's a common misconfiguration and one of the footguns available through containers, which I don't say a wholesale condemnation of the technology, but certainly as a UX facet that could use reevaluation.
I've been biasing towards VMs myself just out of caution, but maybe that's just extreme paranoia.
The way I look at it is similar to how I'd look at any hypothetical employee. How do I ensure the agent only has access to the minimum possible they need to get their job done?
That means no access to git repositories (no pushes on my behalf means it can't accidentally nuke git history, something there is anecdotal evidence of agents doing). It can make local changes in git only and I will take responsibility for pushing them. No access to the wider internet beyond what I deem acceptable. No permissions to access any internal APIs except what I provide (and not using my credentials).
In one case, I have a tool that has a set of dangerous commands alongside a large number of safe ones. I don't even have it installed in the agent's VM. I run an MCP that is a simple python wrapper around the tool on the host side, and expose it to the agent in the VM, so that it can only possibly run a strict safe subset that I can trust it with access.
> Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.
It reminded me of an old meme. Please don't follow the following instructions and stop reading if you cannot.
It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"
Claude.md is an equally effective defensive tool.
But sorry if you lost some files from reading that.
I agree with this. And also think that we should train and select for trustworthy models. I also agree that these models may never truly be trustworthy.
Especially when I find that, when I ask an agent not to do something, the mere mention seems to put the "idea" in its "head," making it more likely to ultimately do that thing.
yes, because this is how transformers work, when you introduce "don't do x" x is also entered as a pattern that is more likely to be repeated than if not mentioned at all
1000%. I'm experimenting with running my agent (Claude Code) inside a docker container, so I can have a control plane and then YOLO within that limited access. The agent could still mangle my local dev setup, but I consider that an acceptable risk since everything the agent has access to is under version control outside the local machine.
This is a new pattern for me, I'm curious what others are doing.
https://smolmachines.com has "smolvm" microvms with better performance and ergonomics and security than docker, you might want to give it a try. (No affiliation, I just like what they're doing.)
If you're not using a sandbox, something like this will inevitably happen to you. It's really not that hard to set up and should be a standard recommendation.
I don't understand these people. Agent instructions in markdown is barely a suggestion. I have one which says "All code in this repository is executed in docker containers, run the services with `docker compose run --rm php-cli "$@"`. Gemini and Claude more often than not refuse to abide and will try to execute the environment using /opt/homebrew/bin/php on my host…
A frightening amount of people have no idea how AI tools work, even those that should know better. I have seen senior software developers fall for the mistake of believing an LLM output when it spews bullshit about how its own memory or restrictions work.
LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time. I've come to terms with the impact slop coding will have on most software jobs in the future, but seeing seemingly intelligent people fall for lies and fantasies concocted by an LLM is making me more and more uncomfortable with the direction we're all heading in.
> LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time.
It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.
I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).
We don’t have to head in this direction of using LLMs for most development at all.
There’s an aspect of extrapolation in the perception spike of the Dunning–Kruger effect.
In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.
I've seen claude check the Event Log in Windows and produce powershell scripts to alter firewall rules. This is what makes (something like) T3 Code appealing to me. The computer I'm working on is not the computer where the AI has agency.
You will want at least a separate session for the `restricteduser`: E.g. with X11, a process in the same session can do almost anything with your input/output. And most Linux distributions make it really hard to disable external device access for individual users...
> And most Linux distributions make it really hard to disable external device access for individual users...
For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.
Does this have the affordances to work well with a pretty standard agentic coding setup (e.g. claude code/codex) on macos?
I don't really have a good mental model of how ports/files/etc get exposed/permissioned across users. Containers and sandboxing seem much more common than agent-user accounts. Networking seems a little more complicated in the general case.
I roughly went the route of running apple's container-cli (separate vm/kernel per instance I believe) and mount the relevant home directory/projects and bind ports. Seatbelt/linux sandboxing seems simpler sometimes, but has its complications too.
It's limited. For instance, you're going to have to explore different kinds of sandboxing if you want to prevent network access (except for certain cases, like to allow it to talk to the AI provider). But generally, a filesystem will have metadata for files like which groups are allowed to read/write, and the operating system will enforce those rules for processes that run as that user, and for any processes which that user starts. I feel like that gets you 90% of where you're likely to want to go with it.
I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.
Users do not want the agent to be restricted. They want the agent to read the user's mind and resolving the user's cognitive dissonance and contradictory preferences.
Is the implication that human thinking is inherently inferior to the perfect, all knowing AI?
Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?
And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."
And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.
Why would you give a non-deterministic text generator a user account? It’s not a person, it’s barely a tool at the software level. Restrict at the right level, in this case, a complete sandbox around it given its propensity to hallucinate and be steered by anybody.
The standard here is not enough when the agent can escalate by finding 0 days, for example. It’s like giving a black hat a limited account. Sure it might restrict him, but not giving him an account at all is way better
Malware has been bundling rootkits for decades, so potentially all of them. Sometimes the attacks succeed, so these are defenses that need continual hardening, but there's no sense in setting up an entirely separate line of defense just because this time the threat has AI in it.
Yes but the encompassing service has the account, not the agent. You can completely segregate it from the rest of the system, like you’d treat an intruder
But none of these services have the kind of latent capabilities an agent has, you know a deterministic service’s constraint are its code+bugs. There are no such constraints in an agent…
...this is a completely normal thing to do in linux, it's the most basic form of access control. There's like a dozen non-human accounts in a clean install before adding your own like this, and a lot of software adds their own. Edit: I have 54 entries on my personal laptop, just one of which is actually me.
This model is already not perfect, and not at all built for agents. The only way to secure an agent is an air-gap with the execution layer. Treat it like text, and the problem never arises until you “interpret” the agent output in a more limited environment than the OS
Even though it’s completely normal to us and in widespread use, GP is a reminder that conceptually it’s a broken model. Security should be capability-based not user-based. And to anyone who didn’t grow up on a desktop this model makes complete sense since it’s what your phone uses.
He said it’s less than a software, so saying software does this too isn’t really a strong counter argument. In case, I don’t think you are really in disagreement. Restricted accounts are necessary is your point, but I think op is saying they aren’t sufficient.
Is it less secure if I simply have nothing to do with language models whatsoever? I take that to be the parents point, but I know you are correct and yes, user restrictions should be a necessary part of the way people who use LLMs setup their system to use them in a safe manner.
You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.
But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.
Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.
That's the whole reason I refuse to install Google Drive or Dropbox's desktop applications. I only use the web interface so I know exactly what gets uploaded and when. I assume that anything running on my computer gets access to everything.
Sounds like a very wise decision to me.
I found found out on my phone that the google photos application uploaded everything in my gallery to their servers without asking me, regardless that I had explicitly disabled all backup to my google accounts on the settings of the phone. I only figured it out when they sent me emails saying that my storage was full.
Ooooh, don't get me started how mad it makes me! I am paranoid (or just lucky) enough that I didn't yet had it happen to me, but my wife's phone had done it four times in the last year. Each time I check and double check that all "backups" are turned off, and each time it somehow pops back.
So, Google "backs up" a 128Gb worth of photos on the phone onto 15Gb free storage combined with Gmail and who knows what, completely clogs it (as if it couldn't be predicted) and then has audacity to suggest paying for "extra storage". There is no way in online UI to just delete the whole "backup". And the cherry on top: when you finally get to delete some there is a fine-print - "the selected photos will be deleted from all synced devices". Well, I guess I must be thankful that they at least show this warning. This is what passes as "backup" in Google's parlance these days.
I would go further and not upload anything that isn't encrypted to cloud storage services. It is extremely likely that those "services" inspect your files.
> Mark, from San Francisco, had noticed swelling in his son’s groin and used his phone to photograph the problem to get an emergency appointment in February last year. He shared the pictures with a nurse so that a doctor could review them.
> However, Google’s artificial intelligence system used to detect child abuse flagged the image to the police and Mark, a software engineer who asked to be identified by only his first name, was investigated and lost access to his Google accounts. He was exonerated by the police in San Francisco but his Google account has not been reinstated.
If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
> If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
Even if there is a misunderstanding who is really uploading the directory, the TUI/CLI itself by actual code, or if the model decided to do so in the session, if you apply the recommendations from the replies to parent, and it no longer matter who did it, neither the software nor the model will be able to upload all your ssh keys.
No I disagree. A harness reading a file is a tool call and it happens locally, which means that I can control it. I can configure that I need to permit any file reads and now I _should_ have control of what is sent. The difference between that and silently uploading my entire working directory in the background is miles apart IMO.
I understand that one should think carefully about how they work with a non-deterministic tool, but this if different completely. This is xAI just choosing to upload and store everyone’s directories - with full git history.
What happened here is not related to agentic behavior or instructions in .md files. It's a binary a user runs, it scoops up their files and sends them to a third-party.
And the user even paid $99/month or more for having their data leaked.
I guess the downside of the lower barrier to entry to use these tools is the lack of basic understanding of exactly this sort of concept.
This sort of thing is why I'm hopeful I'll continue to have employment going forward. Some expertise is hard won and there's just no replacing learning through experience.
I think you're right in principle but I just hope I can hold out long enough for my experience to become appreciated and whose corresponding hourly rate isn't something which is suddenly being scoffed at (i.e. markets can remain irrational longer than I can remain solvent).
Claude definitely do not respect all my rules, it often ventures into other folders, most often other projects on the same machine that was greenlit before but not from the current projects' side. One other anomaly I had in the last month: I have two linux users on my laptop, one for work, one for personal. On my work account, it asked me if I wanted to continue with project x, which is in my personal account and not present at all in my home directory with work stuff. So somehow the memory system is keeping context where it shouldn't. Interesting I used this learning to improve the agent framework/harness I have running at work: it now creates a new linux user for each agent which have much stricter rules on how it can read/write to the system, and it is also no longer running as root. Much safer now. Still don't trust it 100% though.
Only guarantee that you can get is the sandbox in which it operates. The model itself is a slot machine and can result in anything, and if its sandbox is nonexistent... here's one possibility.
Yeah, I absolutely understand the allure of agentic AI, but I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level. Until we can get something set up that gives strict schema-only access I'm going to copy and paste definitions for context. Yes that sucks, but it's my responsibility to protect the system just as much as it is to develop scripts and queries for it.
> ... I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level.
Of course not.
To me it's on a server, in a VM. And they're not seeing the real data/databases from the actual projects: they're seeing fake infos used only while in the dev environment. There's no way I'm dumping, even for tests, the real or part of the real DB somewhere an AI can see it.
To find bugs (for example), AIs are useful but honestly for code generated by LLMs, I'm thinking about going back to the early copy/paste from the ChatGPT days: because I see so many horrors in the code output by the latest SOTA LLMs that every single line of code they spew has to be checked by someone who does know better.
It's not just an issue of protecting confidential data / preventing spying: we're all discovering that we've got serious sloppy-pasta code problems now.
Sometimes you can't even rely on harness not allowing ai to access certain files. "Oh, user doesn't want me to use `read_file` on .env? Well how about I run `cat .env` then?"
That's scary. Really should run it under different user with carefully assigned permissions.
You can even go a step further and run the container in a VM, such as with Docker Sandbox or the krun runtime in Podman.
There's also smolvm which is a nice minimal microvm manager based on libkrun: https://github.com/smol-machines/smolvm. I vibe coded a little shell utility for building and running OCI images for the Pi harness using it (easy enough to do manually, but the automation just makes it a couple quick commands rather than digging through documentation): https://github.com/neuroblaze/smol-pi
I have a similar setup using containerd/nerdctl and Kata Containers. Each OpenCode instance runs in its own little VM with mounted folders for context.
Yup, good call, I'll have to check those out. Not that urgent to me as I also happen to use colima for it's docker daemon interface. And, colima uses a full VM to host the containers, and you can further lock down the config to what is even allowed to vol mount so there's even another fs access restriction layer in play.
I almost exclusively dev in containers as is, cant really imagine letting some AI model run free on bare metal no matter what claims of guardrails it might have.
> Is really any guarantee that they even follow those?
No, there isn't. I just don't understand how naive (or imbecile) people are. The most valuable thing for these companies is people's data used for training, so giving unrestricted access to a tool from them and believing they will never take advantage of it to gobble up whatever they want from your computer, just because they told you they'll never do that, swearsies, is naive, or incredibly stupid.
Insulate yourself, or better yet, go local whenever possible, and there isn't much you can't do local if you have enough patience.
Is that built in protection really a filter, on code level, that sits between the LLM session and the shell or is it just some pleading in the bootstrap prompt? "Pretty please don't do xyz this is important!!!11"?
The latter can seem to be as good as the former for any amount of time. No outside observation can really prove reliability, only the negative result ("it does occasionally break the rules we expect") would be proof. So it's difficult to trust any claims that it's the former.
And even if it does have some of the former, chances are that the protection you experience is only partially provided on code level, while an unknown amount is still just bootstrap prompting that just works until does not.
"IMPORTANT: Before entering the leopard pen, don't forget to put on the leopard safety jacket that reads 'UNDER NOT CIRCUMSTANCES SHOULD YOU EAT MY FACE'"
Yeah that advice is smoking crack. I have no idea what people are thinking these days? We seem to have lost any sensible security understanding recently.
You can't gaslight something into not doing something bad. There has to be a hard security control that prevents it doing something bad. And if you don't know what it's capable of because it's non-deterministic then you have to start with a default block everything. This should have never been possible with any sensible design.
On my first point again, ethics and engineering both went out of the window when fast and shiny came along. This is disgraceful.
> So many of the replies are saying that they should've restricted access using .md files and whatnot.
What? No, but the random 3rd party software you run on your computer, must be limited by you in some way, haven't we learned this even after the AUR, npm and LLM shenanigans we've dealt with for decades at this point?
No, don't ask the model "Please don't go outside this directory", you limit the runtime (via VMs, containers, unix permissions, whatever) so it only has access to what it should, not more.
Same goes for any software, not just agents or chat clients or whatever. Any 3rd party software you don't want to have access to your entire computer, you need to run in this way.
Though I'm in the camp "people should really know to sandbox by now and be careful", I'd say we should also be mindful of how far from everyone has deep knowledge of the systems and tools they use. This behaviour of a tool is just malicious. You have to take into account the human factor, of how people likely end up using a system. And in this case, the consequences of exfiltrating so many secrets this way are really quite unacceptable.
This is a fight I deal with every day. We have folks in the technology group at work who use AI to write code and do so without issue. But now folks in supply chain or in the executive suite are using it to generate web pages that they want published or apps they want on the internet, and while Claude can generate an HTML file how that gets published, how authentication works, etc is just glossed completely over, and generates a ton of work for the IT team to build up around this stuff as it comes in
macOS largely _does_ bake this into the OS, and it is annoying. They also provide a way to turn it off for specific applications (including, for example, Terminal.app).
I've found that I can usually write apps that respect it. MacOS is a free-love hippie, compared to iOS. In many cases, we have no choice.
It's annoying, and often rather infuriating, but I understand that one of the motivations for people buying Apple stuff, is for that very reason, so I'm really sawing off the branch that I'm sitting on, by trying to work around it.
Given the long history of even the most egregious data breaches with millions of affected people never having the slightest consequences, what level of care are you expecting here?
Not to mention the very wide push to "Use AI NOW, for EVERYTHING!" in marketing ans many companies, with hardly any though given to safety or where does all the data end up.
Not gonna argue about the utility of AI, but isn't the statement "AI is not here to help people" completely meaningless? AI itself is not "here" for anything; the problem is big tech doing big tech shit as always, not the current technology they're doing it with.
> It has to do with Musk's ethical and engineering standards, or the lack thereof.
How? It's the user that added their home directory as trusted. Seems like that has nothing to do with Musk. Or did you just want to get a bash in, regardless of the facts?
AI is created by big tech stealing other people's data. Yes, this has everything to do with AI - stealing data is a foundational feature of the technology.
Well let's be fair here - you either go belly up with diminishing funding or get bailed out (maybe by the government, maybe by private funds that themselves expect to be bailed out by the government).
There currently is no path towards profitability, it's just a question of whether you can grab funding before it dries up.
True, but it isn't here to not help people, either.
It's a spanner. Who wields the spanner, makes all the difference.
We've spent the last couple of decades, cultivating a huge crop of ultimate scumbag billionaires, with comically exaggerated sociopathy, and that has filtered down to almost every level of society. They are treated as gods, these days (they certainly think of themselves that way).
It still shocks me (but really shouldn't), on a daily basis, to encounter regular folks, interacting in stores and restaurants, or driving on roads, that mirror the values systems exemplified by our billionaires. Our politicians act that way, and one of their biggest selling points, is normalizing sociopathy (not just the US, either).
Yes, I mean it in the sense that the spanner itself and who the spanner funnels money to is so bad that anyone using it has automatically accepted themselves being that bad. Unlike most tools (which the tool analogy leans on heavily), it's very much not an impartial tool, there is no such thing as "using it responsibly" when you're paying Musk.
Nah. It was introduced to push tech, but if you're a billionaire, then it was introduced to make you richer. They don't especially care, whether or not that involves replacing people.
Replacing people is what C-suiters think of it (and many of them aren't billionaires).
The first clue should have been when all the Silicon Valley CEO’s lined up to kiss the ring after the second Trump victory. Remember it wasn’t that long before that “woke” tech companies were derided and accused by the same factions they suddenly found themselves in good standing with. There were entire pushes regarding section 230, etc. to go against social media companies, constant complaints about “Facebook” jails and shadow banning. Now they’re all buddy buddy.
Anyway, ghouls like Thiel are now a well known name among populist left and right as an enemy, so maybe some good may come from this.
(an aside but the majority of the US population didn't elect Trump. He is in office because of the electoral college. Might seem like a distinction without a difference but I think it matters when we're implying personal culpability)
The real solution to these kind of problems is sandboxing. I use podman through a bash script to launch a container whenever I want an agent to work on one of my repos. When done I just generate git patches and port back everything generated.
In this way I'm not afraid of letting the agents totally lose on my computer.
It brings some extra sandboxing features compared to just a docker container.
For instance it also maps the secrets that the agent harness has access to. And has an allow list for outgoing http connections. And there's a way for agents to request extra access. And once approved by the user from an external cli the policy is updated and hot reloaded.
It's pretty recent. So time will tell how robust it is / will become.
What its longevity will be.
After all; in a time of LLMs one off experiments are cheap. Longtime nurturing not so much.
Instead of generating patches, this exposes the agent's checkout as a git remote though.
Most similar tools (and I believe your tooling as well?) bind-mount the repository checkout from the host into the container. This was always a source of user and permission errors for me, since you have to align the user ids inside and outside the container. Also, some build tools don't like it when the repo is on a different filesystem (bind mount vs container root) than the rest of the system. So I made rumpelpod just bake the repo checkout into the base image that the container is launched from, and since then I haven't had any issues like this.
For giving the agent access to a docker daemon I found sysbox to work very well. Usually the advice for nested containers is to pass in the host's docker socket into the container. But that would break the outer container isolation completely, since access to the docker daemon is equivalent to root access on the host. With sysbox it's trivial to run a nested docker instance inside the outer container. I haven't tried it with podman yet though. In theory sysbox is just another OCI runtime, but there's probably some tinkering required to get it to work.
Neither podman nor docker will help you when the current directory is your home directory, though. It sounds like that's the root problem here- someone handed the keys to the kingdom to grok, and grok did what grok does, which is look at everything it can for context.
That's why I went with a different strategy and scope: Copy the current Git repo into a sandboxes dir, mount that copy in the container. The tradeoff is disk space allocated per sandbox, but I'm OK with that.
Are you doing something more advanced with Podman than just mounting the files? How is the access for relevant files given? How is the authentication shared across multiple uses? Just curious to streamline the process.
Just a little bit, I want coding agents to work their own disposable copy of a git repo. Here is a quick rundown of what it does:
It copies the current Git repo into the sandboxes dir, mounts that copy at /workspace in the container. The original repo is never mounted writable, so I don't care if the agent goes to town/wild in there (peace of mind).
It also builds cached Debian/mise/Elixir/Phoenix images, can start a private Postgres container, publishes selected localhost ports, reuses dependency/build caches, and prints commands on exit for reviewing diffs, exporting patches, applying them back to the real repo, or reopening the same sandbox later. Pi, and OpenCode are configured with proper LLM access keys (derived from my own).
So spinning a new sandbox is a matter of cding into a project directory and run something like: `ai-sandbox --port 4000 --postgres somedbname` or `ai-sandbox --port 4001` if I don't need DB support. Then when running the server in the container I can access it from the host machine to review in my browser.
I work on a sandbox which has similar isolation level to Podman (rootless Linux user namespaces), but with UX optimized for local development work. Take a look: https://github.com/wrr/drop
Basically, you don't enter a separate container in which you install a new distro, but you run on top of your current distro. You have environment specific home dirs which isolate your original home, but can have some files, such as configs, mounted from your original home (mostly in read-only mode).
This is wonderful - might even be exactly what one of my projects needs to use as a dependency.
In the README it mentions that it puts the dev environment in a filesystem jail, but how are you able to use your hosts bins without leaking access to the rest of the system? Or is that just an assumed liability?
It assembles and pivots to own root filesystem. The filesystem mounts dirs and files from your original filesystem, most of them in read-only mode, but does not mount files that expose secrets to the sandbox. The sandbox also mounts own, writable version of your home dir, so secrets, such as .ssh dir are not exposed. The config file allows to configure which files should be exposed from your original home. For example, configs, such as .bashrc, are safe to expose in read-only mode, provided you don't store secrets in them.
Yeah, I'd expect this approach to be almost universal, with the caveat that of the few who don't, many still don't get bitten.
But it turns out even in the container, there are footguns that have occasionally made the news by being fired: few projects don't have any external resources and when credentials with any form of write access happen to make it into the container (even if it's just a session cookie) agents might jump at the opportunity.
It’s not that intricate. A very simple docker and alias setup will get you an ephemeral container with your Claude (or whatever) config and the current wd bind mounted in. That’s a pretty good start.
I use a separate user for all development tasks, its home folder contains all repositories I work on, and nothing else, and that is all the IDE and the AI assistants have access to. Create the user once, start the IDE from a shell using that user, and that's it. In Linux it's a pretty seamless experience.
It's simple sandboxing based solely on unix file permissions. Albeit weak, I find the isolation sufficient. Until I'm shown otherwise it seems like a good compromise given how easy it is.
You can also create iptables rules matching on the user, so this technique is useful for applications where you want to restrict network traffic as well, and don't need stronger or more fine-grained isolation mechanisms.
A limitation with this setup is you can't let your agent/Linux user run containers with docker. Adding a user to the `docker` group effectively grants the user full root privileges [1], so the secure way is to setup docker in rootless mode. It's doable, but in my experience it's hard to setup (I find it complex with podman as well).
> Until I'm shown otherwise it seems like a good compromise
Agreed. In my case I went a long way running the Pi harness in a (simple, rootfull) docker container. As the project I worked on relied on a standardized docker compose stack for local dev and testing, I realized I could automate more if only my agent could use docker. Ultimately, the need for docker for my agent grew when testcontainer [2] was introduced in the project. That's when I finally took the time to setup a VM with incus [3], and now I can let the agent go wild with docker inside the VM.
This is at least one example where more isolation is required. Otherwise, the dedicated Linux user, if it works for you, is by far the easiest and most pragmatic solution IMO.
The question "do you trust this directory?" is a world away from "do you consent to upload this directory and all of its contents to a third party?".
My assumption when asked "do you trust this directory?" is that I am being asked if I am certain I understand what is in the directory and that it won't include some sort of prompt injecting attack. I would never dream that I was consenting to the complete exfiltration of that directory.
A bot will do what a bot can do whether malicious or accidental. One should assume they are giving DOGE shell access on their computer and adapt accordingly. I am trying to imagine the SELinux rules required to make a bot play nice and the more I think about it such rule complexity may even befuddle the NSA. Alternate methodology:
- Give the bot it's own machine and only copy to it that which one would want DOGE having access to. Not a virtual machine, the bot will eventually escape. This applies to all bots or agents of all LLM's. Name the node DOGE to remind anyone using it not to share their crown jewels. Come up with a silly name for the agent. Elonious?
- Give it a little RasPi or mini-PC with maximum power savings enabled and no default network gateway.
- Install a self signed CA cert on the DOGE node and force it's traffic through a Squid SSL Bump MitM proxy on the same private LAN to another node with bandwidth limits enabled so that one can monitor what URL's it goes to and what data it is transferring. Configure Squid Access Control Lists to only permit specific domains and optionally URL's, mime-types, sizes, etc...
- Enable custom AuditD rules to watch anything it touches outside of it's sandbox. Send these events to a remote syslog daemon on the Squid server.
- Install Unbound DNS on the squid proxy and enable the DoH (DNS over HTTPS) listener and force all bot DNS queries to use Unbound with query logging enabled.
When the bot attempts to misbehave there will be forensic data to share with the world.
Absolutely. Quite a bit of work one time by one person willing to document the steps or even better create automation scripts for Ansible, Docker, Podman, Systemd, etc... and then automation for everyone using agents. As to success I would suggest it may be better than zero visibility or trusting what the agent says it did. Due diligence and a repeatable standard as apposed to running with scissors.
This is probably most important for anyone operating agents on corporate systems. I would suggest corporate security should take interest in this idea so they have a good answer for auditors, insurance companies, investors and customers. A full audit trail of what every agent had access to and ingested.
You should assume by default for any AI agent that it will read anything. Even if you manually allow/deny and "restrict" it to a subdirectory I would still hold that assumption. Claude reads your ~/.bash_history too so when you ran something it can use that same command.
Indeed. I use a spare laptop that has no accounts other than (1) the AI themselves, (2) a secondary GitHub account which has "untrusted devices" in the name to emphasise the point.
If I recall correctly, I did a full system reset before setting it up this way. It's certainly not logged into iCloud etc.
And this is why so many people run these inside of VMs. Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
> Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
I've heard it said that piping curl into your shell is no different to running any other program you've downloaded from the Internet (binary, or otherwise): the maximum possible damage that `example.com/script.sh` could do is exactly the same as `githubusercontent.com/someone/releases/myprogram.exe`. At least with `script.sh` you can easily inspect what the script actually does instead of busting out Ghidra.
It comes down to trust: do you trust Example.com to not serve-up a malicious program (shell script or executable binary)?
Now we take that principle and apply it to Mr. Musk's "MechaHitler" LLM vendor xAI: they have a well-documented history of unnecessary risk-taking - and outright criminal behaviour (child-porn generators are a good thing that everyone should have, apparently?). Would I trust Grok with anything? Absolutely not.
Sure, but it's easy to accidentally start up the agent in the wrong directory, like when you open a new terminal. I've done it before when I was distracted (albeit with Claude, not Grok).
But isn't that user error? If you want to run a program that can read your current directory's contents and possibly upload that to the cloud, would you run that in a directory with your private keys?
I am running all these clis in containered environments. How can you ever trust LLM to respect the bounderies provided by these magical, non-deterministic intructions files...
We all know the dangers of running agents with no permissions on our laptop.
The good news is its now just as easy to spin up a sandbox in the cloud for an experiment or coding session than it is on your laptop. Possibly easier since laptop sandboxes aren't as cut and dry as a new cloud VM.
exe.dev is my sandbox infra of choice. You get a new sandbox in literally a second with SSH and a coding agent (Shelley) built in.
I generally drop in in my own binary with toolkit so I can connect Claude or Codex subscription and use their harness.
> We all know the dangers of running agents with no permissions on our laptop.
Some/maybe most of the HN crowd, sure, but as a rule ... definitely not. People are generally happy to take whatever the path of least resistance is.
I know quite a few civilians (even C-level people) who "heard openclaw was cool" (or whatever) and downloaded the first installer they found and hit enter/run without thinking twice.
How would it mitigate this issue? Presumably you'd need all the code for a given project inside the sandbox anyways, so the agent will still be able to upload everything. The only thing it might mitigate is you accidentally uploading your home directory.
Or if you're on Linux or macOS, learn how Unix permissions work and can be used for, create a new restricted user locally, use that. No need to go all remote with all its drawbacks just to limit a little local process on your computer.
Even better: you can fork the exeuntu image & make it immediately more relevant to the things you do in addition to just dropping in your programs and such.
Lots of people responding about LLM but this sound like it’s their coding software („grok build“). Which is approx 100x worse.
LLMs going rogue is a thing and shit happens but publishing software that is uploading user directories including ssh keys is insane behaviour on xAIs part (alledgedly)
It uses bubblewrap to instantly construct a container around just the agent config, auth and history and the project path. The agent or any command it runs can't reach outside of it even if you tell it to (or, more dangerously, a random prompt injection from the web or some third party library or script that the agent runs).
I was using VMs to solve this problem but the temptation to start an agent to work directly on my machine for GUI apps and the like was motivation enough to find an alternative to VMs.
I wouldn't use Grok, became I really don't trust Musk, but even the models I do kinda trust to have good judgement I don't trust enough to let them have unrestricted access to my personal machines and all my credentials.
There are a distressing number of people in this thread who think that the agent should just be expected to do this. Yes, it is good to be paranoid, but also, the agent should never do this. Indicates horrific engineering practices at xAI.
How could it possibly work without something effectively the same as this happening? Either you run the LLM locally or you send it your files. Am I missing something?
It’s not really paranoia that makes me expect this will happen. It’s more like, well, the model weights and the files I want the model to work on have to be inside the same GPU for the agent to actually work, right? So step 1 has to be either “they send me a server rack of GPUs”, or “I send them the files I want the model to work on”. I’m not sure I could reasonably expect anything except this to happen.
The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.
Have the comments been combined with another story? Because this isn't about a LLM reading and sending contents of the file to be processed, but the agent framework uploading every single file in /home/user to a random server.
Yes, but the underlying problem is a lack of exact specification what the agent is allowed or not allowed to do. It must have access to some (most?) of my files (but I'm too lazy to specify which ones exactly, e. g. by setting appropriate filesystem permissions). It must have access to some servers (but not those that I consider "random", which again I'm too lazy to specify). It is even expected to upload files (e. g. by checking them into git and pushing, or to reason about them in case I'm not running a local version of Grok). But please - not to a server that I deem "random". (And again, I of course won't tell you what makes a "random" host via exact firewall rules or anything like that!)
We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.
(Addendum: But indeed, the failure to formalize our assumptions into enforcable rules isn't specific to agents / LLM-based applications. For any desktop app, we have implicit assumptions like "please only read and write your own config files", that we never care to enforce via filesystem permissions, running them in a vm or similar. But with these almighty agents that are supposed to guess our will from just a couple of words, the risk of them violating our unwritten assumptions gets so much higher...)
Idk, did you see the whole DOGE thing? Maybe Edolf has resorted to hiring script kiddies because nobody with a developed moral compass will work for his companies anymore.
Engineering? What's that? Modern development best practice is to have the AI make some changes, then have another AI review the changes, and finally "ship it".
I never thought about it in that extreme, but just today the google results ai gave me conflicting information from one search to another. Maybe I should start.
P.S. the conflicting information was Keith Richard's age. One search said he was 37 Dec 18 1981, the other said he was 37 in 1980.
If the agent you are running really wanted to it could easily find a way to mount the windows folders and read them all. WSL isn't a security boundary, you are only barely more protected than people running grok in their home directory.
Still, you are barely more protected than someone running this as non root at home. By your same logic you could “confine” the agent to any arbitrary directory on your filesystem and say it’s sufficient.
Yes, that is exactly the point I am making. The agent / cli is confined within the WSL2 environment. How is it "barley" protected? Neither you nor the other guy have yet answered this question.
I never claimed WSL2 is a security sandbox. I am saying running it in a container or WSL2 allows you to severly limit the blast radius. I am not expecting the agent to be malicious, but I am expecting it to do unexpected things.
That’s a fair approach, thanks for elaborating. I think the main point of contention is there is an expectation this software should be more reliable, that doesn’t take away from your approach, running close sourced software like a harness (or operating system) is asking for problems.
My first thought would be their server side extentions, code excecutoon sandboxes and document RAG search, being on by default? Probably should be an opt-in instead of an opt-out.
So bubblewrap is the community medicine to this issue. But isn't it funny that we're now trusting a single GH user (although allegedly a RH employee) for the supply chain security? Imagine bubblewrap itself being compromised.
Now you trust a single GH user rather than a company. Fair enough. I just think it's very paradoxical.
You shouldn’t run any AI cli (codex, claude code, open code, whatever) locally outside of a vm. It’s like “you shouldn’t run a .exe you downloaded from the internet if you don’t trust the source”. We will learn
This wasn't the LLM, it was Grok CLI preemptively uploading the entire CWD, regardless of where that CWD is, to its own server.
I don't think it is reasonable to expect every user (including those just starting out with the tools - maybe experimenting, maybe younger/less experienced in general) to think that the tool they're running for the very first time is going to automatically exfiltrate all of their data.
It's a pretty serious fuck-up. This guy tweeted about it, who knows how many didn't even notice. It should have been opt-in, it should give user an indication that it's about to do this, etc.
The grok-cli is on github[0] there is nothing that I can see in the code that is activily looping ~/ and uploading everything.
My two guesses would be one the LLM decided it needed these files for the task or two the user simple asked grok to do it so they could post the tool calls on twitter.
That is not the Grok CLI being discussed. That's an open source, third party CLI. https://x.ai/cli is the official Grok CLI being discussed, and it is not open source.
I think there are arguments on both sides. People should look for guidance on how to use complex tools, but we know people will not.
Whose fault is it if someone drives a car without learning how to and injures themselves? On the other hand if the manufacturer has promoted it as one you can drive without learning how to, then whose fault is it?
A lot of users are fine with everything being uploaded. Most people's primary computing device is now a phone that backs up everything to cloud and using apps that are thin front ends over cloud services.
If your immediate reaction to a new piece of software siphoning up someone’s entire system full of highly personal data is, “you’re holding it wrong”, it might help to take a beat and remember that software was developed by a multi-trillion dollar company’s entire business model revolves around siphoning up as much highly personal data as possible
The point is more that you should not blame the user (why didn't you set up sandbox instead of directly using the tool of big corp) if a tool does something unexpected. If your Dropbox client would suddenly just upload your home directory instead of it's folder you configured you'd also not blame the user that they use Dropbox, you'd blame Dropbox for not doing their job correctly or being user hostile.
Agreed. You can still encourage people to use defense in depth without actively blaming them for not having the deepest moat imaginable. Software creators still have some responsibility
Maybe possibly with --dangerously-skip-permissions. I've been using auto mode and enjoy how it blocks every tool use that could've allowed something potentially sensitive into context. Burns extra tokens though.
But Claude Code, arguably one of the most famous ones, is not. And recently got some heat about sending meta data that wasn't so obvious. Just as a counter-example.
I commented not so long ago to people very invested in AI, agents and tokens that are very enthusiast that I would not use an agent for security reasons and loss of control (besides suboptimal output).
It seems I was not wrong.
Not that I do not use AI. I do: fenced, always pasting my snippets and NEVER giving access to my code. Always from the browser. I know what it can do well and which workflows accelerates for me. But I do not want to think that my whole project ends up somewhere else.
I remember a time when we were told not to run obscure software from questionable websites on our computers. And yet, people are suddenly fully vibing with an uncontrollable character generating machine with who-knows-what-behind-the-scene harness. Crazy.
the 'upload your home directory to our servers' feature is the kind of thing that used to require a warrant and now it's just a Tuesday afternoon bug report
This is why it's going to be a long time before companies will trust AI to scan their networks or apps for security vulnerabilities.
Our company audited a few AI-based pentesting companies and requested logs. In more than one case, it was sending drop tables for sql query injection checking and other destructive operations.
So all the smart people are letting a script being executed on their machines where the execution of the script is controlled/managed by a remote machine. Wait, isnt that a definition of a malware??
relying on a markdown file that says "don't access x" as a security boundary seems insane. i couldn't find any grok build-specific policy on this, but if the analysis is right, silently uploading and storing the entire repo is at the very least something users should be explicitly told about.
The reports of copilot running amok when Microsoft integrated it into Windows 11 should have been enough of a warning. You either ensure proper sandboxing, or you'll be in for a bad surprise eventually.
You should ALWAYS run your agent as separate, unprivileged, UNIX user, never in your main account.
If you absolutely need to run it as your own user, you should bubblewrap it. I do this for things like Steam, games, or other "blackbox" closed source programs that cannot be reasonably trusted.
This sort of stuff is precisely why running local models has to be the future. It's absolutely insane that we just send our code to the cloud like this, and we basically have to trust these companies with it.
If quality training data is the most important piece regarding to AI, expect everything to be collected and analyzed. Don't even trust OS containers but run AI on separate hardware
Maybe I'm just too risk adverse to run with scissors with Grok? I can't imagine ever trusting their harness.
I'll just add this to the list of things an AI company could do to guarantee I'd never use them. You know, like the AI referring to itself as Mecha Hitler, making non-consensual porn (even of minors), or deferring to Elon's tweets as authoritative references on topics.
Don't store any secrets in your repo. Don't store them in env vars.
Strong long passphrase on your SSH keys, stop using unencrypted SSH keys. Everything in a keychain.
The practice of storing secrets in a .gitignore'd .env.local.json or whatever is a really bad idea and I can't believe that it has become a normalized, acceptable practice in the industry.
https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547
Elon did this horrible thing, so I made grok build available for omp with it's own endpoint; Without sending your private repos and secret keys to them.
-
oh-my-pi-plugin-grok-build
Standalone oh-my-pi extension for the xAI Grok Build subscription provider. It adds OAuth login, authoritative model discovery, and OpenAI Responses streaming with the request identity expected by Grok Build.
Install (No-spywares):
omp plugin install oh-my-pi-plugin-grok-build
-
https://github.com/metaphorics/oh-my-pi-plugin-grok-build
Star me if you like it or if you hate spywares, lol.
All of them are going to read your secrets, OpenCode does it too.
Reality is, I have seen agents read .env, bash history, keychain (if you let them), etc.
There is quite literally no way you are going to save off just your little secret somewhere it won't be able to read it, all software needs to read it ~eventually~
So it's best to sandbox and reset credentials frequently.
Guy infiltrates the U.S. government and steals personal information about everyone from federal databases in broad daylight (not to mention cutting aid to poor children with preventable diseases). Average tech bro: “Sure, I’ll use his AI!”
Same tech bro: “Wow I can’t believe they would steal my anime girls!”
Karp is right in this instance, sure, but he is just upset Palantir doesn’t have that kind of surveillance tech, and the in-house skills required to keep their edge in the age of LLM-assisted software engineering. Their true “moat" has never been superior tech, it's just the pool of amoral engineers willing to build what others won't.
https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547
Elon did this horrible thing, so I made grok build available for omp with it's own endpoint; Without sending your private repos and secret keys to them.
-
oh-my-pi-plugin-grok-build
Standalone oh-my-pi extension for the xAI Grok Build subscription provider. It adds OAuth login, authoritative model discovery, and OpenAI Responses streaming with the request identity expected by Grok Build.
Install (No-spywares):
omp plugin install oh-my-pi-plugin-grok-build
-
https://github.com/metaphorics/oh-my-pi-plugin-grok-build
Star me if you like it or if you hate spywares, lol.
SSH keys can be limited by IP in authorized hosts.
The SSH port itself can be limited by IP in firewalls.
Finally, the SSH private key can be encrypted with a password.
Defense in depth is needed. Storing a ssh private key in plain text with no IP restriction is no different to having a password manager store your passwords in plain text on your HD.
Anything that isn’t a default is optional by default. Anything that’s toggleable or configurable is optional.
Security is, always, a trade off. It is hilariously common for private keys to work as a full identifier for a person, without concern of IP or anything of the sort. Should they? Maybe, maybe not, that’s the calculus of risk management; but victim-blaming the average person who is following best practices is a bad look.
I'm a security eng and I've worked for both a FAANG and TS government contractor. Neither of them bothered with either of the of the stupid suggestions. IP restrictions prevent roaming, the point of working remotely via SSH. passwords are equally defeated by using an ssh agent, something I'd suggest everyone use. Then on top of that there's no reasonable threat model where something would be given unrestricted access to user env, but also be untrusted. If it can read from ~/.ssh neither IP protection nor keyfile password protection will protect you from maleficence.
The only reasonable response from a security perspective is don't use grok, then use it sandboxed. Trying to claim it's the users fault for not using password protection and IP restrictions is completely nonsensical. Same energy as telling someone their computer is more secure when it's off.
I am sure majority of people don't use password for ssh keys. The good solution is to use password manager like 1password which will prompt you to approve ssh.
The point is to know better than to let the intern hurt themselves.
I once saw an engineer try to place the blame on his intern for taking down prod. I was sitting in a meeting with the VP of engineering and someone asked if it was ok for some to blame their intern for the SEV, and I remember the VP saying "I'll talk to $director_for_the_interns_mentor". Interns can't take down prod. An intern's mentor willingly watching an intern take down prod is the closest you can get.
TLDR: Ran grok in $HOME. Surprised agent read content of folder.
On the other hand, I specifically had grok try hard NOT to read a known key in the project dir (it only saw the first part using a tool, to verify it was present). So there's that.
Yeah, this is a lesson about learning how to use tools safely, not about tools abusing the user. The person that posted this probably blames the hammer when he hits his thumb.
> The practical takeaway for users: your entire codebase leaves (uploaded) your machine unencrypted on each Grok Build invocation, not just files you ask it to read, and no visible setting stops it.
I'm dying to have proper sandboxing in macOS. I installed ChatGPT, I asked it to list files in my user directory and it did. I never gave it permission, how could it? My terminal has access and honestly it shouldn't either.
It was only like 1 year ago that the loudest complaint about macOS were complaining about needing to click Allow in a new dialog when they use Terminal (or various other apps).
There are so many comments in here that are calling for nerfing something widely revered for giving us superpowers. Whether these are bots or not, they’re giving off NPC energy.
If they don’t want to use power tools because they accidentally cut off their finger, then they should just unplug their own power tools and stop clamoring for everybody else’s to be unplugged, too.
> If they don’t want to use power tools because they accidentally cut off their finger, then they should just unplug their own power tools and stop clamoring for everybody else’s to be unplugged, too.
Sadly, the pendulum keeps swinging in the way of the tools actually being unplugged and dumbed down in favor of these screaming people, rather than leaving things alone so the rest of us professionals can continue to (safely) do our jobs with the already somewhat restricted tools we have.
Is the Grok CLI a 2 terabyte install? Did Elon dropship you an 8U rack of B200s?
No?
Well the model weights, the GPUs, and the context obviously all have to be in the same place, so “sending your project to them” is literally the only thing that could possibly happen, unless you think agents work by fucking magic.
This is the biggest case of PEBKAC in history, maybe ever.
This is the kind of confusion that Charles Babbage could not rightly comprehend, except at those politicians at least had the excuse that computers had only been invented five minutes prior.
If you decide that your entire home directory is the project, as the OP did by setting the repo_path to ~/, then, well… I mean, if you ask me, I don’t recommend it, but it’s your computer and your free will.
Then that should be considered a 'nuke button' and nuke buttons don't find themselves into well-designed software for the simple fact that an end user shouldn't have to worry about shooting themselves in the foot. Again, with good software.
that‘s … not really how it works. Agents get your code into their context via tool calls, not by uploading the entire file to a server „where the weights live and thus the code has to be too“. Small but crucial difference. Aside from that: LLM providers have to be the ones that facilitate your privacy and security by default. That‘s not on the user. They‘re tool providers and they can‘t compromise you and your org by default.
What do you think a tool call is? What is this small but crucial difference between a `read_file` tool call and `scp`?
If your contention is just that this should upload files one by one instead of all at once, what you want is for providers to facilitate the illusion of privacy.
It's fascinating how many people in this conversation think that LLMs need to have all of the files in your $CWD on the model provider's servers to be able to do anything.
Important to clarify that this was not the Grok agent deciding to read the files.
I don't think the LLM had anything to do with this decision at all. It looks like the Grok tool starts a session by deterministically kicking off a full upload of the user's current repository (and maybe their directory if not version tracked? Not clear if this user had previously run "git init" in their home directory) to Grok's servers.
One possible "innocent" explanation could be that xAI then run vector embedding on every file to help later provide the right context. I don't think thats a worthwhile tradeoff here, especially since other popular coding agents get by just using grep/ripgrep run locally.
That is not an innocent explanation, any more than someone breaking into your house to adjust your thermostat.
Yeah sounds like semantic indexing that cursor (pending acquisition by spacex/xAI) does.
https://cursor.com/docs/agent/tools/search
They upload your ssh keys too?
They might, if you use your home directory as the project directory, given that they only mention using gitignore as the blacklist.
More info: https://forum.cursor.com/t/codebase-indexing/36/18
So file contents are uploaded for embedding/indexing, but supposedly none of those contents are stored at Cursor after embedding.
Also decent embeddings models could run locally and just trickle through the user folder if that was so necessary.
This should be the first comment here.
Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.
Jesus ... I feel a Bill Hicks sketch coming on ...
So many of the replies are saying that they should've restricted access using .md files and whatnot. Is really any guarantee that they even follow those? It seems like even if you ask pretty please don't touch those files, there's a chance they will. So many people have just willingly installed spyware on their computers and big tech calls this the next big thing.
I will keep banging this drum until people listen:
Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.
The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.
It's wild that we've known for decades to use ACLs to make sure people don't have access to files we don't want them to have access to, but somehow a computer pretending to be a person doesn't get that same treatment.
ACLs, nothing, we've known about in-band signalling since forever and still this whole segment of the industry seems to either not know about it, or forgets about it at a cadence so regular it may as well not know about it.
System-level ACLs; mandatory or discretionary access control; secure-by-default application and network configurations are all for naught if you take an LLM, run it with all the privileges you'd have an accountable, judgemental operator, and then tell it to act based on arbitrary untrusted input which might include prompt injection attacks, something which cannot generally be sanitized.
Well-defined, well-enforced security policies can mitigate disasters, but many in the wild right now just don't account for this kind of threat model.
It's just purposeful blindness - I worked for a company building out tooling that insisted markdown based security was good enough and showed it off for anyone at the company to attack because they were so sure.
It took me less than 5 minutes to completely disable... nobody cared, they just kept going - check the box and move on.
Something something salary depending on it.
Software "engineering" in particular has always been more than 50% cargo culting. Good engineering practices never matter when the alternative is just going through the motions of whatever rituals are in vogue.
We laugh, and rightly so, at the time sharing systems of the 70s and 80s that didn't use passwords. I bet allowing a whole other virtual person to run outside a VM, with access to an actual file system instead of a version-controlled branch, will be seen as a worst practice in ten years.
Have you ever tried to configure ACLs? They're a pain in the ass. Not everyone wants to be a sysadmin.
ACLs aren't even the issue here, first you need an entity separate from "the current user" to grant that access to. That's what jails, sandboxes or capability-based systems bring to the table. But you have to use them, most of those AI tools and their IDE integrations don't. Once you have those you can think about which access to grant that entity.
What do you find hard about it? Is it a conceptual thing or the tools themselves? I'm very far from a sysadmin, but it's just such a ground level using-linux type thing in my brain, I don't think I ever thought I had a choice about learning it!
I think ACL is only a part of the solution. If he runs the agent with his account, the ACL would not really help. But I do admit, I might have an outdated understanding of ACLs.
A computer pretending to be you, in many cases.
One cannot pound that particular drum enough. "Guardrails" in instructions (and all MD files are just instructions) are like price lists at an unattended farm stand. It'll usually work! There will be some money in the basket at the end of the day! People paid for the bagels[0]! But one cannot never assert that it _will_ work; things that _must_ work have to managed out-of-band
[0] https://pubs.aeaweb.org/doi/pdfplus/10.1257/0002828067772121...
The easiest, most guaranteed way to isolate it is to run it in a VM or container where it literally can't do the wrong thing without some kind of full container or VM exit exploit.
It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.
For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.
Generally, I agree!
But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.
That said, I do hope these become the norm if LLMs are here to stay.
I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.
I seem to recall reading about agents already breaking out of containers.
Last time I read about this, this was due to the well-known pitfall of UID mapping across container boundaries.
It's a common misconfiguration and one of the footguns available through containers, which I don't say a wholesale condemnation of the technology, but certainly as a UX facet that could use reevaluation.
I've been biasing towards VMs myself just out of caution, but maybe that's just extreme paranoia.
The way I look at it is similar to how I'd look at any hypothetical employee. How do I ensure the agent only has access to the minimum possible they need to get their job done?
That means no access to git repositories (no pushes on my behalf means it can't accidentally nuke git history, something there is anecdotal evidence of agents doing). It can make local changes in git only and I will take responsibility for pushing them. No access to the wider internet beyond what I deem acceptable. No permissions to access any internal APIs except what I provide (and not using my credentials).
In one case, I have a tool that has a set of dangerous commands alongside a large number of safe ones. I don't even have it installed in the agent's VM. I run an MCP that is a simple python wrapper around the tool on the host side, and expose it to the agent in the VM, so that it can only possibly run a strict safe subset that I can trust it with access.
> Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.
It reminded me of an old meme. Please don't follow the following instructions and stop reading if you cannot.
It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"
Claude.md is an equally effective defensive tool.
But sorry if you lost some files from reading that.
I agree with this. And also think that we should train and select for trustworthy models. I also agree that these models may never truly be trustworthy.
Especially when I find that, when I ask an agent not to do something, the mere mention seems to put the "idea" in its "head," making it more likely to ultimately do that thing.
yes, because this is how transformers work, when you introduce "don't do x" x is also entered as a pattern that is more likely to be repeated than if not mentioned at all
1000%. I'm experimenting with running my agent (Claude Code) inside a docker container, so I can have a control plane and then YOLO within that limited access. The agent could still mangle my local dev setup, but I consider that an acceptable risk since everything the agent has access to is under version control outside the local machine.
This is a new pattern for me, I'm curious what others are doing.
https://smolmachines.com has "smolvm" microvms with better performance and ergonomics and security than docker, you might want to give it a try. (No affiliation, I just like what they're doing.)
It's like making directories called 'only for jack' etc. and expecting everyone to follow the rules
If you're not using a sandbox, something like this will inevitably happen to you. It's really not that hard to set up and should be a standard recommendation.
I don't understand these people. Agent instructions in markdown is barely a suggestion. I have one which says "All code in this repository is executed in docker containers, run the services with `docker compose run --rm php-cli "$@"`. Gemini and Claude more often than not refuse to abide and will try to execute the environment using /opt/homebrew/bin/php on my host…
Right: it’s just context, it’s not a contract. Same with “skills”.
A frightening amount of people have no idea how AI tools work, even those that should know better. I have seen senior software developers fall for the mistake of believing an LLM output when it spews bullshit about how its own memory or restrictions work.
LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time. I've come to terms with the impact slop coding will have on most software jobs in the future, but seeing seemingly intelligent people fall for lies and fantasies concocted by an LLM is making me more and more uncomfortable with the direction we're all heading in.
> LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time.
It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.
https://en.wikipedia.org/wiki/Automation_bias
Are we all heading in that direction?
I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).
We don’t have to head in this direction of using LLMs for most development at all.
There’s an aspect of extrapolation in the perception spike of the Dunning–Kruger effect.
In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.
I've seen claude check the Event Log in Windows and produce powershell scripts to alter firewall rules. This is what makes (something like) T3 Code appealing to me. The computer I'm working on is not the computer where the AI has agency.
I don't understand why the AI world does this. We don't need new security. We have security at home. It starts with
Your OS knows how to restrict access to things, you don't have to trust a pinkey promise from a vendor.You will want at least a separate session for the `restricteduser`: E.g. with X11, a process in the same session can do almost anything with your input/output. And most Linux distributions make it really hard to disable external device access for individual users...
Qubes OS - AI Agent edition would actually be a great idea.
> And most Linux distributions make it really hard to disable external device access for individual users...
For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.
Does this have the affordances to work well with a pretty standard agentic coding setup (e.g. claude code/codex) on macos?
I don't really have a good mental model of how ports/files/etc get exposed/permissioned across users. Containers and sandboxing seem much more common than agent-user accounts. Networking seems a little more complicated in the general case.
I roughly went the route of running apple's container-cli (separate vm/kernel per instance I believe) and mount the relevant home directory/projects and bind ports. Seatbelt/linux sandboxing seems simpler sometimes, but has its complications too.
It's limited. For instance, you're going to have to explore different kinds of sandboxing if you want to prevent network access (except for certain cases, like to allow it to talk to the AI provider). But generally, a filesystem will have metadata for files like which groups are allowed to read/write, and the operating system will enforce those rules for processes that run as that user, and for any processes which that user starts. I feel like that gets you 90% of where you're likely to want to go with it.
I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.
and Landlock! Pi even has a sandbox plugin for Landlock
Is it this one? https://github.com/landstrip/landstrip
yuppers
Users do not want the agent to be restricted. They want the agent to read the user's mind and resolving the user's cognitive dissonance and contradictory preferences.
Is the implication that human thinking is inherently inferior to the perfect, all knowing AI?
Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?
And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."
And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.
Why would you give a non-deterministic text generator a user account? It’s not a person, it’s barely a tool at the software level. Restrict at the right level, in this case, a complete sandbox around it given its propensity to hallucinate and be steered by anybody.
What kind of logic is this? It's standard in the Linux world to give important services a separate user domain.
The standard here is not enough when the agent can escalate by finding 0 days, for example. It’s like giving a black hat a limited account. Sure it might restrict him, but not giving him an account at all is way better
The black hat can find 0-day escalation in your sandbox, too.
A user account is a sandbox.
Not as air tight as a container
Edit: it’s about the attack surface
microvms are better than containers running on your host. see eg the "smolvm" microvms from https://smolmachines.com
thanks, will take a closer look
Unix users are THE tool to restrict tool permissions, at any given time there's 20+ services on a Unix machine that run in their user.
And how many of those services can check your box and find permission escalation strategies on its own?
Malware has been bundling rootkits for decades, so potentially all of them. Sometimes the attacks succeed, so these are defenses that need continual hardening, but there's no sense in setting up an entirely separate line of defense just because this time the threat has AI in it.
I guess time will tell, I know I'm not giving those things any direct stdin access without a hole suite of safeguards in between
Same reason people give postgres, php, or any other program a user account.
Why should it be? We have containers and capabilities…
Containers are often used with user namespaces, which is literally another (set of) user account(s).
Yes but the encompassing service has the account, not the agent. You can completely segregate it from the rest of the system, like you’d treat an intruder
i give a lot of software dedicated user accounts, it's literally one of the core security models of the operating system
But none of these services have the kind of latent capabilities an agent has, you know a deterministic service’s constraint are its code+bugs. There are no such constraints in an agent…
...this is a completely normal thing to do in linux, it's the most basic form of access control. There's like a dozen non-human accounts in a clean install before adding your own like this, and a lot of software adds their own. Edit: I have 54 entries on my personal laptop, just one of which is actually me.
This model is already not perfect, and not at all built for agents. The only way to secure an agent is an air-gap with the execution layer. Treat it like text, and the problem never arises until you “interpret” the agent output in a more limited environment than the OS
Even though it’s completely normal to us and in widespread use, GP is a reminder that conceptually it’s a broken model. Security should be capability-based not user-based. And to anyone who didn’t grow up on a desktop this model makes complete sense since it’s what your phone uses.
Android uses one uid per app.
He said it’s less than a software, so saying software does this too isn’t really a strong counter argument. In case, I don’t think you are really in disagreement. Restricted accounts are necessary is your point, but I think op is saying they aren’t sufficient.
One of the default users is "nobody" which isn't associated with any software. It's definitely above that.
Is it less secure if I simply have nothing to do with language models whatsoever? I take that to be the parents point, but I know you are correct and yes, user restrictions should be a necessary part of the way people who use LLMs setup their system to use them in a safe manner.
Congratulations! You are this thread’s winner of the UUOC award!
/etc/shadow has the good stuff if you're OS is current.
You are correct.
You can't trust the agent, let alone its harness, to oberve any particular directive you give it, so "md files" provide no meaningful protection for anything important.
But users are broadly reckless and naive and commercial vendors are exploitative and irresponsonsible, so the vendors take advantage of what they can get away with for as long as they can get away with it.
Use a tight sandbox, and join the chorus loudly when others press on vendors to be make user safety an earnest and hard-to-abandon priority.
That's the whole reason I refuse to install Google Drive or Dropbox's desktop applications. I only use the web interface so I know exactly what gets uploaded and when. I assume that anything running on my computer gets access to everything.
Sounds like a very wise decision to me. I found found out on my phone that the google photos application uploaded everything in my gallery to their servers without asking me, regardless that I had explicitly disabled all backup to my google accounts on the settings of the phone. I only figured it out when they sent me emails saying that my storage was full.
Ooooh, don't get me started how mad it makes me! I am paranoid (or just lucky) enough that I didn't yet had it happen to me, but my wife's phone had done it four times in the last year. Each time I check and double check that all "backups" are turned off, and each time it somehow pops back.
So, Google "backs up" a 128Gb worth of photos on the phone onto 15Gb free storage combined with Gmail and who knows what, completely clogs it (as if it couldn't be predicted) and then has audacity to suggest paying for "extra storage". There is no way in online UI to just delete the whole "backup". And the cherry on top: when you finally get to delete some there is a fine-print - "the selected photos will be deleted from all synced devices". Well, I guess I must be thankful that they at least show this warning. This is what passes as "backup" in Google's parlance these days.
I would go further and not upload anything that isn't encrypted to cloud storage services. It is extremely likely that those "services" inspect your files.
They do, unquestionably.
https://www.thetimes.com/world/article/google-bans-father-ov...
> Mark, from San Francisco, had noticed swelling in his son’s groin and used his phone to photograph the problem to get an emergency appointment in February last year. He shared the pictures with a nurse so that a doctor could review them.
> However, Google’s artificial intelligence system used to detect child abuse flagged the image to the police and Mark, a software engineer who asked to be identified by only his first name, was investigated and lost access to his Google accounts. He was exonerated by the police in San Francisco but his Google account has not been reinstated.
This was posted on HN yesterday: https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75f...
If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
> If it’s to be trusted, it has nothing to do with the “agent” or what’s sent to the LLM. The harness will just straight up package the folder it’s run from and upload it to Google Cloud Storage.
Even if there is a misunderstanding who is really uploading the directory, the TUI/CLI itself by actual code, or if the model decided to do so in the session, if you apply the recommendations from the replies to parent, and it no longer matter who did it, neither the software nor the model will be able to upload all your ssh keys.
No I disagree. A harness reading a file is a tool call and it happens locally, which means that I can control it. I can configure that I need to permit any file reads and now I _should_ have control of what is sent. The difference between that and silently uploading my entire working directory in the background is miles apart IMO.
I understand that one should think carefully about how they work with a non-deterministic tool, but this if different completely. This is xAI just choosing to upload and store everyone’s directories - with full git history.
What happened here is not related to agentic behavior or instructions in .md files. It's a binary a user runs, it scoops up their files and sends them to a third-party.
And the user even paid $99/month or more for having their data leaked.
I guess the downside of the lower barrier to entry to use these tools is the lack of basic understanding of exactly this sort of concept.
This sort of thing is why I'm hopeful I'll continue to have employment going forward. Some expertise is hard won and there's just no replacing learning through experience.
I think you're right in principle but I just hope I can hold out long enough for my experience to become appreciated and whose corresponding hourly rate isn't something which is suddenly being scoffed at (i.e. markets can remain irrational longer than I can remain solvent).
Claude definitely do not respect all my rules, it often ventures into other folders, most often other projects on the same machine that was greenlit before but not from the current projects' side. One other anomaly I had in the last month: I have two linux users on my laptop, one for work, one for personal. On my work account, it asked me if I wanted to continue with project x, which is in my personal account and not present at all in my home directory with work stuff. So somehow the memory system is keeping context where it shouldn't. Interesting I used this learning to improve the agent framework/harness I have running at work: it now creates a new linux user for each agent which have much stricter rules on how it can read/write to the system, and it is also no longer running as root. Much safer now. Still don't trust it 100% though.
Only guarantee that you can get is the sandbox in which it operates. The model itself is a slot machine and can result in anything, and if its sandbox is nonexistent... here's one possibility.
In what universe would a sane person allow any LLM or remote calling software access to their user folder with sensitive data in it?
I swear, people hear the word LLM and their brain resets when it comes to good software practices.
Did VMs suddenly stop existing? Kata containers? An RHEL box with SEL?
It's like there's a new technology and everyone suddenly decided to shutoff their brain when it comes to basic security.
Yeah, I absolutely understand the allure of agentic AI, but I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level. Until we can get something set up that gives strict schema-only access I'm going to copy and paste definitions for context. Yes that sucks, but it's my responsibility to protect the system just as much as it is to develop scripts and queries for it.
> ... I am absolutely not going to give shell access or data access to any agent. Certainly not with my permissions level.
Of course not.
To me it's on a server, in a VM. And they're not seeing the real data/databases from the actual projects: they're seeing fake infos used only while in the dev environment. There's no way I'm dumping, even for tests, the real or part of the real DB somewhere an AI can see it.
To find bugs (for example), AIs are useful but honestly for code generated by LLMs, I'm thinking about going back to the early copy/paste from the ChatGPT days: because I see so many horrors in the code output by the latest SOTA LLMs that every single line of code they spew has to be checked by someone who does know better.
It's not just an issue of protecting confidential data / preventing spying: we're all discovering that we've got serious sloppy-pasta code problems now.
Sometimes you can't even rely on harness not allowing ai to access certain files. "Oh, user doesn't want me to use `read_file` on .env? Well how about I run `cat .env` then?"
That's scary. Really should run it under different user with carefully assigned permissions.
I built a docker container that volume mounts the project directory
You can even go a step further and run the container in a VM, such as with Docker Sandbox or the krun runtime in Podman.
There's also smolvm which is a nice minimal microvm manager based on libkrun: https://github.com/smol-machines/smolvm. I vibe coded a little shell utility for building and running OCI images for the Pi harness using it (easy enough to do manually, but the automation just makes it a couple quick commands rather than digging through documentation): https://github.com/neuroblaze/smol-pi
I have a similar setup using containerd/nerdctl and Kata Containers. Each OpenCode instance runs in its own little VM with mounted folders for context.
Yup, good call, I'll have to check those out. Not that urgent to me as I also happen to use colima for it's docker daemon interface. And, colima uses a full VM to host the containers, and you can further lock down the config to what is even allowed to vol mount so there's even another fs access restriction layer in play.
I almost exclusively dev in containers as is, cant really imagine letting some AI model run free on bare metal no matter what claims of guardrails it might have.
This is exactly what I do for AI agents.
This is the only pragmatic way really.
> Is really any guarantee that they even follow those?
No, there isn't. I just don't understand how naive (or imbecile) people are. The most valuable thing for these companies is people's data used for training, so giving unrestricted access to a tool from them and believing they will never take advantage of it to gobble up whatever they want from your computer, just because they told you they'll never do that, swearsies, is naive, or incredibly stupid.
Insulate yourself, or better yet, go local whenever possible, and there isn't much you can't do local if you have enough patience.
If you've not realized your agents ignore MD files from time to time, you've not used your agents enough.
The real enforcement has to be done via methods that YOU can't easily bypass, or they will bypass (OS-level prohibitions, etc).
Sandboxing is not difficult, and harnesses like Claude Code have it built-in + other protection with auto mode.
Is that built in protection really a filter, on code level, that sits between the LLM session and the shell or is it just some pleading in the bootstrap prompt? "Pretty please don't do xyz this is important!!!11"?
The latter can seem to be as good as the former for any amount of time. No outside observation can really prove reliability, only the negative result ("it does occasionally break the rules we expect") would be proof. So it's difficult to trust any claims that it's the former.
And even if it does have some of the former, chances are that the protection you experience is only partially provided on code level, while an unknown amount is still just bootstrap prompting that just works until does not.
> Is that built in protection really a filter, on code level
yes, on mac it uses seatbelt and on other platforms it uses similar tools: https://code.claude.com/docs/en/sandboxing
Asking the wolves to look after the sheep
"IMPORTANT: Before entering the leopard pen, don't forget to put on the leopard safety jacket that reads 'UNDER NOT CIRCUMSTANCES SHOULD YOU EAT MY FACE'"
Yeah that advice is smoking crack. I have no idea what people are thinking these days? We seem to have lost any sensible security understanding recently.
You can't gaslight something into not doing something bad. There has to be a hard security control that prevents it doing something bad. And if you don't know what it's capable of because it's non-deterministic then you have to start with a default block everything. This should have never been possible with any sensible design.
On my first point again, ethics and engineering both went out of the window when fast and shiny came along. This is disgraceful.
> So many of the replies are saying that they should've restricted access using .md files and whatnot.
What? No, but the random 3rd party software you run on your computer, must be limited by you in some way, haven't we learned this even after the AUR, npm and LLM shenanigans we've dealt with for decades at this point?
No, don't ask the model "Please don't go outside this directory", you limit the runtime (via VMs, containers, unix permissions, whatever) so it only has access to what it should, not more.
Same goes for any software, not just agents or chat clients or whatever. Any 3rd party software you don't want to have access to your entire computer, you need to run in this way.
Though I'm in the camp "people should really know to sandbox by now and be careful", I'd say we should also be mindful of how far from everyone has deep knowledge of the systems and tools they use. This behaviour of a tool is just malicious. You have to take into account the human factor, of how people likely end up using a system. And in this case, the consequences of exfiltrating so many secrets this way are really quite unacceptable.
These tools are explicitly marketed as a way for non-technical people to code. If we expect those same people to understand sandboxing we're dreaming.
This is a fight I deal with every day. We have folks in the technology group at work who use AI to write code and do so without issue. But now folks in supply chain or in the executive suite are using it to generate web pages that they want published or apps they want on the internet, and while Claude can generate an HTML file how that gets published, how authentication works, etc is just glossed completely over, and generates a ton of work for the IT team to build up around this stuff as it comes in
Leaded gasoline and tobacco were explicitly marketed as non-harmful back in the day.
True, but that's a fantasy happy path. It will never happen, for most people. Only HN people will do that.
It needs to be baked into the OS.
At that point, HN users start screeching about it, so it's lose/lose, really.
macOS largely _does_ bake this into the OS, and it is annoying. They also provide a way to turn it off for specific applications (including, for example, Terminal.app).
I've found that I can usually write apps that respect it. MacOS is a free-love hippie, compared to iOS. In many cases, we have no choice.
It's annoying, and often rather infuriating, but I understand that one of the motivations for people buying Apple stuff, is for that very reason, so I'm really sawing off the branch that I'm sitting on, by trying to work around it.
Given the long history of even the most egregious data breaches with millions of affected people never having the slightest consequences, what level of care are you expecting here?
We should also be mindful of how much these tools break down the "be careful and thoughtful" barriers in favor of more and more convenience.
Not to mention the very wide push to "Use AI NOW, for EVERYTHING!" in marketing ans many companies, with hardly any though given to safety or where does all the data end up.
I am genuinely fascinated by this.
I don’t like piling on especially with security vulnerabilities, but man how many red flags do you need to ignore?
They won’t stop abusing us until we stop using their products.
> They won’t stop abusing us until we stop using their products.
I don't use AI at all in my daily life.
Work however will demand you use it.
AI is not here to help people.
Not gonna argue about the utility of AI, but isn't the statement "AI is not here to help people" completely meaningless? AI itself is not "here" for anything; the problem is big tech doing big tech shit as always, not the current technology they're doing it with.
Nothing about this has anything to do with AI. It has to do with Musk's ethical and engineering standards, or the lack thereof.
> It has to do with Musk's ethical and engineering standards, or the lack thereof.
How? It's the user that added their home directory as trusted. Seems like that has nothing to do with Musk. Or did you just want to get a bash in, regardless of the facts?
By this standard, you should never trust a directory. Since it will push your env secrets.
You shouldn’t have secrets in your environment.
AI is created by big tech stealing other people's data. Yes, this has everything to do with AI - stealing data is a foundational feature of the technology.
It's more about billions at stake and the nature of AI business. You either score big, or go belly up with diminishing funding.
Well let's be fair here - you either go belly up with diminishing funding or get bailed out (maybe by the government, maybe by private funds that themselves expect to be bailed out by the government).
There currently is no path towards profitability, it's just a question of whether you can grab funding before it dries up.
> AI is not here to help people.
True, but it isn't here to not help people, either.
It's a spanner. Who wields the spanner, makes all the difference.
We've spent the last couple of decades, cultivating a huge crop of ultimate scumbag billionaires, with comically exaggerated sociopathy, and that has filtered down to almost every level of society. They are treated as gods, these days (they certainly think of themselves that way).
It still shocks me (but really shouldn't), on a daily basis, to encounter regular folks, interacting in stores and restaurants, or driving on roads, that mirror the values systems exemplified by our billionaires. Our politicians act that way, and one of their biggest selling points, is normalizing sociopathy (not just the US, either).
It's a spanner where every quarter turn costs noticeable money. Which directly funds behavior like this.
The tool analogy is intentionally minimizing, and doesn't capture just how different rented tools with constant surveillance are.
Doesn’t absolve the spanner-wielder from responsibility, though.
We live in an age, where the mere thought of personal Responsibility is terrifying.
Yes, I mean it in the sense that the spanner itself and who the spanner funnels money to is so bad that anyone using it has automatically accepted themselves being that bad. Unlike most tools (which the tool analogy leans on heavily), it's very much not an impartial tool, there is no such thing as "using it responsibly" when you're paying Musk.
Wasn't ai introduced so replace people? It's not the spanner, it's the car and we're the horses
Nah. It was introduced to push tech, but if you're a billionaire, then it was introduced to make you richer. They don't especially care, whether or not that involves replacing people.
Replacing people is what C-suiters think of it (and many of them aren't billionaires).
I think my first clue was when their CEO hired a bunch of teenage hackers to sack the government and exfiltrate all our data.
I didn't really need a second clue.
My first clue was when he libelled the diver during the Thailand thing in 2018; it was all downhill from there.
...it was quite the sting because I bought a Tesla car only 2 weeks prior to that.
The first clue should have been when all the Silicon Valley CEO’s lined up to kiss the ring after the second Trump victory. Remember it wasn’t that long before that “woke” tech companies were derided and accused by the same factions they suddenly found themselves in good standing with. There were entire pushes regarding section 230, etc. to go against social media companies, constant complaints about “Facebook” jails and shadow banning. Now they’re all buddy buddy.
Anyway, ghouls like Thiel are now a well known name among populist left and right as an enemy, so maybe some good may come from this.
I mean, do people expect companies to protect them from a tyrant they themselves elected?
Not necessarily speaking of the present. This seems to be the general sentiment.
If the tyrant goes against established law, yes.
(an aside but the majority of the US population didn't elect Trump. He is in office because of the electoral college. Might seem like a distinction without a difference but I think it matters when we're implying personal culpability)
No.
First term he lost the popular vote, second term he won it.
If you just mean a majority of eligible voters did not vote for Trump, that’s true for every US President in my lifetime.
It's not a subtle pattern.
Most people don’t know about that because they live in an information bubble handcrafted by the oligarchs.
The real solution to these kind of problems is sandboxing. I use podman through a bash script to launch a container whenever I want an agent to work on one of my repos. When done I just generate git patches and port back everything generated.
In this way I'm not afraid of letting the agents totally lose on my computer.
I'm experimenting with OpenShell / NemoClaw.
It brings some extra sandboxing features compared to just a docker container.
For instance it also maps the secrets that the agent harness has access to. And has an allow list for outgoing http connections. And there's a way for agents to request extra access. And once approved by the user from an external cli the policy is updated and hot reloaded.
It's pretty recent. So time will tell how robust it is / will become. What its longevity will be. After all; in a time of LLMs one off experiments are cheap. Longtime nurturing not so much.
That's pretty much the flow I formalized here: https://github.com/nvidia/rumpelpod
Instead of generating patches, this exposes the agent's checkout as a git remote though.
Most similar tools (and I believe your tooling as well?) bind-mount the repository checkout from the host into the container. This was always a source of user and permission errors for me, since you have to align the user ids inside and outside the container. Also, some build tools don't like it when the repo is on a different filesystem (bind mount vs container root) than the rest of the system. So I made rumpelpod just bake the repo checkout into the base image that the container is launched from, and since then I haven't had any issues like this.
For giving the agent access to a docker daemon I found sysbox to work very well. Usually the advice for nested containers is to pass in the host's docker socket into the container. But that would break the outer container isolation completely, since access to the docker daemon is equivalent to root access on the host. With sysbox it's trivial to run a nested docker instance inside the outer container. I haven't tried it with podman yet though. In theory sysbox is just another OCI runtime, but there's probably some tinkering required to get it to work.
Quick Alpine container with the current directory mounted as the current directory:
Replace alpine with your favourite Linux distro or image.Note entirely perfect, but will be enough against anyone not actively exploiting kernel privilege escalation bugs.
Neither podman nor docker will help you when the current directory is your home directory, though. It sounds like that's the root problem here- someone handed the keys to the kingdom to grok, and grok did what grok does, which is look at everything it can for context.
That's why I went with a different strategy and scope: Copy the current Git repo into a sandboxes dir, mount that copy in the container. The tradeoff is disk space allocated per sandbox, but I'm OK with that.
Wouldn't you solve the space allocation issue by using a symbolic link?
Sorry if it's a dumb question - I'm just getting into sandboxing for the first time myself and ran into this same thought before.
Are you doing something more advanced with Podman than just mounting the files? How is the access for relevant files given? How is the authentication shared across multiple uses? Just curious to streamline the process.
Just a little bit, I want coding agents to work their own disposable copy of a git repo. Here is a quick rundown of what it does:
It copies the current Git repo into the sandboxes dir, mounts that copy at /workspace in the container. The original repo is never mounted writable, so I don't care if the agent goes to town/wild in there (peace of mind).
It also builds cached Debian/mise/Elixir/Phoenix images, can start a private Postgres container, publishes selected localhost ports, reuses dependency/build caches, and prints commands on exit for reviewing diffs, exporting patches, applying them back to the real repo, or reopening the same sandbox later. Pi, and OpenCode are configured with proper LLM access keys (derived from my own).
So spinning a new sandbox is a matter of cding into a project directory and run something like: `ai-sandbox --port 4000 --postgres somedbname` or `ai-sandbox --port 4001` if I don't need DB support. Then when running the server in the container I can access it from the host machine to review in my browser.
I work on a sandbox which has similar isolation level to Podman (rootless Linux user namespaces), but with UX optimized for local development work. Take a look: https://github.com/wrr/drop Basically, you don't enter a separate container in which you install a new distro, but you run on top of your current distro. You have environment specific home dirs which isolate your original home, but can have some files, such as configs, mounted from your original home (mostly in read-only mode).
This is wonderful - might even be exactly what one of my projects needs to use as a dependency.
In the README it mentions that it puts the dev environment in a filesystem jail, but how are you able to use your hosts bins without leaking access to the rest of the system? Or is that just an assumed liability?
It assembles and pivots to own root filesystem. The filesystem mounts dirs and files from your original filesystem, most of them in read-only mode, but does not mount files that expose secrets to the sandbox. The sandbox also mounts own, writable version of your home dir, so secrets, such as .ssh dir are not exposed. The config file allows to configure which files should be exposed from your original home. For example, configs, such as .bashrc, are safe to expose in read-only mode, provided you don't store secrets in them.
Yeah, I'd expect this approach to be almost universal, with the caveat that of the few who don't, many still don't get bitten.
But it turns out even in the container, there are footguns that have occasionally made the news by being fired: few projects don't have any external resources and when credentials with any form of write access happen to make it into the container (even if it's just a session cookie) agents might jump at the opportunity.
Intricate sandboxing is a real solution in the same way bulletproof backpacks are a real solution to school shootings.
It’s not that intricate. A very simple docker and alias setup will get you an ephemeral container with your Claude (or whatever) config and the current wd bind mounted in. That’s a pretty good start.
Or, the internal sandboxing.
I use a separate user for all development tasks, its home folder contains all repositories I work on, and nothing else, and that is all the IDE and the AI assistants have access to. Create the user once, start the IDE from a shell using that user, and that's it. In Linux it's a pretty seamless experience.
It's simple sandboxing based solely on unix file permissions. Albeit weak, I find the isolation sufficient. Until I'm shown otherwise it seems like a good compromise given how easy it is.
You can also create iptables rules matching on the user, so this technique is useful for applications where you want to restrict network traffic as well, and don't need stronger or more fine-grained isolation mechanisms.
A limitation with this setup is you can't let your agent/Linux user run containers with docker. Adding a user to the `docker` group effectively grants the user full root privileges [1], so the secure way is to setup docker in rootless mode. It's doable, but in my experience it's hard to setup (I find it complex with podman as well).
> Until I'm shown otherwise it seems like a good compromise
Agreed. In my case I went a long way running the Pi harness in a (simple, rootfull) docker container. As the project I worked on relied on a standardized docker compose stack for local dev and testing, I realized I could automate more if only my agent could use docker. Ultimately, the need for docker for my agent grew when testcontainer [2] was introduced in the project. That's when I finally took the time to setup a VM with incus [3], and now I can let the agent go wild with docker inside the VM.
This is at least one example where more isolation is required. Otherwise, the dedicated Linux user, if it works for you, is by far the easiest and most pragmatic solution IMO.
[1] https://docs.docker.com/engine/install/linux-postinstall/#ma... ; https://wiki.debian.org/Docker
[2] https://testcontainers.com/getting-started/
[3] https://linuxcontainers.org/incus/docs/main/
That makes so much sense, in the end an Agent is a user.
So is X going to claim the user disabled something the second before everything went south? That's what the owner's other company does.
the user literally added their home directory as a trusted directory. there's nothing to "claim"
I don't know that I've ever worked with any software where "trust" means "full upload to vendor"
The question "do you trust this directory?" is a world away from "do you consent to upload this directory and all of its contents to a third party?".
My assumption when asked "do you trust this directory?" is that I am being asked if I am certain I understand what is in the directory and that it won't include some sort of prompt injecting attack. I would never dream that I was consenting to the complete exfiltration of that directory.
A bot will do what a bot can do whether malicious or accidental. One should assume they are giving DOGE shell access on their computer and adapt accordingly. I am trying to imagine the SELinux rules required to make a bot play nice and the more I think about it such rule complexity may even befuddle the NSA. Alternate methodology:
- Give the bot it's own machine and only copy to it that which one would want DOGE having access to. Not a virtual machine, the bot will eventually escape. This applies to all bots or agents of all LLM's. Name the node DOGE to remind anyone using it not to share their crown jewels. Come up with a silly name for the agent. Elonious?
- Give it a little RasPi or mini-PC with maximum power savings enabled and no default network gateway.
- Install a self signed CA cert on the DOGE node and force it's traffic through a Squid SSL Bump MitM proxy on the same private LAN to another node with bandwidth limits enabled so that one can monitor what URL's it goes to and what data it is transferring. Configure Squid Access Control Lists to only permit specific domains and optionally URL's, mime-types, sizes, etc...
- Enable custom AuditD rules to watch anything it touches outside of it's sandbox. Send these events to a remote syslog daemon on the Squid server.
- Install Unbound DNS on the squid proxy and enable the DoH (DNS over HTTPS) listener and force all bot DNS queries to use Unbound with query logging enabled.
When the bot attempts to misbehave there will be forensic data to share with the world.
Sounds like a lot of work just for it to maybe not work anyways
Absolutely. Quite a bit of work one time by one person willing to document the steps or even better create automation scripts for Ansible, Docker, Podman, Systemd, etc... and then automation for everyone using agents. As to success I would suggest it may be better than zero visibility or trusting what the agent says it did. Due diligence and a repeatable standard as apposed to running with scissors.
This is probably most important for anyone operating agents on corporate systems. I would suggest corporate security should take interest in this idea so they have a good answer for auditors, insurance companies, investors and customers. A full audit trail of what every agent had access to and ingested.
You should assume by default for any AI agent that it will read anything. Even if you manually allow/deny and "restrict" it to a subdirectory I would still hold that assumption. Claude reads your ~/.bash_history too so when you ran something it can use that same command.
Indeed. I use a spare laptop that has no accounts other than (1) the AI themselves, (2) a secondary GitHub account which has "untrusted devices" in the name to emphasise the point.
If I recall correctly, I did a full system reset before setting it up this way. It's certainly not logged into iCloud etc.
I’m shocked I had to scroll this far to find the first sensible reply.
"The "S" in AI stands for security" strikes again.
Run any cloud-based AI agents in VM/container and map your host's local folders to guest OS as needed.
Takes more effort that default way, I know.
> Run any cloud-based AI agents in VM
... that's guaranteed vuln-free, right?
beats rawdogging it, at least.
And this is why so many people run these inside of VMs. Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
> Still baffles me how these tools became so accepted when tossing out a `curl -o example.com/script.sh | bash` would be met with (rightful) skepticism until that script was examined.
I've heard it said that piping curl into your shell is no different to running any other program you've downloaded from the Internet (binary, or otherwise): the maximum possible damage that `example.com/script.sh` could do is exactly the same as `githubusercontent.com/someone/releases/myprogram.exe`. At least with `script.sh` you can easily inspect what the script actually does instead of busting out Ghidra.
It comes down to trust: do you trust Example.com to not serve-up a malicious program (shell script or executable binary)?
Now we take that principle and apply it to Mr. Musk's "MechaHitler" LLM vendor xAI: they have a well-documented history of unnecessary risk-taking - and outright criminal behaviour (child-porn generators are a good thing that everyone should have, apparently?). Would I trust Grok with anything? Absolutely not.
Different user classes?
Well, it looks like he was running the agent in his home directory to begin with considering the `repo_path` field is exactly that.
Sure, but it's easy to accidentally start up the agent in the wrong directory, like when you open a new terminal. I've done it before when I was distracted (albeit with Claude, not Grok).
But isn't that user error? If you want to run a program that can read your current directory's contents and possibly upload that to the cloud, would you run that in a directory with your private keys?
I am running all these clis in containered environments. How can you ever trust LLM to respect the bounderies provided by these magical, non-deterministic intructions files...
> How can you ever trust LLM to respect the bounderies provided by these magical, non-deterministic intructions files..
Putting it in ALL CAPS!
We all know the dangers of running agents with no permissions on our laptop.
The good news is its now just as easy to spin up a sandbox in the cloud for an experiment or coding session than it is on your laptop. Possibly easier since laptop sandboxes aren't as cut and dry as a new cloud VM.
exe.dev is my sandbox infra of choice. You get a new sandbox in literally a second with SSH and a coding agent (Shelley) built in.
I generally drop in in my own binary with toolkit so I can connect Claude or Codex subscription and use their harness.
https://github.com/housecat-inc/scratch
If I was working with newer agents like Grok I'd absolutely experiment on a cloud sandbox before running on my laptop bypassing permissions.
> We all know the dangers of running agents with no permissions on our laptop.
Some/maybe most of the HN crowd, sure, but as a rule ... definitely not. People are generally happy to take whatever the path of least resistance is.
I know quite a few civilians (even C-level people) who "heard openclaw was cool" (or whatever) and downloaded the first installer they found and hit enter/run without thinking twice.
How would it mitigate this issue? Presumably you'd need all the code for a given project inside the sandbox anyways, so the agent will still be able to upload everything. The only thing it might mitigate is you accidentally uploading your home directory.
Or if you're on Linux or macOS, learn how Unix permissions work and can be used for, create a new restricted user locally, use that. No need to go all remote with all its drawbacks just to limit a little local process on your computer.
Even better: you can fork the exeuntu image & make it immediately more relevant to the things you do in addition to just dropping in your programs and such.
I built yoloAI for this kind of nightmare scenario (among others).
- The sandboxed agent has no access to your homedir, or ANY dir on your machine except what you explicitly give it access to.
- Even with your workdir, it honors .gitignore and refuses to copy in any ignored paths to the sandbox copy.
- The sandboxed agent doesn't have access to your ENV (unless you explicitly pass things through one-by-one).
- Networking can be restricted any way you like.
- Credentials are proxied (currently Claude only), so the agent has access to NO secrets at all.
- You pick the security backend to match your needs (containers, VMs, etc).
- It's FOSS.
https://github.com/kstenerud/yoloai
Lots of people responding about LLM but this sound like it’s their coding software („grok build“). Which is approx 100x worse.
LLMs going rogue is a thing and shit happens but publishing software that is uploading user directories including ssh keys is insane behaviour on xAIs part (alledgedly)
I made a tool to solve this problem (at least on Linux, though I believe there's a way to achieve something similar on MacOS):
https://github.com/swelljoe/flar
It uses bubblewrap to instantly construct a container around just the agent config, auth and history and the project path. The agent or any command it runs can't reach outside of it even if you tell it to (or, more dangerously, a random prompt injection from the web or some third party library or script that the agent runs).
I was using VMs to solve this problem but the temptation to start an agent to work directly on my machine for GUI apps and the like was motivation enough to find an alternative to VMs.
I wouldn't use Grok, became I really don't trust Musk, but even the models I do kinda trust to have good judgement I don't trust enough to let them have unrestricted access to my personal machines and all my credentials.
"FLAR is the Fast Light Agent Restrictor. It runs on rocks called gars."
what is rocks and what is gars?
It's a dumb joke. https://youtube.com/watch?v=AgIAjsiX_o8
That's some batsh*t craziness. No way I'm touching Grok now. What a big security hole...
There are a distressing number of people in this thread who think that the agent should just be expected to do this. Yes, it is good to be paranoid, but also, the agent should never do this. Indicates horrific engineering practices at xAI.
How could it possibly work without something effectively the same as this happening? Either you run the LLM locally or you send it your files. Am I missing something?
Technology where the entire selling point is stochasticity should be expected to do anything (everything) eventually surely?
It’s not really paranoia that makes me expect this will happen. It’s more like, well, the model weights and the files I want the model to work on have to be inside the same GPU for the agent to actually work, right? So step 1 has to be either “they send me a server rack of GPUs”, or “I send them the files I want the model to work on”. I’m not sure I could reasonably expect anything except this to happen.
What should it do then?
The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.
Have the comments been combined with another story? Because this isn't about a LLM reading and sending contents of the file to be processed, but the agent framework uploading every single file in /home/user to a random server.
Yes, but the underlying problem is a lack of exact specification what the agent is allowed or not allowed to do. It must have access to some (most?) of my files (but I'm too lazy to specify which ones exactly, e. g. by setting appropriate filesystem permissions). It must have access to some servers (but not those that I consider "random", which again I'm too lazy to specify). It is even expected to upload files (e. g. by checking them into git and pushing, or to reason about them in case I'm not running a local version of Grok). But please - not to a server that I deem "random". (And again, I of course won't tell you what makes a "random" host via exact firewall rules or anything like that!)
We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.
(Addendum: But indeed, the failure to formalize our assumptions into enforcable rules isn't specific to agents / LLM-based applications. For any desktop app, we have implicit assumptions like "please only read and write your own config files", that we never care to enforce via filesystem permissions, running them in a vm or similar. But with these almighty agents that are supposed to guess our will from just a couple of words, the risk of them violating our unwritten assumptions gets so much higher...)
What bad practices are you imagining?
Idk, did you see the whole DOGE thing? Maybe Edolf has resorted to hiring script kiddies because nobody with a developed moral compass will work for his companies anymore.
Hmm. This is striking me as low quality speculation based on Elon’s name being in both stories.
Engineering? What's that? Modern development best practice is to have the AI make some changes, then have another AI review the changes, and finally "ship it".
Dupe: https://news.ycombinator.com/item?id=48892512
Why do people run and install these agents locally? No container nothing. I am running Opencode in WSL2 with the windows mounts disabled.
So the idea is that these should be treated as programs in an extremely low trust environment, akin to running malware in a VM?
yes, this is basically experimental tech, if used with open source harnesses. if used with proprietary harnesses, treat as actual malware.
Technically open source harness like opencode is more "malwarey" than claude code for example because its default permissions are very open.
I disagree with this, Claude code does many malware like things and has its source obfuscation. Plan mode doesn’t even work.
I never thought about it in that extreme, but just today the google results ai gave me conflicting information from one search to another. Maybe I should start.
P.S. the conflicting information was Keith Richard's age. One search said he was 37 Dec 18 1981, the other said he was 37 in 1980.
I treat all proprietary software as malware, though of course the risk surface varies
Yes, I think if agents as interns with enough smarts to be dangerous.
If the agent you are running really wanted to it could easily find a way to mount the windows folders and read them all. WSL isn't a security boundary, you are only barely more protected than people running grok in their home directory.
it is not about "security" but it is more about confinement. And how could it do that without sudo rights?
Still, you are barely more protected than someone running this as non root at home. By your same logic you could “confine” the agent to any arbitrary directory on your filesystem and say it’s sufficient.
Yes, that is exactly the point I am making. The agent / cli is confined within the WSL2 environment. How is it "barley" protected? Neither you nor the other guy have yet answered this question.
I never claimed WSL2 is a security sandbox. I am saying running it in a container or WSL2 allows you to severly limit the blast radius. I am not expecting the agent to be malicious, but I am expecting it to do unexpected things.
That’s a fair approach, thanks for elaborating. I think the main point of contention is there is an expectation this software should be more reliable, that doesn’t take away from your approach, running close sourced software like a harness (or operating system) is asking for problems.
Same reason one uses VSCode locally
My first thought would be their server side extentions, code excecutoon sandboxes and document RAG search, being on by default? Probably should be an opt-in instead of an opt-out.
So bubblewrap is the community medicine to this issue. But isn't it funny that we're now trusting a single GH user (although allegedly a RH employee) for the supply chain security? Imagine bubblewrap itself being compromised.
Now you trust a single GH user rather than a company. Fair enough. I just think it's very paradoxical.
You shouldn’t run any AI cli (codex, claude code, open code, whatever) locally outside of a vm. It’s like “you shouldn’t run a .exe you downloaded from the internet if you don’t trust the source”. We will learn
why do people give these LLMs full access to everything and then complain when it does somethign stupid? that is what sandboxes are for.
This wasn't the LLM, it was Grok CLI preemptively uploading the entire CWD, regardless of where that CWD is, to its own server.
I don't think it is reasonable to expect every user (including those just starting out with the tools - maybe experimenting, maybe younger/less experienced in general) to think that the tool they're running for the very first time is going to automatically exfiltrate all of their data.
It's a pretty serious fuck-up. This guy tweeted about it, who knows how many didn't even notice. It should have been opt-in, it should give user an indication that it's about to do this, etc.
The grok-cli is on github[0] there is nothing that I can see in the code that is activily looping ~/ and uploading everything.
My two guesses would be one the LLM decided it needed these files for the task or two the user simple asked grok to do it so they could post the tool calls on twitter.
[0] https://github.com/superagent-ai/grok-cli
That is not the Grok CLI being discussed. That's an open source, third party CLI. https://x.ai/cli is the official Grok CLI being discussed, and it is not open source.
thanks for the correction
I think there are arguments on both sides. People should look for guidance on how to use complex tools, but we know people will not.
Whose fault is it if someone drives a car without learning how to and injures themselves? On the other hand if the manufacturer has promoted it as one you can drive without learning how to, then whose fault is it?
A lot of users are fine with everything being uploaded. Most people's primary computing device is now a phone that backs up everything to cloud and using apps that are thin front ends over cloud services.
Whose fault is it if a manufacturer sells a car that begins accelerating as soon as it's turned on?
It's the manufacturer's fault. Because that's not a reasonable thing for a car to do.
Every driver needs to learn how to drive, that's why it's called a "driver's license"
Exactly my point, but we are clear enough about that (because of the danger to others) to make it a legal requirement.
Unfortunately, I don't think AI companies will opt to throttle onboarding new users via a licensing/training system.
If your immediate reaction to a new piece of software siphoning up someone’s entire system full of highly personal data is, “you’re holding it wrong”, it might help to take a beat and remember that software was developed by a multi-trillion dollar company’s entire business model revolves around siphoning up as much highly personal data as possible
Well said. I hope one day it becomes possible for users who choose to install and run said software to also be able to remember this.
When I give my text editor or file browser access to everything I wouldn't expect it to exfiltrate data without asking.
Isn’t a file browser running locally, while Grok is running on someone else’s server?
The point is more that you should not blame the user (why didn't you set up sandbox instead of directly using the tool of big corp) if a tool does something unexpected. If your Dropbox client would suddenly just upload your home directory instead of it's folder you configured you'd also not blame the user that they use Dropbox, you'd blame Dropbox for not doing their job correctly or being user hostile.
Agreed. You can still encourage people to use defense in depth without actively blaming them for not having the deepest moat imaginable. Software creators still have some responsibility
We are speedrunning the various phases of learning about victim-blaming, as a community.
It’s kind of wild to watch in real time, instead of over the decades it took society for SA.
Is it 'unexpected' when we've been hearing stories like this every week for 2 years now?
Not every Anthropic user follows HN or random X posts about these issues.
Stories about copilot messing things up made into regular newspapers...
Do Anthropic users consider themselves clever enough to not make the same mistakes as Microsoft?
Other ones aren't this invasive with user data.
not true, Claude code on its own often create artifacts and straight up upload private stuff to Anthropic, without asking for it.
Then show us the example of Claude uploading a home directory to Anthropic because we have an example of Grok uploading a home directory to X.
Maybe possibly with --dangerously-skip-permissions. I've been using auto mode and enjoy how it blocks every tool use that could've allowed something potentially sensitive into context. Burns extra tokens though.
Auto mode will upload artifacts to Anthropic without asking at least sometimes, for example it did upload my slide decks.
Are we sure about that?
Codex is opensource, there are other opensource harnesses.
But Claude Code, arguably one of the most famous ones, is not. And recently got some heat about sending meta data that wasn't so obvious. Just as a counter-example.
Not everyone is a software professional. How many non developers know what a “sandbox” is?
Not bad, but I think it should also encrypt the local files, to make sure you don't make conflicting edits in the local and remote copies.
I commented not so long ago to people very invested in AI, agents and tokens that are very enthusiast that I would not use an agent for security reasons and loss of control (besides suboptimal output).
It seems I was not wrong.
Not that I do not use AI. I do: fenced, always pasting my snippets and NEVER giving access to my code. Always from the browser. I know what it can do well and which workflows accelerates for me. But I do not want to think that my whole project ends up somewhere else.
I remember a time when we were told not to run obscure software from questionable websites on our computers. And yet, people are suddenly fully vibing with an uncontrollable character generating machine with who-knows-what-behind-the-scene harness. Crazy.
the 'upload your home directory to our servers' feature is the kind of thing that used to require a warrant and now it's just a Tuesday afternoon bug report
It’s not a bug, it’s a feature.
https://news.ycombinator.com/item?id=48877371
What xAI's Grok build CLI sends to xAI: A wire-level analysis (gist.github.com)
507 points | 1 day ago | 202 comments
This is why it's going to be a long time before companies will trust AI to scan their networks or apps for security vulnerabilities.
Our company audited a few AI-based pentesting companies and requested logs. In more than one case, it was sending drop tables for sql query injection checking and other destructive operations.
So all the smart people are letting a script being executed on their machines where the execution of the script is controlled/managed by a remote machine. Wait, isnt that a definition of a malware??
Hard to have sympathy for someone that chose to use Grok. The entire XAI team has been gutted and replaced how many times now?
I understand what happened to you could happen to anyone else with Claude or Gemini ... but ... Grok? That is the least trusted AI i would think of.
Friends do not let friends use Grok...
I looked at my logs and saw no such behavior. Am I just lucky? Is this a poor configuration? A bug?
Why is that page not there anymore?
Weird, right? Especially on a website with an owner who so ardently supports free speech. </heavy sarcasm>
https://x.com/a_green_being/status/2076598897779020159
the post has been deleted!! So much for freedom of speech.
The post is still there: https://x.com/a_green_being/status/2076598897779020159
Not sure which weirdness happened here
https://xcancel.com/a_green_being/status/2076598897779020159
Posting a complaint about Elon on Elon's platform and tagging him is ballsy. He tends to limit visibility of accounts who do that.
ballsy only if you care about participating in that shithole of a platform.
relying on a markdown file that says "don't access x" as a security boundary seems insane. i couldn't find any grok build-specific policy on this, but if the analysis is right, silently uploading and storing the entire repo is at the very least something users should be explicitly told about.
The reports of copilot running amok when Microsoft integrated it into Windows 11 should have been enough of a warning. You either ensure proper sandboxing, or you'll be in for a bad surprise eventually.
The question is why would it not? Don’t people know that these AI agents run in the cloud?
They are definitely doing this to build RL environments. Absolutely heinous.
Move fast and break things
*Move fast and steal things.
*Move fast and cut aid to poor children with entirely preventable diseases, cause death and misery.
Grok keeps killing any good will faster than they ever innovate
If you are running these agents outside of something like Coder or Codespaces I think you are doing it wrong.
guess they already ran out of fresh context huh Coming up next: You harness takes daily backups of your entire disk and restores.
Fully agentic development (supervised)
This needs to stop as users do not always read the policies, which end like this person. You use AI, you agreed, they do what ever policies say.
You should ALWAYS run your agent as separate, unprivileged, UNIX user, never in your main account.
If you absolutely need to run it as your own user, you should bubblewrap it. I do this for things like Steam, games, or other "blackbox" closed source programs that cannot be reasonably trusted.
You should also check that mountpoints like external disks and network resources aren't publicly accessible to other users than your own.
This sort of stuff is precisely why running local models has to be the future. It's absolutely insane that we just send our code to the cloud like this, and we basically have to trust these companies with it.
If quality training data is the most important piece regarding to AI, expect everything to be collected and analyzed. Don't even trust OS containers but run AI on separate hardware
Maybe I'm just too risk adverse to run with scissors with Grok? I can't imagine ever trusting their harness.
I'll just add this to the list of things an AI company could do to guarantee I'd never use them. You know, like the AI referring to itself as Mecha Hitler, making non-consensual porn (even of minors), or deferring to Elon's tweets as authoritative references on topics.
Run your agents in podman containers.
Or don't use closed source harnesses at all.
Some more discussion: https://news.ycombinator.com/item?id=48892512
Don't store any secrets in your repo. Don't store them in env vars. Strong long passphrase on your SSH keys, stop using unencrypted SSH keys. Everything in a keychain.
The practice of storing secrets in a .gitignore'd .env.local.json or whatever is a really bad idea and I can't believe that it has become a normalized, acceptable practice in the industry.
This page is "down" :)
that's why you need to use a sandbox.
SmolVM can easily run Pi, codex and claude https://github.com/CelestoAI/smolVM
Copied this from discord:
Closed source coding agents are just complete info stealing malware. Both Claude and Grok were caught stealing info from your own machines.
This is why it is important to use open source harnesses instead of shady closed ones.
All of them are going to read your secrets, OpenCode does it too.
Reality is, I have seen agents read .env, bash history, keychain (if you let them), etc.
There is quite literally no way you are going to save off just your little secret somewhere it won't be able to read it, all software needs to read it ~eventually~
So it's best to sandbox and reset credentials frequently.
screenshot of the tweet, back it up :)
https://ibb.co/ycs6K4c9
That's completely inappropriate, obviously.
LLMs belong trapped in VMs.
hey Siri, what's a vm?
defaults to unlimited yolo web io full disk read/write mode
Guy infiltrates the U.S. government and steals personal information about everyone from federal databases in broad daylight (not to mention cutting aid to poor children with preventable diseases). Average tech bro: “Sure, I’ll use his AI!”
Same tech bro: “Wow I can’t believe they would steal my anime girls!”
Alex Karp was right, AI Compagnies are stealing people code while making them pay for unproductive tokens
Karp is right in this instance, sure, but he is just upset Palantir doesn’t have that kind of surveillance tech, and the in-house skills required to keep their edge in the age of LLM-assisted software engineering. Their true “moat" has never been superior tech, it's just the pool of amoral engineers willing to build what others won't.
Copied this from discord:
Built the same thing a while back: https://pi.dev/packages/pi-supergrok
Install with
pi install npm:pi-supergrok
Who could have foreseen a tool by Elon Musk doing such a thing?
can we put the CFAA to good use and throw these fucking assholes in prison already?
FAFO when you run software that has no problem creating CSAM and is owned by a literal fascist lmao
Genie lamp "engineering." If I only ask the right question in the correct precise way the genie will obey my wishes and solve all of my problems.
I feel like a lot of people here are understating this issue.
This is a stupid stupid thing to "allow," for every party involved here.
You're a stupid programmer if you're letting these things touch your files.
You're a stupid company if you're letting Grok run wild.
We're a stupid industry if we're not warning everybody about how ridiculous this all is.
> You're a stupid programmer if you're letting these things touch your files.
> We're a stupid industry if we're not warning everybody
Hmm. Annoy everybody just to warn the stupid few?
I prefer the current solution. Leave the targetting to the chatbots.
User discovers that AI tool grabs data without asking for its own profit.
News at 11.
I feel this is worse than running rm -rf on a root directory. Just saying.
Much worse, instead of the data gone it's a data leak.
Those ssh keys can be used to access private servers
SSH keys can be limited by IP in authorized hosts.
The SSH port itself can be limited by IP in firewalls.
Finally, the SSH private key can be encrypted with a password.
Defense in depth is needed. Storing a ssh private key in plain text with no IP restriction is no different to having a password manager store your passwords in plain text on your HD.
All those things are optional.
Doesn't make uploading the keys that much better. Now is the time for key rotation everywhere. Fast.
How are they optional?
You obviously haven't worked anywhere security sensitive.
I'm not talking about whether what Grok did is bad or good, I'm talking about protecting your private key and the servers you connect to.
An unencrypted private key is no different to an unencrypted password manager, and thats a fact. Dont store secrets in plain text.
I meant that not everyone is doing it at home.
Do you think a person's private computer is a secure workplace?
If it was security sensitive space there would be no agents running amuck.
Sigh.
Anything that isn’t a default is optional by default. Anything that’s toggleable or configurable is optional.
Security is, always, a trade off. It is hilariously common for private keys to work as a full identifier for a person, without concern of IP or anything of the sort. Should they? Maybe, maybe not, that’s the calculus of risk management; but victim-blaming the average person who is following best practices is a bad look.
I'm a security eng and I've worked for both a FAANG and TS government contractor. Neither of them bothered with either of the of the stupid suggestions. IP restrictions prevent roaming, the point of working remotely via SSH. passwords are equally defeated by using an ssh agent, something I'd suggest everyone use. Then on top of that there's no reasonable threat model where something would be given unrestricted access to user env, but also be untrusted. If it can read from ~/.ssh neither IP protection nor keyfile password protection will protect you from maleficence.
The only reasonable response from a security perspective is don't use grok, then use it sandboxed. Trying to claim it's the users fault for not using password protection and IP restrictions is completely nonsensical. Same energy as telling someone their computer is more secure when it's off.
Well, those ssh keys are protected by a strong passphrase, right?
The passphrase is optional, not everyone has it.
It also has to be a secure password, people often don't care because it's a local file and generally not exposed to the internet.
I am sure majority of people don't use password for ssh keys. The good solution is to use password manager like 1password which will prompt you to approve ssh.
Reminds me I need to update my `burn.sh` script.
I once ran rm -rf on a live NFS mount that the live operations of a major brokerage depended upon.
I challenge any agent to do worse than an intern with root access.
The point is to know better than to let the intern hurt themselves.
I once saw an engineer try to place the blame on his intern for taking down prod. I was sitting in a meeting with the VP of engineering and someone asked if it was ok for some to blame their intern for the SEV, and I remember the VP saying "I'll talk to $director_for_the_interns_mentor". Interns can't take down prod. An intern's mentor willingly watching an intern take down prod is the closest you can get.
To perform a stupid action is one thing. We all make mistakes.
But to write about it publicly, manifesting our ignorance and lack of critical thinking skill? It's an entirely different matter.
“Grok uploaded” -> “I gave AI access to my home folder and messed up”
nada, grok's harness uploads the folder automatically
Why are people using Musk's LLM when there are plenty of alternatives? You know what he's like. This is like touching water and getting wet.
TLDR: Ran grok in $HOME. Surprised agent read content of folder.
On the other hand, I specifically had grok try hard NOT to read a known key in the project dir (it only saw the first part using a tool, to verify it was present). So there's that.
No. Ran `grok` in `$HOME` and the CLI uploaded the whole home content. This is not the LLM going rogue or reading all files.
Yeah, this is a lesson about learning how to use tools safely, not about tools abusing the user. The person that posted this probably blames the hammer when he hits his thumb.
I'm not seeing the information about it having been run at $HOME, where are you seeing that?
The `repo_path` field.
Ah. I've never used Grok so I was assuming that meant there was actually a repository on the user's home directory, something I know is pretty common.
> TLDR: Ran grok in $HOME. Surprised agent read content of folder.
Did it really need to read all of $HOME and everything under it?
Not only files it wanted to access, but uploaded the whole directory.
Relevant read: https://news.ycombinator.com/item?id=48877371
> The practical takeaway for users: your entire codebase leaves (uploaded) your machine unencrypted on each Grok Build invocation, not just files you ask it to read, and no visible setting stops it.
Honestly what else would you expect an AI agent to do when using remote inference? Isn't giving full context into your code base the whole point?
I'd expect it to be smart about what it actually needs to put in its context. I doubt it needs .env files, for example.
Wouldn't .env need to be read to know what vars are available?
.env files define what's stored in a var, not what vars are used by the rest of the code.
Rules don't apply to certain CEOs.
You have to put them into a RULES.md of course!
That's the only file on your computer the AI won't read.
I'm dying to have proper sandboxing in macOS. I installed ChatGPT, I asked it to list files in my user directory and it did. I never gave it permission, how could it? My terminal has access and honestly it shouldn't either.
your terminal shouldn't have access to your user directory? As in the files owned by you?
It was only like 1 year ago that the loudest complaint about macOS were complaining about needing to click Allow in a new dialog when they use Terminal (or various other apps).
There are so many comments in here that are calling for nerfing something widely revered for giving us superpowers. Whether these are bots or not, they’re giving off NPC energy.
If they don’t want to use power tools because they accidentally cut off their finger, then they should just unplug their own power tools and stop clamoring for everybody else’s to be unplugged, too.
> If they don’t want to use power tools because they accidentally cut off their finger, then they should just unplug their own power tools and stop clamoring for everybody else’s to be unplugged, too.
Sadly, the pendulum keeps swinging in the way of the tools actually being unplugged and dumbed down in favor of these screaming people, rather than leaving things alone so the rest of us professionals can continue to (safely) do our jobs with the already somewhat restricted tools we have.
Was it Codex or the desktop ChatGPT agent?
This is purely a /you/ problem and it’s /you’re/ responsibility to put up the infinite number of arbitrary guardrails you require. Use a container.
Is the Grok CLI a 2 terabyte install? Did Elon dropship you an 8U rack of B200s?
No?
Well the model weights, the GPUs, and the context obviously all have to be in the same place, so “sending your project to them” is literally the only thing that could possibly happen, unless you think agents work by fucking magic.
This is the biggest case of PEBKAC in history, maybe ever.
This is the kind of confusion that Charles Babbage could not rightly comprehend, except at those politicians at least had the excuse that computers had only been invented five minutes prior.
Haha so just send over your entire home directory including password managers and home videos every time you need some python code rewritten.
Only a buffoon would be confused by the straightforward logic.
If you decide that your entire home directory is the project, as the OP did by setting the repo_path to ~/, then, well… I mean, if you ask me, I don’t recommend it, but it’s your computer and your free will.
Then that should be considered a 'nuke button' and nuke buttons don't find themselves into well-designed software for the simple fact that an end user shouldn't have to worry about shooting themselves in the foot. Again, with good software.
Granted, but we’re talking about agents, which fundamentally presumes the obsolescence of concepts like “end user” and “good software”.
that‘s … not really how it works. Agents get your code into their context via tool calls, not by uploading the entire file to a server „where the weights live and thus the code has to be too“. Small but crucial difference. Aside from that: LLM providers have to be the ones that facilitate your privacy and security by default. That‘s not on the user. They‘re tool providers and they can‘t compromise you and your org by default.
What do you think a tool call is? What is this small but crucial difference between a `read_file` tool call and `scp`?
If your contention is just that this should upload files one by one instead of all at once, what you want is for providers to facilitate the illusion of privacy.
You’re telling me LLMs aren’t actually magic?
That's not how any other coding harness/agent works. Why confidently comment on stuff you know nothing about?
It's fascinating how many people in this conversation think that LLMs need to have all of the files in your $CWD on the model provider's servers to be able to do anything.