As much as I like Claude Code, Boris has done a lot of harm by encouraging software engineering practices that lead to slopware. We have two camps of people at work, the first camp are the agent goes brrr. They don't understand the code they write. They have loops running, agent orchestrators or agent hype du jour. The second camp is people who are inundated with PRs, are holding the line on quality, and just exhausted. We've also had some management pressures where they think people are wasting time looking at code. Perhaps because some podcast they might be listening to, somebody says coding is largely solved.
> I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.
This is going to be a net negative on software quality for people who take this up, in my opinion.
I call out Boris but I also don't think he's being malicious. He's at the center of an important technological revolution and it would be hard not to get excited. I just wished he advocated for a more balanced and a realistic perspective.
Reading through the thread, it's striking how many people are feeling the same mix of excitement and exhaustion you describe. I'm in that camp too... the tools are incredible, but the pace and expectations around them can feel overwhelming.
I fully agree with what you say regarding Boris, but I would emphasize that I don't think he has malicious intention either. He still is doing his job, to showcase the features their product offers.
I think at the end of this we'll have a new software engineering paradigm.
Mostly nobody now worry about binaries or instructions because those are for the compilers, even undefined behavior are mostly ignored.
You can either tailor the development pattern for LLM, or have LLM come write for the same old development pattern. I think there is going to be a difference.
We will have a new software engineering paradigm and it is called quitting to become a day drinker.
Burnout can be so all-encompassing that it can ruin your life. It can make you feel like you are losing your mind and your basic abilities. A lot of that is coming for a lot of people who are currently stressed but excited.
Yes, I am exhausted. Most of my company is obsessed with agents, because everyone wants to be seen as AI first. There is little thought going into usage. No care for long term maintainability and quality. Our product is actively worse by many metrics, but no one cares because we marketing can say “agents”.
The sad part is that this technology is incredible. It’s us choosing to turn it into a slop cannon (and the labs sure seem to encourage this).
> I call out Boris but I also don't think he's being malicious.
From a market perspective, he's acting completely rationally in his own interests. Bottom line is that these companies need to do whatever they can to keep growing token consumption because that's their goal.
If the nation's drinking skyrocketed, we wouldn't be sitting here wondering why the CEO of Budweiser isn't advocating for temperance. His job is to move kegs, just like Boris' job is to move tokens.
I never understood this perspective. Just because a person's behavior is market-rational, it does not mean they can't be criticized for externalities.
That is, in fact, an important thing to do. It turns those externalities into public perception, which turns into market forces that adjust the behavior, if you want to think purely in market terms.
The analogy with Budweiser is not a good one. This would be the CEO of Budweiser actively pushing more drinking while the nation's drinking was increasing. And yes, people would be right, and effective, to oppose this (see Oxycontin).
As soon as you open up the externalities discussion, the wider question of increasing electricity prices and turbocharging global warming comes up, not to mention RAM prices. AI is a machine for turning negative externalities into stock prices.
FWIW, the CEO doesn't seem to be the CEO? If I understand what I read properly, he has one direct report — his "chief of staff". So arguably the true chief executive in a functional sense (the person to whom the organisation ultimately reports through hierarchy) is his sister, who appears to be grounded in reality.
I suspect that the reason Anthropic is generating such developer-negative, toxic, insensitive influencer sentiments like these from Boris has to do with Dario being a fantasy-fiction-reading quasi-mascot who thinks it's his job to tell scary stories, who is being allowed to do just that by a sibling who perhaps prefers he's not involved in the day-to-day.
Respectfully disagree. From my view, the tech industry hasn't behaved in a way that regards long-term interests over short-term interests in a very, very long time. Much of the innovation is simply finding new and creative ways to shrink this loop even further, and vibe-coding/slop is just the latest manifestation of that.
Quoting the creator of CC holds little value in my opinion. I too call my product good.
> opting out of this fully machine-driven future may not be an option.
I am contemplating whether I want to stay inside this rat race.
I completely agree with the conclusion of this blog post, by the way. I feel uneasy, and I do not enjoy the work I deliver using LLMs. I think OP did a really good job on capturing at least my current state.
I and my friends go back and forth, every day, on whether coding with LLMs is a net plus or a net negative.
I'm at the point where I think it's dumb to not do it but also dumb to do it. I have no real answer.
I have settled on using LLMs for everything but to spend more time honing the quality and cleanliness with LLM passes afterwards than I generally would have taken to write it well myself in the first place. This is in some ways the worst of both worlds, but it somehow lets me bypass akrasia while still getting pretty good code out, so I consider it superior to how I worked before. I get more done in three months even if I get less done in a day.
I am with you here but don't get overly pessimistic: devising hooks and stopgaps and flows and constantly tuning what to watch out for does not only improve the quality of the LLM-output code. It hones and refines your own abilities.
CC has made some pretty dumb stuff in my projects but I don't resent those occurrences. They taught me (more accurately: reminded me, because I already knew but was not applying that knowledge too often) very valuable lessons on code quality -- that's still a dark area to this day and every ray of light on it is valuable for the future programming.
To me programming with LLMs made me a better programmer. But yes, I don't just rubber-stamp PRs.
It also finally allowed me to be less of a code monkey and more of an architect and a backend lead than before. Which I was really missing.
I'm still trying to figure that out. Something diplomacy-adjacent would be interesting to me; maybe conflict studies, or international relations.
Based on word from my friends who work in the field, a lot of it is people who have a lot of respect for the field, and a lot of professional respect for eachother. It's also a field I feel is unlikely to suffer from the same kind of scams that are taking over software while still offering an engaging environment.
It's also work with real-world impact, which is nice, though obviously comes with its challenges.
> I am contemplating whether I want to stay inside this rat race.
Same. I'm currently trying to find _my next thing_ and all anyone wants to talk about is how I'm using AI and it's absolutely maddening. It's become a lazy, lossy proxy for productivity. I've had a few intros for the types of orchestration engineering roles which are described in this post and they're just completely unappealing -- especially the prescriptive aspects. Like, the sort of JDs I'm seeing are variants of, "we want a back-end developer who has experience with XYZ but they must use agentic harnesses to do their work." Why does any serious person give a flying fuck how the end result is reached? The flip side of all this is that rates are also being driven through the floor by loop cowboys who are generating steaming piles of shit which are _good enough_ ... until they aren't. I'm being completely serious when I say that stocking shelves at Tractor Supply is becoming more appealing by the day and I also just thought to myself, "Maybe I should just join the Army while they'll still take me?"
> I feel uneasy, and I do not enjoy the work I deliver using LLMs.
I have basically stopped writing code in my spare time since the advent of AI. Before I felt like I was working on a classic car. Was it a practical use of my time? No. I could go out and download software that did what I wanted. Did I have fun doing it? Yes, the act of working on it was important, I felt I was still learning and improving as I did.
Nowadays I see people doing far more in a month than I could in a year and I feel like its all a waste, like I just spent the past few years transcribing a phonebook while standing next to a photocopier.
I don't know if that'll ever change. I can't even pretend I was doing something prestigious and artisan like watchmaking because I wasn't a good programmer beforehand.
Before I would just throw prompts at the LLM and it'd end up building a pile of crap (but semi-working crap, and 100x faster than I ever could) - it was pretty depressing. Using tools like `grill-me` (or `grill-with-docs`) I feel like I'm actually building my understanding of the system and helping shape it, and the results are much better.
The fun part about that `grill-me` command is that when the questions are over, I've found that I can go right into implementation without needing to dump a PRD or some sort of broken up plan. Now this is obviously completely predicated on what you are asking it to grill you on. But for tasks that are semi complicated, it's fantastic.
I used to think I'll be into coding for the long haul, contributing to open source, and working on multi-year side projects.
Nearly all of that passion vanished this year, and I've been struggling to replace it. I know I'm much better than the machine now, but the lines are starting to blur, and some of the small puzzles of day-to-day have been completely automated away.
We've birthed a lot of puzzle solvers that enjoyed programming, and I'm sure many of them will move on to something else that scratches the same itch. I'm keen on learning what that will turn out to be.
I was misunderstood you if you intend to write code by hand, I still did, I use AI to learn by example, but I write the real code myself, AI can help me improve the code. I learned a lot.
I'm the opposite, couldn't be bothered to work on code outside of work. Barely did at work because I was more focused on wrangling a small army of shitty contractors (thanks strategic partner initiative for firing all of our small shop contractors and replacing them with morons from "offshore").
Now with LLMs I find myself doing small projects that interest me or have some utility for me outside of work, and doing a lot more development in the codebases at work outside of just review/docs/arch than I was before. Also making small tools that I find pleasant/useful but were not important enough to spend time on before.
Agreed - there was always a set of things I wanted to do that I knew the magic core for, but wanted a team of implementers for the curft, the 100k of actual testing harnesses, hyperparameter exploration, etc.. . I now have that team of implementers. All the problems seem research-y though - optimal binary transport systems that are zero-copy and compatible with languages, fast physical simulation optimizers, etc etc... So, things that all had a _LOT_ of busywork around the magic core.
I am so over this. I cannot take anyone seriously that claims inevitability of their ideas, and how you must adopt them without "being left behind". If these tools are so good and so capable the result should be able to speak for themselves rather than this FOMO inducing, emotional language.
I couldn't agree more. Thus far I'm still objectively more productive than all of the AI enthusiasts I've worked with. I think a lot of the activity with these tools is coming from people who just enjoy using them more than they enjoyed coding. They feel more productive not because they are producing more but because they are producing somewhat less with much effort. It takes them roughly the same amount of time even if it changes the distribution of time spent on each task.
> and in recent weeks it has started to dominate the Twitter discourse.
As a general rule, I don't waste my time with the advice of people who still think Twitter is a source of wisdom.
You're being uncharitable. I don't read it as intentionally FOMO inducing. I read it as the exhausted sigh of resignation from someone who sees where the wind is blowing whether they like it or not. I see it as someone watching tech management and execs listening rapt as Boris pours the poison of AI maximalism in their ears. I read it as someone who sees developers around them either drinking from that same poisoned well or bowing under the pressure from those leaders to adopt AI or lose their livelihoods.
It is true that the author is incorrect: you can certainly opt out, but you won't be opting out of AI, you'll be opting out of the industry.
"the industry" is not some monolith, and treating it as such is no productive. There are all types of software and many ways in which it is created. If the companies that are "AI enabled" are so much better we should see some big changes soon. But I'm still waiting for products I use from "AI enabled" companies to start churning out features at unprecedented speed.
Loops work when you spend the proper amount of time to understand what you want ahead of time. The prerequisite is clarity — enough clarity that you could write a careful specification that you could hand off to a junior colleague.
Often, it takes 5-6 broken crappy versions of a thing until you understand that. There is no accelerating the 5-6 broken crappy versions - there’s no agent tech that’s going to help your meat brain avoid thinking time.
So most of my time is iterating between these two phases: I don’t understand what I want, I need to read and write and play with code, okay it’s been long enough I think I know what I want (it is extremely easy to deceive yourself) … okay now I do actually know what I want and I can write a loop.
Many people think they can jump ahead with agents. You cannot fake understanding or clarity. It is painfully obviously when someone skipped that meat brain understanding phase.
I had codex write a tool to extract all my pi sessions. (Had to filter out my prompts from the agents talking to subagents).
Then I had it analyze the patterns i was making and turned that into the flowchart for the outer guidance-creating-prompt.
I didn't have to spend too much time thinking what i wanted. I wanted it to do that.
The result is still mixed, and i'm not trusting it with delicate code bases, but for a game i've been building i dropped my check-in time to 1/5th i was previously spending on it.
Thats not a good thing per-se. I'm sure i'm missing good ideas by _not_ spending time with it. But previously I really had stagnated with my prompts becoming mechanical #now-do-this and #now-review-that with 90% of its suggestions being correct.
Just need to (automatically) remind it to "do the hard stuff first, clean up & refactor as you go" as well as a "reflect on your work" after its first return to get it to spill the beans on any crap left behind, and then process that in the guidance-creating-prompt to dish out new work.
> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.
Context-smithing can help to a degree and cyclomatic-like complexity rules tend to make matters worse. So, you either roll up your sleeves or close your eyes and hope for the best. I've had limited success with the latter.
I've run into the issue a lot; I know it happens. I handled it manually for a while by just having a fresh instance inspect the code - "review this for DRY violations" and "how would you re-write this into a global architecture instead of a bunch of local code".
Eventually the list ended up long enough that I've got an agent that handles it. You've just got to treat "write code that works" and "write elegant code" as two separate tasks - either a fresh instance or an Agent will work
>Yet even with a lot of manual steering, that type of code does not come out of LLMs naturally, and even if the code comes out naturally like that, they will still attempt to handle now impossible errors.
This is something I’ve struggled to fight against in many PR reviews. Especially once already written, convincing someone that their excessive null checking is harmful is an uphill battle. Short of better modeling (and languages that allow for sum types to enable it), I haven’t been able to come up with a universally convincing argument against this kind of “shotgun parsing.”
Maybe it really just isn’t that big of a deal? But when actually reading through and refactoring a codebase I’ve always found it frustrating to manage these unnecessary checks. Sometimes they’re nearly impossible to delete safely once present without first adding some kind of logging or broad investigation.
And AI code reviews encourage overly delusional defensive paranoia. triple null checking deep inside a function is technically a real risk, but in practice should never be hit because you've checked for nulls in every function that calls or could call the function in question and is thus not necessarily worth guarding against.
Part of the problem is that models don't have a strong sense of taste, part of the problem is that the context in which projects exist is incompletely represented in the LLM context, and part of the problem is that LLMs tend to be myopic.
The lack of taste can be mitigated to some degree by improved training, though taste is not a stationary distribution in humans (see trends/fads/etc), we can at least better track the cutting edge. I think this area still has low hanging fruit but frontier labs are more concerned with being able to solve problems than the style of the solution right now (for evidence of this just look at the Opus 4.5 -> 4.8 arc).
The problem of incomplete context is partly a human problem and partly a harness/interconnectivity problem.
LLM Myopia is a harder problem to solve just by virtue training models on question/answer pairs. Countering this requires emphasizing RL on solution paths rather than just prompt/response, which is doable but harder.
> the right fix is not "handle every malformed case." ... [LLMs] will still attempt to handle now impossible errors.
This is the number one code smell from LLMs and I don't know why they are so obsessed with it. In python, it often comes as `hasattr` checks on types that are defined to have that attribute, in a code base that is fully type-checked.
Why do they do that? Is it from pre-training or re-enforcement? If that latter, can the labs please fix this?
I suspect it's mostly the training data. I am also on team "make illegal states unrepresentable". It may get talked about a lot on HN, but I'm still at the point that I'm surprised when I see a code base that I didn't write in the wild that does a really good job of it, either open source or at work. Most programmers still think in terms of picking up pieces and fixing errors at the point where the error message pops out rather than making it so the error can't happen and the data reflects that.
I say "mostly" because I think there's also a problem with AIs thinking this way in their current state. That last level of human understanding of a code base, where the human holistically understands the flow of those guarantees, is a challenge to give them right now. On the raw code level, this sort of thing often involves enough code to easily blow out their context window. Trying to summarize it in memories-style files has its own problems; just because there is text written down about the guarantees doesn't mean that the AI is going to get the right info out of it, any more than a human might from just reading the code. I won't say it's "impossible" to give an AI this understanding because I'm not sure it is, but it is a level of understanding of the code that even if you get them to have it, their practices tend to fight against it.
My own solution to this problem has largely been to give up on them getting this. I prompt a solution to the problem the way that most people do, then if I want to make bad illegal states unrepresentable I prompt the AI through the process of the necessary refactorings, unless it's so small that I just do it myself. Given a lot of code that uses maps/dicts and arrays and strings and ints, if you prompt it through making those more thoroughly typed, it's actually pretty good at it. I've not had a lot of luck getting good designs out of single prompts, even when I get detailed. Treating it as two separate tasks seems to work out well.
And watch the diffs on the types carefully; AI loves to sneak past a ".JustSetItAndIgnoreAllThePreAndPostConditions(string)" method. After all, I suspect there's plenty of training data of "types that are nicely structured to make error states unrepresentable and then a later maintainer came along and added a 'JustEffingDoIt' method that broke everything" in the field. One of the best defenses is to make sure that the type implementing these things is in its own file and you can easily look at all the methods it adds on those types and smack it when it does that. I've tried slathering warnings about not doing this and explaining the pre- and post-conditions being maintained in the docs but the change seems marginal.
Sorry to say but the solution is to stop using python. The models are trained to code defensively assuming historically representative python codebases. The models trust the types a lot more in languages where the canonical historical examples trust the types because the language is constructed around that premise.
It’s because it matches the patterns they are trained to follow. They don’t understand the code. They can’t reason about the actual logic flow. They can only work with patterns.
I keep thinking about at which point I should not force myself into the loop. As a developer I really like working on the code structure, making it clearer, thinking about good abstraction, breaking into modules, etc. I really take pleasure in it. At the same time I understand that at some point I am becoming the limiting factor.
If the point of the software is benefit people, should I still care about how the code looks.
Right now, I still think that the answer is yes, but in 3 years? in 10 years?
> My current status is that I have not had much success with this way of working for code I deeply care about
If something is judgement heavy, "code i care deeply about", then i don't really agree with the direction of travel here. Don't try to delegate decisions you care deeply about.
I do like the framing of agent loop vs harness loop, but only delegate stuff that you can accurately specify in advance, that usually means stuff that's repeatable in my case ("hey go see how i did X, do that but for Y"), and that inherently means stuff that's predictable.
For stuff where lack of my judgement as input is just going to cause me to say "no", we're down to collaborating in the "agent loop" as Armin puts it. And that's totally fine. It's fast, but also safe.
Remember before AI coding assistants, sometimes you'd get an engineer join your team who was SUPER productive, your peers would be jealous "oh yeah but you guys only got all that done because you have X on your team!" - they didn't live the curse of having that kind of person around - if you don't have them PERFECTLY aligned, then they run off at break neck speed in the wrong direction.
I'm a software developer from way back, using tools and languages that coding agents are far less familiar with.
So when I use an agent to write code, it's in languages I'm less familiar with, and often using libraries I know nothing about.
All to say, my part of the process often ends up being:
1. "Here's what I'm looking for, in detail"
2. "That's not right. Here's one way it's not right, and a specific example. Please fix that."
3. Sometimes I give suggestions for how what is going wrong might be happening, or conceptually how to work around the issue.
4. And iterate on 2-3 until the result is close enough.
This is really terrible advice right now for most people.
I've had to rip out a lot of pretty terrible code made by engineers who have tried this.
I don't disagree that eventually, "loops" when combined with unlimited tokens and amazing models in the hands of people who know how to set them up right will be amazing. But for the typical Claude Code user, it's disaster.
The problem is not that loops write bad code once. Humans do that too. The problem is that loops apply local pressure repeatedly: add a fallback, add a guard, special-case the failing input, quiet the exception, satisfy the test. Over time that selects for code that is more survivable in the short term but less intelligible in the long term.
We used a “loop” before it was called that to drive MS-DOC support into Tritium. Based on that experience, I take issue with this:
“There are already impressive examples of large automatic porting efforts, including the reported work around moving parts of Bun from Zig to Rust.” (Emphasis added.)
It will be impressive if/when the Bun team is able to pick up and continue extending and supporting Bun. For us, MS-DOC remains read-only and probably perpetually buggy until we reimplement with a better understanding. Until then, it’s definitely not “impressive”. Functional? Maybe. Impressive, no.
In my own ham-fisted experiments with coding loops, one pathology I have noticed is that the LOC just spirals out of control. That's likely because of the layers of defensive fixes, etc., that get built. That inevitably causes context bloat (or at least navigational friction) and results in quality decline.
I wonder how many loop-related issues could be addressed by simply fixing a LOC budget, or assigning a cost in some way. Unclear how you would dial in the right numbers, though.
The issue is that whilst the loops will initially lead to good results they will be less and less as context gets bigger and bigger and tougher to understand for human and AI.
We've had great success with agents thus far at my job. A year into Clauding and all our dev metrics are up while our downtime has remained steady.
Being an iOS engineer, much of my engineering cycle these days is going from Figma/PRD → spec → code. After being handed off to QA, we handle the bugs and product slips as they come through, while we simultaneously build/spec the upcoming addition. This is basically the same agile style that's been popular for 20y, just super-powered with agents.
How might someone accomplish the same goals using loops instead?
I personally have not had good luck with loops due to similar issues as the post author - but if you were to port your flow to "looping" it would be something like:
- An automation that periodically checks for PRD's at a given location that have not yet been implemented.
- If it sees one not implemented, it puts a lock on it (so other agents later don't pick it up while its still working) and implements the PRD in code, assuming it has the figma link and all specs required.
- When its done it makes a PR, waits for if it passes and even in some cases automatically merges into your staging/preview enironments and just pings you with a build/URL. You can then leave feedback or something and it can also also poll for pending feedback. Or you just mark it looks good, the agent then merges the PR, moves the PRD to implemented status, maybe even writes/updates docs and cleans up any temporary work.
- Repeat checking for new PRD's every T unit time. (10 minutes, 1 hour, etc)
This is how people say you should be looping - you never even cared or looked at the code, and also never prompted the agent yourself.
But I find most agents are often pretty bad still at replicating UI vs making something from scratch and most design specs are still not as detailed around how things look at all sizes, in all scenarios etc. Design seems to be one of those things that still requires a human to validate. And then all the things the post author mentions about it not being willing to apply hard constraints, minimize impossible states, validate at edges and prevent horrendous overchecking of things. etc.
Use appium or XCTest or swift testing; generate the tests first (failing) from the spec.
The loop is basically then a while loop:
While (tests fail) { trigger agent: spec, failures list }
for bugs, write failing tests.
Its basically TDD.
Loops do nothing useful beyond making the “spec -> code” step more “hands off” and let you be confident that the code you write does what is intended.
Obviously you see the issue: writing the loop harness is > effort than not having it…
…but the idea is that you run “spec first” and are totally hands off on the code, just updating the validation step and then waiting while the agent iterates over and over to solve for some solution that passes the loop harness.
People suggest that it is possible to go, eg. directly figma/jira to harness via (random tool here), saving even more time and invoking even fewer humans, but thats currently, as far as I can tell, actually just hype.
No one is actually doing that effectively.
Loops are currently carefully hand crafted, which makes them tedious and of questionable value imo.
Would you have a breakdown of costs/benefit? Can you say with certainty that this workflow has increased productivity so much that you are seeing profit increases that you wouldn't have otherwise noticed just by hiring more people?
Asking with no ill intention, I just crave for actual business cases that make sense, and yet no-one seems to be able to reliably produce that.
Generally interesting reflections here, yet I see the same kind of myopia and fatalism that is rampant in our (fashion) industry:
> yet I have no doubts that this looping future is going to be our future despite the fact that I presently resent it
Why would anyone concluded this? LLMs are just one kind of application of MLs to software production. There is a vast solution space for automating parts of software production. The idea that slop loops are the inevitable future because they happen to be accelerating output at the moment just seems profoundly short-sighted and lacking in vision.
These new AI trends are very tiresome, very similar to 2021 crypto mania - both trigger a lot of FOMO. If we have loops that write code and we don't need to verify anything, why are the devs still here? What's point of even learning this new trick as a dev if you truly believe that this can be used without any intervention? If loops work then it follows that a loop of loop works too - why hire any people at all? Just run a bunch of loops and build a profitable business, but then what's your moat? Any person can now launch loops on top of loops.
The post suggests fear about a surge of increasing amounts of code by loops and loops of agents.
I don’t know if I like the current world without it though.
80% of different teams code the code is poorly tested. The code doesn’t handle data consistency or asynchronous code properly because the engineers don’t know better (and frankly don’t care enough).
Dependency handling is poorly managed leading to low quality operations with improper dashboards, alarms, and ops.
Badly managed processes leads to people doing monkey work signing off checklists rather than automation.
Frankly… why is keeping any of that good? It really pisses me off seeing people accept any of that low quality but that standard is the default and not the outlier.
I think this is a common sentiment among heavy users of AI that also still cares about code quality.
I've built up a skill harness and review flow that makes Opus generate slop-free code 90% of the time. But the remaining 10% requires me to stay at the helm. Especially in the early stages.
I would love to use loops to automate more, but I couldn't do it with the current generation models.
And on the back of my mind I'm still evaluating the possible future where we are forced to API pricing. I'm currently paying $400 for Opus, and use around 1.5-2 billion tokens per day. This will cost around $20k/m with API pricing. And I don't want to even imagine the possible scenario of getting locked out of frontier models because of politics.
Will the models get better to cut me out of the loop completely? I believe so.
Will the open source models catch up tho SOTA models, and diversify from China-only? I hope so. Otherwise 2 superpowers will wield a soft power that can cripple the tech industries of all other countries.
This is a very fatalistic take. While I understand where it's coming from, I try not to share the same mindset: engineers getting increasingly distant from how things are getting built is not something that will "undoubtedly happen, whether we like it or not".
Also:
> Now there is obviously a question if this desire to understand the code is one that I will still have a few years from now.
I do not think we should be having doubts like this. Either you consider understanding the code you ship and allowing your future self to be able to work on the system you're building to be a value, or you don't. I, for one, do, and I do not think using LLMs and coding agents will affect my point of view on that.
I think there's 2 important, but separate, ideas in this post:
- Models are not good at or getting better at creating strong invariants, which his fundamental to good software
- It is unclear how to keep tabs on what the agent is doing, so you, a human, can intervene.
These are related, obviously: one of the highest-leverage things you can do is force you agent to use a strong, minimal set of types or data invariants or other constraints. They get much better when your codebase broadly supports this!
I do suspect they're separable, though.
If you had the right levers and visibility, you should be able to get the model to produce code that doesn't feel like slop. But every time I've had a model try to keep me in the loop, it inundates me with irrelevant decisions and busywork. Its inability to see what's structurally important still shows up, just differently.
[If the models get better at defining and respecting invariants, maybe there's a new flavor of slop, that's less obvious today.]
I honestly wonder if this kind of stuff really brings something to the table. Like I use opus for sometime and certainly I can put it to good use and optimize some parts of my day to day job (programmer). But it fails so hard in such simple tasks that it seems to me that putting it in loop can't just magically make everything better, unassisted. Does anyone actually uses agents and loops to create new software, new technology? Has anyone created with those systems, software they couldn't produce otherwise technologically wise? Or is it at best just an accelerator, cutting off on the building time?
A friendly reminder to just do 9 to 5 and touch lots of grass. None of this shit represents industry trends, majority of people still use chat interfaces and copy blocks of code. There’s zero early adopter advantage here, only FOMO and lots of anxiety.
> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.
It’s almost as though these models were trained on a vast corpus of largely mediocre code. They will never outperform the median Github user - it is all they know, it is all they can do.
(I mean, OK, so I can't just write that, because of some rule or another. But really, just no, at this point, surely? How is it we are so actively turning software development into some weird blend of cargo cults, homeopathy, prosperity gospel and penal treadmills? Chasing trend after trend from corporate influencers with motivated reasoning who are trying to create new cultures that entrench their weird inscrutable metered intelligence taps? Handing over this level of control to and for the benefit of the sociopathic or solipsistic executives of two companies that cannot demonstrate they will ever make a profit, backed by a handful of other companies who are stirring money around in a pot to make it look like they are still generating value? Can we stop? Please? Please?)
No.
(And also no: it was not, in fact, this bad before. No matter how you try to retcon it. It was not good. But it was also not this insanely nihilistic)
Theres a deep insight in this post about the value of looping for throw away code to explore a problem space, rather than brute force a problem by just applying more tokens and hoping.
The more I play in this space, the more I’m drawn to the idea that some kind of back tracking constraint solver is a better solution than then the current naive while loop / brute force approach here.
The results I see are similar to what you get from a greedy brute force constraint solver; solves trivial problems, sometimes solves harder problems after a long time, takes too long to solve really hard problems; solutions are increasingly non optimal on average as complexity goes up.
We have so much existing knowledge about building good constraint solvers, if we could just figure out how to apply it here somehow.
Show me the billion dollar solopreneur startup, or the profit increase for companies and at that point I’ll start thinking that this tasteless high level wanking might make sense in some way
One car went Mach 1, ever, apparently. Anyway, I don’t think the analogy fits. Ford or whoever didn’t loudly and frequently predict Mach 1 cars, right?
The situation is more like: Altman & co are predicting their new car will replace all vehicles: horses, trains, planes, motorcycles, there’s a real possibility the concept of vehicles will not exist other than cars, in the future. Meanwhile it hasn’t really done highway speeds yet. It does some impressive runs on curated tracks, and people use it around their farms (it seems to work ok for some of them).
Yes, one car did Mach 1. And the first production car, the Benz Velo, could only go 12mph. It's an apt analogy.
As I mentioned to OP, applying future aspirations to the current space is incorrect. Some people are able to understand the progression of industrial automation, some people aren't. But if you look at the current batch of frontier models and say, "I just don't see how this is going to be useful", then you're in the camp of those in the 80's who didn't understand personal computers, or in the 90's who didn't understand the web. In hindsight, the technologies evolved massively and found routine use cases that no one initially predicted.
It is a terrible analogy that shows terrible thinking.
After all, there's one thing we can bet with more confidence on: delegating thinking to this mediocrity machines is affecting the ability to do the same in scores and scores of previously smart people.
Terrible thinking? Delegating thinking to mediocrity machines? "Previously" smart people? I don't think you can project your own limited and malformed understanding of the world any harder. Please go touch grass and learn how to engage with others on the internet without being so negative.
No, you mentioned his discussion of the "billion dollar one person startup", which we can both agree is a fanciful idea and more of an eventual "possibility" that will of course not occur as once anyone can be a billionaire, the whole system is going to change.
However, your "tasteless high level wanking" is not a quote from Altman, it's a vague and directionless insult that manages to sweep quite a lot of legitimate discussion about the future of automation and professional work under its thumb.
It's wrong because you're saying, "where are the billion-dollar one-man startups?" in the same way that I might look at a Benz Velo and go, "But it's so slow! Horses can go faster than that! Everyone saying cars are going to change the fabric of society are just tasteless wankers!"
The point is that you are applying future aspirations on the present-day relatively brand-new model space and getting upset that we aren't there yet.
> For now I have not moved past the point of comprehension being important to me.
Ah ! This is me too... at least for what I have to ship at work. Not so much for my toy/weekend projects. But it turns out agents are also good at explaining.
I think it's insane to suggest that software developers should ever get to the point where they don't even comprehend their code.
Before someone else says it, no I don't read the assembly code that is produced by my compilers. However, I can generally predict what kind of assembly will be produced, and the result is deterministic unlike LLMs. It seems like most vibe coders scoff at the idea of even looking at the code, and it just seems untenable to me when we're working with (usually correct) stochastic parrots.
I'm willing to be persuaded otherwise: Looping seems to (currently) be a side effect of token subsidies.
If token costs are nil, then you can afford to run verification and generation through the same models. If token costs are high, then you will go broke verifying code sprawl.
Currently costs are (mostly) absent from the conversation, even though costs are what decide the limits which shape experience.
Also: Firms can be held liable for the products they sell, so if code cannot be reviewed then that code is essentially a law suit waiting to happen. I believe this is what customers will be demanding in the future: someone to hold accountable when things go wrong.
This commit seems to mostly be grammar fixes? If someone used a spell/grammar checker it might produce a diff similar to this. Why does the fact it was Claude and not Microsoft Word or other matter in this case?
There's _way_ more than one way to do "loops". I just asked one of my superviors/auditors to document how it's been working while monitoring a few other agents that have long-term goals:
Yeah I don't know. Don't get me wrong, the article points makes sense. But sometimes I think that we're going to stay near this current point of productivity for a little while.
Currently my org of 8 people use around 1000 euro worth of tokens per month. We've recently had a discussion near the water-cooler, that if the cost climbs 5x-10x it may be just more worth it to get more developers (we're EU based). While the tools work and are definitely nice, even in our little org with our little budget, using Opus 4.8 we've noticed code quality going down.
If I had to bet money, I'd bet that the models will get 30-50% more nice, around 2x more expensive and we will settle into some mode where we'll use llms for some tasks, manually doing others and calling places focusing on speed at any cost some funny name like "gulags, 996, sweatshops, etc" and collectively try to somewhat avoid those places, which will need to offer a premium to attract talent. Wishful thinking.
The way I answer this question is, I don't because my boss is saying to basically accept whatever LLMs spit out which is fine by me because I have explained the risks of doing this to our only product and it was shrugged off. I am both disgusted and excused and just take the salary.
As much as I like Claude Code, Boris has done a lot of harm by encouraging software engineering practices that lead to slopware. We have two camps of people at work, the first camp are the agent goes brrr. They don't understand the code they write. They have loops running, agent orchestrators or agent hype du jour. The second camp is people who are inundated with PRs, are holding the line on quality, and just exhausted. We've also had some management pressures where they think people are wasting time looking at code. Perhaps because some podcast they might be listening to, somebody says coding is largely solved.
> I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.
This is going to be a net negative on software quality for people who take this up, in my opinion.
I call out Boris but I also don't think he's being malicious. He's at the center of an important technological revolution and it would be hard not to get excited. I just wished he advocated for a more balanced and a realistic perspective.
This sums up the dynamic: https://x.com/danhockenmaier/status/2021617680525172840
Right, which basically means that judgment compounds. AI just makes the slope steeper in both directions.
Reading through the thread, it's striking how many people are feeling the same mix of excitement and exhaustion you describe. I'm in that camp too... the tools are incredible, but the pace and expectations around them can feel overwhelming.
I fully agree with what you say regarding Boris, but I would emphasize that I don't think he has malicious intention either. He still is doing his job, to showcase the features their product offers.
I think at the end of this we'll have a new software engineering paradigm.
Mostly nobody now worry about binaries or instructions because those are for the compilers, even undefined behavior are mostly ignored.
You can either tailor the development pattern for LLM, or have LLM come write for the same old development pattern. I think there is going to be a difference.
We will have a new software engineering paradigm and it is called quitting to become a day drinker.
Burnout can be so all-encompassing that it can ruin your life. It can make you feel like you are losing your mind and your basic abilities. A lot of that is coming for a lot of people who are currently stressed but excited.
Yes, I am exhausted. Most of my company is obsessed with agents, because everyone wants to be seen as AI first. There is little thought going into usage. No care for long term maintainability and quality. Our product is actively worse by many metrics, but no one cares because we marketing can say “agents”.
The sad part is that this technology is incredible. It’s us choosing to turn it into a slop cannon (and the labs sure seem to encourage this).
I want to leave the industry as soon as I can.
> I call out Boris but I also don't think he's being malicious.
From a market perspective, he's acting completely rationally in his own interests. Bottom line is that these companies need to do whatever they can to keep growing token consumption because that's their goal.
If the nation's drinking skyrocketed, we wouldn't be sitting here wondering why the CEO of Budweiser isn't advocating for temperance. His job is to move kegs, just like Boris' job is to move tokens.
I never understood this perspective. Just because a person's behavior is market-rational, it does not mean they can't be criticized for externalities.
That is, in fact, an important thing to do. It turns those externalities into public perception, which turns into market forces that adjust the behavior, if you want to think purely in market terms.
The analogy with Budweiser is not a good one. This would be the CEO of Budweiser actively pushing more drinking while the nation's drinking was increasing. And yes, people would be right, and effective, to oppose this (see Oxycontin).
> can't be criticized for externalities
As soon as you open up the externalities discussion, the wider question of increasing electricity prices and turbocharging global warming comes up, not to mention RAM prices. AI is a machine for turning negative externalities into stock prices.
His job is to do what’s in the long term interests of his company.
[Edit: was thinking of the ‘CEO’. This doesn’t apply as cleanly to Boris.]
FWIW, the CEO doesn't seem to be the CEO? If I understand what I read properly, he has one direct report — his "chief of staff". So arguably the true chief executive in a functional sense (the person to whom the organisation ultimately reports through hierarchy) is his sister, who appears to be grounded in reality.
I suspect that the reason Anthropic is generating such developer-negative, toxic, insensitive influencer sentiments like these from Boris has to do with Dario being a fantasy-fiction-reading quasi-mascot who thinks it's his job to tell scary stories, who is being allowed to do just that by a sibling who perhaps prefers he's not involved in the day-to-day.
Respectfully disagree. From my view, the tech industry hasn't behaved in a way that regards long-term interests over short-term interests in a very, very long time. Much of the innovation is simply finding new and creative ways to shrink this loop even further, and vibe-coding/slop is just the latest manifestation of that.
> we wouldn't be sitting here wondering why the CEO of Budweiser isn't advocating for temperance
But we would start to wonder if the CEO of Budweiser started advocating binge drinking.
> This is going to be a net negative on software quality for people who take this up, in my opinion.
The silver lining appears to be that long term most people won't be able to afford producing slop at current rates.
Like Mario tried to do... But no one listened
Quoting the creator of CC holds little value in my opinion. I too call my product good.
> opting out of this fully machine-driven future may not be an option.
I am contemplating whether I want to stay inside this rat race.
I completely agree with the conclusion of this blog post, by the way. I feel uneasy, and I do not enjoy the work I deliver using LLMs. I think OP did a really good job on capturing at least my current state.
I and my friends go back and forth, every day, on whether coding with LLMs is a net plus or a net negative.
I'm at the point where I think it's dumb to not do it but also dumb to do it. I have no real answer.
I have settled on using LLMs for everything but to spend more time honing the quality and cleanliness with LLM passes afterwards than I generally would have taken to write it well myself in the first place. This is in some ways the worst of both worlds, but it somehow lets me bypass akrasia while still getting pretty good code out, so I consider it superior to how I worked before. I get more done in three months even if I get less done in a day.
I am with you here but don't get overly pessimistic: devising hooks and stopgaps and flows and constantly tuning what to watch out for does not only improve the quality of the LLM-output code. It hones and refines your own abilities.
CC has made some pretty dumb stuff in my projects but I don't resent those occurrences. They taught me (more accurately: reminded me, because I already knew but was not applying that knowledge too often) very valuable lessons on code quality -- that's still a dark area to this day and every ray of light on it is valuable for the future programming.
To me programming with LLMs made me a better programmer. But yes, I don't just rubber-stamp PRs.
It also finally allowed me to be less of a code monkey and more of an architect and a backend lead than before. Which I was really missing.
I'm not enthusiastic about the field anymore, which sucks, because I used to love working in programming.
What are you going to do? Asking for a friend.
I'm still trying to figure that out. Something diplomacy-adjacent would be interesting to me; maybe conflict studies, or international relations.
Based on word from my friends who work in the field, a lot of it is people who have a lot of respect for the field, and a lot of professional respect for eachother. It's also a field I feel is unlikely to suffer from the same kind of scams that are taking over software while still offering an engaging environment.
It's also work with real-world impact, which is nice, though obviously comes with its challenges.
> I am contemplating whether I want to stay inside this rat race.
Same. I'm currently trying to find _my next thing_ and all anyone wants to talk about is how I'm using AI and it's absolutely maddening. It's become a lazy, lossy proxy for productivity. I've had a few intros for the types of orchestration engineering roles which are described in this post and they're just completely unappealing -- especially the prescriptive aspects. Like, the sort of JDs I'm seeing are variants of, "we want a back-end developer who has experience with XYZ but they must use agentic harnesses to do their work." Why does any serious person give a flying fuck how the end result is reached? The flip side of all this is that rates are also being driven through the floor by loop cowboys who are generating steaming piles of shit which are _good enough_ ... until they aren't. I'm being completely serious when I say that stocking shelves at Tractor Supply is becoming more appealing by the day and I also just thought to myself, "Maybe I should just join the Army while they'll still take me?"
> I feel uneasy, and I do not enjoy the work I deliver using LLMs.
I have basically stopped writing code in my spare time since the advent of AI. Before I felt like I was working on a classic car. Was it a practical use of my time? No. I could go out and download software that did what I wanted. Did I have fun doing it? Yes, the act of working on it was important, I felt I was still learning and improving as I did.
Nowadays I see people doing far more in a month than I could in a year and I feel like its all a waste, like I just spent the past few years transcribing a phonebook while standing next to a photocopier.
I don't know if that'll ever change. I can't even pretend I was doing something prestigious and artisan like watchmaking because I wasn't a good programmer beforehand.
This piece changed how I work with LLMs and made me much more optimistic about how "fun" it can be to work with them: https://nolanlawson.com/2026/05/25/using-ai-to-write-better-...
Before I would just throw prompts at the LLM and it'd end up building a pile of crap (but semi-working crap, and 100x faster than I ever could) - it was pretty depressing. Using tools like `grill-me` (or `grill-with-docs`) I feel like I'm actually building my understanding of the system and helping shape it, and the results are much better.
The fun part about that `grill-me` command is that when the questions are over, I've found that I can go right into implementation without needing to dump a PRD or some sort of broken up plan. Now this is obviously completely predicated on what you are asking it to grill you on. But for tasks that are semi complicated, it's fantastic.
I used to think I'll be into coding for the long haul, contributing to open source, and working on multi-year side projects.
Nearly all of that passion vanished this year, and I've been struggling to replace it. I know I'm much better than the machine now, but the lines are starting to blur, and some of the small puzzles of day-to-day have been completely automated away.
We've birthed a lot of puzzle solvers that enjoyed programming, and I'm sure many of them will move on to something else that scratches the same itch. I'm keen on learning what that will turn out to be.
Be your customer, write the software just for you, AI is so effective that you could do something meaningful for you just in spare tine.
Here is the similar perspective: https://isene.org/2026/05/Audience-of-One-Numbers.html
I was misunderstood you if you intend to write code by hand, I still did, I use AI to learn by example, but I write the real code myself, AI can help me improve the code. I learned a lot.
I'm the opposite, couldn't be bothered to work on code outside of work. Barely did at work because I was more focused on wrangling a small army of shitty contractors (thanks strategic partner initiative for firing all of our small shop contractors and replacing them with morons from "offshore").
Now with LLMs I find myself doing small projects that interest me or have some utility for me outside of work, and doing a lot more development in the codebases at work outside of just review/docs/arch than I was before. Also making small tools that I find pleasant/useful but were not important enough to spend time on before.
Agreed - there was always a set of things I wanted to do that I knew the magic core for, but wanted a team of implementers for the curft, the 100k of actual testing harnesses, hyperparameter exploration, etc.. . I now have that team of implementers. All the problems seem research-y though - optimal binary transport systems that are zero-copy and compatible with languages, fast physical simulation optimizers, etc etc... So, things that all had a _LOT_ of busywork around the magic core.
> You Cannot Quite Opt Out
I am so over this. I cannot take anyone seriously that claims inevitability of their ideas, and how you must adopt them without "being left behind". If these tools are so good and so capable the result should be able to speak for themselves rather than this FOMO inducing, emotional language.
I couldn't agree more. Thus far I'm still objectively more productive than all of the AI enthusiasts I've worked with. I think a lot of the activity with these tools is coming from people who just enjoy using them more than they enjoyed coding. They feel more productive not because they are producing more but because they are producing somewhat less with much effort. It takes them roughly the same amount of time even if it changes the distribution of time spent on each task.
> and in recent weeks it has started to dominate the Twitter discourse.
As a general rule, I don't waste my time with the advice of people who still think Twitter is a source of wisdom.
In my experience, some language like this is the result of witnessing it speak for itself.
You're being uncharitable. I don't read it as intentionally FOMO inducing. I read it as the exhausted sigh of resignation from someone who sees where the wind is blowing whether they like it or not. I see it as someone watching tech management and execs listening rapt as Boris pours the poison of AI maximalism in their ears. I read it as someone who sees developers around them either drinking from that same poisoned well or bowing under the pressure from those leaders to adopt AI or lose their livelihoods.
It is true that the author is incorrect: you can certainly opt out, but you won't be opting out of AI, you'll be opting out of the industry.
"the industry" is not some monolith, and treating it as such is no productive. There are all types of software and many ways in which it is created. If the companies that are "AI enabled" are so much better we should see some big changes soon. But I'm still waiting for products I use from "AI enabled" companies to start churning out features at unprecedented speed.
Loops work when you spend the proper amount of time to understand what you want ahead of time. The prerequisite is clarity — enough clarity that you could write a careful specification that you could hand off to a junior colleague.
Often, it takes 5-6 broken crappy versions of a thing until you understand that. There is no accelerating the 5-6 broken crappy versions - there’s no agent tech that’s going to help your meat brain avoid thinking time.
So most of my time is iterating between these two phases: I don’t understand what I want, I need to read and write and play with code, okay it’s been long enough I think I know what I want (it is extremely easy to deceive yourself) … okay now I do actually know what I want and I can write a loop.
Many people think they can jump ahead with agents. You cannot fake understanding or clarity. It is painfully obviously when someone skipped that meat brain understanding phase.
I had codex write a tool to extract all my pi sessions. (Had to filter out my prompts from the agents talking to subagents).
Then I had it analyze the patterns i was making and turned that into the flowchart for the outer guidance-creating-prompt.
I didn't have to spend too much time thinking what i wanted. I wanted it to do that.
The result is still mixed, and i'm not trusting it with delicate code bases, but for a game i've been building i dropped my check-in time to 1/5th i was previously spending on it.
Thats not a good thing per-se. I'm sure i'm missing good ideas by _not_ spending time with it. But previously I really had stagnated with my prompts becoming mechanical #now-do-this and #now-review-that with 90% of its suggestions being correct.
Just need to (automatically) remind it to "do the hard stuff first, clean up & refactor as you go" as well as a "reflect on your work" after its first return to get it to spill the beans on any crap left behind, and then process that in the guidance-creating-prompt to dish out new work.
This.
> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.
Context-smithing can help to a degree and cyclomatic-like complexity rules tend to make matters worse. So, you either roll up your sleeves or close your eyes and hope for the best. I've had limited success with the latter.
I was really surprised by this part!
I've run into the issue a lot; I know it happens. I handled it manually for a while by just having a fresh instance inspect the code - "review this for DRY violations" and "how would you re-write this into a global architecture instead of a bunch of local code".
Eventually the list ended up long enough that I've got an agent that handles it. You've just got to treat "write code that works" and "write elegant code" as two separate tasks - either a fresh instance or an Agent will work
>Yet even with a lot of manual steering, that type of code does not come out of LLMs naturally, and even if the code comes out naturally like that, they will still attempt to handle now impossible errors.
This is something I’ve struggled to fight against in many PR reviews. Especially once already written, convincing someone that their excessive null checking is harmful is an uphill battle. Short of better modeling (and languages that allow for sum types to enable it), I haven’t been able to come up with a universally convincing argument against this kind of “shotgun parsing.”
Maybe it really just isn’t that big of a deal? But when actually reading through and refactoring a codebase I’ve always found it frustrating to manage these unnecessary checks. Sometimes they’re nearly impossible to delete safely once present without first adding some kind of logging or broad investigation.
And AI code reviews encourage overly delusional defensive paranoia. triple null checking deep inside a function is technically a real risk, but in practice should never be hit because you've checked for nulls in every function that calls or could call the function in question and is thus not necessarily worth guarding against.
Part of the problem is that models don't have a strong sense of taste, part of the problem is that the context in which projects exist is incompletely represented in the LLM context, and part of the problem is that LLMs tend to be myopic.
The lack of taste can be mitigated to some degree by improved training, though taste is not a stationary distribution in humans (see trends/fads/etc), we can at least better track the cutting edge. I think this area still has low hanging fruit but frontier labs are more concerned with being able to solve problems than the style of the solution right now (for evidence of this just look at the Opus 4.5 -> 4.8 arc).
The problem of incomplete context is partly a human problem and partly a harness/interconnectivity problem.
LLM Myopia is a harder problem to solve just by virtue training models on question/answer pairs. Countering this requires emphasizing RL on solution paths rather than just prompt/response, which is doable but harder.
> the right fix is not "handle every malformed case." ... [LLMs] will still attempt to handle now impossible errors.
This is the number one code smell from LLMs and I don't know why they are so obsessed with it. In python, it often comes as `hasattr` checks on types that are defined to have that attribute, in a code base that is fully type-checked.
Why do they do that? Is it from pre-training or re-enforcement? If that latter, can the labs please fix this?
Likely just that they err on the unnecessary error handling than missing error handling. They likely penalize runtime errors harshly in the training
I suspect it's mostly the training data. I am also on team "make illegal states unrepresentable". It may get talked about a lot on HN, but I'm still at the point that I'm surprised when I see a code base that I didn't write in the wild that does a really good job of it, either open source or at work. Most programmers still think in terms of picking up pieces and fixing errors at the point where the error message pops out rather than making it so the error can't happen and the data reflects that.
I say "mostly" because I think there's also a problem with AIs thinking this way in their current state. That last level of human understanding of a code base, where the human holistically understands the flow of those guarantees, is a challenge to give them right now. On the raw code level, this sort of thing often involves enough code to easily blow out their context window. Trying to summarize it in memories-style files has its own problems; just because there is text written down about the guarantees doesn't mean that the AI is going to get the right info out of it, any more than a human might from just reading the code. I won't say it's "impossible" to give an AI this understanding because I'm not sure it is, but it is a level of understanding of the code that even if you get them to have it, their practices tend to fight against it.
My own solution to this problem has largely been to give up on them getting this. I prompt a solution to the problem the way that most people do, then if I want to make bad illegal states unrepresentable I prompt the AI through the process of the necessary refactorings, unless it's so small that I just do it myself. Given a lot of code that uses maps/dicts and arrays and strings and ints, if you prompt it through making those more thoroughly typed, it's actually pretty good at it. I've not had a lot of luck getting good designs out of single prompts, even when I get detailed. Treating it as two separate tasks seems to work out well.
And watch the diffs on the types carefully; AI loves to sneak past a ".JustSetItAndIgnoreAllThePreAndPostConditions(string)" method. After all, I suspect there's plenty of training data of "types that are nicely structured to make error states unrepresentable and then a later maintainer came along and added a 'JustEffingDoIt' method that broke everything" in the field. One of the best defenses is to make sure that the type implementing these things is in its own file and you can easily look at all the methods it adds on those types and smack it when it does that. I've tried slathering warnings about not doing this and explaining the pre- and post-conditions being maintained in the docs but the change seems marginal.
Sorry to say but the solution is to stop using python. The models are trained to code defensively assuming historically representative python codebases. The models trust the types a lot more in languages where the canonical historical examples trust the types because the language is constructed around that premise.
It’s because it matches the patterns they are trained to follow. They don’t understand the code. They can’t reason about the actual logic flow. They can only work with patterns.
I keep thinking about at which point I should not force myself into the loop. As a developer I really like working on the code structure, making it clearer, thinking about good abstraction, breaking into modules, etc. I really take pleasure in it. At the same time I understand that at some point I am becoming the limiting factor.
If the point of the software is benefit people, should I still care about how the code looks.
Right now, I still think that the answer is yes, but in 3 years? in 10 years?
You will always be able to ask the agent to do refactors for you - and it can do mega ones that exhaust you to think about!
Agentic refactoring is very questionable if you want to maintain quality, as it will rewrite all your code to be more average.
> My current status is that I have not had much success with this way of working for code I deeply care about
If something is judgement heavy, "code i care deeply about", then i don't really agree with the direction of travel here. Don't try to delegate decisions you care deeply about.
I do like the framing of agent loop vs harness loop, but only delegate stuff that you can accurately specify in advance, that usually means stuff that's repeatable in my case ("hey go see how i did X, do that but for Y"), and that inherently means stuff that's predictable.
For stuff where lack of my judgement as input is just going to cause me to say "no", we're down to collaborating in the "agent loop" as Armin puts it. And that's totally fine. It's fast, but also safe.
Remember before AI coding assistants, sometimes you'd get an engineer join your team who was SUPER productive, your peers would be jealous "oh yeah but you guys only got all that done because you have X on your team!" - they didn't live the curse of having that kind of person around - if you don't have them PERFECTLY aligned, then they run off at break neck speed in the wrong direction.
I'm a software developer from way back, using tools and languages that coding agents are far less familiar with.
So when I use an agent to write code, it's in languages I'm less familiar with, and often using libraries I know nothing about.
All to say, my part of the process often ends up being:
1. "Here's what I'm looking for, in detail" 2. "That's not right. Here's one way it's not right, and a specific example. Please fix that." 3. Sometimes I give suggestions for how what is going wrong might be happening, or conceptually how to work around the issue. 4. And iterate on 2-3 until the result is close enough.
That's a loop I'd love to automate.
This is really terrible advice right now for most people.
I've had to rip out a lot of pretty terrible code made by engineers who have tried this.
I don't disagree that eventually, "loops" when combined with unlimited tokens and amazing models in the hands of people who know how to set them up right will be amazing. But for the typical Claude Code user, it's disaster.
The problem is not that loops write bad code once. Humans do that too. The problem is that loops apply local pressure repeatedly: add a fallback, add a guard, special-case the failing input, quiet the exception, satisfy the test. Over time that selects for code that is more survivable in the short term but less intelligible in the long term.
We used a “loop” before it was called that to drive MS-DOC support into Tritium. Based on that experience, I take issue with this:
“There are already impressive examples of large automatic porting efforts, including the reported work around moving parts of Bun from Zig to Rust.” (Emphasis added.)
It will be impressive if/when the Bun team is able to pick up and continue extending and supporting Bun. For us, MS-DOC remains read-only and probably perpetually buggy until we reimplement with a better understanding. Until then, it’s definitely not “impressive”. Functional? Maybe. Impressive, no.
In my own ham-fisted experiments with coding loops, one pathology I have noticed is that the LOC just spirals out of control. That's likely because of the layers of defensive fixes, etc., that get built. That inevitably causes context bloat (or at least navigational friction) and results in quality decline.
I wonder how many loop-related issues could be addressed by simply fixing a LOC budget, or assigning a cost in some way. Unclear how you would dial in the right numbers, though.
The issue is that whilst the loops will initially lead to good results they will be less and less as context gets bigger and bigger and tougher to understand for human and AI.
So it depends really on the size of your project.
We've had great success with agents thus far at my job. A year into Clauding and all our dev metrics are up while our downtime has remained steady.
Being an iOS engineer, much of my engineering cycle these days is going from Figma/PRD → spec → code. After being handed off to QA, we handle the bugs and product slips as they come through, while we simultaneously build/spec the upcoming addition. This is basically the same agile style that's been popular for 20y, just super-powered with agents.
How might someone accomplish the same goals using loops instead?
I personally have not had good luck with loops due to similar issues as the post author - but if you were to port your flow to "looping" it would be something like:
- An automation that periodically checks for PRD's at a given location that have not yet been implemented.
- If it sees one not implemented, it puts a lock on it (so other agents later don't pick it up while its still working) and implements the PRD in code, assuming it has the figma link and all specs required.
- When its done it makes a PR, waits for if it passes and even in some cases automatically merges into your staging/preview enironments and just pings you with a build/URL. You can then leave feedback or something and it can also also poll for pending feedback. Or you just mark it looks good, the agent then merges the PR, moves the PRD to implemented status, maybe even writes/updates docs and cleans up any temporary work.
- Repeat checking for new PRD's every T unit time. (10 minutes, 1 hour, etc)
This is how people say you should be looping - you never even cared or looked at the code, and also never prompted the agent yourself.
But I find most agents are often pretty bad still at replicating UI vs making something from scratch and most design specs are still not as detailed around how things look at all sizes, in all scenarios etc. Design seems to be one of those things that still requires a human to validate. And then all the things the post author mentions about it not being willing to apply hard constraints, minimize impossible states, validate at edges and prevent horrendous overchecking of things. etc.
Use appium or XCTest or swift testing; generate the tests first (failing) from the spec.
The loop is basically then a while loop:
While (tests fail) { trigger agent: spec, failures list }
for bugs, write failing tests.
Its basically TDD.
Loops do nothing useful beyond making the “spec -> code” step more “hands off” and let you be confident that the code you write does what is intended.
Obviously you see the issue: writing the loop harness is > effort than not having it…
…but the idea is that you run “spec first” and are totally hands off on the code, just updating the validation step and then waiting while the agent iterates over and over to solve for some solution that passes the loop harness.
People suggest that it is possible to go, eg. directly figma/jira to harness via (random tool here), saving even more time and invoking even fewer humans, but thats currently, as far as I can tell, actually just hype.
No one is actually doing that effectively.
Loops are currently carefully hand crafted, which makes them tedious and of questionable value imo.
Would you have a breakdown of costs/benefit? Can you say with certainty that this workflow has increased productivity so much that you are seeing profit increases that you wouldn't have otherwise noticed just by hiring more people? Asking with no ill intention, I just crave for actual business cases that make sense, and yet no-one seems to be able to reliably produce that.
Generally interesting reflections here, yet I see the same kind of myopia and fatalism that is rampant in our (fashion) industry:
> yet I have no doubts that this looping future is going to be our future despite the fact that I presently resent it
Why would anyone concluded this? LLMs are just one kind of application of MLs to software production. There is a vast solution space for automating parts of software production. The idea that slop loops are the inevitable future because they happen to be accelerating output at the moment just seems profoundly short-sighted and lacking in vision.
These new AI trends are very tiresome, very similar to 2021 crypto mania - both trigger a lot of FOMO. If we have loops that write code and we don't need to verify anything, why are the devs still here? What's point of even learning this new trick as a dev if you truly believe that this can be used without any intervention? If loops work then it follows that a loop of loop works too - why hire any people at all? Just run a bunch of loops and build a profitable business, but then what's your moat? Any person can now launch loops on top of loops.
The post suggests fear about a surge of increasing amounts of code by loops and loops of agents.
I don’t know if I like the current world without it though.
80% of different teams code the code is poorly tested. The code doesn’t handle data consistency or asynchronous code properly because the engineers don’t know better (and frankly don’t care enough).
Dependency handling is poorly managed leading to low quality operations with improper dashboards, alarms, and ops.
Badly managed processes leads to people doing monkey work signing off checklists rather than automation.
Frankly… why is keeping any of that good? It really pisses me off seeing people accept any of that low quality but that standard is the default and not the outlier.
I think this is a common sentiment among heavy users of AI that also still cares about code quality.
I've built up a skill harness and review flow that makes Opus generate slop-free code 90% of the time. But the remaining 10% requires me to stay at the helm. Especially in the early stages.
I would love to use loops to automate more, but I couldn't do it with the current generation models.
And on the back of my mind I'm still evaluating the possible future where we are forced to API pricing. I'm currently paying $400 for Opus, and use around 1.5-2 billion tokens per day. This will cost around $20k/m with API pricing. And I don't want to even imagine the possible scenario of getting locked out of frontier models because of politics.
Will the models get better to cut me out of the loop completely? I believe so. Will the open source models catch up tho SOTA models, and diversify from China-only? I hope so. Otherwise 2 superpowers will wield a soft power that can cripple the tech industries of all other countries.
This is a very fatalistic take. While I understand where it's coming from, I try not to share the same mindset: engineers getting increasingly distant from how things are getting built is not something that will "undoubtedly happen, whether we like it or not".
Also:
> Now there is obviously a question if this desire to understand the code is one that I will still have a few years from now.
I do not think we should be having doubts like this. Either you consider understanding the code you ship and allowing your future self to be able to work on the system you're building to be a value, or you don't. I, for one, do, and I do not think using LLMs and coding agents will affect my point of view on that.
I think there's 2 important, but separate, ideas in this post:
- Models are not good at or getting better at creating strong invariants, which his fundamental to good software
- It is unclear how to keep tabs on what the agent is doing, so you, a human, can intervene.
These are related, obviously: one of the highest-leverage things you can do is force you agent to use a strong, minimal set of types or data invariants or other constraints. They get much better when your codebase broadly supports this!
I do suspect they're separable, though.
If you had the right levers and visibility, you should be able to get the model to produce code that doesn't feel like slop. But every time I've had a model try to keep me in the loop, it inundates me with irrelevant decisions and busywork. Its inability to see what's structurally important still shows up, just differently.
[If the models get better at defining and respecting invariants, maybe there's a new flavor of slop, that's less obvious today.]
I honestly wonder if this kind of stuff really brings something to the table. Like I use opus for sometime and certainly I can put it to good use and optimize some parts of my day to day job (programmer). But it fails so hard in such simple tasks that it seems to me that putting it in loop can't just magically make everything better, unassisted. Does anyone actually uses agents and loops to create new software, new technology? Has anyone created with those systems, software they couldn't produce otherwise technologically wise? Or is it at best just an accelerator, cutting off on the building time?
A friendly reminder to just do 9 to 5 and touch lots of grass. None of this shit represents industry trends, majority of people still use chat interfaces and copy blocks of code. There’s zero early adopter advantage here, only FOMO and lots of anxiety.
This is the best essay on agentic coding I've read. Clear thinking and writing, pragmatic about the future of agent-led coding.
If you usually skip straight to the comments, you might want to actually read this one.
> Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery. Worse though: I so far see very little progress of this improving.
It’s almost as though these models were trained on a vast corpus of largely mediocre code. They will never outperform the median Github user - it is all they know, it is all they can do.
No.
(I mean, OK, so I can't just write that, because of some rule or another. But really, just no, at this point, surely? How is it we are so actively turning software development into some weird blend of cargo cults, homeopathy, prosperity gospel and penal treadmills? Chasing trend after trend from corporate influencers with motivated reasoning who are trying to create new cultures that entrench their weird inscrutable metered intelligence taps? Handing over this level of control to and for the benefit of the sociopathic or solipsistic executives of two companies that cannot demonstrate they will ever make a profit, backed by a handful of other companies who are stirring money around in a pot to make it look like they are still generating value? Can we stop? Please? Please?)
No.
(And also no: it was not, in fact, this bad before. No matter how you try to retcon it. It was not good. But it was also not this insanely nihilistic)
Theres a deep insight in this post about the value of looping for throw away code to explore a problem space, rather than brute force a problem by just applying more tokens and hoping.
The more I play in this space, the more I’m drawn to the idea that some kind of back tracking constraint solver is a better solution than then the current naive while loop / brute force approach here.
The results I see are similar to what you get from a greedy brute force constraint solver; solves trivial problems, sometimes solves harder problems after a long time, takes too long to solve really hard problems; solutions are increasingly non optimal on average as complexity goes up.
We have so much existing knowledge about building good constraint solvers, if we could just figure out how to apply it here somehow.
my experimental looping build on top of pi and zx mostly pi deep seek and some skills ;-) https://github.com/topce/pizx
Show me the billion dollar solopreneur startup, or the profit increase for companies and at that point I’ll start thinking that this tasteless high level wanking might make sense in some way
We've just invented the car and you're upset it hasn't achieved Mach speed yet.
One car went Mach 1, ever, apparently. Anyway, I don’t think the analogy fits. Ford or whoever didn’t loudly and frequently predict Mach 1 cars, right?
The situation is more like: Altman & co are predicting their new car will replace all vehicles: horses, trains, planes, motorcycles, there’s a real possibility the concept of vehicles will not exist other than cars, in the future. Meanwhile it hasn’t really done highway speeds yet. It does some impressive runs on curated tracks, and people use it around their farms (it seems to work ok for some of them).
We’ll see, I guess.
Yes, one car did Mach 1. And the first production car, the Benz Velo, could only go 12mph. It's an apt analogy.
As I mentioned to OP, applying future aspirations to the current space is incorrect. Some people are able to understand the progression of industrial automation, some people aren't. But if you look at the current batch of frontier models and say, "I just don't see how this is going to be useful", then you're in the camp of those in the 80's who didn't understand personal computers, or in the 90's who didn't understand the web. In hindsight, the technologies evolved massively and found routine use cases that no one initially predicted.
It is a terrible analogy that shows terrible thinking. After all, there's one thing we can bet with more confidence on: delegating thinking to this mediocrity machines is affecting the ability to do the same in scores and scores of previously smart people.
Funny enough it is a great analogy if you think these modes will never really “get there” at scale.
Terrible thinking? Delegating thinking to mediocrity machines? "Previously" smart people? I don't think you can project your own limited and malformed understanding of the world any harder. Please go touch grass and learn how to engage with others on the internet without being so negative.
I was literally quoting Sam Altman
No, you mentioned his discussion of the "billion dollar one person startup", which we can both agree is a fanciful idea and more of an eventual "possibility" that will of course not occur as once anyone can be a billionaire, the whole system is going to change.
However, your "tasteless high level wanking" is not a quote from Altman, it's a vague and directionless insult that manages to sweep quite a lot of legitimate discussion about the future of automation and professional work under its thumb.
It's wrong because you're saying, "where are the billion-dollar one-man startups?" in the same way that I might look at a Benz Velo and go, "But it's so slow! Horses can go faster than that! Everyone saying cars are going to change the fabric of society are just tasteless wankers!"
The point is that you are applying future aspirations on the present-day relatively brand-new model space and getting upset that we aren't there yet.
One of the biggest problems with LLMs has turned out to be the cost of actually running them and this strategy functions as a usage multiplier.
> For now I have not moved past the point of comprehension being important to me.
Ah ! This is me too... at least for what I have to ship at work. Not so much for my toy/weekend projects. But it turns out agents are also good at explaining.
I think it's insane to suggest that software developers should ever get to the point where they don't even comprehend their code.
Before someone else says it, no I don't read the assembly code that is produced by my compilers. However, I can generally predict what kind of assembly will be produced, and the result is deterministic unlike LLMs. It seems like most vibe coders scoff at the idea of even looking at the code, and it just seems untenable to me when we're working with (usually correct) stochastic parrots.
I'm willing to be persuaded otherwise: Looping seems to (currently) be a side effect of token subsidies.
If token costs are nil, then you can afford to run verification and generation through the same models. If token costs are high, then you will go broke verifying code sprawl.
Currently costs are (mostly) absent from the conversation, even though costs are what decide the limits which shape experience.
Also: Firms can be held liable for the products they sell, so if code cannot be reviewed then that code is essentially a law suit waiting to happen. I believe this is what customers will be demanding in the future: someone to hold accountable when things go wrong.
My own thoughts on this, with examples https://github.com/nfcampos/loop-dev/blob/main/README.md
Are those your own thoughts? Or Claude's? I tried to read but had to stop a few paragraphs in because of Claude's annoying authorial voice.
No, they really are my thoughts, I've been doing this stuff pretty much every day for the past 6 months
No offense, but this sentiment is diluted by commits like this:
https://github.com/nfcampos/loop-dev/commit/e28b1fce0078e605...
I assume that GP was just saying that they would prefer to read these thoughts written by a human author (preferably you). I agree.
That's a perfect use case for an LLM: grammar-aware find and replace (spicy find and replace.)
That commit happens to be a find replace because i usually write for my company’s blog (we)
I would be more willing to believe you only used Claude for minor editing tasks if you disclosed your usage of Claude upfront.
This commit seems to mostly be grammar fixes? If someone used a spell/grammar checker it might produce a diff similar to this. Why does the fact it was Claude and not Microsoft Word or other matter in this case?
Because Microsoft Word doesn't write blog posts for you
Edit: I just realized that Microsoft Word probably does do that now, and I hate it.
Copilot is the new clippy meme. Copilot is hiding in every MS product and is ready to write an email for you, anytime.
Yes but that commit doesn't support the fact that it has written it for the user.
There's _way_ more than one way to do "loops". I just asked one of my superviors/auditors to document how it's been working while monitoring a few other agents that have long-term goals:
https://gist.github.com/rcarmo/4922b550ab48bf0b4246c77e606a5...
I'm glad HN prefers downvoting actual work/demos to discussing them.
Yeah I don't know. Don't get me wrong, the article points makes sense. But sometimes I think that we're going to stay near this current point of productivity for a little while.
Currently my org of 8 people use around 1000 euro worth of tokens per month. We've recently had a discussion near the water-cooler, that if the cost climbs 5x-10x it may be just more worth it to get more developers (we're EU based). While the tools work and are definitely nice, even in our little org with our little budget, using Opus 4.8 we've noticed code quality going down.
If I had to bet money, I'd bet that the models will get 30-50% more nice, around 2x more expensive and we will settle into some mode where we'll use llms for some tasks, manually doing others and calling places focusing on speed at any cost some funny name like "gulags, 996, sweatshops, etc" and collectively try to somewhat avoid those places, which will need to offer a premium to attract talent. Wishful thinking.
How do you control code quality?
The way I answer this question is, I don't because my boss is saying to basically accept whatever LLMs spit out which is fine by me because I have explained the risks of doing this to our only product and it was shrugged off. I am both disgusted and excused and just take the salary.