I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.
By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.
But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.
It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.
Same. I let the LLM work inside a framework that I made.
Of course I only arrived at the framework after trial and error with an LLM showed me the shape of the problem and the framework presented itself to me.
So, in other words, we save time and money when it came to creating stuff, but we've made a whole bunch of new work around verification and testing of said stuff?
What have we gained aside from shifting where time and effort is spent in the design loop?
I find the problem with "lots and lots of testing" as a solution - (assuming by that you mean lots and lots of automated testing) is it invites lots and lots of lines of code - making code reviews even more challenging.
If someone finds it burdensome to review my 50-line AI-generated bug fix, adding 200 lines of tests only gives them more lines to review.
Might be a little controversial to say this but automated tests can be reviewed at a lower bar than production code.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them. It’s better to spend more time on the core implementation.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them.
I respectfully disagree. IME writing a good test suite is one of the areas where current LLMs perform worst. They certainly like to write plenty of tests but those tests aren't necessarily adding much value and they don't necessarily cover the important functionality.
Given how heavily LLMs and specifically agentic workflows still rely on producing something vaguely correct and then iteratively applying guard rails to reach something that actually works I humbly suggest that trusting a very substandard test suite to be one of those guard rails might be unwise.
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
The point is that you're not verifying if you don't even review the code.
Even if your test suite tested every single possible behavior and code path, tests usually can't intercept all possible unintended behavior that an LLM might have added.
In all likelihood, most test suites don't intercept any unintented behavior (such as unintended internet connections).
That's not a big problem when a honest human has written or reviewed the code, because unintended code is usually easy to spot.
But an LLM might have dropped any whatsoever code: maybe some of your project's code resembles the one of a trojan in its training, so it completes it with the trojan.
Or maybe it found that if your server is unreachable, it can upload the client's data to a public Github repo; or that if there's no space left on the device, it can delete some of the user's folders, easy!
Would your clients be understanding if that happened?
Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.
In most workplaces, when supervisor/boss says "we need you working over here now; you start tomorrow", you either embrace or quit. Again, nothing fundamentally different.
How much understanding is needed? If it works as specified (you did a complete spec and occasional check-ins to ensure things are on track and then tested the delivery, right?), then what more is needed?
> exactly how the unit and integration tests are made
There should be enough detail in the spec that it's pretty obvious. And unlike - many - human devs, agents don't shy away from writing tests. Add a coverage guardrail and tell them to use red/green TDD and you're already a good way there. The main thing is to ensure shared understanding, so they also know business expectations and write appropriate tests. And... nothing here is really specific to an agent, as a human dev also needs sufficient guidance (tailored to their experience level).
Noone understood the code to begin with. People understand the concepts, the primitives, the "theory" as Naur said, the basis upon which a project is built. Noone ever understood a codebase worth understanding to begin with.
1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).
2) Ask agents to do more than just code.
They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.
It would be an insane amount of work for humans, but AI can do it easily for each PR.
Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.
If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.
I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
Yes, compilers are a good candidate for verification through thorough testing. If a comprehensive test suite passes, reviewing every line of code isn’t so important.
Very similar here. For internal projects the test is - if it works, unit and integration tests were created, and it follows our architecture correctly.
I am not in META but for what I read on blind, assuming it is true, they are not spending any time in long planning sessions or in getting into details of the implementation plan at all, and they could not anymore considering how many PRs in parallel they are requested to manage
Yeah, I'm retired. The only person I have to answer to regarding progress is me. I'm a stubborn son of a bitch when it comes to choosing quality over speed...
That doesn't mean I don't like progress mind :) This morning I got the preliminary results of a 'C' front-end to the compiler (it uses clang to compile to LLVM IR, then translates to our IR, then feeds that IR into the autovectoriser as if it had parsed the code itself). So for the exact same source code I now have:
Those are some pretty fine speedups, from naive C, with no alterations from the standard "C" code - it automatically finds places that can be made data-parallel, tracks the data-hazards to make sure its safe to "GPU" and writes all the kernels, boilerplate and data-stream-management if necessary. Obviously the checksums have to match, so I know it did the work correctly too :)
Hacker News isn't interested - any post I make linking to the webpages is immediately blocked/dead so I'll wait until the C++ front-end is ready (some time next week) and announce it somewhere else. A compiler that takes standard C/C++ code and speeds it up by automatically finding data-parallel areas and scheduling them on the GPU with no programming overhead is pretty rare.
Oh, and it manages the local network as well, so you can run xcd on machines on your lan, or via the TB4/5 bridge between Macs and it'll schedule on other GPUs, just like it manages multiple GPUs on the local machine. It characterises both latency and compute-time to figure out which devices are worth scheduling on.
So, the real input to your autovectorizer is LLVM IR; you could soon enough support any language that has an LLVM front-end? (I say soon enough instead of immediately because I’m sure there’s some manual wiring up to do). That seems like a pretty big deal. Do Fortran next, pls. :)
Upvotes don't appear publicly on HN so I'll comment to agree with this. I'm sorry if earlier information about this compiler got lost in the whirlwind. Data parallelism is a hard problem and these results look genuinely impressive so I for one would definitely be interested in hearing more about it and maybe discussing it further when there are interesting developments.
I’m being a little cynical - I’ve submitted it twice (though the ‘C’ front end is new) and it bombed both times - the second time within seconds of submission.
I think it’s my own fault, I’ve posted the link to the compiler pages several times in what I thought were relevant posts, and I think HN thinks I’m spamming. So I’m not posting the home page for a while and we’ll see if a future submission “takes”
Yep. I’d give the link but I think HN thinks I’m spamming because I’ve mentioned it too many times before in posts - hence immediate death of any submission, so I’m not mentioning it for a while to see if that tails off.
Bottom line: GPL3 for the compiler, standard library exclusions for derived works, so use it for anything but feed me back improvements to the compiler itself, and you don’t get to redistribute it as your own.
Kind of, except as product manager you're held accountable for delivering the right product, while your engineering team is held accountable for things like performance, incidents, and security issues caused by their code.
It's hard to hold an LLM accountable for anything.
Accurate but manageable. Just like a pm works with properties an engineer also does. The Product person wants to deliver the right product properties so they model their process around identifying the ones they want and then creating the right framework to get them. On the engineering side its the same thing. Define what you want, give agents an environment they can succeed in, iterate.
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
Which is - of course - absurd. Accepting responsibility without demanding the corresponding authority is a fool's game.
I have never been much of an advocate for unionisation in the software development industry. I didn't think it was necessary to protect the interests of the workers because for most of the industry's history the workers have largely been able to stand up for themselves anyway. Likewise I've never been a big believer in accreditation and licensing practitioners in our field - because who is qualified to judge someone else worthy in such a young industry with so little hard data about what works or doesn't work? I find my views on these subjects are evolving now.
It's not about protecting jobs if they truly aren't needed any more or protecting people working in the industry who refuse to move with the times. The Luddite philosophy isn't going to get us anywhere here. And besides - if AI is as effective as its advocates have been claiming then there should soon be more demand for capable developers to take advantage of the new opportunities. (The Jevons Paradox is relevant here.) But it is about protecting standards. If the tools aren't doing a good enough job then expecting professional developers to ship the code produced by those tools even though they know it's junk is something that needs to be challenged. And if developers are being expected to ship anything important without having the slightest idea whether it even works then that's even worse.
We had a lot of silliness going on when the very low interest rates and huge amounts of VC funding were propping up ideas and practices that would never have survived otherwise. Today we have a different kind of funny money distorting the industry but it's still propping up ideas and practices that we wouldn't normally support or perhaps even tolerate. And just like before no individual is going to be able to hold back the tide and it's going to have to be a collective effort to push back - hence my evolving views on topics like unionisation and accreditation. We've all seen the drop in quality across much of the industry and we all know why it's really happening because we see how the sausage is now being made.
Not all mistakes are code, a lot are feature design mistakes, and if SWEs would always be responsible about PM mistakes, then why not use Claude anyway?
Maybe the new meta should be reviewing new code at the end of the week/month rather than at each merge? But even that might be too much of a bottleneck. The alternative is to talk to your agent to understand changes at a high level without reading every line of code.
It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.
The assumption was that some engineers knew how each part of the system worked, or at least the system was comprehensible because it was created by humans for other humans. There were comments explaining difficult places, etc. With AI this assumption is broken. Now nobody understands how things work, and it cannot be understood in principle because the system is a mountain of slop created by some alien intelligence.
I've been spending a lot of time iterating on feature implementation plans with Claude (at work) and GPT (for personal projects).
I then make sure they write a lot of tests and that if functionality is changed or added/removed then tests are updated alongside.
Generally I try to review changes myself and ask it to explain the changes.
It's not perfect but I feel like the planning step being iterative isn't all too different from my old research -> think about implementation -> implement loop.
At that point, reviewing any code changes goes hand in hand with how I coded before LLMs.
The major difference is I usually have it make any changes for things I don't like.
as someone using AI, it is mania, leadership has decided it is so and software is getting worse. we could have just accepted our 20% bump and been happy, fixed bugs, made things better quality, nope, ship half assed features at 2x speed, pretend claude is infallible and will solve all our problems, brand reputation? user research? pixel perfect designs? what are you a dinosaur?
Except in this case the label on the eggs says "Eggs may occasionally be poisonous. Users should not rely on eggs to be safe, and should carefully scrutinise eggs before and during cooking. Bob's Farm accepts no liability for injuries resulting from making omelets with our eggs."
I lost my ability to fully explain the systems I work on around the time I transitioned from next-edit-prediction AI running in my IDE (early Copilot) and working primarily via Codex/Claude CLI's.
The further away I got from the implementation details, the fuzzier the semantics became. To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.
The solution to this, in my mind, is not to revert back to reviewing implementation details. The economics of that from a business/productivity perspective don't play out well.
The only viable solution in my mind is adapting code/languages themselves to the shift by making programs formally verifiable. That way you can be sure that the code "does what it says on the tin", and provably so.
The bridge is built. How do we test the load? Drive bigger and bigger trucks until it fails, note the weight, print the sign and rebuild the bridge the same way.
I'd guess they started developing the project back in 2021 coding by hand, and are now improving/maintaining via agenetic engineering. This is not uncommon.
The point is that it's easy to lose understanding of the finer details, even of a codebase you lovingly created, once this switch occurs and an LLM gets its hands on it.
I agree with you with one caveat. Professionally, your obligation is to be qualified for your job and ship work that you stand behind. Complete comprehension is absolutely necessary.
Personally, with side projects and tooling, go nuts. There's no reason to read all of that code.
If the agent has written tests and crucially - witnessed them fail before starting the work to make them pass - you have a lot of confidence.
Review of your own work's easier too - test and production code tend to mirror each other, giving you a much more complete picture. Confused by the production code? Go read its tests for a bit, then come back.
It's also easier to understand the change if you were there for the build, watching the transcript, checking in every so often. Doesn't have to be your primary task - leave it on a second monitor, it'll catch your eye naturally.
And finally, on top of test-driven development, have the agent first (before the first test driven development cycle) build out an end-to-end test covering the work it'll do. Once that test is green (test driven word for "is passing") you're done with your work and can clean up at will.
Play with it before shipping, get it real nice. It's never been easier to clean the tech debt up and make the UI perfect.
Eventually you'll be ready to package it up into a PR!
Now's you can leverage agents and get creative. Make your colleagues froth to review your work. Consider making a Loom video to walk them through it. Consider reworking it into a commit series; it only takes a few second for your agent and you can tell a story for humans. Maybe you want to sequence it according to your struggles; maybe it's better to show the different layers emerge with each commit - it's up to you. Consider also that your agent can easily do one-off visualizations of the change for your colleagues if you think it'll help them, and you can attach those too - mermaid charts or even whimsical AI generated images. All this is because motivation is driven entirely by other humans' motivation and you want them frothing with excitement every time you drop a review. Make it fun, it takes seconds and gets your stuff merged fast while making your teammates smile.
This is a brand new job and we will work out how to do it together :)
In my experience you're just replicating the problem.
Slop code doesn't stop being slop code when you add in slop tests to account for the sloppiness.
If you generate code at a certain velocity, you eventually get to a point where you generate tests the same way you generate code. The model does its best effort in creating tests, but you'll inevitably get to a point in your work where the tests themselves are faulty and unreliable, the same sloppiness you were trying to control for.
There have been multiple times where my software dev team has gone back to prior implemented tests they had the model generate and found that after some time eventually the model got frustrated with the test no longer passing eventually modifying it to fit current needs around a feature push.
You can probably anticipate how we came to discover this, as it ended up introducing a regression around multiple edge cases which the test was meant to control for.
The only answer is to inject actual engineering into the process as much as possible without giving it all up to the models to generate more slop in response to slop.
tests are a great answer, but only if you treat them with serious intentionality.
Your engineers weren't doing what I'm describing I promise.
There are multiple places in my post where I explicitly describe the process by which you intimately:
- make the agent do verified work (the e2e tests and test driven thing)
- watch the agent work (the second monitor thing)
- read your own code (the tests/prod mirror thing
- play with the work and perfect it (the tech debt and UI perfect thing)
Plus a ton of other places in my post I very clearly describe being intimately involved in building and selling the work.
That to me is basic god damn competency. What your engineers did is a fad, and it won't last, and I'm sorry they abdicated their professional duty.
If you wish to screen for that sort of shit in future/current hires, show them my post and see if they turn up their noses or if they get excited and creative. You want only the latter.
I've been playing with this https://getchronosphere.com. It will (attempt to) rebuild a PR in a coherent journey rather than a massive blob. It's been somewhat helpful up to this point, and still improving.
I was struggling with this today. Working on a feature and AI review flagged something so I need to fix that. I iterated multiple times throughout the day and was completely exhausted cause each iteration was always complicated with the fix being ~1k lines. Finally landed on a one I felt at least a little comfortable submitting for review but a work day like this is super draining and I felt like crap the entire evening (I am about to go to bed, I am probably gonna have nightmares about pressing enter and enabling auto mode on plans that make no sense in a true nightmare sense)
I can think of so many times this has happened before. People getting used to MVC, Dependency Injection, ORMs. Sometimes the code is so large you only work on a part of it, so you have to get used to a lot of not knowing.
So how do we build confidence anyhow when we can't know everything?
I think first, standard checks - does it compile? What do static analysis tools say - code quality, passes SAST analysis? Linters? Style checkers?
Do the tests run? I'm not comfortable with the tests I write let alone the AI.. by adding fault injection (fuzzing, chaos), and mutation testing, I have a better degree of confidence in the tests and code.
Spend even more time getting to know the product instead. Maybe I don't know the code as well as I used to, but I know the product backwards and forwards. That helps me have some new confidence that I didn't really have before. Knowing the greater whole.
Your comment made me think that people have been happily importing hundreds to thousands of dependencies into their projects that were written by some random guys on the internet, but they've just now started to become concerned they don't understand the code they're shipping.
I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.
It’s very disconcerting. I think this will largely be solved by going all in on AI and use it to test and review the code in an adversarial manner. That won’t be cheap but it’s the only way I see this work. But a lot of companies will probably save money and use AI mainly for writing code and then skimp on review and testing. I bet we will see some huge blow ups with AI written code and hopefully the industry will learn.
It also took the industry quite a while to accept that server code needs to be secure even if that requires a lot of extra effort. AI will probably be the same.
Im currently building a ui framework for unreal. I dont enjoy ui work so I wanted a way to have to write less ui.
For me I dont look too closely at the implementation but I look at the api where if a human was using my framework how is the code they'd have to write and engage look at.
Then I also do plenty of performance against a comparable target in my case slate (default unreal) and keep overhead minimal.
So good performance and reasonable public facing api are the 2 main pieces im focused on. Less with the internals.
Care about the seams and interaction and the things you can measure or count.
I'm currently working on a combined Aero sim/CAD/CAM software for designing, simulating, and performing DFM optimization specifically for Group 1 to 3 winged UAVs.
What I do is ask Opus/Gemini to explain the parts I don't understand. And I do some of my own research to verify accuracy of the explanations.
In addition:
* Every new feature has a Feature Design document which I review, modify, and approve. And a corresponding Implementation Plan which I also review and approve.
* Every fix has a RCA document and fix Implementation Doc which I review, modify, and approve.
* There's a global high level Architecture and Design document with additional sub component design documents which are always kept in synch after a new feature or bug fix is done. Synching it is always a step in the Implementation plan.
This does adds some overhead but it allows me to have have check point gates. This has often allowed me to catch misunderstandings and align properly before any code gets edited by the AI agent.
Having the above docs also helps when it comes to referencing them as context to properly ground new features or bug fixes.
Product doesn’t care. And most of us will get replaced by agents. I guess around 30% of the engineering workforce will keep their jobs which will be mainly to keep the product up and running, doing whatever it takes (so using agents to fix stuff basically).
The ones who get to keep their jobs will be rebranded as Maintenance engineers with a lower salary. VPs and Heads of will keep their cozy job (bc there are not so many of them), and they are not that dumb to fire themselves.
I don't ship code I don't understand. I fully review every line if it's going to prod.
My dumb little utils in ~/.local/bin? Those get merged after a quick functional test without my having read a single line of it. Still, the skill I maintain for such a job directs agents to write such throwaway scripts in anything but shellscript because I might want to extend some day and fuck shellscript
>This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever.
"This new way of piling bricks is so fast that I can no longer afford to check the buildings integrity".
I just don't understand how the fuck we ended up with this narrative. Somehow the machine making things twice as fast forbids us from spending time ensuring quality? Surely if it does things faster you have more time and tools to check!
I'm half convinced this frenzy is the marketing work of AI companies at this point.
Short-sightedness is quite literally the reason people are using these systems to begin with. If you care about the health and integrity of your code base years down the road you wouldn't be shipping endless amounts of slop you don't understand, Fred Brooks taught us this half a century ago.
I don't understand these types of comments. Aren't the CVEs epic today with human written code? What difference does it make then? You want no CVEs, make software engineering into a real gate-kept engineering professions with accountability chains, government regulations, formal education, etc.
> You want no CVEs, make software engineering into a real gate-kept engineering professions with accountability chains, government regulations, formal education, etc.
This will not only not prevent CVEs, it will also make everything a giant pain in the ass
Of course it will. But at the same time I don't see building falling left and right like CVEs in software. So don't tell me agents write bad code, because humans do as well. The only way to prevent this (or minimize), is to turn software engineering into real engineering.
"If it works, ship it" is very different from "If it works, and I understand it because I wrote it, ship it".
Does the LLM understand it, in detail? Can you trust its interpretation of the information and instructions you provided? Did it independently make an adverse decision you haven't noticed, because you didn't read the code? Was the model missing any information it required -- and would you know if it was?
LLMs depend on humans and human oversight to ensure they have the information required to make the right decisions, to catch when they don't, and to steer them based on your much more complete understanding of the broader context and requirements. You can't rely on them to do that thinking for you (yet?).
I’d argue that most people also don’t understand code they read from their colleagues. Or if they do, then they don’t understand the entire context of the app.
You said LLM need human oversight and it’s true, but it does not imply you have to read the code.
I think we are going towards a place where code no longer makes sense, you just describe what you want in English, and the LLM just builds directly in machine code. You then build different verification tests to make sure the program is doing what it’s suppose to do.
It's simple: you don't. You slow down. It's essential to understand what you ship, before you ship, as there are real world consequences for not doing so, ranging from wasting the time and energy of yourself and others, up to failures which cause a great deal of harm to people and the world. To knowingly ship code you don't understand is grossly negligent, regardless of how common it becomes.
> How to deal with shipping AI-written code you don't understand?
Stop shipping code you don't understand.
Is there really any other solution? You can either ignore it and pretend like it's the new era of software like half the other commenters are, or you can sit down and understand the code and what you're shipping.
I wouldn't be surprised if the meta person has a gun to their head to increasingly keep shipping like a maniac (how decade-long bugs remain unfixed don't ask me), so this advice may not be very useful.
Generally speaking I brain storm the feature first and produce a dev plan. This includes asking it to reuse code/libraries we’ve already got! After it’s gone off ah done its thing we go back over and do a QA on what it’s built, why that approach, what can we do better etc etc?
You're going to have to understand it one way or another, either at writing or at debugging. Even if you delegate all debugging to AI, you'll still need to determine what is "right".
If you don't understand the code at writing or debugging, you'll pay the architectural price (where it degrades). That price may be worth paying, but that depends on the system you are building.
It feels like another layer of "conceptual compression". I'm not sure who/when that phrase came around but in context of Rails is where I heard it and its things like ActiveRecord or really any ORM. You aren't writing the SQL directly, something else is and for the most part it works. Then it doesn't work or your database grows too much and you need to optimize table structures with indexes, etc.
My point is if you don't understand the fundamentals of the underlying code (SQL in the example above), you're more likely to have issues in the future. Or maybe not if that app doesn't reach a usage pattern or complexity where the nuances under the hood matter.
It really depends on what you're shipping. Meta ships social media apps where in the grand scheme of things if something is broken software wise it largely doesn't matter. What's the worst case? lost ad revenue? Users can't do things? If that were software for MRI machines or aerospace where software issues/bugs can result in lives lost then it matters much more in understanding what is going on at all levels of the software and potentially hardware.
Anyone who trusts the output of LLMs uncritically, even after the impressive level models and harnesses have gotten to, is borrowing against their future.
Models and harnesses have gotten really good, but they still are hard-capped by the operators that use them.
The sycophancy of models is perhaps their greatest weakness, as when a user who might be experienced in one area attempts to use AI to build within an area they are less familiar with, the model does not take a step back and examine the "why" of what is being built from an architectural standpoint, it simply performs the expected task in a direct manner, ultimately creating sloppy work that may be easy to ship but difficult to maintain.
A good example of this was a recent interaction I had with a Dev who was very familiar with how to develop embedded and system level applications. He and every other dev are being asked for increasingly more due to AI, and he had to bring something he had built as a single-user desktop application into a distributed cloud environment.
He ended up rsyncing a copy of his working directories including multiple credentials and sensitive internal project documents to a virtual machine binding it to a public ip address. Thankfully our tooling detected this before he could get any farther but it illustrates how the AI model isn't doing any real thinking or engineering at the end of the day.
Unless the operator knew to instruct the model about proper cloud architecture and understood best practices around deployment of public systems, the model simply did what it was being told to do by a user who didn't know any better.
Another example of this is the sheer amount of unsophisticated users we've been detecting using Javascript injection into their web browser sessions as a form of automation for web services they may use. These users would never know to ask how to request an API token which may allow them to accomplish things in a much easier and approved way, and instead are relying on the frailest possible automations that expose them to risks of credential exposure.
We've got to be careful with how we engineer things with AI models. Just because we've made boilerplate easier to generate doesn't mean that we can avoid the actual engineering of what we do. And I can't help but think that firms which are fully committed to AI code generation, won't just be introducing an avalanche of slop into their product which will explode the costs of maintenance over time.
If it's important code (and much isn't), then I spend a lot of time using Matt Pocock's /grilling first, and then I get the agent to kick off peer-review with other agents, and surface any disagreements. Some models are better at it than others. This usually forces me to broadly understand the architecture, and I spend most of my time answering the design questions for which there isn't an obvious solution. I have been pretty happy with the code I've launched to production, as measured by the ways it breaks, and by how agents who weren't part of the original committee describe it.
This is the state that founders and decision makers are usually in at a startup. Sure, sometimes you have the coder founder who wrote the first iteration of the product, but many times the founders have no clue how the product works under the hood. They entrust that to the engineers. That was then. Now, much of that engineering layer has been removed, so the founder just needs a smaller team of people who understand how to structure the software and interact with the LLM. LLMs do the rest.
The problem here is that you're trying to do a job that is already being done for you. Maybe you don't yet trust it to do a good job, but that is quickly changing.
Replace "AI agents" with "a bunch of human coders" and you'll see the answer. Your product needs decent testing and it's your job as the responsible person in charge to make sure that's in place and working properly. So if anything you probably want to be reviewing the test code. But often that isn't necessary since you can do things like introduce bugs deliberately and see if tests fail; check test logs for information that confirms the tests were run as expected; try some manual testing once in a while; ask for new more advanced tests (e.g. test performance not just functionality); and so on.
The part that makes me uneasy is when the AI writes both the code and the tests. They might agree with each other and still miss the same thing. How are people here checking for that without doing the whole job twice?
The author of the code was physically touching the internals and potentially intuiting obvious shortcomings as they worked on it. That very crucial part is now gone.
I like fault injection (fuzzing, chaos), and mutation testing. I think introducing these helps the AI write better tests and helps having some confidence in the tests that are written.
You should be using two (or even three) different LLMs. For example: Claude to do the coding, and Codex to do an adversarial review. This can go back and forth for several iterations before converging on an acceptably "correct" version of the code...
Occasionally, Claude may not be able to fix the issue – and it may turn out that Codex can, because it has a different approach.
Either way, at least two LLMs is the only way to get (mostly) reliable code.
I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.
By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.
But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.
It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.
Same. I let the LLM work inside a framework that I made.
Of course I only arrived at the framework after trial and error with an LLM showed me the shape of the problem and the framework presented itself to me.
This is where myself and a colleague have settled. We design for longevity but it can still type, search, and debug faster than we can.
Testing.
Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.
In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.
On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.
Nobody's perfect. Trust; but verify.
So, in other words, we save time and money when it came to creating stuff, but we've made a whole bunch of new work around verification and testing of said stuff?
What have we gained aside from shifting where time and effort is spent in the design loop?
Shipping features and fixing bugs much, much faster.
Better tested software duh
Isn't this the perfect classical capitalist advertisement? Every number goes up.
- code? up - build budget? Up - developer output? Up - developer oprrational costs? Waaay up!
Why is this Bad? Every one involved profits somehow.
I find the problem with "lots and lots of testing" as a solution - (assuming by that you mean lots and lots of automated testing) is it invites lots and lots of lines of code - making code reviews even more challenging.
If someone finds it burdensome to review my 50-line AI-generated bug fix, adding 200 lines of tests only gives them more lines to review.
Might be a little controversial to say this but automated tests can be reviewed at a lower bar than production code.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them. It’s better to spend more time on the core implementation.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them.
I respectfully disagree. IME writing a good test suite is one of the areas where current LLMs perform worst. They certainly like to write plenty of tests but those tests aren't necessarily adding much value and they don't necessarily cover the important functionality.
Given how heavily LLMs and specifically agentic workflows still rely on producing something vaguely correct and then iteratively applying guard rails to reach something that actually works I humbly suggest that trusting a very substandard test suite to be one of those guard rails might be unwise.
How do you test code that you don't understand?
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
Are you just trusting the AI to do it all?
Nothing a "make no mistakes" at the end of a query cannot fix. /s
The point is that you're not verifying if you don't even review the code.
Even if your test suite tested every single possible behavior and code path, tests usually can't intercept all possible unintended behavior that an LLM might have added.
In all likelihood, most test suites don't intercept any unintented behavior (such as unintended internet connections).
That's not a big problem when a honest human has written or reviewed the code, because unintended code is usually easy to spot.
But an LLM might have dropped any whatsoever code: maybe some of your project's code resembles the one of a trojan in its training, so it completes it with the trojan.
Or maybe it found that if your server is unreachable, it can upload the client's data to a public Github repo; or that if there's no space left on the device, it can delete some of the user's folders, easy!
Would your clients be understanding if that happened?
Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.
Why would you need to embrace something forced upon you?
There is no escape from Dario's basement. It's best just to accept our fate. Rub the lotion on our skin or else we get the hose again.
In most workplaces, when supervisor/boss says "we need you working over here now; you start tomorrow", you either embrace or quit. Again, nothing fundamentally different.
Perhaps it will make it easier to cope with? Fighting something beyond your control is pretty exhausting
So you can pay rent?
Then whose role is it now to understand the code?
How much understanding is needed? If it works as specified (you did a complete spec and occasional check-ins to ensure things are on track and then tested the delivery, right?), then what more is needed?
So you just click around the UI, assuming there is one?
Can you break down for me exactly how the unit and integration tests are made? Do you ask the AI what to assert?
> exactly how the unit and integration tests are made
There should be enough detail in the spec that it's pretty obvious. And unlike - many - human devs, agents don't shy away from writing tests. Add a coverage guardrail and tell them to use red/green TDD and you're already a good way there. The main thing is to ensure shared understanding, so they also know business expectations and write appropriate tests. And... nothing here is really specific to an agent, as a human dev also needs sufficient guidance (tailored to their experience level).
> (you did a complete spec and occasional check-ins to ensure things are on track and then tested the delivery, right?)
I have never in my career been afforded the time to make a complete spec.
Ah yes the Santa dev fallacy.
Noone understood the code to begin with. People understand the concepts, the primitives, the "theory" as Naur said, the basis upon which a project is built. Noone ever understood a codebase worth understanding to begin with.
Project managers just take the engineers word for it.
And engineers have a lot of leeway to just make shit up and the PM would have no idea until it blows up.
Except you can't blame Claude when sh-- hits the fan.
1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).
2) Ask agents to do more than just code.
They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.
It would be an insane amount of work for humans, but AI can do it easily for each PR.
Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.
If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.
I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
See the first paragraph here: https://alexgaynor.net/2026/sep/07/is-it-correct/
If your definition of "works" includes "doesn't have vulnerabilities", you're going to need to do more than just poke at it and add tests.
Yes, compilers are a good candidate for verification through thorough testing. If a comprehensive test suite passes, reviewing every line of code isn’t so important.
Very similar here. For internal projects the test is - if it works, unit and integration tests were created, and it follows our architecture correctly.
I am not in META but for what I read on blind, assuming it is true, they are not spending any time in long planning sessions or in getting into details of the implementation plan at all, and they could not anymore considering how many PRs in parallel they are requested to manage
Yeah, I'm retired. The only person I have to answer to regarding progress is me. I'm a stubborn son of a bitch when it comes to choosing quality over speed...
That doesn't mean I don't like progress mind :) This morning I got the preliminary results of a 'C' front-end to the compiler (it uses clang to compile to LLVM IR, then translates to our IR, then feeds that IR into the autovectoriser as if it had parsed the code itself). So for the exact same source code I now have:
Those are some pretty fine speedups, from naive C, with no alterations from the standard "C" code - it automatically finds places that can be made data-parallel, tracks the data-hazards to make sure its safe to "GPU" and writes all the kernels, boilerplate and data-stream-management if necessary. Obviously the checksums have to match, so I know it did the work correctly too :)Hacker News isn't interested - any post I make linking to the webpages is immediately blocked/dead so I'll wait until the C++ front-end is ready (some time next week) and announce it somewhere else. A compiler that takes standard C/C++ code and speeds it up by automatically finding data-parallel areas and scheduling them on the GPU with no programming overhead is pretty rare.
Oh, and it manages the local network as well, so you can run xcd on machines on your lan, or via the TB4/5 bridge between Macs and it'll schedule on other GPUs, just like it manages multiple GPUs on the local machine. It characterises both latency and compute-time to figure out which devices are worth scheduling on.
So, the real input to your autovectorizer is LLVM IR; you could soon enough support any language that has an LLVM front-end? (I say soon enough instead of immediately because I’m sure there’s some manual wiring up to do). That seems like a pretty big deal. Do Fortran next, pls. :)
Ssshhh. I like to try things before promising them :)
Awesome, this is genuinely-cool HN material. Thanks for sharing.
Upvotes don't appear publicly on HN so I'll comment to agree with this. I'm sorry if earlier information about this compiler got lost in the whirlwind. Data parallelism is a hard problem and these results look genuinely impressive so I for one would definitely be interested in hearing more about it and maybe discussing it further when there are interesting developments.
Hell yea I love that computers are getting another chance at becoming compute fabric, just using each others GPUs. Sick.
> Hacker News isn't interested
I wouldn't be too sure about that :)
I’m being a little cynical - I’ve submitted it twice (though the ‘C’ front end is new) and it bombed both times - the second time within seconds of submission.
I think it’s my own fault, I’ve posted the link to the compiler pages several times in what I thought were relevant posts, and I think HN thinks I’m spamming. So I’m not posting the home page for a while and we’ll see if a future submission “takes”
That is pretty adventurous, but it sounds really cool! Will it be open source?
Yep. I’d give the link but I think HN thinks I’m spamming because I’ve mentioned it too many times before in posts - hence immediate death of any submission, so I’m not mentioning it for a while to see if that tails off.
Bottom line: GPL3 for the compiler, standard library exclusions for derived works, so use it for anything but feed me back improvements to the compiler itself, and you don’t get to redistribute it as your own.
I like it! The Rust crew would love this I bet.
> I’m slowly coming around to the idea that if it works, I’m good with that.
A.k.a. "The concrete didn't crumble when I hit it with a hammer."
Process quality has been a thing for ages because it is essential.
This is how every product manager feels every day.
If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.
Agreed. The implication is, get really good at defining acceptance criterion.
Kind of, except as product manager you're held accountable for delivering the right product, while your engineering team is held accountable for things like performance, incidents, and security issues caused by their code.
It's hard to hold an LLM accountable for anything.
Accurate but manageable. Just like a pm works with properties an engineer also does. The Product person wants to deliver the right product properties so they model their process around identifying the ones they want and then creating the right framework to get them. On the engineering side its the same thing. Define what you want, give agents an environment they can succeed in, iterate.
What nonsense! If a product manager ships something that breaks then the software engineer who submitted the code gets the blame!
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
Which is - of course - absurd. Accepting responsibility without demanding the corresponding authority is a fool's game.
I have never been much of an advocate for unionisation in the software development industry. I didn't think it was necessary to protect the interests of the workers because for most of the industry's history the workers have largely been able to stand up for themselves anyway. Likewise I've never been a big believer in accreditation and licensing practitioners in our field - because who is qualified to judge someone else worthy in such a young industry with so little hard data about what works or doesn't work? I find my views on these subjects are evolving now.
It's not about protecting jobs if they truly aren't needed any more or protecting people working in the industry who refuse to move with the times. The Luddite philosophy isn't going to get us anywhere here. And besides - if AI is as effective as its advocates have been claiming then there should soon be more demand for capable developers to take advantage of the new opportunities. (The Jevons Paradox is relevant here.) But it is about protecting standards. If the tools aren't doing a good enough job then expecting professional developers to ship the code produced by those tools even though they know it's junk is something that needs to be challenged. And if developers are being expected to ship anything important without having the slightest idea whether it even works then that's even worse.
We had a lot of silliness going on when the very low interest rates and huge amounts of VC funding were propping up ideas and practices that would never have survived otherwise. Today we have a different kind of funny money distorting the industry but it's still propping up ideas and practices that we wouldn't normally support or perhaps even tolerate. And just like before no individual is going to be able to hold back the tide and it's going to have to be a collective effort to push back - hence my evolving views on topics like unionisation and accreditation. We've all seen the drop in quality across much of the industry and we all know why it's really happening because we see how the sausage is now being made.
Not all mistakes are code, a lot are feature design mistakes, and if SWEs would always be responsible about PM mistakes, then why not use Claude anyway?
Oh, don't worry. Someone is also blaming the product manager!
Maybe the new meta should be reviewing new code at the end of the week/month rather than at each merge? But even that might be too much of a bottleneck. The alternative is to talk to your agent to understand changes at a high level without reading every line of code.
It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.
The assumption was that some engineers knew how each part of the system worked, or at least the system was comprehensible because it was created by humans for other humans. There were comments explaining difficult places, etc. With AI this assumption is broken. Now nobody understands how things work, and it cannot be understood in principle because the system is a mountain of slop created by some alien intelligence.
I've been spending a lot of time iterating on feature implementation plans with Claude (at work) and GPT (for personal projects).
I then make sure they write a lot of tests and that if functionality is changed or added/removed then tests are updated alongside.
Generally I try to review changes myself and ask it to explain the changes.
It's not perfect but I feel like the planning step being iterative isn't all too different from my old research -> think about implementation -> implement loop.
At that point, reviewing any code changes goes hand in hand with how I coded before LLMs.
The major difference is I usually have it make any changes for things I don't like.
I'm miles away from all of this as a non-user of AI and it seems like a mania to me.
as someone using AI, it is mania, leadership has decided it is so and software is getting worse. we could have just accepted our 20% bump and been happy, fixed bugs, made things better quality, nope, ship half assed features at 2x speed, pretend claude is infallible and will solve all our problems, brand reputation? user research? pixel perfect designs? what are you a dinosaur?
I expect the guy sticking to his horse and cart, as the first cars started to overtake him on the roads, felt similar.
You make eat omelets using sourced material you have no way to verify.
You trust the label. The person. The company. You hope someone did their due diligence.
Sometimes taste the shell. That is an acceptable risk.
Except in this case the label on the eggs says "Eggs may occasionally be poisonous. Users should not rely on eggs to be safe, and should carefully scrutinise eggs before and during cooking. Bob's Farm accepts no liability for injuries resulting from making omelets with our eggs."
You should probably try it.
I lost my ability to fully explain the systems I work on around the time I transitioned from next-edit-prediction AI running in my IDE (early Copilot) and working primarily via Codex/Claude CLI's.
The further away I got from the implementation details, the fuzzier the semantics became. To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.
The solution to this, in my mind, is not to revert back to reviewing implementation details. The economics of that from a business/productivity perspective don't play out well.
The only viable solution in my mind is adapting code/languages themselves to the shift by making programs formally verifiable. That way you can be sure that the code "does what it says on the tin", and provably so.
I feel so strongly about this I've written on it:
https://gavinray97.github.io/blog/design-by-contract-and-eff...
You get it.
The bridge is built. How do we test the load? Drive bigger and bigger trucks until it fails, note the weight, print the sign and rebuild the bridge the same way.
> To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.
You weren't programming with LLMs in 2021 or 2022, be real. More like 2024 (that's already pushing it) or 2025.
I'd guess they started developing the project back in 2021 coding by hand, and are now improving/maintaining via agenetic engineering. This is not uncommon.
The point is that it's easy to lose understanding of the finer details, even of a codebase you lovingly created, once this switch occurs and an LLM gets its hands on it.
Because telling lies on the internet attached to my real name is so beneficial to my reputation, right?
https://i.imgur.com/CNAUczi.png
Receipts!
You poor soul. You were gambling against roulette odds.
Well, it's simple. Don't do it.
I agree with you with one caveat. Professionally, your obligation is to be qualified for your job and ship work that you stand behind. Complete comprehension is absolutely necessary.
Personally, with side projects and tooling, go nuts. There's no reason to read all of that code.
> Complete comprehension is absolutely necessary.
I feel like an increasing amount of orgs out there have long abandoned this thinking.
Your world will get smaller and smaller . I don’t think you can stop that wave.
Test driven development and the testing pyramid.
If the agent has written tests and crucially - witnessed them fail before starting the work to make them pass - you have a lot of confidence.
Review of your own work's easier too - test and production code tend to mirror each other, giving you a much more complete picture. Confused by the production code? Go read its tests for a bit, then come back.
It's also easier to understand the change if you were there for the build, watching the transcript, checking in every so often. Doesn't have to be your primary task - leave it on a second monitor, it'll catch your eye naturally.
And finally, on top of test-driven development, have the agent first (before the first test driven development cycle) build out an end-to-end test covering the work it'll do. Once that test is green (test driven word for "is passing") you're done with your work and can clean up at will.
Play with it before shipping, get it real nice. It's never been easier to clean the tech debt up and make the UI perfect.
Eventually you'll be ready to package it up into a PR!
Now's you can leverage agents and get creative. Make your colleagues froth to review your work. Consider making a Loom video to walk them through it. Consider reworking it into a commit series; it only takes a few second for your agent and you can tell a story for humans. Maybe you want to sequence it according to your struggles; maybe it's better to show the different layers emerge with each commit - it's up to you. Consider also that your agent can easily do one-off visualizations of the change for your colleagues if you think it'll help them, and you can attach those too - mermaid charts or even whimsical AI generated images. All this is because motivation is driven entirely by other humans' motivation and you want them frothing with excitement every time you drop a review. Make it fun, it takes seconds and gets your stuff merged fast while making your teammates smile.
This is a brand new job and we will work out how to do it together :)
In my experience you're just replicating the problem. Slop code doesn't stop being slop code when you add in slop tests to account for the sloppiness. If you generate code at a certain velocity, you eventually get to a point where you generate tests the same way you generate code. The model does its best effort in creating tests, but you'll inevitably get to a point in your work where the tests themselves are faulty and unreliable, the same sloppiness you were trying to control for.
There have been multiple times where my software dev team has gone back to prior implemented tests they had the model generate and found that after some time eventually the model got frustrated with the test no longer passing eventually modifying it to fit current needs around a feature push. You can probably anticipate how we came to discover this, as it ended up introducing a regression around multiple edge cases which the test was meant to control for.
The only answer is to inject actual engineering into the process as much as possible without giving it all up to the models to generate more slop in response to slop. tests are a great answer, but only if you treat them with serious intentionality.
Your engineers weren't doing what I'm describing I promise.
There are multiple places in my post where I explicitly describe the process by which you intimately:
- make the agent do verified work (the e2e tests and test driven thing)
- watch the agent work (the second monitor thing)
- read your own code (the tests/prod mirror thing
- play with the work and perfect it (the tech debt and UI perfect thing)
Plus a ton of other places in my post I very clearly describe being intimately involved in building and selling the work.
That to me is basic god damn competency. What your engineers did is a fad, and it won't last, and I'm sorry they abdicated their professional duty.
If you wish to screen for that sort of shit in future/current hires, show them my post and see if they turn up their noses or if they get excited and creative. You want only the latter.
I've been playing with this https://getchronosphere.com. It will (attempt to) rebuild a PR in a coherent journey rather than a massive blob. It's been somewhat helpful up to this point, and still improving.
I was struggling with this today. Working on a feature and AI review flagged something so I need to fix that. I iterated multiple times throughout the day and was completely exhausted cause each iteration was always complicated with the fix being ~1k lines. Finally landed on a one I felt at least a little comfortable submitting for review but a work day like this is super draining and I felt like crap the entire evening (I am about to go to bed, I am probably gonna have nightmares about pressing enter and enabling auto mode on plans that make no sense in a true nightmare sense)
I can think of so many times this has happened before. People getting used to MVC, Dependency Injection, ORMs. Sometimes the code is so large you only work on a part of it, so you have to get used to a lot of not knowing.
So how do we build confidence anyhow when we can't know everything?
I think first, standard checks - does it compile? What do static analysis tools say - code quality, passes SAST analysis? Linters? Style checkers?
Do the tests run? I'm not comfortable with the tests I write let alone the AI.. by adding fault injection (fuzzing, chaos), and mutation testing, I have a better degree of confidence in the tests and code.
Spend even more time getting to know the product instead. Maybe I don't know the code as well as I used to, but I know the product backwards and forwards. That helps me have some new confidence that I didn't really have before. Knowing the greater whole.
Your comment made me think that people have been happily importing hundreds to thousands of dependencies into their projects that were written by some random guys on the internet, but they've just now started to become concerned they don't understand the code they're shipping.
> So how do we build confidence anyhow when we can't know everything?
Avoid known unreliable tools?
I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.
It’s very disconcerting. I think this will largely be solved by going all in on AI and use it to test and review the code in an adversarial manner. That won’t be cheap but it’s the only way I see this work. But a lot of companies will probably save money and use AI mainly for writing code and then skimp on review and testing. I bet we will see some huge blow ups with AI written code and hopefully the industry will learn.
It also took the industry quite a while to accept that server code needs to be secure even if that requires a lot of extra effort. AI will probably be the same.
Force to always use TTD, at least validate the tests.
Recommend reading (All though a little self serving): https://www.linkedin.com/pulse/ai-code-quality-jorge-varas-a...
Formal verification, pick a language that has it integrated and prove relevant post conditions.
AI still does not know how to even write code that makes impossible state unpresentable so testing won’t help.
Im currently building a ui framework for unreal. I dont enjoy ui work so I wanted a way to have to write less ui.
For me I dont look too closely at the implementation but I look at the api where if a human was using my framework how is the code they'd have to write and engage look at. Then I also do plenty of performance against a comparable target in my case slate (default unreal) and keep overhead minimal.
So good performance and reasonable public facing api are the 2 main pieces im focused on. Less with the internals.
Care about the seams and interaction and the things you can measure or count.
I shipped code I didn't understand when I copy pasted from Stackoverflow before. At a level if it works it works.
I'm currently working on a combined Aero sim/CAD/CAM software for designing, simulating, and performing DFM optimization specifically for Group 1 to 3 winged UAVs.
What I do is ask Opus/Gemini to explain the parts I don't understand. And I do some of my own research to verify accuracy of the explanations.
In addition:
* Every new feature has a Feature Design document which I review, modify, and approve. And a corresponding Implementation Plan which I also review and approve.
* Every fix has a RCA document and fix Implementation Doc which I review, modify, and approve.
* There's a global high level Architecture and Design document with additional sub component design documents which are always kept in synch after a new feature or bug fix is done. Synching it is always a step in the Implementation plan.
This does adds some overhead but it allows me to have have check point gates. This has often allowed me to catch misunderstandings and align properly before any code gets edited by the AI agent.
Having the above docs also helps when it comes to referencing them as context to properly ground new features or bug fixes.
Hey this is very cool, reach out to me https://alexanderaghili.com/ because Id love to learn more.
By taking responsibility of it, maybe by saying "Hey this is not a good idea", and if that doesn't work perhaps by switching jobs if that's an option?
The mental gymnastics developers are willing to go through just so they can avoid standing up for something is quite perplexing.
Product doesn’t care. And most of us will get replaced by agents. I guess around 30% of the engineering workforce will keep their jobs which will be mainly to keep the product up and running, doing whatever it takes (so using agents to fix stuff basically).
The ones who get to keep their jobs will be rebranded as Maintenance engineers with a lower salary. VPs and Heads of will keep their cozy job (bc there are not so many of them), and they are not that dumb to fire themselves.
Code understanding doesn’t matter anymore (or won’t matter soon)
I had same problem, but after a while I just accepted that is inevitable
"This code is really ugly."
That sucks. Ship it.
That's the neat part, you don't.
Or at least, I don't think people should.
To me this completely goes against the whole point of engineering.
IMO If you don't understand the code, then you're just a manager now. Soon your salary may also reflect as such, assuming you get to keep your job.
I don't ship code I don't understand. I fully review every line if it's going to prod.
My dumb little utils in ~/.local/bin? Those get merged after a quick functional test without my having read a single line of it. Still, the skill I maintain for such a job directs agents to write such throwaway scripts in anything but shellscript because I might want to extend some day and fuck shellscript
>This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever.
"This new way of piling bricks is so fast that I can no longer afford to check the buildings integrity".
I just don't understand how the fuck we ended up with this narrative. Somehow the machine making things twice as fast forbids us from spending time ensuring quality? Surely if it does things faster you have more time and tools to check!
I'm half convinced this frenzy is the marketing work of AI companies at this point.
> this frenzy is the marketing work of AI companies
Worse. "AI" is a self-marketing contagious delusion, driven by an inexhaustible power supply. Human gullibility.
>How are you handling this?
easy, don't write code with LLMs. If my job forced me to do so I would quit.
I have many questions. This seems shortsighted.
Short-sightedness is quite literally the reason people are using these systems to begin with. If you care about the health and integrity of your code base years down the road you wouldn't be shipping endless amounts of slop you don't understand, Fred Brooks taught us this half a century ago.
You just stop caring. Everyone else did about three months ago. Time to find something new to care about.
Claude to Prod. Every day.
Amen. If it works, its good enough.
The CVEs are going to be epic.
I don't understand these types of comments. Aren't the CVEs epic today with human written code? What difference does it make then? You want no CVEs, make software engineering into a real gate-kept engineering professions with accountability chains, government regulations, formal education, etc.
> You want no CVEs, make software engineering into a real gate-kept engineering professions with accountability chains, government regulations, formal education, etc.
This will not only not prevent CVEs, it will also make everything a giant pain in the ass
Of course it will. But at the same time I don't see building falling left and right like CVEs in software. So don't tell me agents write bad code, because humans do as well. The only way to prevent this (or minimize), is to turn software engineering into real engineering.
"If it works, ship it" is very different from "If it works, and I understand it because I wrote it, ship it".
Does the LLM understand it, in detail? Can you trust its interpretation of the information and instructions you provided? Did it independently make an adverse decision you haven't noticed, because you didn't read the code? Was the model missing any information it required -- and would you know if it was?
LLMs depend on humans and human oversight to ensure they have the information required to make the right decisions, to catch when they don't, and to steer them based on your much more complete understanding of the broader context and requirements. You can't rely on them to do that thinking for you (yet?).
I’d argue that most people also don’t understand code they read from their colleagues. Or if they do, then they don’t understand the entire context of the app.
You said LLM need human oversight and it’s true, but it does not imply you have to read the code.
I think we are going towards a place where code no longer makes sense, you just describe what you want in English, and the LLM just builds directly in machine code. You then build different verification tests to make sure the program is doing what it’s suppose to do.
It's simple: you don't. You slow down. It's essential to understand what you ship, before you ship, as there are real world consequences for not doing so, ranging from wasting the time and energy of yourself and others, up to failures which cause a great deal of harm to people and the world. To knowingly ship code you don't understand is grossly negligent, regardless of how common it becomes.
> How to deal with shipping AI-written code you don't understand?
Stop shipping code you don't understand.
Is there really any other solution? You can either ignore it and pretend like it's the new era of software like half the other commenters are, or you can sit down and understand the code and what you're shipping.
I wouldn't be surprised if the meta person has a gun to their head to increasingly keep shipping like a maniac (how decade-long bugs remain unfixed don't ask me), so this advice may not be very useful.
An engineer from Meta not knowing what they're doing is exactly how I imagined engineers at Meta. Thank you for confirming it's an actual reality.
On the other hand, not even surprised anymore most people even try to provide serious answers to this. Welcome to the pure insanity.
Don't.
By doing so, you debase your own profession.
Generally speaking I brain storm the feature first and produce a dev plan. This includes asking it to reuse code/libraries we’ve already got! After it’s gone off ah done its thing we go back over and do a QA on what it’s built, why that approach, what can we do better etc etc?
I spend time to add extra observability, cross-checking and alerting at runtime to offset the risk.
You are accountable.
You're going to have to understand it one way or another, either at writing or at debugging. Even if you delegate all debugging to AI, you'll still need to determine what is "right".
If you don't understand the code at writing or debugging, you'll pay the architectural price (where it degrades). That price may be worth paying, but that depends on the system you are building.
It feels like another layer of "conceptual compression". I'm not sure who/when that phrase came around but in context of Rails is where I heard it and its things like ActiveRecord or really any ORM. You aren't writing the SQL directly, something else is and for the most part it works. Then it doesn't work or your database grows too much and you need to optimize table structures with indexes, etc.
My point is if you don't understand the fundamentals of the underlying code (SQL in the example above), you're more likely to have issues in the future. Or maybe not if that app doesn't reach a usage pattern or complexity where the nuances under the hood matter.
It really depends on what you're shipping. Meta ships social media apps where in the grand scheme of things if something is broken software wise it largely doesn't matter. What's the worst case? lost ad revenue? Users can't do things? If that were software for MRI machines or aerospace where software issues/bugs can result in lives lost then it matters much more in understanding what is going on at all levels of the software and potentially hardware.
If you don't understand it, then you haven't built it.
Anyone who trusts the output of LLMs uncritically, even after the impressive level models and harnesses have gotten to, is borrowing against their future.
Models and harnesses have gotten really good, but they still are hard-capped by the operators that use them. The sycophancy of models is perhaps their greatest weakness, as when a user who might be experienced in one area attempts to use AI to build within an area they are less familiar with, the model does not take a step back and examine the "why" of what is being built from an architectural standpoint, it simply performs the expected task in a direct manner, ultimately creating sloppy work that may be easy to ship but difficult to maintain.
A good example of this was a recent interaction I had with a Dev who was very familiar with how to develop embedded and system level applications. He and every other dev are being asked for increasingly more due to AI, and he had to bring something he had built as a single-user desktop application into a distributed cloud environment. He ended up rsyncing a copy of his working directories including multiple credentials and sensitive internal project documents to a virtual machine binding it to a public ip address. Thankfully our tooling detected this before he could get any farther but it illustrates how the AI model isn't doing any real thinking or engineering at the end of the day. Unless the operator knew to instruct the model about proper cloud architecture and understood best practices around deployment of public systems, the model simply did what it was being told to do by a user who didn't know any better.
Another example of this is the sheer amount of unsophisticated users we've been detecting using Javascript injection into their web browser sessions as a form of automation for web services they may use. These users would never know to ask how to request an API token which may allow them to accomplish things in a much easier and approved way, and instead are relying on the frailest possible automations that expose them to risks of credential exposure.
We've got to be careful with how we engineer things with AI models. Just because we've made boilerplate easier to generate doesn't mean that we can avoid the actual engineering of what we do. And I can't help but think that firms which are fully committed to AI code generation, won't just be introducing an avalanche of slop into their product which will explode the costs of maintenance over time.
If it's important code (and much isn't), then I spend a lot of time using Matt Pocock's /grilling first, and then I get the agent to kick off peer-review with other agents, and surface any disagreements. Some models are better at it than others. This usually forces me to broadly understand the architecture, and I spend most of my time answering the design questions for which there isn't an obvious solution. I have been pretty happy with the code I've launched to production, as measured by the ways it breaks, and by how agents who weren't part of the original committee describe it.
This is the state that founders and decision makers are usually in at a startup. Sure, sometimes you have the coder founder who wrote the first iteration of the product, but many times the founders have no clue how the product works under the hood. They entrust that to the engineers. That was then. Now, much of that engineering layer has been removed, so the founder just needs a smaller team of people who understand how to structure the software and interact with the LLM. LLMs do the rest.
The problem here is that you're trying to do a job that is already being done for you. Maybe you don't yet trust it to do a good job, but that is quickly changing.
We just don't do that, I don't know why people talk like that is a given
Is it a company culture thing or a domain specific thing?
Do not
Replace "AI agents" with "a bunch of human coders" and you'll see the answer. Your product needs decent testing and it's your job as the responsible person in charge to make sure that's in place and working properly. So if anything you probably want to be reviewing the test code. But often that isn't necessary since you can do things like introduce bugs deliberately and see if tests fail; check test logs for information that confirms the tests were run as expected; try some manual testing once in a while; ask for new more advanced tests (e.g. test performance not just functionality); and so on.
> How to deal with shipping AI-written code you don't understand?
Recognise you aren't the one shipping it.
It's Meta. Shipping crap is par for the course. You're fine.
The part that makes me uneasy is when the AI writes both the code and the tests. They might agree with each other and still miss the same thing. How are people here checking for that without doing the whole job twice?
Not sure about that. This was structurally the same before AI: the author of the code wrote the tests. The tests still helped.
The author of the code was physically touching the internals and potentially intuiting obvious shortcomings as they worked on it. That very crucial part is now gone.
I like fault injection (fuzzing, chaos), and mutation testing. I think introducing these helps the AI write better tests and helps having some confidence in the tests that are written.
You should be using two (or even three) different LLMs. For example: Claude to do the coding, and Codex to do an adversarial review. This can go back and forth for several iterations before converging on an acceptably "correct" version of the code...
Occasionally, Claude may not be able to fix the issue – and it may turn out that Codex can, because it has a different approach.
Either way, at least two LLMs is the only way to get (mostly) reliable code.
You can automate this entire orchestration with zeroshot.sh, mentioning it since it is relevant here.
Claude, ChatGPT, Google Gemini, then Replit...