This isn't an issue for engineers only, it is basically ingrained in the current system in the world. The problem is in simple terms; the fact that being a polymath is absurdly hard in today's world.
You can't not reinvent something if you don't know it exists in the first place.
The thing about software is that it slots nicely into every other field, making it a _very_ good base to work off of to get concepts (aka the modern bootleg polymath) that have likely been invented in other fields (with different names).
I'm a hands on engineering leader for a team of about 20 engineers and I've been spending the past 6 months trying to get my team to understand just this.
On Friday we had a coffee hour to share how we've been working recently and they all seemed perplexed at the workflows I've been adopting. It seems natural to me as someone who's been a manager for some time now, but very alien to all those who've never gone down that path.
That's not to say my workflows are superior, but they're extremely different to some of my teams now. In reality it's just leaning heavily on things like prds, limiting communication between agents, etc.
Out of curiosity, what was the path that the engineers have been taking? The limiting communication between agents to my mind is obvious to anyone who hasn't swallowed orthodoxy Agile development completely whole. And in my experience the people who have, are rarely the devs. Context switching, between agents, people or whoever, require ramp up time to relearn context. It's always slower and more expensive, other potential upsides about long term training or developers being more replaceable notwithstanding. But no agent gets long term training, not in a 1M context window.
Relatedly, I fully expect the software business to fail to learn from (nearly) literally every other industry how to operate when your marginal costs are no longer zero. It drives me nuts the number of people in software who think that people in other industries are slow because they’re just not as smart as us, rather than because when it takes months to get a cast part made, it better be right the first time.
It's even funnier when you realize that not too long ago a decent chunk of the software industry did have to deal with the realities of having to press CDs, print boxes and ship them to retailers in time for the holiday season. The idea that you can ship a half-baked product directly to your end users, charge for it and promise it will get better over time (but only if you manage to secure enough VC funding in the meantime) is a relatively new invention.
IDK, I think it's been a bit of a (reverse?) bell curve of sorts.
30+ years ago, You had to be a -lot- more careful about how you handled releasing software. Your stuff was in boxes on shelves, most people didn't have internet access to 'download an update' so fixing anything had a long tail support cost (i.e. paying for the media containing the fix and shipping).
But there's a fine line. As an example, If your shop is having a vendor do brand new stuff, -compartmentalized from your other stuff-, using technologies that are actually out of support vs an in-support version is a free/zero effort update versus using the outdated stuff... Something went wrong on the management side.
How many updates did one apply to Windows 3.0 and Word in those days??
Sure, there might have been a “hotfix” available for download over a long distance BBS, but virtually nobody would have ever done so (even if they were lucky enough to have a modem). Maybe large companies applied such when impacted.
Even in the Windows XP days,
I think windowsupdate.com was still a manually triggered process…
For a while, bundled drivers were a steaming pile of crap in quality and several generations of Windows suffered greatly until Microsoft got vendors in line and updates became common.
Yes, the future of software as an institution, if there is any, is driven by the loudest voices. Those loudest voices are thus driven by selected personalities as opposed to any accomplishment or title. The people who are the highest achievers are rarely the loudest voices as they tend to be the people spending time solving real problems as opposed to the people who just talk about themselves, the forest for the trees.
Yet this is part correct, part wrong. Indeed every industry has some unique challenges that are not directly solvable. However, while a cast is absurdly expensive, the problem is being solved in large part by software in the FDM (3D printing) space and it's changing how things are built.
edit: this doesn't invalidate the OP thesis, which I strongly agree with. Except that it's not just engineers, this is a more general thing across many fields.
Do you work in this field or are you just reading that? I’ve been out of the game for over five years now, but as far as I can tell from people I know still in that world, if you’re going to make more than like … 3 of a part, casting is going to be much cheaper. And while you can do some prototyping on 3d prints, the nature of the material winds up different from a cast part.
That's totally untrue. You clearly do not have any experience designing, building, manufacturing, testing, and certifying either cast metal or PBF parts. BTW, PBF is powerbed fusion and it is how you make 3D printed metal parts. FDM is a plastic manufacturing process and has nothing at all to do with replacing metal castings.
For a certain aircraft part that I designed, that measured approximately 16"x8"x10", if you needed up to 12 of them it was cheapest to 3D print them using powerbed fusion from AlSi10Mg, but with a 25% weight penalty due to the lower mechanical properties of 3D printed aluminum compared to cast. If you needed between 12 and 50 of them, it was cheapest to use 3D printing to make a wax preform then use the preform to make a traditional metal investment casting from A356. For more than 50 of them, it was cheapest to invest in permanent closed die tooling.
Other than for prototyping and extremely low volume production, additive has been a total flop for real-world manufacturing, and has not at all lived up to any of it's lofty promises.
Do you see that changing as parts are designed for additive manufacturing? Things that basically cannot be made another way, but have strength, weight, and even functional benefits over a traditionally made part?
I’m thinking of say cooling channels built into walls of rocket combustion chambers, or car brake/axle assemblies, and some of the „organic“ designs that are lighter and stronger than a cast/machined part.
>You can make lots of money by making something appear novel and undiscovered, and consequently make yourself sound smart and cutting edge. Nobody raises a round to apply a well understood discipline correctly.
I'm curious about this. I thought investors preferred safe bets?
On the other hand, I know that if you're too early, it can be impossible to make a business work (or even to pitch the idea in the first place).
It's really funny, I've read this essay before a long time ago and it's really interesting playing back the many times both in realising it in retrospect and having seen it playing out before my eyes, that the last part really comes to the fore
> One final point: I expect none of you to really get what I’m talking about here, because this principle also applies to itself. But I fully expect I’ll get the occasional email saying “Oh! so that’s what you meant.” or “Why didn’t you tell me that?” I did, but you can’t tell people anything.
In my youthful naivety I thought I understood this, but now years later I can more deeply appreciate the message that you really can't tell people anything
A few years ago a software engineer asked a few Chemical/Civil/Mechanical engineers and the majority of "real" engineers seemed to think we were [1]. I'm not aware of a better study but happy to be proven wrong, since I generally refer to myself as a Software Developer
Not sure it’s obvious reasons, but understandable, sure. When someone questions your credentials the default is to tell them to jump, whether that’s a good idea or not.
Because Engineering is an incredibly rigorous discipline and title that requires so much work to obtain. It’s like a massage therapist calling themself a doctor. Computer Science isn’t Engineering.
Absolutely. Engineering is defined by it learning from history, it's an art build up from thousands of years of human attempts to alter the world around them. In a modern sense engineers inevitably require a certain amount of schooling, certification, and most critically of all professional and ethical standards.
He cites Brooks as proving that communication friction is quadratic in headcount. The exponent does not need to be exactly, or even nearly, 2; 1 + epsilon is already fatal. This is very closely related to Coase's ceiling.
This is mostly wrong as far as motivation is concerned. Some people just by default figure stuff out with what they know. Others accumulate knowledge. If you go too far in either direction you are probably going to struggle, but it's just different people having different nature.
Man I love and agree with nearly everything about this post except that I wouldn't take the advice to follow waterfall or PMBOK literally.
The post laments that it's hard to find Dr. Royce's original waterfall paper. That's probably true, I have my copy from a compilation book “Ideas that Created the Future: Classic Papers of Computer Science” edited by Lewis [1].
I do agree that it's important to do things like scope out your demands of your AI agent, check-in on progress, give as clear a requirement and test cases as you can. But you've always been able to do that with agile methods, and LLMs are fast enough that you don't need to go full waterfall (and if anything it would be counter productive).
Dr. Royce's paper talks about literally thousands of pages of documentation being needed for any reasonably useful system. Good luck fitting that into even a 1M context window :P.
But the main point to thesis, that you can't just let your coders loose to do whatever and expect the right results even pre-dates Fred Brooks. I'd argue it goes all the way back to the beginning, to the comments about Baggage's computing machine where British politicians asked if it would generate the correct answers even with incorrect inputs.
The answer then is the same answer today: of course not, and expecting anything different is foolishness.
As someone that has studied engineering, designing a solution is very different than managing the resources around the design and implementation of that solution. Books like The Mythical Man Month and Making Things Happen deals with the management side, (from a team level) and The Pragmatic Programmer is from a personal level.
But TFA is ignoring the actual engineering side where you have to grapple with some primitives and assemble them in a way that do something valuable. And so in a way that is cost effective. That part is always answered by hand waves.
Nobody gets rewarded for rejecting the newfangled thing. Even when the newfangled thing should be rejected. Same as scientific papers are very biased against reporting nulls.
There’s a good list of reasons why we do this, but it leaves out the biggest one, at least for me: building stuff is fun. Reinventing stuff is fun. It’s the most natural hammer to reach for whenever I encounter a nail. Not necessarily the best, but such is life.
The author missed one key motivation: giant fucking ego. I don’t know why every damn software engineer thinks they are gods gift and can derive anything from first principles. The OP hinted at it with the physicist XKCD but I think it is far more dominant in software because the payouts for mediocrity are just astronomical.
I don’t quite understand the management analogy. Agents aren’t people, they’re code generation machines. Part of managing is delegating and trusting the output of your employees without having to verify everything yourself. But if you’re “managing” an agent, how does that work? You can’t exactly hold an LLM responsible if it fucks something up.
Or are people really just yoloing and not even verifying that the code generation is correct? I know it’s a bit of a meme, but are people actually doing the meme in irl where there are actual consequences??
Another obnoxious behaviour I’ve observed lately, is trying to attach whatever pre-existing pet methodologies one had to the AI-hype bandwagon like some sort of personal vindication orgy.
The joke at my school was when the engineering students failed (mech, EE, civ) out, they went into business, when they failed out of business they went into software engineering.
This isn't an issue for engineers only, it is basically ingrained in the current system in the world. The problem is in simple terms; the fact that being a polymath is absurdly hard in today's world.
You can't not reinvent something if you don't know it exists in the first place.
The thing about software is that it slots nicely into every other field, making it a _very_ good base to work off of to get concepts (aka the modern bootleg polymath) that have likely been invented in other fields (with different names).
> You can make lots of money by making something appear novel and undiscovered, and consequently make yourself sound smart and cutting edge.
This is pretty much the crux of it. It's very similar to the strategy of undercutting a market with VC subsidies until it dies and can be replaced.
VC subsidies are no longer enough, now we need IPOs to keep it going...
An IPO and subsequent offers is hardly a temporary subsidy. That can keep going until the economy dies.
Mate, its so fucked, people are going to cry way worse than 2008.
people in these fields we pre-crying, building the torment nexus, because if they dont, they'll be in the "permanent underclass".
FOMO is going to kill more americans than COVID.
As an MCP engineer, how dare you
Sounds like Apple.
I'm a hands on engineering leader for a team of about 20 engineers and I've been spending the past 6 months trying to get my team to understand just this.
On Friday we had a coffee hour to share how we've been working recently and they all seemed perplexed at the workflows I've been adopting. It seems natural to me as someone who's been a manager for some time now, but very alien to all those who've never gone down that path.
That's not to say my workflows are superior, but they're extremely different to some of my teams now. In reality it's just leaning heavily on things like prds, limiting communication between agents, etc.
Out of curiosity, what was the path that the engineers have been taking? The limiting communication between agents to my mind is obvious to anyone who hasn't swallowed orthodoxy Agile development completely whole. And in my experience the people who have, are rarely the devs. Context switching, between agents, people or whoever, require ramp up time to relearn context. It's always slower and more expensive, other potential upsides about long term training or developers being more replaceable notwithstanding. But no agent gets long term training, not in a 1M context window.
> obvious to anyone who hasn't swallowed orthodoxy Agile development completely whole.
What? One of the first principles in doing Lean and Agile correctly is limiting Work in Progress and avoiding context switching whenever possible.
Can you share some details of the workflows that work for you?
Is anyone on your team not using multi agent?
Relatedly, I fully expect the software business to fail to learn from (nearly) literally every other industry how to operate when your marginal costs are no longer zero. It drives me nuts the number of people in software who think that people in other industries are slow because they’re just not as smart as us, rather than because when it takes months to get a cast part made, it better be right the first time.
It's even funnier when you realize that not too long ago a decent chunk of the software industry did have to deal with the realities of having to press CDs, print boxes and ship them to retailers in time for the holiday season. The idea that you can ship a half-baked product directly to your end users, charge for it and promise it will get better over time (but only if you manage to secure enough VC funding in the meantime) is a relatively new invention.
> did have to deal with the realities of having to press CDs, print boxes and ship them to retailers in time
This is because the we have these periodic influxes of scammers into the IT the industry, with their empty promises and trust me bro culture.
IDK, I think it's been a bit of a (reverse?) bell curve of sorts.
30+ years ago, You had to be a -lot- more careful about how you handled releasing software. Your stuff was in boxes on shelves, most people didn't have internet access to 'download an update' so fixing anything had a long tail support cost (i.e. paying for the media containing the fix and shipping).
But there's a fine line. As an example, If your shop is having a vendor do brand new stuff, -compartmentalized from your other stuff-, using technologies that are actually out of support vs an in-support version is a free/zero effort update versus using the outdated stuff... Something went wrong on the management side.
Exactly!
How many updates did one apply to Windows 3.0 and Word in those days??
Sure, there might have been a “hotfix” available for download over a long distance BBS, but virtually nobody would have ever done so (even if they were lucky enough to have a modem). Maybe large companies applied such when impacted.
Even in the Windows XP days, I think windowsupdate.com was still a manually triggered process…
For a while, bundled drivers were a steaming pile of crap in quality and several generations of Windows suffered greatly until Microsoft got vendors in line and updates became common.
> Your stuff was in boxes on shelves...
Have you ever heard of an SLA? This "used to be" story about software has never been true. If you can't meet the agreement, then or now, you're done.
Software has always been a lot more diverse than just consumer goods. Why is that your only evidence of how you think software projects are managed?
Yes, the future of software as an institution, if there is any, is driven by the loudest voices. Those loudest voices are thus driven by selected personalities as opposed to any accomplishment or title. The people who are the highest achievers are rarely the loudest voices as they tend to be the people spending time solving real problems as opposed to the people who just talk about themselves, the forest for the trees.
As a buddy in construction used to say, concrete erasers are expensive.
Yet this is part correct, part wrong. Indeed every industry has some unique challenges that are not directly solvable. However, while a cast is absurdly expensive, the problem is being solved in large part by software in the FDM (3D printing) space and it's changing how things are built.
edit: this doesn't invalidate the OP thesis, which I strongly agree with. Except that it's not just engineers, this is a more general thing across many fields.
Do you work in this field or are you just reading that? I’ve been out of the game for over five years now, but as far as I can tell from people I know still in that world, if you’re going to make more than like … 3 of a part, casting is going to be much cheaper. And while you can do some prototyping on 3d prints, the nature of the material winds up different from a cast part.
That's totally untrue. You clearly do not have any experience designing, building, manufacturing, testing, and certifying either cast metal or PBF parts. BTW, PBF is powerbed fusion and it is how you make 3D printed metal parts. FDM is a plastic manufacturing process and has nothing at all to do with replacing metal castings.
For a certain aircraft part that I designed, that measured approximately 16"x8"x10", if you needed up to 12 of them it was cheapest to 3D print them using powerbed fusion from AlSi10Mg, but with a 25% weight penalty due to the lower mechanical properties of 3D printed aluminum compared to cast. If you needed between 12 and 50 of them, it was cheapest to use 3D printing to make a wax preform then use the preform to make a traditional metal investment casting from A356. For more than 50 of them, it was cheapest to invest in permanent closed die tooling.
Other than for prototyping and extremely low volume production, additive has been a total flop for real-world manufacturing, and has not at all lived up to any of it's lofty promises.
Do you see that changing as parts are designed for additive manufacturing? Things that basically cannot be made another way, but have strength, weight, and even functional benefits over a traditionally made part?
I’m thinking of say cooling channels built into walls of rocket combustion chambers, or car brake/axle assemblies, and some of the „organic“ designs that are lighter and stronger than a cast/machined part.
> It drives me nuts the number of people in software who think that people in other industries are slow because they’re just not as smart as us. . .
Ha! I studied computer science and became a programmer because I knew I was too dumb for med. school and the other engineering paths.
That’s because most Software Engineers aren’t Engineers, they’re computer science majors. It’s a completely different discipline.
>You can make lots of money by making something appear novel and undiscovered, and consequently make yourself sound smart and cutting edge. Nobody raises a round to apply a well understood discipline correctly.
I'm curious about this. I thought investors preferred safe bets?
On the other hand, I know that if you're too early, it can be impossible to make a business work (or even to pitch the idea in the first place).
Related: You can't tell people anything (2004)
https://web.archive.org/web/20091025030730/https://habitatch...
It's really funny, I've read this essay before a long time ago and it's really interesting playing back the many times both in realising it in retrospect and having seen it playing out before my eyes, that the last part really comes to the fore
In my youthful naivety I thought I understood this, but now years later I can more deeply appreciate the message that you really can't tell people anythingSeems like this could be categorized as yet another reason why software developers are not engineers.
A few years ago a software engineer asked a few Chemical/Civil/Mechanical engineers and the majority of "real" engineers seemed to think we were [1]. I'm not aware of a better study but happy to be proven wrong, since I generally refer to myself as a Software Developer
[1] https://www.hillelwayne.com/post/are-we-really-engineers/
I never understood why actual engineers(eg. Civil) have not complained more that programmers/coders stole that title.
they have, loudly and repeatedly, it doesn't really make it very far on HN for obvious reasons.
Not sure it’s obvious reasons, but understandable, sure. When someone questions your credentials the default is to tell them to jump, whether that’s a good idea or not.
Why would they care? And a lot of them turned to coding anyway.
Because Engineering is an incredibly rigorous discipline and title that requires so much work to obtain. It’s like a massage therapist calling themself a doctor. Computer Science isn’t Engineering.
Absolutely. Engineering is defined by it learning from history, it's an art build up from thousands of years of human attempts to alter the world around them. In a modern sense engineers inevitably require a certain amount of schooling, certification, and most critically of all professional and ethical standards.
He cites Brooks as proving that communication friction is quadratic in headcount. The exponent does not need to be exactly, or even nearly, 2; 1 + epsilon is already fatal. This is very closely related to Coase's ceiling.
This is mostly wrong as far as motivation is concerned. Some people just by default figure stuff out with what they know. Others accumulate knowledge. If you go too far in either direction you are probably going to struggle, but it's just different people having different nature.
A comments section about an HN comment about a post. We can recurse even deeper.
More then just engineers, we all need to study history
Man I love and agree with nearly everything about this post except that I wouldn't take the advice to follow waterfall or PMBOK literally.
The post laments that it's hard to find Dr. Royce's original waterfall paper. That's probably true, I have my copy from a compilation book “Ideas that Created the Future: Classic Papers of Computer Science” edited by Lewis [1].
I do agree that it's important to do things like scope out your demands of your AI agent, check-in on progress, give as clear a requirement and test cases as you can. But you've always been able to do that with agile methods, and LLMs are fast enough that you don't need to go full waterfall (and if anything it would be counter productive).
Dr. Royce's paper talks about literally thousands of pages of documentation being needed for any reasonably useful system. Good luck fitting that into even a 1M context window :P.
But the main point to thesis, that you can't just let your coders loose to do whatever and expect the right results even pre-dates Fred Brooks. I'd argue it goes all the way back to the beginning, to the comments about Baggage's computing machine where British politicians asked if it would generate the correct answers even with incorrect inputs.
The answer then is the same answer today: of course not, and expecting anything different is foolishness.
[1] https://www.amazon.com/dp/0262045303
He says "engineers" but he means code monkeys. Actual engineering is all about learning from past failures
Solo programming is like managing a team of developers that can only communicate and contribute in one temporal direction.
Sometimes two or more developers may appear to coexist, but it's actually a simulated concurrency achieved by rapid context switching
Every time you close a file to open another, you are setting up a different context, therefore you are different agents.
Most HUMANS will do anything to avoid learning from history. Engineers are merely an example. Sigh.
Take my money. This is better than "The Pragmatic Engineer".
As someone that has studied engineering, designing a solution is very different than managing the resources around the design and implementation of that solution. Books like The Mythical Man Month and Making Things Happen deals with the management side, (from a team level) and The Pragmatic Programmer is from a personal level.
But TFA is ignoring the actual engineering side where you have to grapple with some primitives and assemble them in a way that do something valuable. And so in a way that is cost effective. That part is always answered by hand waves.
Nobody gets rewarded for rejecting the newfangled thing. Even when the newfangled thing should be rejected. Same as scientific papers are very biased against reporting nulls.
There’s a good list of reasons why we do this, but it leaves out the biggest one, at least for me: building stuff is fun. Reinventing stuff is fun. It’s the most natural hammer to reach for whenever I encounter a nail. Not necessarily the best, but such is life.
The author missed one key motivation: giant fucking ego. I don’t know why every damn software engineer thinks they are gods gift and can derive anything from first principles. The OP hinted at it with the physicist XKCD but I think it is far more dominant in software because the payouts for mediocrity are just astronomical.
I don’t quite understand the management analogy. Agents aren’t people, they’re code generation machines. Part of managing is delegating and trusting the output of your employees without having to verify everything yourself. But if you’re “managing” an agent, how does that work? You can’t exactly hold an LLM responsible if it fucks something up.
Or are people really just yoloing and not even verifying that the code generation is correct? I know it’s a bit of a meme, but are people actually doing the meme in irl where there are actual consequences??
Another obnoxious behaviour I’ve observed lately, is trying to attach whatever pre-existing pet methodologies one had to the AI-hype bandwagon like some sort of personal vindication orgy.
We don’t have long before it is all made irrelevant so need to make one final dash for glory before it’s too late.
See also: people.
You know the old saying: "Those who can't do, teach. Those who can't teach, go into software engineering."
I think it needs to be updated to just say "Those who can't do, go into software engineering". Teachers don't deserve that crap.
What? The joke saying about that is "those who can't teach, teach gym".
Never heard anyone say that about software engineering.
The joke at my school was when the engineering students failed (mech, EE, civ) out, they went into business, when they failed out of business they went into software engineering.
And what happened when they failed out of "software engineering"?
Among many teachers it is, "those who can't teach become education consultants/researchers."
That's the joke. That software engineering is the new "teach gym"