Overall a good collection on the complex systems, somehow it missed the part how complex system came into the existence in the first place itself.
> Human expertise in complex systems is constantly changing
I feel this is single most important factor responsible to both making system complex and at the same time improving them depending on who the people are and how they take failures and breakdowns. I find it funny but we are in an era where folks building ML systems do not seem to remember the direction in which to open the screws. They do have clear expertise in something new but clearly lack in some other areas
I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing.
The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure state. Naturally, the "root cause" seems like lock system resiliency. But definitionally a metastable failure is one that persists after the inciting condition is resolved. Now you have two "root causes", the lock failure and the metastability of the deployment system fault. Keep looking and you'll find more.
But to me the biggest brick to the forehead in this piece is further observation that random things are failing all the time in any complex system. "Complex systems run in degraded mode". Resilient components are good, but it's the resiliency of the overall process that orchestrates the whole system that determines whether things are going to blow up.
All practitioner actions are gambles. I should have that inked somewhere.
> I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing.
If you want a bit of cheese to go with that wine, this article pairs nicely with The Grug-Brained Developer: https://grugbrain.dev/
> Failure free operations require experience with failure.
This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.
I've always struggled to apply this to the systems I work on. If the system fails, someone potentially dies, though in practice they've never been more than hospitalized. To avoid that, huge amounts of effort are expended on failure modelling and testing, but that doesn't eliminate unknown unknowns. That discrepancy has made front page news a couple times.
When I was at Netflix and reddit, one thing I said often was, "Luckily, we are not a bank". And it sounds like you are working on even more critical systems than that.
Chaos engineering doesn't really apply to data critical or safety critical systems. You can't just break them in the real world to see how they fail.
You have it exactly right -- it has to be modeled and tested in lab conditions. Safety critical systems are not a place for YOLO development.
It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be unexpected, related to your decisions and really hurt so that you can learn from them.
PS. Btw, I am certain that this is the main cause of the mental health crisis amongst young people.
> One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures.
Its cause we already learned from our failures and make the world a safer place. In my youth we climbed on trees, one of my friend fall down broke his arm, the doctor couldn't fix it and he can not move the hand for the rest of his live. My friend would be happy do not been allowed to climb, or at least under safer conditions but have his hand moveable still. I burned myself really heavy with fireworks as teenager and have a huge scar from it. I never touched any firework after this but also never bought it for my kids, so they never had the chance to "learn" from making the the same mistake.
Keep in mind that while claiming the world is too safe you say this from a perspective of an survivor.
I may have shared this before on a different submission: John Gall’s books are really good on this topic: General Systemantics [https://en.wikipedia.org/wiki/Systemantics]
Gall's law is amongst my favorite ones and with decades of experience in software development, I have to say it holds absolutely true:
> “A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.” — John Gall, Systemantics (1975)
Richard Cook was a UChicago anaesthesiologist who took up safety systems research after studying patient safety; some of his work is rooted in Three Mile Island, and some of it comes from aviation safety.
And industrial accident investigation (except the ones with low regulation or whatever). And market or supply chain collapse, and civilization collapse (late Bronze Age anyone?)
I think, part of the point is that it’s not possible to have a recipe to prevent failures. They are cascades of many events coming together to fail in an a priori non obvious way.
Overall a good collection on the complex systems, somehow it missed the part how complex system came into the existence in the first place itself.
> Human expertise in complex systems is constantly changing
I feel this is single most important factor responsible to both making system complex and at the same time improving them depending on who the people are and how they take failures and breakdowns. I find it funny but we are in an era where folks building ML systems do not seem to remember the direction in which to open the screws. They do have clear expertise in something new but clearly lack in some other areas
I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing.
The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure state. Naturally, the "root cause" seems like lock system resiliency. But definitionally a metastable failure is one that persists after the inciting condition is resolved. Now you have two "root causes", the lock failure and the metastability of the deployment system fault. Keep looking and you'll find more.
But to me the biggest brick to the forehead in this piece is further observation that random things are failing all the time in any complex system. "Complex systems run in degraded mode". Resilient components are good, but it's the resiliency of the overall process that orchestrates the whole system that determines whether things are going to blow up.
All practitioner actions are gambles. I should have that inked somewhere.
> I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing.
If you want a bit of cheese to go with that wine, this article pairs nicely with The Grug-Brained Developer: https://grugbrain.dev/
> Failure free operations require experience with failure.
This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.
I've always struggled to apply this to the systems I work on. If the system fails, someone potentially dies, though in practice they've never been more than hospitalized. To avoid that, huge amounts of effort are expended on failure modelling and testing, but that doesn't eliminate unknown unknowns. That discrepancy has made front page news a couple times.
When I was at Netflix and reddit, one thing I said often was, "Luckily, we are not a bank". And it sounds like you are working on even more critical systems than that.
Chaos engineering doesn't really apply to data critical or safety critical systems. You can't just break them in the real world to see how they fail.
You have it exactly right -- it has to be modeled and tested in lab conditions. Safety critical systems are not a place for YOLO development.
It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be unexpected, related to your decisions and really hurt so that you can learn from them.
PS. Btw, I am certain that this is the main cause of the mental health crisis amongst young people.
> One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures.
Its cause we already learned from our failures and make the world a safer place. In my youth we climbed on trees, one of my friend fall down broke his arm, the doctor couldn't fix it and he can not move the hand for the rest of his live. My friend would be happy do not been allowed to climb, or at least under safer conditions but have his hand moveable still. I burned myself really heavy with fireworks as teenager and have a huge scar from it. I never touched any firework after this but also never bought it for my kids, so they never had the chance to "learn" from making the the same mistake. Keep in mind that while claiming the world is too safe you say this from a perspective of an survivor.
I may have shared this before on a different submission: John Gall’s books are really good on this topic: General Systemantics [https://en.wikipedia.org/wiki/Systemantics]
Gall's law is amongst my favorite ones and with decades of experience in software development, I have to say it holds absolutely true:
> “A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.” — John Gall, Systemantics (1975)
One of the great documents of our civilisation
The definitive work on this topic is Normal Accidents, with a modern retelling in Meltdown.
https://en.wikipedia.org/wiki/Normal_Accidents
https://en.wikipedia.org/wiki/Meltdown_(Clearfield_and_Tilcs...
All of this sounds just like any air crash investigation I ever read
Richard Cook was a UChicago anaesthesiologist who took up safety systems research after studying patient safety; some of his work is rooted in Three Mile Island, and some of it comes from aviation safety.
And industrial accident investigation (except the ones with low regulation or whatever). And market or supply chain collapse, and civilization collapse (late Bronze Age anyone?)
I think there are a few common themes to the failure reasons, but cannot get my hands on it.
This seems like a list of reasons while I am looking for more abstract directions on how to prevent them.
---
I am trying not to use AIs to just do that for me to tinkle my neurons.
I think, part of the point is that it’s not possible to have a recipe to prevent failures. They are cascades of many events coming together to fail in an a priori non obvious way.
Or so I read the [site? article?]
I can't tell if the article is describing how complex systems fail or if they are using failure characteristics to define complex systems.
It’s more about failure. It’s right there on top.