Rendered at 17:15:23 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jedberg 41 minutes ago [-]
> Failure free operations require experience with failure.
This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.
AlotOfReading 15 minutes ago [-]
I've always struggled to apply this to the systems I work on. If the system fails, someone potentially dies, though in practice they've never been more than hospitalized. To avoid that, huge amounts of effort are expended on failure modelling and testing, but that doesn't eliminate unknown unknowns. That discrepancy has made front page news a couple times.
feyman_r 1 hours ago [-]
I may have shared this before on a different submission: John Gall’s books are really good on this topic: General Systemantics [https://en.wikipedia.org/wiki/Systemantics]
littlecranky67 1 hours ago [-]
Gall's law is amongst my favorite ones and with decades of experience in software development, I have to say it holds absolutely true:
> “A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.” — John Gall, Systemantics (1975)
yipinwong 27 minutes ago [-]
I think there are a few common themes to the failure reasons, but cannot get my hands on it.
This seems like a list of reasons while I am looking for more abstract directions on how to prevent them.
---
I am trying not to use AIs to just do that for me to tinkle my neurons.
shash 22 minutes ago [-]
I think, part of the point is that it’s not possible to have a recipe to prevent failures. They are cascades of many events coming together to fail in an a priori non obvious way.
Or so I read the [site? article?]
1 hours ago [-]
rowyourboat 55 minutes ago [-]
All of this sounds just like any air crash investigation I ever read
shash 21 minutes ago [-]
And industrial accident investigation (except the ones with low regulation or whatever). And market or supply chain collapse, and civilization collapse (late Bronze Age anyone?)
mohamedkoubaa 31 minutes ago [-]
I can't tell if the article is describing how complex systems fail or if they are using failure characteristics to define complex systems.
This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.
> “A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.” — John Gall, Systemantics (1975)
This seems like a list of reasons while I am looking for more abstract directions on how to prevent them.
---
I am trying not to use AIs to just do that for me to tinkle my neurons.
Or so I read the [site? article?]