How complex systems succeed
Dr. Richard Cook was a system safety researcher. He researched and wrote about how complex systems fail. The kinds of systems he looked at were things like surgical operations, commercial aeroplanes, power stations. All systems that are obviously complex and which you wouldn’t want to fail.
What we’ve come to realise more recently, is that everything is a complex system. Every aspect of our lives is interwoven into complex systems. Food, manufacturing, banking, education, health, economy, transport, climate. When anyone talks about the polycrisis we are facing, they are talking about how when crises in many of these systems become entangled, they produce harms greater than the sum of those the crises would produce in isolation. Being interconnected makes things worse. And yet, for how failure prone these systems are, they seem to keep working. Why?
How do complex systems succeed?
Let’s consider a particular kind of complex system; a fictional social media platform. Let’s call it Mutter. It starts out small, just a few people using it to message people they know. It grows. It grows more, and within four years fifty million Mutts are being Muttered every day. News broadcasters find stories in real time and share them with a large audience. Media companies use it to create a buzz about TV shows. Advertisers promote every kind of product and service. Communities spring up where people share knowledge and learn from each other. Politicians use it to communicate. Activists highlight and fight oppression. Bullies and bigots attack people. Businesses are built and customers serviced. The more interwoven into life it becomes, the more it is depended upon. It’s no longer just a social media platform, its a complex system interwoven into modern digital life.
It could have failed at any point. But it didn’t. It still could. But maybe Cook can help us understand a bit about why it won’t.
All systems are flawed, but they run nonetheless - Catastrophe requires multiple failures, individual flaws and single point failures are not enough to bring down the system. Not just technical flaws like the Remutt button not working, but flaws in how people interact, how some messages get more attention than they should, how things that shouldn’t be on Mutter, are. And, because the system is wider than just what happens on the platform, there are flaws in how a businesses customer service team isn’t able to help customers, how people are affected by barrages of hate, how adverts targeted inappropriately might create distress. Mutter continues to run successfully because despite these many flaws because people find a way around them. People are the most adaptable element of complex systems, and so multiple small failures over time that are learned from and improved on create a more successful system overall.
Every action is a gamble, but with unpredictable consequences - The consequences of small actions are impossible to predict, but because of the size and complexity of the system, every new change introduces new and unexpected ways for the system to succeed. The effect a Mutt sent by a politician has on the stock market, is a gamble. A change to Mutter’s algorithm that sees posts about dogs getting more visibility than cats, is a gamble. Hiring more people than Mutter can afford to pay in the hope they’ll be able to increase income, is a gamble. Communities using Mutter to connect, is a gamble. Businesses investing in building an audience, is a gamble. Dealing in uncertainties doesn’t make the system any less resilient, in fact the more of those unpredictable interactions going on the more likely the system is to succeed.
Systems defend themselves, but in well established ways - The high consequences of failure lead to the construction of multiple layers of defences. For Mutter, because of how interwoven it is into so many things, the defences are well established and well proven. They includes laws on how people can be fired, the essential institutional knowledge contained in people heads, the backup servers, the commitment of organisations, the number of people using the platform. All of these things exist to protect the system and ensure its continued success.
So, because Mutter has existed for long enough to become interwoven into complex political, commercial, social systems, learned how to deal with many small failures and accept the uncertainty of the consequences it creates, and has multiple layers of protection, it becomes extremely resilient.
The only measure of success for a complex system is that it continues to function. Mutter will.