Three kinds of chaos

There are three kinds of chaos. They’re fundamentally different from each other. Don’t mix them up…

(For a quick overview of the themes in this post, see the video ‘Three kinds of chaos‘.)

I was going to start this off by asking about the relationship between order, chaos and entropy, and how can we best use that to help make sense of our enterprise architectures. However, from the response I’d had to previous posts on entropy – particularly ‘Reframing entropy in business‘ and ‘More on reframing entropy in business‘ – I’d best leave that part mostly out of the picture for now. Let’s just say there’s total chaos there, masked by various illusions of order, and not much that’s useful at all? A more useful place to start, and that would still allow us to circle back to entropy in business, would be a well-known quote by Douglas Hoftstadter:

It turns out that an eerie type of chaos can lurk just behind a façade of order – and yet, deep inside the chaos lurks an even eerier type of order.

The ‘chaos’ that he’s referring to there is the form explored mostly in science, with a strong mathematical base. To me that’s best understood as just another variant of what so many people seem most to desire, which a sense of order or ‘control’. The reality of chaos is a bit more complex – and made yet more complex by the fact that there at least three colloquial meanings of the term, which are each fundamentally different from the others.

The first of these is chaos before ‘order’ – literally, ‘primordial chaos’. Classically, this is ‘the void’, the realm of infinite possibilities; in current cosmology, this is the universe at the moment just after the ‘Big Bang’, which, again, is a moment of infinite possibilities. This is also the direction we move towards when we need to find new possibilities, as illustrated by Nietzsche’s famous quote:

You must have chaos within you to give birth to a dancing star.

Within that kind of chaos, order can arise – the turbulence settles out into patterns of varying complexity, as ‘self-order‘. Or we can seek to impose our own notions of ‘order’ upon the seeming chaos. When our imposed-order breaks down, we’d tend to call that ‘a collapse into chaos’ – chaos as counterpart to order. But it’s actually not ‘chaos’ in that first sense at all: it’s just a reversion to a form of self-order that happens to be different to what we’d want. Or, in short, ‘undesired self-order’.

And the final outcome for that natural process of self-ordering is, confusingly, also called ‘chaos’ – but in this case, chaos after ‘order’. It’s the exact opposite of primordial-chaos: a state of ultimate self-order, a state of infinite-impossibility, an utterly non-negotiable absence of choice. In current cosmology, we arrive at this state at ‘the heat-death of the universe’, the ending of any possibility for change. And the harder we push towards ‘order’, towards an often spurious sense of certainty, the faster we move everything towards that end-point. Too much of the wrong kind of ‘order’ will literally be the death of us all…

We can summarise all of this visually as follows:

This also helps in making sense of a key dilemma about entropy, the so-called ‘arrow of time’. In mainstream descriptions, the assertion is that entropy goes from order to chaos. But from what we’ve seen above, we could also validly assert that it goes from chaos to order – from chaos as infinite-possibility, to ultimate self-order. So which of these descriptions is right?

What the diagram above shows is that the only valid answer is ‘Yes’: that ‘arrow of time’ actually moves from primordial-chaos, through various form of ‘order’ with perhaps a bit of perceived-chaos along the way, and onward towards end-chaos. In short, that confusion about the direction of the arrow arises solely from the fact that we’re using the same term to mean three completely different things. Not A Good Idea…?

Right now we’re still stuck without good terms for any of these. In honour of Terry Pratchett, we could perhaps use ‘kaos‘ to denote that primordial form, but that still doesn’t leave us anything for the other two. Mainstream entropy would describe the last as ‘disorder’, but that seems to add even more confusion than just ‘chaos’ alone: maybe call it ‘end-chaos’ and leave it at that for now?

—

So far, so academic, no doubt. But this does have very real practical implications for enterprise-architecture and the like. To me, there are two parts to this, of which the first can be summarised in colloquial terms as follows:

• chaos increases possibility
• order reduces possibility

(That’s ‘chaos’ as in “…to give birth to a dancing star”, back upward towards kaos; and ‘order’ as in the drive towards end-chaos or disorder. Yeah, it’s still confusing. Oh well…)

In short, which way we need to move depends on whether we want possibility or not. If we want to reduce possibility, reduce uncertainty, we’ll need order – though not too much, or it’ll drag us downward ever faster towards the end-chaos of disorder. Yet if we need new ideas, new possibilities, then we’ll need just the right amount of chaos – and too much of the wrong kind of order will only get in the way. So the decision is never as simple as ‘order or chaos’ – it’s always just the right amount of both. Hence the importance of the Architect’s Mantra, with its emphasis on ‘It depends‘ and ‘Just enough‘.

The other part is somewhat trickier, and relates to that crucial theme of going ‘upward’ to find ‘useful chaos’, new useful-possibility. In principle, entropy only flows one way, from primordial-chaos to the end-chaos of ultimate-self-order. If that simplistic concept of entropy were correct, there would never be any way back towards kaos, to find new ideas – which clearly isn’t the case, because we can do so. Sometimes. Somehow. But just exactly how isn’t obvious at all…

The way round that is that the flow of entropy is sort-of… well, ‘lumpy’, would perhaps be the right word. Everything goes downward, so to speak, but there are ways to navigate through the flow that in effect can go upward. It’s a bit like that scene in the film Kung Fu Panda, where all the blocks are falling downward, but the tiger escapes by jumping upward from one block to another. In practice, all living things do this: they ‘cheat’ entropy this way, for a while as individuals, and for a long while collectively as a species. We do it as organisations too: a business-model lasts only for some while, until conditions change, but we can refresh and re-pivot to keep business as a whole continuing on. And we do that by turning towards kaos, towards ‘useful-chaos’, as a conscious, deliberate choice. And then find ‘useful-order’ from there, to bring it back down towards certainty again. And repeat the process, and repeat the process, and repeat the process, again and again and again. Always in the opposite directions to those that the common notions of entropy as ‘from order to chaos’ would suggest.

There’s an analogy for this that came up in a recent conversation with Bard Papegaaij: look at how a glider works. A glider is always descending downwards the ground: it has no power-source of its own, so it has no choice in that. It can pull some kind of short-term manoeuvre pushing itself upwards into a climb, but in practice those would only make the descent even faster than it would otherwise be. And yet a skilled pilot can keep a glider up in air for hours, even days sometimes. How does that work?

The trick is that the glider’s descent is relative to the surrounding air – not just relative to the ground. So whilst the glider is going downward relative to the air, the air itself may be going up – such as when it’s pushed upward as a hot-air thermal, or by flow over a mountain-ridge. Conversely, the air may go downward, pushing the glider down faster than before. In which case, a glider-pilot who wants to stay up in the air needs to seek out the rising air-currents, and avoid the downward-falling ones – and also avoid the ‘undesired self-order’ of local turbulence, too. Doing it right takes skill, experience, practice – a learned ability to sense out the currents, often an almost literal seat-of-the-pants awareness of when to stay in the current, and when to pull away:

Exactly the same applies in business and elsewhere. We need to seek out and sense the right currents, reaching up out of the everyday into that ‘useful-chaos’ of new ideas, then let go and drop downward again into ‘useful-order’ to make use of those new ideas. Again, doing it right takes skill, experience, practice – a learned ability to sense out the currents, often an almost literal seat-of-the-pants awareness of when to stay in the current, and when to pull away.

And a first requirement for development of that skill is to build a clear understanding of the differences between these three kinds of chaos – and to know when we need each of them, and when we don’t.

Anyway, leave it at that for now – over to you for your comments, if you would?