More on reframing entropy in business

My previous post on ‘Reframing entropy in business‘ kinda triggered off a veritable storm of correspondence, both in the comments and offline – hence seems it’s worth summarising and revisiting those themes here.

Perhaps the first point is that yes, it’s confusing – to say the least. There are several different ways to view entropy in business, and often they clash horribly with each other. Probably the best advice here came in a comment from Ivo Velitchkov:

it seems we are not dealing with entropy but with the “story of entropy”, and our co-creating based on our mental models is what matters

In other words, think of it not as about establishing which theory or view is ‘true’, but much more about what kind of story is useful for enterprise-architecture and the like. A rather important difference…

The real source of confusion is that there are at least two fundamentally-different things going on:

  • the strict physics view of entropy as ‘the winding-down of the universe’, a steady(ish) leveling-out and decay of (usable) energy-difference
  • the notion of ‘order’ in the universe, often contrasted with ‘disorder’

Hence we have the notion of (desirable) order decaying to (undesirable) disorder, as part of a loss of available energy. That’s a useful way to describe some of the issues that enterprise-architecture would need to address in business – a useful story of entropy. Yet that’s only one interpretation we can derive from that mix of themes: there are quite a few others. And sorting out which story is which, and which story to use in each type of context, is not helped by the fact that ‘order’ is just one of several key terms here that each have at least two radically-different meanings or interpretations:

  • ‘order’ as something inherent, something that naturally exists – the ‘laws of science’ and suchlike – and that describes and enables predictability and certainty
  • ‘order’ as something artificial, a constraint or filter arbitrarily imposed upon the broader view of the whole
  • ‘disorder’ as the collapse of order, the decay after a breakdown in order
  • ‘disorder’ as an unfiltered condition before sensemaking or imposition of order-constraints takes place
  • ‘chaos’ as the absence of order, or the extreme end-point after disorder-as-breakdown-of-order, a state of total differentiation where nothing is possible
  • ‘chaos’ as the extreme start-point before order, a state of total differentiation where anything is possible
  • ‘variety’ as the total number of possible states that a context can present (context-oriented view – hence ‘requisite-variety’, the total number of states that a system would need in order to ‘control’ that context)
  • ‘variety’ as the total number of states that a system of order can impose on the context (system-oriented view)

Different combinations of these definitions can imply very different stories. For example, the classic deterministic view, typified by Taylorism, would view order as inherent, natural, certain and desirable. Disorder is a failure of order, and undesirable; chaos is the absence of order, and highly undesirable. The quest is always to create a system with sufficient control-variety as to render the context under complete, perfect and predictable order, viewed always as an ideal yet at least theoretically-achievable goal. In this story, entropy is essentially a story of failure: the loss of certainty, the loss of control.

Contrast that with the story implied by the Cynefin framework. As I understand it (and I may be wrong, of course), ‘disorder’ here is not the collapse of order, but more a label to apply to a context before any sensemaking or ‘ordering’ takes place. Order is not necessarily something inherent, but more a view or filter that’s imposed on a context – particularly as seen in Cynefin’s ‘simple’ and ‘complicated’ domains. Yet deterministic-order itself can at times be misleading, because contexts may often include elements whose variety can change in often-unpredictable ways, and that can be inherently non-linear, self-altering (such as in ‘wicked problems‘), random or unique. These latter elements are not amenable to conventional linear concepts of ‘order’ – hence why Cynefin co-originator Cynthia Kurtz coined the term ‘unorder‘ – and are described in Cynefin as ‘complex’ or ‘chaotic’.

In essence, the Cynefin story both includes and extends the Taylorist story. Entropy could still be seen as a natural decay of order, yet not necessarily a ‘failure’, because the existence of unorder – the Complex and the Chaotic – always presents the possibility of counter-entropy refresh.

To me, as an enterprise-architect, this view as suggested by Cynefin is a more useful story than the classic Taylorist one of a futile struggle for ‘control’ against the tides of decay.

The view or story of entropy is different again in the SCAN framework. Here the emphasis is probably more on the relationship between the effective-variety and ‘variety-weather‘ in the context-in-scope (the first meaning of ‘variety’ in that bullet-list above), versus the available variety in the means we have to interact with that context (the second meaning of ‘variety’) – and hence whether the would-be system of ‘control’ has the requisite-variety to impose its control on the context.

In SCAN, the effective limit of requisite-variety is represented by the ‘Inverse-Einstein boundary’ – the vertical red-line. To the left of the line, doing the same thing leads to the same results: the control-system has requisite-variety to deliver the desired outcomes in the context. To the right of the line, doing the same thing may lead to different results, or we may need to do something different in order to achieve the same results. There, the control-system either does not have requisite-variety, or else it needs assistance from ‘outside’ of itself in order to adapt the control-mechanisms so as to create a localised sort-of-requisite-variety for that specific context.

There’s also a crucial distinction between theory and practice, that’s often expressed in terms of the time-dimension in SCAN – the horizontal dotted-line boundary between ‘considered’ intent (above the line) versus real-time action (below the line):

What theory expects or asserts is ‘controllable’ can often omit or ignore elements of variety that make the context much less controllable in real-practice: the variety of the control-system will seem sufficient to provide requisite-variety for the context, but there’s more variety in the real context than theory expects. This often gives rise to a skewed perception at the theory-level – the Complicated domain – of just how much ‘control’ is available from the current variety in the would-be control-system, and/or a dangerous delusion either that everything ‘should’ be in our control and that the real-world (or the operator who has to work on that real-world context) is somehow ‘wrong’ for failing to align itself with our expectations, or that we have control anyway because anything that doesn’t fit our expectations can be safely ignored:

The only way out of that trap is to pay rigorous attention to what the real-world is telling us about the actual variety in the context, and to keep a careful rein on our expectations and wishful-thinking in the desire for certainty and ‘order’. (Both of these challenges are represented respectively by the two cross-cutting boundaries in the SCAN frame: the horizontal boundary between theory (above) and practice (below); and the vertical Inverse-Einstein boundary between repeatable (left) versus potentially or actually non-repeatable (right).) At times we need to be very careful to remember that ‘science’ is just another belief-system, with all the limitations and constraints that that implies – especially where people’s emotions and feelings are facts in the context.

In terms of entropy, I’d argue the same principle as in the previous post, that entropy represents a decay from chaos to ‘useful-order’ to ‘non-useful order’ and onward to decrepitude. In a sense, any form of ‘order’ is over on the left-side of the SCAN frame – and as soon as it gets there, it’s already decaying towards ‘non-useful order’. The distinction between ‘useful’ versus ‘non-useful’ is less about the variety of the context in itself, but more about the available-variety in the system for ‘control’ of the context – so it’s actually more that the relationship between the ‘control’-system and the real variety in the context, rather than that the context itself is decaying.

In many cases the variety in the context actually increases during this apparent decay  – for example, parts within the control-system become more worn (hence capable of more variety of movement etc), or raw materials have more variance than the control-system expects. The decay into so-called ‘disorder’ or ‘chaos’ is more that the control-system can’t deliver the desired results any more – not something inherent in the context itself.

So as soon as we have ‘order’, or ‘control’, over the context, the fact of entropy should warn us that we’re already starting to lose it. Once the loss of ‘order’ or ‘control’ moves far enough towards a tipping-point, we’re likely to be pushed over the Inverse-Einstein boundary into uncertainty and ‘unorder’, whether we like it or not. The key here is to realise that that far side of the Inverse-Einstein boundary is the only space where counter-entropy becomes possible –  in other words, a place where we can leverage the uncertainty itself to reframe the structure and capability of the variety in our ‘control’-system, and thence to revitalise our ‘useful-order’ in the context.

In the longer term, what entropy really tells us is that the only way to maintain order is to let go of order – and to know when (and under what conditions, and so on) to hold onto order, and when to let it go. That’s a real skill in itself… for which the key trick is to choose to let go before it’s forced upon us by that decay of entropy. For example, at a more conceptual or theoretical level, we can explore ‘It depends…‘, which places us for the while over in the ‘Ambiguous but Actionable’ space (and respective tactics) of the SCAN frame; or we can face the deeper challenges of ‘I don’t know‘, making use of the tactics summarised for the ‘Not-known, None-of-the-above’ space in the frame.

I’d emphasise again, though, that it’s best to assume that none of these views into entropy and the like are ‘the truth’, or suchlike – merely that they can be useful, in different ways, in different contexts, for different people, and so on.

What I’ve found that definitely is useful, in enterprise-architecture practice and elsewhere, is to be able to switch between these views or ‘lenses’ according to need – what Richard Veryard, Nigel Green and others describe as ‘lenscraft’. I hope this is useful to you in that sense at least, perhaps?

6 Comments on “More on reframing entropy in business

  1. “So as soon as we have ‘order’, or ‘control’, over the context, the fact of entropy should warn us that we’re already starting to lose it.”

    Indeed. I have the impression that, particularly at the enterprise level, ‘order’ is a historical snapshot rather than a possible current/future state. In keeping with its chaotic nature (chaos theory rather than colloquial usage), we can tease out the deterministic nature of these snapshots, but not predict future state with any real fidelity.

    • @Gene: “particularly at the enterprise level, ‘order’ is a historical snapshot rather than a possible current/future state” – yep, agreed, that’s a good way to put it. ‘Order’ provides the dangerous delusion that we can predict the future – which we can’t. (Not in the complete-certainty sense that’s often implied by colloquial usages of ‘order’, anyway.)

  2. Tom,

    “Increasing entropy does not mean increasing disorder, it means increasing possibilities or options.”

    I beg you to stop describing entropy in terms of order vs. disorder and begin describing it in terms of decreasing vs. increasing possibilities and localized vs. diffused energy.

    For more on increasing entropy as increasing possibilities, see . This is the only way to understand how increasing entropy can actually increase “order.”

    For more on increasing entropy as increasing the diffusion of energy, see . This is the only way to see that the energy dispersed in a reaction is not “waste heat.”

    Applied to enterprises, these emerging views of entropy make it easier to understand why an enterprise must increase entropy in order to increase agility and adaptability.

    • @Nick: “I beg you to stop describing entropy in terms of order vs. disorder and begin describing it in terms of decreasing vs. increasing possibilities and localized vs. diffused energy.”

      I don’t (the former) and I do (the latter) – honest! All I was doing above was illustrating the ‘order vs disorder’ story as one example of a story that can be drawn from the conflicting definitions listed earlier above. As can be seen from my comments further down, I don’t think it’s a good story (or a useful story, rather – or a wise one to use, for that matter), but it is a story that can be derived from specific choices from those definitions, and it’s one that ‘makes sense’ to many people. (Which is often a problem for us, of course…)

      Thanks for the pointer to your Facebook post and, in turn, to the nanostructure article – very useful. It definitely does illustrate the point that the notion of ‘disorder’ as the inherent absence of order is not a useful one. I really don’t find that the notion of ‘order’ as something inherent or not-inherent is useful at all: to me it’s always a view that is imposed rather than inherent, and that one person’s idea of ‘order’ is not necessarily always valid for anyone else (or even for themself at some other time).

      In a strict physics sense, increasing entropy inherently implies decreasing opportunities: there’s less energy-difference, for example, which inherently decreases the opportunities to leverage those energy-differences. I’d strongly agree with you, though, that often that it’s much more a matter of perception rather than of nature: for example, the notion of ‘waste heat’ usually stems more from a lack of imagination than from the constraints of physics – as can be seen in system-designs that re-use the so-called ‘waste’ in more imaginative and effective ways.

      @Nick: “Applied to enterprises, these emerging views of entropy make it easier to understand why an enterprise must increase entropy in order to increase agility and adaptability.”

      I take your point, though to me that’s a somewhat skewed view: entropy is going to increase anyway, and we don’t always have much choice about it. 🙂 What we do have is choice about how we can leverage options for counter-entropy – in terms of the strict physics sense of increasing-entropy as a ‘levelling-down’ of energy-difference (or ‘levelling-up‘ of energy-sameness, if we want to keep the terms aligned with each other). The point I’ve been trying to make in these posts is that a conventional concept of ‘order’ actually makes it (much) more difficult to see or access those opportunities for counter-entropy refresh – by which, again, I mean a local increase in energy-difference in some form or other, which in turn increases opportunities in some form or other.

      To get to that understanding, we first need to acknowledge that there are many stories implied by the relationships between ideas such as ‘entropy’, ‘order’, ‘disorder’, ‘chaos’ and suchlike, depending on which of the various definitions we choose to use – and that some stories that arise from different combinations of those definitions can be a lot more useful in practice than others. What I’m also saying here is that the perhaps most common story – that entropy is a decay from order to disorder to chaos – is actually the least-useful story in enterprise-architecture and the like: so as you’ve said above, we usually need to choose an alternate combination of definitions, to give us a more useful story.

      So overall, I don’t disagree with you here – especially about “I beg you to stop describing entropy in terms of order vs disorder”. I’d just suggest that there are also other entropy-stories beyond your chosen ones of “decreasing vs increasing possibilities” and “localized vs diffused energy” – and that each of those stories has their usefulness, too. Which brings us back to the architects’ dreaded phrase “It depends…”, doesn’t it? 🙂

  3. I think of Entropy as ‘order in the universe’ where EVERYTHING starts and reverts to star mater over time and that everything in-between is structuring of entropy in a repeatable and widely accepted way till it too becomes star mater.

    • Peter – yes, agreed, that’s the physics big-picture. The perhaps more practical point I’m aiming for here is to explore how to make use of that big-picture in the rather smaller picture of our enterprise-architectures? 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *