Complexity, chaos and enterprise-architecture
Courtesy of a link by fellow enterprise-architect Sally Bean, I’ve just spent the past couple of hours viewing and then reviewing an online seminar on complexity by one of the thought-leaders on complexity-theory and practice, Dave Snowden:
From Induction to Abduction: a new approach to research and productive enquiry
This seminar will provide a summary of both the theory and practice of a new approach to research based on the large scale capture of self-interpreted micro-narrative. The approach has been described as the first technique for distributed ethnography and has been developed over the past decade with project based funding from the US, UK and Singapore Governments in the context of risk assessment, horizon scanning, cultural mapping and weak signal detection. It allows the linkage of research with knowledge management and impact based measurement. Current projects involve measuring the impact of development projects in Africa, narrative based knowledge management for the US Army in Afghanistan and cultural mapping of various inner city communities within the UK.
The theoretical origins lie in the application of complex adaptive systems theory to social systems together with new understanding about the nature of human decision making from the cognitive sciences. The seminar will summarise the theory, but will also use a series of projects to combine theory with practice. One of the goals is to create learning systems that work on continuous capture of material in the field as it happens linked with a capacity for feedback loops and sophisticated representations that allow people to learn by doing, building on the micro-narratives of day to day experience. Narrative forms of knowledge lie between the experiential and the symbolic, allowing complex interactions and interventions in multiple social situations.
Abductive reasoning is, as Dave explains, “the logic of hunches”, and plays a key role in helping to develop understanding of how themes emerge in social contexts such as in business and elsewhere. It’s all fascinating stuff – very strongly recommended. The depth and versatility of the techniques will be a real eye-opener to anyone who hasn’t previously seen Dave’s work, and its applicability to whole-of-enterprise architecture and the like should be self-evident.
I will admit I do have mixed feelings about the way Dave develops and presents his work. On the one hand, he has a brilliant mind and is a brilliant presenter, and there’s no doubt at all that his tools and techniques, such as Cynefin and the theory and practice behind his Sensemaker software-suite represent real paradigm-shifts in the way we think about organisations and enterprises (in the broadest sense of those terms). But I do find it beyond tedious that he spends so much effort denigrating other people’s work – for example, Nonaka, Weick and Six Sigma (endless derided by Dave as ‘sick stigma’) all come in for attack in the first few minutes of the seminar. And I too have been on the receiving end of that same… well, I would have to describe it as an odd kind of sort-of-scientific bigotry… which is more than just annoying at times. And annoying not least because pretty much everything I’d tried to explain to him and that he’d dismissed with such vehemence – such as the nature of ‘magical’ processes and the role of ritual – Dave in fact now incorporates (though probably unconsciously) as significant if unacknowledged sub-themes in his work (as can be seen in various places in the video). But we have to take the ‘Dave Snowden’ package as a whole, I guess: and most of the contents of that package are important – definitely.
But there’s one slide, right at the start of the presentation, that I find especially fascinating:
There is indeed a clear historical sequence here, paralleling the shifts in the underlying scientific paradigms, from Newtonian to hard-systems to complex-systems in the present day. But there are two important points that are easy to miss here:
- in each case the old dominant paradigm remains useful, though is seen to describe a distinct set of special-cases rather than a grand ‘Theory of Everything’;
- the sequence does not stop here, with Dave’s ‘sense-making’ – it continues on to at least one more layer, and possibly two.
The reasoning for those assertions comes from Cynefin itself, plus a cross-map to this diagram above. As Shawn Callahan of Australian consultancy Anecdote explains in his excellent intro on the same web-page, Cynefin has a central (and ‘initial-state’) domain of ‘disorder’, and has four distinct domains of sensemaking and action:
- Simple: assumes simple cause-effect rules; sensemaking tactic is ‘sense -> categorise -> respond’
- Complicated: assumes linear causality, but accepts that these may include many factors, delays, feedback-loops etc; sensemaking tactic is ‘sense -> analyse -> respond’
- Complex: accepts that cause and effect are intertwined, leading to non-linearity and non-reversibility; sensemaking tactic is ‘probe -> sense -> respond’
- Chaotic: no identifiable cause-effect relationships; sensemaking tactic is ‘act -> sense -> respond’
Dave links the first three of these to the respective S-curves: Scientific Management is Simple, ‘classic’ Systems-Theory is Complicated, and his version of Sense-Making is Complex. Yet he provides no equivalent linkage for the Chaotic domain, and the listed tactic of ‘act -> sense -> respond’ literally consists of running away from the problem. Which is hardly a valid approach if the chaos insists on being sustained. Which, in the real world, it all too often does…
It’s often struck me that Dave’s great strength in the Complex domain seems also to create a real inability to describe any means to tackle the Chaotic domain. In this sense it does seem that, to use his own words, “the old dominant paradigm suppresses the new idea” – where the ‘new idea’ is that the Chaotic domain does need to be respected in exactly the same depth as we now do for Complex and the others.
To me there are two key clues here.
The first is a direct warning from the classical tradition and elsewhere that running away is not a viable response in the Chaotic domain. The clue here is the Greek-derived word ‘panic’, which is what many people will experience when facing any kind of chaos. But the ‘pan-‘ root literally means ‘the everything’ (hence panorama, pandemonium, and so on): what’s happening in panic is that everything and nothing is true at the same time. Yet that’s exactly what we need when we’re striving for innovation, or any other kind of search for new ideas: we need the ability to bring apparently unrelated themes together in new ways. And in practice we do that by not running away, but instead by ‘holding to the centre’, ‘finding the still-point’, ‘the calm at the centre of the storm’, and so on.
To find that centre, we turn to the other clue in what’s not in Dave’s diagram above.
The Simple domain is about control of Function, says Dave – in other words, the physical world, the physical dimension. Hence scientific-management came to the fore in the heyday of the assembly-line – and it still makes perfect sense within that type of context, where everything remains exactly predictable, exactly the same, just like most physical objects do.
The Complicated domain, says Dave, is about control of Information, the conceptual dimension. Hence ‘hard-systems’ thinking came to the fore in the heyday of the mainframe and the supercomputer, massive number-crunching and the like – and it still makes perfect sense in that context, of massively complicated cause-effect relationships between information-items.
The Complex domain, says Dave, is about control (or ‘situating’, rather) of the Network – otherwise known as the relational dimension. Hence Dave’s ‘Sensemaker’ and the like come to the fore in genuinely-complex social contexts, where meaning and ‘truth’ emerge from the interweaving between the individual and the collective in the respective physical, conceptual, social and aspirational milieu, in which everything and anything may become both cause and effect of everything else.
But what’s not there in Dave’s model is any consistent framework to tackle the Chaotic domain – instead, we’re just told to run away back to the safety of one of the other domains. And yet, following that same logic above, we can see straight away what its base would be: the aspirational dimension, the explicit choice of meaning and purpose – otherwise known in the enterprise-architecture as vision, values and principles.
From my discussions with him, Dave seems to dismiss this whole domain because to him it appears to have no identifiable science behind it. Yet I would suggest that this too may be the result of a too-close identification with his own concept of ‘science’, because as soon as we allow ourselves to move outside of the constraints of Western tradition of science, we will immediately find other traditions with at least the same levels of precision and discipline, if not more. The Australian Aboriginal concept of the Dreaming is one obvious example, an extremely sophisticated study of relationship with the land that is only now beginning to be understood in Western terms; likewise the Tibetan research into the period immediately before and after death; or, to give a more tangible example, the Polynesian science of navigation:
‘The Wayfinders’ [lecture-series] is a profound celebration of the wonder of human genius and spirit as brought into being by culture.
The entire science of wayfinding is based on dead-reckoning. You only know where you are by knowing where you have been and how you got to where you are…that your position at any one time is determined solely on the basis of distance and direction travelled since leaving the last known point… If you took all of the genius that allowed us to put a man on the moon and applied it to an understanding of the ocean, what you would get is Polynesia.
The consistent theme in each of these traditions is a very strong sense of purpose, intentionally embedded by and within the individual to act as a personal ‘guiding star’ that provides a known, certain ‘still-point’ under conditions on uncertainty and chaos. Hence, although Dave somewhat characteristically dismisses and debunks vision in the business context, I do believe he’s missed the point. Those are genuine skills, genuine sciences, every bit as valid as as the sciences behind scientific-management, systems-thinking and complexity-theory – and their niche of greatest applicability is the Chaotic domain.
Which, once we think of it that way, makes Cynefin complete.
Or rather, there’s one more layer to this. Each of the Cynefin domains has its own respective science, its own technologies and so on. But there’s also a need for a ‘meta-discipline’ to switch between the Cynefin domains, linking them together into a unified whole.
Checklists can provide some of that discipline; likewise a consistent iterative methodology such as the extended TOGAF-like cycle that I use in my own enterprise-architecture work. An explicit multi-dimensional model such as the tetradian can also help in this. And we have much that we could learn from the many non-Western traditions – or even from a better understanding of how science really works in practice.
But perhaps more, for here, we perhaps need to note that whilst Dave’s complexity-theory is useful – very useful indeed – it’s unlikely to be ‘the last word’ in the sciences that we need in enterprise-architecture. There’s still some way to go: and a more consistent, more honest approach to how we handle the Chaotic domain would seem to be the necessary next step in that journey.