Enterprise-architecture is wicked

How do we cope with the wickedness of enterprise-architecture?

I don’t mean here the implicit wickedness of people promoting term-hijacks and other half-baked notions as ‘the truth’ of enterprise-architecture – though heaven-knows there’s enough of that wickedness around.

I also don’t mean here the more explicit wickedness of unethical business-practices, such as those that aim to get clients addicted to delusions that don’t work – though heaven-knows there’s more than enough of that wickedness around, too.

What I do mean here is the kind of ‘wickedness’ that underpins wicked-problems – because as I understand enterprise-architecture, most of the issues we face in enterprise-architecture are wicked-problems.

If we don’t face that kind of inherent wickedness, and learn to work with it rather than fight against it – or worse, pretend that it doesn’t exist – then it becomes inevitable that any architecture we create is guaranteed to fail in ‘unexpected’, unpredictable ways. In short, this matters…

Tame-problems and wicked-problems

[The Wikipedia page on wicked-problems gives a very good summary of the characteristics of wicked-problems and the types of contexts in which they occur – including almost any kind of systems-development. Please do read that before continuing here.]

Let’s look at this first in terms of the SCAN frame:

The horizontal axis here is a measure of modality in the context – in effect, the extent of its ambiguity and uncertainty. The red dividing-line represents the Inverse-Einstein Test: to the left (blue domains), doing the same thing gives the same results; whilst to the right (red domains) doing the same thing may give different results, or we may need to do different things to get to the same results – the whole point is that it’s inherently uncertain.

In effect, the real world bounces us around all of that frame, all of the time. But if we can constrain it into the blue-zone in some way, or develop tactics that will still allow us to ‘make sense’ even in the red-zone, we can ‘solve’ the problem that the world presents us. We can ‘control’ it, at least well enough to get something done.

Where tame-problems and wicked-problems differ is in their expectations about what happens next:

• the context of a tame-problem will always stay the same – so once solved, a tame-problem stays solved
• the context of wicked-problem is always-changing, in fact the act of ‘solving’ it often changes the nature of the problem – so once solved, a wicked-problem will always need to be repeatedly ‘re-solved‘

In short, there is a single ‘right-answer’ or ‘correct’ solution to a tame-problem, but there is no single ‘solution’ to a wicked-problem. With a wicked-problem, what we aim for is not a fixed, eternal ‘right answer’, but a useful answer, a ‘good answer’ or ‘better answer’, relative to the context at the time:

• for a tame-problem, the focus must always be on ‘truth’, correctness
• for a wicked-problem, the focus must always be on ‘value’, effectiveness

As another view, let’s look at this in terms of fitness-landscapes and variety-weather.

Typical examples of tame-problems are a crossword-puzzle or a mathematical equation. There is just one correct answer, or in some cases a family of correct-answers, such as an English-language crossword-puzzle that allows for ‘-ise’ and ‘-ize’ word-endings. In most cases, especially in mathematics, we should be able to reverse the equation: given the formula and a set of inputs, we should be able to identify the range of possible correct-answers, or even infer the inputs just from the answers.

When working on a crossword-puzzle, we iterate towards that correct-answer, which we might describe as ‘peak-correctness’; some answers will lead us down wrong-paths where we can get stuck on a local peak that’s as correct we can get without retracing our steps. So reframe the state of the crossword-puzzle at any time as a landscape with peaks and troughs of potential-correctness, culminating in that ultimate peak of ‘the correct answer’. That’s a fitness-landscape.

But in effect, all of this relies on stable variety – the total set of parameters for the system, and the total number of states which the system can be in. (For that matter, it assumes that there is such a condition as a ‘state’ – which quite often may not be the case in the real world…) It assumes that there is no such thing as variety-weather – it assumes that nothing in the variety-structure of the context will ever change, that the variety of the variety itself never undergoes any change. Or, more precisely, that the variety-weather will always remain exactly flat-calm for that context – something that only happens in either imaginary or artificially-isolated special-cases.

In any real-world context, though, variety is not stable. Even in something as supposedly-predictable as celestial-mechanics, we’ll still need course-corrections to keep a space-probe exactly on-target – and we won’t know beforehand exactly what those course-corrections would need to be. For space-probes the variations are usually quite small: the fitness-landscape might flex by small amounts, but otherwise remain much the same. But in many real-world contexts it’s not so much a fitness-landscape as a fitness-seascape – often more like a raging storm at sea, where a high peak at one moment may well become a deep trough at the next. And that’s what a real wicked-problem is like: a context with inherently unstable variety.

To be blunt, it shouldn’t take much thought or observation to recognise that most problems we face in enterprise-architecture are wicked-problems. They involve people with different worldviews, the parameters change, the pace of change is faster than linear-paradigm approaches can keep up with, there are interactions and emergent-effects between ‘layers’ or domains: the list goes on and on. Anyone who fails to acknowledge this either isn’t looking, or else perhaps has something to hide…

The difficulty for enterprise-architecture is that common errors such as IT-centrism (or ‘anything-centrism’) and misuses of the linear-paradigm often try to address wicked-problems as if they’re tame-problems – which makes the problem more wicked. There’s then repeated attempts to use the same methods to ‘solve’ that now-changed context – which again fails to work, in fact probably makes things worse again. The only way to prevent a ‘death-spiral’ is to recognise that the failure is not in the problem itself, but in the tactics used to address the problem: tame-problem tactics do not work with wicked-problems – or, more accurately, are too incomplete to work well in those contexts.

But if the tame-problem tactics that everyone knows – such as the classic Taylorist methods, and IT-centric approaches to automation and the like – don’t work well for this, what can we use? The short answer is that we can still use them, if we’re careful about how we do it: but first, and most of all, we need to design for wicked-problems, rather than pretending that they don’t exist.

Designing for wickedness

First, identify whether the context is a wicked-problem or incorporates wicked-problem factors somewhere within itself. (Hint: very few real-world contexts do not include wicked-problem factors. For example, almost by definition, anything involving real-people will almost certainly embody wicked-problem characteristics, because of differences in perceptions and worldviews between people.)

Second, if it is a wicked-problem, acknowledge that it is a wicked-problem – don’t pretend that it’s not. (Hint: if you do pretend that a wicked-problem is ‘tame’, and hence ‘solvable’ by simple tame-problem tactics such as Taylorism, you set yourself up for guaranteed failure.)

Third, model the variety-weather in the context, both of the overall variety-climate and of the context’s various micro-climates. For example:

• What are the inherent conflicts in the context? What paradoxes and dilemmas do these conflicts inherently imply?
• Who are the players in the context? What inherent conflicts arise from their differing worldviews, paradigms, time-perspectives and and suchlike?
• What are the distribution-curves for opportunity, risk and change? What are the implications of the ‘black-swan‘ or long-tail opportunities and fat-tail or kurtosis-risks on the outer edges of those distributions? What are the cycles, patterns and anti-patterns of change?
• Which aspects have inherent constraints placed on them by their nature, and hence forced to rely on specific paradigms or constraints? – for example, within its actual operation, most IT and machine-automation would be inherently constrained to the ‘true/false’ decision-models of the linear-paradigm

Use iterative, recursive, whole-context techniques to elicit and ‘surface’ these themes. Examples of such techniques include:

• the SCAN sensemaking/decision-making framework
• the ‘This’ game for service-oriented modelling
• Nigel Green’s VPEC-T (Values Policies, Events, Content, Trust)
• Sohail Inayatullah’s Causal Layered Analysis (‘poststructuralism as method’)

Just as in the interaction between topology, geography and meteorology, different parts of the context will have different micro-climates of variety-weather. Some parts will always have ‘wild weather’: anything ‘political’, for example, or anywhere that human aspirations and emotions play any significant role. Other parts may seem stable for long periods, but then suddenly undergo stormy disruptions: the impacts of the election-cycle and other social milieux, for example, or business-strategy, or ‘bottom-up’ technological change. And yet other parts may have variety-weather that’s calm enough, or predictable enough, or undergo change slow enough, for Taylorist-type tactics to actually work. We need to be clear which type of microclimate applies in each context, in order to design appropriately for each.

Given all of the above, design for wickedness, not against it. For example:

• acknowledge and design for the conflicts – “everyone is always right, but no-one’s ever right”
• keep the emphasis on overall effectiveness – and minimise arguments about whose view is ‘true’
• manage the overhead of conflict-management – acknowledge the conflicts, yet keep the focus on achieving intended results, and avoid ‘analysis-paralysis’ and suchlike
• design systems and structures to create and develop mutual-support between players – for example, cyclic-design such that the outputs of one player inherently support the input-needs for another player

To accommodate microclimates of variety and ‘wickedness’, we also need:

• acknowledgement of context-specific uncertainty, and contextual variation in uncertainty
• system-design for context-specific variance
• governance for contextual and context-specific variance

The next post in this series – ‘Unbreaking’ – will show a couple of real-life worked-examples that resolve some of the ‘brokenness’ in many common approaches to enterprise-architectures, and address some of these themes on design for – rather than ‘against’ – inherent-wickedness in real-world contexts.

Enough for now: over to you for comment, perhaps?