Metatheory and enterprise-architecture
“What’s the theory of enterprise-architecture?”, a colleague asked the other day. “Is there any kind of coherent and consistent theory behind it that holds it all together?”
Short answer: no.
Slightly longer answer: yes.
Or sort-of, rather. Both no and yes – both at the same time. Which is where this gets kinda weird…
From the outside, the enterprise-architecture discipline – such as it is at present – often looks an absolute shambles: a mess of competing theories and frameworks and self-styled ‘best practices’ that are rarely consistent even within themselves, let alone with anything else.
What’s perhaps hard to explain is that, at the surface level, that’s exactly what it should be. If it wasn’t that seeming mess, that’s when we should be really worried…
But even if there’s no possibility of a single, coherent ‘The Theory Of Enterprise Architecture’ – and believe me, there isn’t – there is the possibility of a single consistent metatheory, or ‘theory of theory’. That distinction between theory and metatheory is somewhat subtle, but utterly crucial.
From a theory-perspective, what we face in enterprise-architecture is a situation much like a reviewer once commented about James Gleick’s book ‘Chaos: Making a new science‘:
It turns out that behind apparent order lies an eerie kind of chaos; yet behind that chaos lies an even eerier kind of order.
Most people expect theory to be consistent, to deliver a set of rules that make sense in real-world practice, to guide that real-world practice. Which actually is true for most theory in enterprise-architecture. The catch, though, is that each specific theory applies to only to a specific subset of the scope of enterprise-architecture – and often doesn’t work well, if at all, with anything else. Hence the mess – especially if we expect a single consistent theory to cover the entire scope.
When we look at the context for enterprise-architecture, it soon becomes clear that it’s built on top of a web of inherent conflicts and trade-offs, including:
- machine and human
- certain and uncertain
- identicality and uniqueness
- predictable and unpredictable
- controllable and uncontrollable
- stability and change
- interweaving rates of change, from very fast to very slow
- interweaving lifecycles, from microseconds and less to centuries and more
- shifting timeframes from past to present to future and back again
- shifting perspectives and levels of detail, from vision to strategy to design to implementation to operation, and back again
- a multiplicity of domains-of-interest, each with their own entity-sets, standards, jargon, governance and more
Each single theory will, at best, only be able to be consistent and descriptive across part of that overall context-space – not all of it. Hence a quest for a single ‘The Theory of Enterprise Architecture’ would not only be futile, but wrong-headed right from the start.
By contrast, a metatheory provides a consistent description of the context-space itself – the parameters and trade-offs underlying that context-space, much as above. As a theory-of-theory, a metatheory provides a frame in which to identify where each type of theory would work well – and where it wouldn’t.
Hence on complexity, for example, Roger Sessions argues that we should aim to eliminate all complexity; John Seddon argues instead that we should aim to embrace complexity, and that trying to eliminate it only makes things worse. Which of them is right? If we were looking for a single consistent theory, one of them surely must be wrong? But actually they’re both right – in the right types of context. Equally, both of them are wrong – for the wrong type of context. To use a well-worn architects’ expression, “It depends“…
The role of a metatheory is to identify the types of context for which each type of theory is ‘right’ or ‘wrong’. For this example about complexity, we can use SCAN as a visual frame for metatheory:
Which, following hints in the respective theories themselves, suggests:
- Roger Sessions’ “eliminate complexity!” would work well with predictable elements such as IT-systems, on the Simple/Complicated side of the ‘edge of uncertainty’ (left-side of the red dotted vertical-line), but not work well with any part of the context on the Ambiguous/Not-known side (right-side of the line)
- John Seddons’ “embrace complexity!” would work well with the inherent-uncertainties on the Ambiguous/Not-known side of the frame, but could well make things worse on the Simple/Complicated side
In effect, the metatheory behind the SCAN frame tells us where to use and not use each of those theories. And although there are some additional complications around fractality and recursion – the same patterns recurring at different levels of granularity and context – the basic principles behind that metatheory would apply, consistently and coherently, in just about every type of context. We can summarise the types of theories and tactics that typically apply best in each SCAN ‘domain’ as follows:
Each of the Tetradian tools or frameworks – such as SCAN, SCORE, Five Element, Enterprise Canvas and the rest – describes a context-space in a different way, and emphasises different types of views and hence different types of theories. In that sense, as someone complained to me a while back, there might not seem to be ‘a single coherent theory of EA’ that links them all together.
But what I’ve come to realise over the past few months is that the various ‘hooks’ and crosslinks between each those frames do make it possible to link them all together as ‘a single coherent metatheory of EA‘. Not that seemingly much-desired ‘a single coherent theory of EA’, of course – but probably closer to what we actually need for real-world practice.
In effect, it’s another illustration of what I’ve long described as the core-principle and core-purpose for all architectures: that things work better when they work together, on purpose. But here, rather than domains or databases or whatever, it’s the various forms of theory that we need to get to work better by getting them to work together, on purpose. And to do that, we need metatheory and metaframeworks that can bridge across the silos formed by each of those clusters of self-consistent but seemingly-incompatible theory.
For example, another Tetradian framework that works well as a visual-frame for metatheory is the maturity-model described in my book Doing Enterprise-Architecture:
Each of the steps presents and makes use of what are actually quite different theories of the enterprise:
- Step 1 (‘Know your business’): There is an underlying human-story – enterprise as ‘a bold endeavour’ – that underpins decision-making and choices for capabilities and services within the enterprise.
- Step 2 (‘Clean up the mess’): There are clear and identifiable rules that determine clean-up activities such as simplification and de-duplication.
- Step 3 (‘Strategy and stuff’): Strategic choices – however complicated – can be determined and implemented via algorithms and analytics.
- Step 4 (‘Work with the real world’): The real-world is inherently uncertain and ambiguous, but actions can be guided by patterns and probabilities.
- Step 5 (‘Pull together’): When faced with uniqueness or the not-known, the most useful guidance is provided by principles that anchor back to the original deep-story.
The architecture-development is not literally step-by-step in that sequence: step-by-step is what we’d prefer to do, and should do wherever practicable, but Reality Department so often gets in the way towards that ideal! What the model does warn us, though, is that activities for each step in effect depend on completion of the respective parts of the ‘previous’ step(s): hence each time that we do have to do things out of step, we’re creating technical-debt – literal or metaphoric – that we’ll have to come back to and clean up again later.
(For a more detailed overview of the maturity-model, see the slidedeck ‘Stepping-stones of enterprise-architecture‘ on Slideshare.)
Each of those maturity-model steps and their respective decision-guides in turn align quite well with the decision-guides of the SCAN domains – the key difference being that Step 1 links with the overall story of the context being described via a SCAN frame:
Note that the paradigm-shift that we need in order to transition between maturity-model Step 3 and Step 4 also aligns with – in fact is demanded by – the transition across SCAN’s ‘edge of uncertainty’, or ‘boundary of effective-certainty’.
As described in the post ‘Context-perspectives and enterprise-architecture maturity‘, the maturity-model steps also align quite well with the distinct perspectives to a given context – inside-in, inside-out, outside-in and outside-out:
Which, in turn, links the perspectives-set to the SCAN frame:
This also identifies probable links between the respective skill-levels (from SCAN) that are needed in order to work with contexts for the respective maturity-model steps and perspectives:
And the perspectives-set also describes what happens between services in a broader shared-enterprise:
Which therefore provides us with further metatheory-links to the Enterprise Canvas service-modelling framework, where the perspectives-set apply to each link and flow, in every type of service:
And the shared-story that links the respective actors together:
Including governance, futures, and the broader context of the shared-story, in the market and beyond:
And thence onward to the service-context model from Enterprise Canvas, both at whole-organisation:
Or in the generic form that applies at any level of service-interaction:
Each of those frames from the Tetradian model-set has its own distinct theory, or more often a theory-set linked together via its own distinct metatheory. But the connections between the frames – as summarised in visual form above – in turn link each of those theories or theory-sets into one unified, complete, consistent metatheory (or, to be pedantic, metametatheory) of enterprise-architecture and beyond.
So no, there isn’t a single ‘The Theory Of Enterprise-Architecture’ – and there can’t be one, ever.
But there can indeed be a single metatheory of enterprise-architecture – and you’re looking at one, right there, in this post.
Hope it’s useful to someone, anyway. Over to you for comments, anyone?
A somewhat bitter addendum… Yesterday evening, on Twitter, someone who darn well ought to know better derided this post as “smells like teen Cynefin”. Anyone who’s read this blog for a while would understand why that snide putdown re-opened some really painful old wounds for me, and gave me yet another sleepless night about it…
For the record, the only part of the blog-post above to which that remark could apply is SCAN: and SCAN is not a “teen Cynefin”. Yes, it’s true that to the unobservant or deliberately-ignorant amongst us, there might seem to be some surface similarities, but SCAN has a different theoretical base than Cynefin, is designed for a different purpose, is much richer and more nuanced, and as explained a few years back in the post ‘ Metaframeworks in practice, Part 4: Context-space mapping and SCAN‘, its origins predate Cynefin by at least a decade or more, and is the outcome of a much longer period of study, research and practice. As also explained in that post, it’s true that somewhen back in the past I had made the mistake of thinking that Cynefin could be used for context-space mapping – the purpose for which most of the Tetradian tools are intended. But it became painfully clear, from many different directions, that it couldn’t and shouldn’t be used in that way, and hence I went on to redesign a more usable framework from scratch, with SCAN and the swamp-metaphor and more.
A polite request: if people can’t be bothered to read this blog-post above properly, or to read any of the hundred or more other posts here on SCAN and context-space mapping and the like, please don’t come up with snarky putdowns such as “smells like teen Cynefin”?
Oh, and I have read and studied a great deal about Cynefin itself – which is why I understand a great deal about where it doesn’t work well, in addition to the (very few) contexts where it does. And don’t ask me what it “smells like”, either – because you probably won’t like the answer…
‘Nuff said, I think?
Hi Tom. Happy 2016!
I thought it might be fun to test your theory about theories and meta-theories. I picked the ‘stratified systems theory’ of Elliott Jacques ( https://en.wikipedia.org/wiki/Elliott_Jaques ) because I have been doing a lot of work with Janne Korhonen, who brought Jacques’ work in Requisite Organization to readers of Journal of Enterprise Architecture not too long ago.
I’m thinking this stratified system idea plays into your maturity model, but I’d be interested to hear your take on that.
As well, I’d be interested in how Adizes’s maturity model plays into your meta-theory.
I also wonder whether Sessions’s and Seddon’s suggestions to eliminate or embrace complexity really qualify as theory? Those statements seem more like prescriptions, maybe with some theoretical underpinning of their own?
Thanks, Doug!
Ah. Oops. As usual, it looks like I’ve gone a long round-about way to describe what is a very simple idea, and ending up probably confusing folks in the process… 🙁 – sorry!
In the sense that I’m using here, as ‘theory-for-action’, ‘theory’ is ‘a consistent description of a context that indicates predictions and tests about activities in that [type of] context’. The keywords here are ‘consistent’ and ‘prediction’ – which in turn imply at least some element of prescription, in order to achieve desired outcomes in that context. So in that sense, both Sessions’ and Seddon’s do qualify as ‘theory’: they are each descriptive, each internally consistent, and each predictive about what will happen in a context if the theory is applied.
But as theory (or, at least, that form of theory) they have no means within themselves to assess their own validity for any given context – their applicability or appropriateness for use in that context. And the tests embedded in the theory are themselves embedded in and built upon the assumptions of that theory – which gives us an immediate risk of circular-reasoning and similar errors. That’s what can lead us, for example, straight into the trap of ‘policy-based evidence‘.
Metatheory is ‘theory of theory’, or ‘theory applied to theory’ – a theory of applicability of theories. We use metatheory to help us select and test which type of theory to use within a context. Metametatheory is a theory of metatheory, which we can use to cross-check the validity of the choices made via a metatheory. And so on… we could go down that rabbit-hole to any depth we might like, but in practice it’s probably best to stop there for now!
I’ll admit I don’t know ‘stratified systems theory’, so I can’t help you there directly. But if we apply the classification above, some parts of it are likely to be theory (consistent description and predictions about a context); some parts of it are likely to be metatheory (consistent description and predictions about applicability of types of theory to types of context), and some may even be metametatheory or beyond. I’ll leave that assessment up to you. 🙂
But from what I know of Adizes (which I’ll admit isn’t all that much), his lifecycle model is both theory (a prediction about the progression of organisations) and metatheory (different theory-types – and hence recommended-actions – apply in different stages of the lifecycle). However, there doesn’t seem to be any metametatheory there: the lifecycle is assumed to apply to all organisations, without any built-in tests as to whether the lifecycle-model is valid in any given context. Hence, again, a risk of circular-reasoning and the ‘policy-based evidence’ trap. Avoiding those traps is really what all of this rambling about theory and metatheory is all about.
Hope that makes (more) sense now?
Actually that DOES help me understand what you are trying to say, which actually is not what I thought you were trying to say.
Actually, I’m embarrassed to admit that I superimposed my own thought process on what you said. What I’m interested in is metatheory that applies to many theories, such as Seddon, Graves, Jaques, Adizes, Goldratt (Theory of Constraints), Giddens (structuration theory), Taylor (scientific management), Snowden, von Bertalanffy, Beer, Checkland, Zachman, Chomsky, Sowa, Wittgenstein, etc., etc.
I see that you actually are applying all these terms and thoughts within the realm of Tetradian. OK, fair enough. The rest of the exercise I had in mind is clearly left to the student 🙂
By the way, when I said “Graves” above, I might have meant either Tom Graves, or Clare Graves, or both 🙂
Tom,
Some comment on a few statements from the post.
> Both no and yes – both at the same time. Which is where this gets kinda weird…
The ambivalence is a norm in social systems. Regarding it as “weird”, or assuming ambivalence and paradox as being a result of an error or being just exceptional, is risky in any attempt to understand what’s going on.
> there is the possibility of a single consistent metatheory, or ‘theory of theory’
I doubt it. It is quite clear that a theory can be either consistent or complete, or at least I’m not aware of a convincing refutation of Gödel’s incompleteness theorems. Having in mind that a meta-theory is a theory, I would need more convincing to even start considering the “possibility of a single consistent metatheory”.
Anyway, any attempt for a meta-theory is encouraging, as it brings new distinctions, which bring new conversations, which bring new distinctions. I have three small comments on your proposal for a meta-theory:
1. What I’m missing in your proposal, are the theoretician and the “users” of the theory. They seem to me excluded, or at least not included in the process of doing what they do, including, developing and/or applying this theory.
2. Any theory is expressed in some language. Shouldn’t then a meta-theory be a theory about language? In fact Doug mentioned some people that worked on that, some of them quite convincingly for me (Wittgenstein), and others not so (Chomsky).
3. There are many theories about what a theory is. I find the following view of Maturana quite convincing:
“We human beings propose theories as systems of explanation of what we distinguish as happening in what we observe
or do in the realization of our living. Theories
are systems of logical deductions that we
propose in order to follow the consequences
that would arise in a particular situation if
we transformed everything in it around the
conservation of some set of basic premises
that we choose to adopt – either because
we accept their validity according to some
logical argument or, a priori, because we like
them.”
Anyway, how do you see the role of an EA meta-theory in general, or your proposal in particular, in the life of an Enterprise Architect?
Ivo – apologies for the delay in replying to this…
“The ambivalence is a norm in social systems.”
Yes, of course you’re right. Note, though, that just two paragraphs later, I wrote: “What’s perhaps hard to explain is that, at the surface level, that’s exactly what it should be. If it wasn’t that seeming mess, that’s when we should be really worried…” The point here is that, unfortunately, many people still expect a single, unified, predictable, predicting theory – which we know is just not going to happen (as per Gödel etc). Hence – again for most people – the all too common was that the lack of an apparent single ‘the theory’ is experienced as something ‘weird’. The point of all this metatheory stuff is to provide a way to get round that sense of ‘weirdness’. (Okay, it may well technically be another variation on a theme of Stewart and Cohen’s ‘lies-to-children’, but it works well enough for that purpose.)
“Having in mind that a meta-theory is a theory, I would need more convincing to even start considering the “possibility of a single consistent metatheory”.”
Yes, of course. The point above about ‘lies-to-children’ likewise applies; also note that very careful use of “a single metatheory” (indefinite-article) rather than a purported “the single metatheory” (definite-article).
“1. What I’m missing in your proposal, are the theoretician and the “users” of the theory.”
Good point. It’s there, in implicit form – such as, for example, that “a metatheory provides a frame in which to identify where each type of theory would work well – and where it wouldn’t” does imply someone who needs to identify types of theory for a given context. But you’re right, it isn’t there explicitly. My bad, I suppose – but is something that I really do need to say explicitly, every time?
“2. Any theory is expressed in some language.”
True. But are words the only ways in which a language can be expressed? I gave a bunch of diagrams, and the relationships between those diagrams: isn’t that a form of language in its own right? (Semioticians would argue so, I think? – likewise Christopher Alexander and the concept of ‘A Pattern Language”?)
“3. There are many theories about what a theory is.”
Agreed; and agreed that Maturana’s view is one useful example – but one amongst many. In a mainstream scientific/technology context, many would argue that the role of a theory is to provide prediction – which is fine for those contexts that fit well with linear-paradigm concepts, and not so good for high-uniqueness or emergence or the like. Given that EA and similar fields tend to incorporate both some of the former – contexts where linear-paradigm may be valid – and a lot of the latter – high-uniqueness, emergence, etc – then the idea that theory is ‘all about prediction’ can be a bit misleading. Hence I view the role of theory – and particularly of metatheory – as a means to point towards potentially-useful insights: technology (usefulness-oriented) more than science (‘truth’-oriented). Hence the nature of this post, and the type of theory/metatheory described within it.
Also the theoretical-base/focus is around relationships between frameworks, rather than purported predictions or ‘proofs’ of a framework: that point should perhaps be emphasised a bit more than I’ve done so above?
“how do you see the role of an EA meta-theory in general, or your proposal in particular, in the life of an Enterprise Architect?”
Probably the main hope is that it might reduce the ‘framework wars’ – that the arguments shift from ‘which one is The One True Framework that applies the everything’, to ‘which frameworks are useful in these specific types of contexts’ – and also be able to explain why this type of framework would be more/less useful than that one, for that purpose.
Also, given my catch-phrase about the role/purpose of architecture as “things work better when they work together, on purpose”, this type of approach would give us a means to link seemingly-disparate frameworks together, consistently, across all types of contexts and at all levels of abstraction. Which would be useful if it works. 🙂 (And which, yeah, I’d hope that this ‘proposal’ above could work as a test-case for that kind of approach. Though do note again the indefinite-article rather than definite-article: it’s a metaframework-set, not ‘the‘ metaframework-set – or worse, ‘the only’ one…)
Hope that makes some degree of sense?
Question: Why do Enterprise Architecture?
My answer: To solve business problems.
Question: If you are not solving business problems, what are you doing?
I find it interesting that the word “problem” does not appear on this page.
Surely any sort of theory must be predicated on a reason for its existence?
Or to put it another way, one test of a theory is if it explains its existence.
Thanks, Bernard.
You’re right, the word ‘problem’ is not there. And yes, in part, that could be construed as a mistake, an important omission. But it’s also because my answer to your question would be slightly different than yours: “We do enterprise-architecture to pre-empt business-problems, wherever practicable”. And almost all of the discussion above – including the nature of theory/metatheory as described above – is about pre-empting problems: such as those that could/would/do arise if we use the wrong type of theory in a given context. So in that sense, yes, it does indeed explain its own existence.
Note also that there’s a crucial difference between business-questions and business-problems. I would strongly argue that we do do enterprise-architecture and the like in order to address a specific business-question. Yet only some (and ideally none) of those questions would be classed as ‘problems’. That distinction is kinda important…
See the post ‘ Selling EA – 1: What do EA clients want?‘ for some examples of what I consider to be the kinds of questions that EA would typically address.
See the post ‘Selling EA – 3: What and how’ for more on why the classic ‘problem -> solution’ dichotomy doesn’t match up well with what we (need to) do, or how we (need to) do it.
Hope that helps?
In terms of problem-think, one way to think of EA is that it addresses the problem of enterprise entropy. Hopefully to avoid; all too often to remediate.
Re Problem/Question.
IMHO, a problem can be expressed as a question. If you can’t express it as a question then it’s not a problem. Value (as perceived and determined by the business) comes from solving that problem.
Not all questions are problems.
So fat I think we are in agreement.
re “We do enterprise-architecture to pre-empt business-problems, wherever practicable”.
That’s an interesting statement.
I assume you don’t mean that EA is only ever done to avoid problems rather than solve existing problems?
I’d agree that it’s one objective but not the only one.
And if you phrase it as “how can we use an EA approach to pre-empt potential business problems?” then that’s a question/problem itself.
re “we do do enterprise-architecture and the like in order to address a specific business-question. Yet only some (and ideally none) of those questions would be classed as ‘problems’”
I think we are into semantics here. I avoid such arguments by defining a business question to be a problem, the solution of which delivers value to the business. If there is no value in answering a business question, then why answer it?
Which brings us back to the simple statement that “the purpose of EA is to solve business problems”.
Or to put it another way, if you don’t understand the problem, any solution will do – just don’t expect it to have value.
And getting back to meta theory – you seem to be answering the business question “how do we avoid using the wrong type of theory?”.
This has enormous value to the business, because, if you get it wrong there is a high probability of ending up with a failed IT project. And there’s been far too many of them, mostly (IMHO) because the wrong approach (theory) has been used.