John Zachman and the curate’s egg

A couple evenings ago the BCS (British Computer Society) held a open question-and-answer session with John Zachman at the EAC-BPM conference in London. How much has the Zachman Framework for enterprise-architecture changed over the past decades – and particularly over the past few years?

If you don’t know the Zachman framework already, here’s a quick visual summary:

In effect, it’s a list, or taxonomy, or ontology, or categorisation, of what Zachman calls ‘primitives’ for enterprise-architecture, laid out in a 6×6 or (as here) 5×6 matrix: a six-column horizontal-axis of ‘Why’, ‘How’, ‘What’, ‘Who’, ‘Where’ and ‘When’ (though in no particular order), and a five- or six-row vertical-axis from most-abstract (or most-‘architectural’) to most-concrete (detail-layer implementations).

The layout has changed back and forth over the years – as can be seen in some detail on the respective Wikipedia page – but the core idea has remained much the same since its first inception back in 1987. In the BCS session, Zachman took us another couple of decades further back, to a bunch of folks at IBM struggling to provide appropriate data-systems support for a huge oil-company merger in the late 1960s onwards.

There’s no doubt at all that the Zachman Framework has been hugely influential – in part because of its simplicity, but in no small part because Zachman himself is unfailingly one of the nicest people we could ever hope to meet. Given that, it does kinda hurt to have to say that, in reality, the framework is a real curate’s egg – “excellent in parts, m’lord”:

Bishop: ‘I’m afraid you’ve got a bad egg, Mr Jones’
Curate: ‘Oh, no, my Lord, I assure you that parts of it are excellent!’

Unlike the original curate’s-egg, yes, there are some definite good-points, with real value – but also some serious howlers, and all underpinned by a metaphor that’s so fundamentally wrong that it’s crippled enterprise-architecture for decades. Yeah, really, that bad… ouch…

To explain why, it’s probably simplest to use some of the quotes from the session.

First, though, a trio of quotes that indicate the character of the man himself:

— (on learning anew) “I’ve learned more about enterprise-architecture in the last five years than in the previous twenty-five.”

It would have been so easy to have stopped with the original framework, capitalised on its huge popularity, and left it at that. Instead, Zachman has had the humility and courage to keep questioning himself, to keep learning; too many others don’t.

— (on use of frameworks) “Please explain to me the business-problem you’re trying to solve, and how you plan to solve it.”

This is the right way to do it: the business-question comes first, not the framework. Far too many people get this one the wrong way round; Zachman doesn’t.

— (on attempts to possess the framework) “Early on, one of the tool-vendors wanted me to sell them the framework as an exclusive, so that they could claim to own enterprise-architecture. That’s wrong: the framework is a fact of nature – it can’t be owned. I do not believe anyone owns the Periodic Table, or the Zachman Framework.”

I’d definitely sing Zachman’s praises here, because he could have got a lot of money for indulging in that all-too-common form of dishonesty. Instead, he’s aligned himself with the other, responsibility-based form of ownership, and acted as a steward of the framework, protecting it as itself on behalf of everyone. Yes, he’s made a good living out of doing so over the decades – but also worked darn hard for it, too, when the dishonest sell-out would have been so much easier. That choice shows real strength of character, real honest steel behind the surface – something well worthy of respect.

Yet this is where things get difficult…

That’s because whilst Zachman himself is a quiet giant in his own way, the framework itself – and even more, its historical and conceptual underpinning – is deeply, deeply flawed, to the extent that it’s often dangerously misleading: a curate’s-egg, arguably “excellent in parts, m’lord”, but so pervaded by subtle rottenness that we might be wise to discard the whole thing and start again. And yeah, I’m well aware that that interpretation could be kinda unpopular, so it’s perhaps best illustrated too by various quotes from the session.

— (on the core basis for architecture) “data, control, infrastructure, systems, planning”

This was a reference back to the ‘Data Processing’ of the late-1960s, but it’s clear that it still holds for his thinking even today. Anyone notice what’s missing from that list? – yep, people. In fact he made it clear that the whole idea was to remove people from any consideration – replace people with automation in every possible way. In other words, classic Taylorism – with all of the errors and failings that that implies.

— (also on core concerns for architecture) “The secret to this whole thing is the categorisation of the data.”

No, it’s not: that’s confusing information about a ‘thing’ with the ‘thing’ itself, often recursively. It’s a classic ontology-error that seems to be endemic among IT-folks, but it does need to be understood and challenged as an error, wherever it may occur. This one error permeates all of the thinking behind the Zachman framework – and is one of the core reasons why the framework is so problematic in real-world practice.

— (on the scope of architecture) “It’s not an IT issue.”

No, it’s not – very much not. (Or not solely an IT-issue, anyway – because no question at all that IT would always be relevant somewhere. Just not the central issue – a too-subtle distinction for some people, it seems, yet utterly crucial.) The catch is that every example that Zachman gives is ultimately an IT-based example – such as that in the ‘What’ column, after ‘Material List’ the only ‘What’ mentioned is data. What about physical things as ‘What’? relations? brands? – don’t they exist? – don’t they matter too? Apparently not…

— (on limits of scope for EA) “He did not think anything with less than $500M (1970) revenue was worth doing architecture for.”

To be blunt, that’s stupid (though we should note that Zachman’s quoting someone else here). There’s perhaps a crucial distinction here between the scope of what IBM would believe worthwhile to sell EA-services for – which, yes, could well be that size and scale – versus the actual limits of need – which are much, much smaller, and seem to apply to organisations of every size, large and small. I’ve worked for larger-sized organisations too – 100,000 person was probably the largest so far – but in practice I’ve also worked with other organisations all the way down to a 30-person small restaurant-chain, a 20-person primary-school and even a 2-person (but very-high-‘value’) investment-team. Others in whole-enterprise EA have said much the same. From a discussion with Philip Hellyer at the conference, it seems likely that whilst EA would be useful and valuable even for a two-person startup, some form of EA becomes all but essential as a guide for communication and integration once an organisation, as a sociotechnical-system, expands in size much beyond the Dunbar Number – around 150 people or so. IT-complexities tend to become critical at somewhat larger organisation-size – particularly after merger/acquisition or major market-change – but as Zachman says above, “It’s not an IT-issue”: the real concerns for EA are about communication between people first, not just machines.

— (on his concept of ‘primitives’ and ‘composites’) “The primitives are the architecture.”

No, they’re not – they’re an aid to architecture, not the architecture itself. As he himself said later, “implementation has to be a composite”: the architecture is in the interaction and interplay between primitives and composites – not just one or the other. That’s a fundamental misunderstanding of the nature and practice of architecture there that is, again, deeply embedded in the framework. Oops…

— (on the role of IT in the enterprise) “We, the IT people, have been driving the enterprise for 70 years – the [IT]-system is the enterprise.”

No, nono:  this is fundamentally wrong – IT-centrism at its worst. IT is, yes, has been and will no doubt continue to be a key enabler for key aspects of the enterprise – no-one doubts that at all. But an enterprise itself is, literally, “a bold endeavour” – and IT-boxes, in themselves, are neither bold, nor do they endeavour. To say that “The [IT]-system is the enterprise” is, by definition, to confuse means with ends – Not A Good Idea… Instead, always, always, the core of the enterprise is people: if we ever get that point wrong, our ‘enterprise’-architecture will likely be worse than useless, a mere distraction from the real concerns of the enterprise. Again, this is yet another fundamental error that is embedded and expressed throughout the framework and most of its popular usages.

— (a question from the audience – from Elizabeth Erwin, I think?) “IT has hijacked EA: how do we get people to understand that rows 1-3 are about business, and just as important as row 4-6 implementation?”

We didn’t get an answer on that from Zachman himself, though we did get a few from the wider audience. The blunt reality is that this is yet another core failing of the framework, arising in part from the muddled concept of ‘primitives’ versus ‘composites’. For what it’s worth, I’ve spent a lot of work over the past several years on this, at first for the whole-of-enterprise rework for the TOGAF ADM (see e.g. ‘Spot the difference‘), but also as a core aspect of Enterprise Canvas – see the posts ‘Services and Enterprise Canvas review – 4: Layers‘ and ‘Services and Enterprise Canvas review – 5: Service-content‘.

— (about the ‘science’ of EA) “When Mendeleev published the Periodic Table, the alchemists were very unhappy.”

Okay, I’ll admit this is a bit of a hobbyhorse of mine – perhaps not so directly related to EA as such? – but I do get annoyed when I see poor grasp of both science and history portrayed as ‘fact’. As a discipline, alchemy had largely died out at least a century before Mendeleev – hence no alchemists to be “unhappy” – though it was popular in previous centuries, even amongst such pillars of science as Isaac Newton. Mendeleev’s achievement was to codify already-existing knowledge, in a way that pointed towards new possibilities: so yes, in that sense, the same sort of idea could be claimed for Zachman’s framework, though not really in the same sense. The big problem, though, is that there are very real limits as to how much mainstream ‘science’ can apply in the sociotechnical-systems in the EA context – see, for example, the post ‘Theory and metatheory in enterprise-architecture‘. And there are also strong grounds to suggest that there’s much that we could still learn from alchemy, too – see the post ‘Enterprise-architect – applied-scientist, or alchemist?‘. In short, don’t be quite so quick to disparage others in the past, as a cheap way to make a present-day idea look better than it actually is – lest others in the future disparage our work in the same way too.

— (on the basis for enterprise-architecture) “Architecture for airplanes, ships, chairs, buildings, it’s all the same – why should it be any different for enterprises?”

This is the howler, the clanger, the one really, really fundamental error that not only cripples the Zachman Framework, but because the framework has been so popular, has also crippled enterprise-architecture itself for way too many decades now. The short-answer to his question is that yes, enterprises are different – fundamentally different – from “airplanes, ships, chairs, buildings”: the latter represent physical ‘things‘ (or virtual ‘things’, if we extend it to data-architecture and the like), whereas an enterprise is a sociotechnical-system, comprised of people as much if not more than of any kind of ‘thing’. Zachman’s metaphor of ‘engineering the enterprise’ could perhaps sort-of work well enough for the solely ‘-technical’ aspects of a sociotechnical-system, but not for the human or ‘socio-‘ aspects of a sociotechnical-system – and the whole point of a sociotechnical-system is that the two types of elements are inherently interwoven with each other.

We do not, cannot and should not attempt to ‘engineer’ people and people’s behaviour, because it doesn’t work – not in the same way as machines, anyway. That was the Taylorist fallacy, that’s been proved false time and time again, in often-disastrous mistakes such as IT-driven ‘business-process reengineering‘. The one-line summary here: enterprises are not technical-systems but sociotechnical-systems – hence the concept of ‘engineering the enterprise’ should never be used as a metaphor for enterprise-architecture.

In short, the Zachman Framework is a ‘curate’s-egg’: it looks good on the surface, but is deeply-flawed, in many different ways – flaws that have been embedded in the framework right from the start, and yet have not been properly addressed at all in the three decades or so since then. Useful though it is in enterprise-architectures, the framework can be dangerously misleading if not used with care, capable of causing very real damage to the respective organisations – and we ignore its flaws at our peril.

Yet another You Have Been Warned item, perhaps?

Posted in Complexity / Structure, Enterprise architecture Tagged with: , , ,
8 comments on “John Zachman and the curate’s egg
  1. Peter Bakker says:

    I’m still wondering how much (EA) frameworks are really used. My (infra) architecture career is now about 15 years old and I worked for several big organizations in the Netherlands but I never saw one not even TOGAF, being used in reality.

    Are there objectives figures available of the usage and effectiveness of EA frameworks?

    • Tom G says:

      My experience is much the same as yours. The one place where I saw a framework formally used, they rejected TOGAF as being uselessly IT-centric (useless for their purposes, anyway), and instead built their own framework from logical extension of some placeholders in FEAF that actually did allow for real-world entities such as people, buildings and physical machines.

      A few industries and contexts are mandated by law to use a specific framework: for example, US defence-industry (DoDAF) and providers to US Federal government (FEAF). From what I’ve heard, they’re rarely used properly as frameworks, but simply as box-ticking exercises – in other words, an expensive waste of time and effort.

      The only frameworks that I’ve seen that are actually used and work well are those developed by the people of the respective organisation, preferably developed bottom-up from the grass-roots, because it rarely works well when imposed top-down. There has to be real value to everyone to use the framework, otherwise it just doesn’t get used.

      @Peter: “Are there objectives figures available of the usage and effectiveness of EA frameworks?”

      I don’t know, but I greatly doubt it… :wry-grin:

  2. Your Zachman piece is brilliant, Tom! I have been scratching my head for years as to why no one has found a way to, in effect, tell the kind, but somewhat naive, “Emperor” that he has been hoodwinked by non-critical praise all these years. It is baffling except that he is so likable.

    But that doesn’t excuse the bizarre conclusion that he stumbled upon the “Periodic Table” of Enterprises/EA, a force of Nature–how peculiar a metaphor and how wrong!

    I wish he had “sold out” because this ethereal”framework”, evangelized by him (rather than being rigorously developed into an actual book!), has set EA back by at least 15 years.

    In 2006, he told me that an airplane (hard) system has the same characteristics as an organization (soft system).

    From that point on, I have tried my hardest to steer people away from his half-baked, obscure, and method-free, generic classification table.

    But I have not succeeded. He continues to charm…and rake in the money on something doing more harm than good.

    As to the question about frameworks, I think good knowledge of TOGAF has proven to be of fundamental importance to so many architects in companies worldwide.

    To me, TOGAF training for certification has been a “no regrets” initiative for thousands. You cannot use TOGAF straight out of the box, but if you don’t know it, how can you customize and evolve it?

    I have trained about 4000 people in EA/TOGAF and I continue to learn from others while teaching TOGAF. It is a great foundation on which to build as an architect, if it is properly taught by experienced architects, and can help achieve, fairly predictably, the acceleration of EA knowledge, skills, and maturity.

    I also train the FEA/F, but I complement the training with TOGAF. I have been asked many times, as a result, why the government is not using TOGAF since it offers so much more, including all the hooks into so many relevant topics a mature Enterprise Architect should also know.

    I think it is important to remember that TOGAF must be tailored.

    Going back to the Zachman approach: Well, it is, after all, the “Periodic Table” for enterprises — to alter it would be to go against the laws of nature, or so we are nicely reminded by a super charming but mistaken EA pioneer.

    It is time to write a critical history of EA. Just because Zachman got it wrong–and still holds center stage, even so–doesn’t mean that all EA Frameworks are a waste of time. We must begin to be objective about the shortcomings of the Zachman Framework without throwing away the baby with the bathwater. Just because the Zachman Framework is a problem, it doesn’t mean knowledge of all EA frameworks is a waste of time.

    Mastering TOGAF, for example, in my view, is a great investment of a week or more and brings one into a community of folks with that same broad education in EA. (By the way, I have been Chief Architect twice and taught EA/TOGAF for 10 years globally).

    But learning TOGAF is a beginning, a license to then begin tailoring it to be actionable, increasingly agile and customizable fo different sectors and increasingly complex challenges.

    • Tom G says:

      Thanks for this, Steve.

      Strong agree re ‘method-free’ – in fact with one client in Australia I had the embarrassing situation that I had to explain that, no, I couldn’t develop their architecture according to ‘the Zachman methodology’, as they’d asked, because it didn’t in fact exist. (This was total news to them, apparently.)

      Unfortunately I can’t agree with your views on TOGAF: to me at least, it’s cause – and still actively causes – even more damage to EA than Zachman, which is quite some achievement. (See the post ‘Spot the difference‘ for more detail on this.) The one saving grace, and the one you do highlight above, is that it is possible to tailor TOGAF in a way that makes it usable (or just-about usable) for real-world EA: the mandatory requirement, though, is that we must drop the rigidly IT-centric ‘three architectures’ from the ADM in order to do so – a requirement that still scarily few people seem to get…

      On “It is time to write a critical history of EA”: again, agreed. I’ve done quite a lot of work on this over the past decade, and so have other folks such as Doug McDavid and Open Group’s Len Fehskens – but yes, it’s time to bring all of those efforts together, into something that uses the past to lay the groundwork for a better future for EA.

      • Ric says:

        Hi Tom,

        The ZF does in fact have a method. Quite a detailed and well worked out one. I did an intensive in it some time back and still have the book sitting on my desk. “Enterprise Architecture for Integration: Rapid Delivery Methods and Technologies” ISBN 1-58053-713-8 It was developed by Clive Finkelstein, and it was Clive himself who conducted the training I attended.

        Clive’s methodology is no rough cut. It is rigorous and very much the work of an engineer. It shows how to go from the ZF atoms to working code.

        Which is the point of this comment, and by and large the point of the ZF.

        The ZF was a product of an era when large firms hired software engineers to develop, integrate and maintain their own business information systems.

        Applied to that task and married to a rigorous software engineering methodology such as Clive’s there is nothing wrong with the ZF. One could pick at details, but assessed against the problem it was conceived to address, it is a more than adequate tool.

        When I did the training Clive was targeting the last problem domain to which the ZF could be applied as intended – large scale enterprise application integration… how to use ZF to manage service orientation of systems using web services to create new applications in the BPM layer.

        In the SaaS/PaaS “smart-end-point-dumb-pipes” world of microservices and REST, it may be that the time for even this application of ZF has now passed.

        My point is this (by way of a deliberately exaggerated analogy)…

        If someone tries to cross a river on a bicycle, it is not a problem with the bicycle’s design that causes the rider to sink rapidly to the bottom.

        • Tom G says:

          Many thanks, Ric – point(s) taken, of course.

          In my defence, there was no official ‘the Zachman method’ at that time (though I gather there is sort-of one now). Your example of Clive Finkelstein’s methodology is no doubt valid, and the methodology itself very good, but (like TOGAF, for that matter) it uses the ZF rather than ‘is based on and endorsed as the method for the ZF’. Which, okay, is a perhaps-subtle difference, but a very important one for anyone who thinks they’re buying ‘the Zachman Method’ – especially when we risk running foul of the Trades Description Act or Copyright Act or whatever if we claim that it is ‘the Zachman Method’.

          And again, the thing that really cripples the ZF – as for so much so-called ‘enterprise’-architecture – is that in most of its versions it was and mostly still is rigidly IT-centric. From what you say above, that wouldn’t have mattered for Finkelstein’s methodology, because the latter is specifically for application-integration and the like; but it does matter – a lot – for anything that has to connect with or work on any part of the scope beyond bare technical-systems and into the sociotechnical-systems space.

          “[I]t may be that the time for even this application of ZF has now passed” – I’d say that the ZF does still have some valid uses, especially when generalised to any kind of technical-system, rather than solely IT-type technical-systems. The problem is that its metaphor of ‘engineering the enterprise’ is fundamentally incompatible with sociotechnical-systems – and it’s the latter, rather than purely technical-systems, that increasingly we have to work with in EA. If we’re going to use it, we do need to use it with care, and with full awareness of its very real limitations.

          “If someone tries to cross a river on a bicycle, it is not a problem with the bicycle’s design that causes the rider to sink rapidly to the bottom” – agreed, of course: ‘a fool with a tool is still a fool’, of course. Yet it doesn’t help if the instruction-manual that comes with the tool makes it very hard to use it in a way that is not foolish, and that instead is so over-hyped that it does quite strongly suggest that this bicycle can indeed cross a river all on its own. Sigh…

          Thanks again, anyway – much appreciated.

Leave a Reply

Your email address will not be published. Required fields are marked *