TOGAF and SOA

This one’s another follow-up to another new comment to a rather old post – namely Manish Joshi’s comment today to the post on ‘Zachman and TOGAF revisited‘ from back in August last year:

My focus is on extending TOGAF to include SOA.

My approach was to add SOA as a cross-cutting architecture (cutting across BA,ISA,TA & also the governance of TOGAF (since SOA also needs SOA governance).

Roland [Ettema] has also suggested a nice approach above of having SOA in place of IA and making IA a horizontal.

How do you guys compare these 2 approaches and is there any other way around for customizing TOGAF for SOA ?

As many people have discovered the hard way, trying to use the standard TOGAF methodology for SOA seems to work well at first, but will suddenly hit intractable problems that just don’t make sense if we come from an IT-centric perspective. (In fact if we come from that perspective alone it’s almost impossible to see why they don’t make sense – which is why SOA so often seems so damn’ hard…) The actual reason is three-fold:

1. an iterative approach – e.g. an Agile methodology – suits SOA developments best, but TOGAF’s methodology (despite its claims to be iterative) is still essentially a ‘big bang’ approach to architecture development
2. the TOGAF methodology purports to describe the whole of ‘enterprise architecture’, but in fact describes a very narrow IT-centric subset – too narrow even for an IT-centric SOA, let alone a full ‘service-oriented enterprise’
3. the TOGAF framework still draws heavily on Zachman, which itself draws on technologies that are at least twenty years old – long before the days of interactive user-interfaces and layered servers, let alone current SOA web-services and service-buses

To resolve items #1 and #2, we need to rework the first half the TOGAF methodology – Phases A to D – with a few lesser amendments in the Preliminary Phase and Phases E to H:

• for item #1 – agile – we split the existing Phase A in half, moving the ‘architecture vision’, documentation, framework skeleton and overall authorisation into the Preliminary Phase, with Phase A amended to focus only on the needs of the specific iteration

• for item #2 – methodology – we expand the scope of TOGAF’s layering of ‘business architecture’, ‘information systems architecture’ and ‘technical architecture’:
  • ‘business architecture’ describes the overlighting business concerns – i.e. Zachman’s ‘contextual’ and ‘conceptual’ layers, with some touch into the ‘logical’ layer, and not the generalised ‘everything not-IT’ grab-bag as described in the present TOGAF 8.1 spec
  • ‘information-systems architecture’ becomes the common interface layer – i.e. Zachman’s ‘logical’ and ‘physical’ layers, but across every aspect of the enterprise, not just the IT [Roland Ettema’s placing of SOA here is just one possible subset of this]
  • ‘technology architecture’ becomes the detail-layers across the whole enterprise, not just the IT – e.g. skills, logistics, vehicles, and much else besides, right down to the real-time operations layers

We could use these amended layers as the content and focus for Phases B to D respectively. The catch is that, once we’re tackling a true whole of enterprise scope, we end up with a requirement for an almost endless stream of stakeholder reviews within each phase. So for purely practical reasons I’d recommend using an alternate split for Phases B to D, as ‘As-Is’, ‘To-Be’ and ‘Gap Analysis’ respectively: with this split, the stakeholder reviews become the phase boundaries.

To resolve item #3, we need to rework Zachman itself – a major rework, since a bit of careful analysis shows not only that he’s missing a layer at the top, but an entire dimension of depth (which I’ve described as ‘segments’), and also the original content and descriptions for several of his columns are, frankly, a scrambled mess. (His latest revision in effect quadruples the complexity of his framework without addressing any of the core underlying problems – and he’s still stuck in the metaphor of ‘engineering the enterprise’, which does not and cannot work with the complexity and dynamics of real-world systems. But that’s another story for another time…)

But in that rework, what we don’t do is add more columns. The basic principle of What, How, Where, Who, When, Why is sound – the catch is that they don’t necessarily correlate directly with anything in the real world. Instead, we focus on Zachman’s valid principle that ‘architecture is primitives; solutions come from composites’. An interface, for example is a composite – it doesn’t sit in a single Zachman cell. (Depending on the context, it’s a composite of How and Who, plus probably When and What as well.) EA-toolset vendors have made this worse by trying to shoehorn all their models into single columns or single cells. If you add ‘interface’ as a column – which I’ve seen several people do – you end up with a Zachman-like muddled mixture of primitives and composites, which is not a good idea. It may look fine at first, and may seem to make solution design easier; but it really does cause utter chaos later down the track, when the technology changes on you, or when you try to model alternate implementations such as for disaster recovery. In short, don’t shoehorn: primitives are primitives, composites are composites – don’t mix them up.

So for SOA, what we need first is an exercise in meta-models and meta-architecture: we start from primitives of the Zachman frame – single cells – and build a layered set of root-composites (meta-patterns) that describe our key entities. (Meta-architecture is a swine to get your head around, but if you want your solutions to be ‘future-proofed’ you can’t afford to skip over this stage.) Services are a good place to start: they’re composites of How (functions) and Who (capabilities), whilst the messages and trigger-events are composites of What and When. We then build cross-linking composites (functional patterns) that are closer to our implementation layer: a web-service, for example, links a service definition with a message definition with an interface definition. That gives us our logical layer, which we can still specify in a true implementation independent manner – which is what we need for, say, disaster recovery planning. Then we can describe entities (implementation patterns) for the the implementation specific physical layer. Then we can do our SOA designs – which may well traverse IT-based and non-IT-based processes. But because they’re all anchored all the way back to the root-primitives, we have a straightforward trail of derivation and dependency to guide redesign when (not ‘if‘…) we get hit by a change of technology, or context or whatever.

To see how this works, have a look at my presentation for the last TOGAF conference: ‘Enterprise architecture and the service oriented enterprise‘ (PDF, 850kb) – it summarises all of this, together with a broader view of ‘service oriented architecture’.

I’m also working on a book on this – Bridging the Silos: enterprise architecture for IT architects – which tackles this general problem of extending TOGAF and Zachman to whole of enterprise architecture. It’s taking a little longer than I’d hoped 🙂 but there’s a PDF of a partial draft already available up on the Tetradian Books website, at tetradianbooks.com/2008/04/silos-ebook/ – any comments much appreciated!

Hope this helps, anyway.