More on TOGAF and certification
Yikes! Talk about misinterpreted in a sound-bite… 🙁
(Before I go any further, a note to all in the TOGAF training/education community: from what you’ve read elsewhere, you may at present believe that I’ve been attacking you personally. As you’ll see below, this is not the case – so please accept my apologies for others’ interpretations of what I wrote. Do read on – and thank you.)
There are a few Tweets going round that suggests I’m attacking TOGAF (again), this time by suggesting that TOGAF training is worse than useless:
harsh deduction by @tetradian: TOGAF certification almost an indication that one is NOT capable of doing #EA
“… close to … a TOGAF certification is an indication that someone is not capable of doing enterprise architecture.” http://bit.ly/uZDZd
I’ll admit my original post summarising the London TOGAF conference last month does indeed include that latter quoted text. But it’s quoted way out of context: so please, do read the whole post before you jump to that conclusion! – because it isn’t what I’m saying at all.
First, it’s not my own ‘deduction’: it’s a near-verbatim report from a broader discussion at the TOGAF conference. From the certification perspective, four key themes came up from the conference, all from leading members of the TOGAF community:
- the reference-architectures (Part VI of the TOGAF spec: ‘Technical Reference Model’ and ‘Integrated Information Infrastructure Reference Model’) are way out of date, and at the least need a complete overhaul, if not dumped altogether [that was from the Open Group’s lead Allen Brown, in one of the plenary sessions]
- “almost no-one” uses the ADM in the form described in the TOGAF specification [in my last post I said I thought that was one of the guys from Deloitte, but my notes indicate it was Mike Lambert from Architecting the Enterprise, one of the lead TOGAF training groups]
- enterprise architecture is much broader than IT, and must now encompass the whole of the enterprise [that theme came up at least a dozen times, in plenary sessions and elsewhere]
- enterprise architecture needs to be understood as a professional discipline, comparable to other professional disciplines such as medicine and building-architecture [again, many people, but particularly Len Fehskens, Open Group VP on Skills and Capabilities]
These are all points that, yes, I have personally pushed hard over the past few years: but you can see from the above that it’s not just me that’s saying it – it’s being echoed now right from the centre of the TOGAF community itself. (Just this once, I’m not ‘the Outsider’ here! <wrygrin>)
So, to the problem of certification. The key point is that certification alone is not an indication of professional competence. Back in my aero-engineering days, it was common knowledge that newly-graduated engineers were a potentially lethal danger to everyone in the place: they knew just enough to think that they knew what they were doing, with that arrogant certainty of the newly-qualified, but had no idea of how to work with the subtle complexities and constraints of the real world of engineering. For example, they would specify components that couldn’t actually be made, or assemblies that could be made but couldn’t physically be assembled. Even for the best, it usually took a year or two at least “to learn how to make my mathematics sufficiently imprecise to be usable”, as one of them put it. Crucially, there were a few who never learnt that lesson, and instead clung on to their certification as ‘proof’ that they were competent – which in practice more proved that they were not competent to be let loose on a real aircraft. Or, in this context, a real enterprise.
On its own – and again I’ll emphasise, on its own – an enterprise architecture certification does not and cannot indicate competence: it needs to be balanced by real-world practice. For which, again, crucially, this profession at present has no means to monitor or measure.
Next, look at what’s actually covered in the existing TOGAF certification: it’s primarily about the ‘standard’ ADM and the reference-models – which are no longer used in that form in practice. And – as also indicated in those themes from the conference – real enterprise architecture is much, much broader than IT: yet everything in the existing certification is centred on IT. So anyone who does slavishly follow the ‘standard’ will be almost guaranteed to create an architecture that might at first seem ‘efficient’, but will be so outdated, so IT-centric and so far off the real mark that it will at best be useless, and possibly much worse than that.
What the old TOGAF 8 certification exam did not cover was how to adapt the ADM to the enterprise, or how to create reference models and use them for compliance-monitoring and risk-management – which is what is actually most needed in those stages of architecture that the TOGAF spec aims to cover. And there’s no way that any of that kind of context-dependent knowledge could be assessed in a simplistic multiple-choice exam such as is still used for TOGAF certification. As I mentioned in the previous post, I nearly failed my TOGAF exam because in many parts of the test, none of the options shown on the screen actually matched what I knew from experience works in practice, and the nearest available guess turned out to be ‘wrong’ according to the specification in the book. Conversations at the conference made it clear that I was far from alone in that experience: in effect, anyone who presents a high score in their TOGAF certification may have the book-knowledge but know nothing about the practice, whereas many will score low because they are competent in practice. So as it stands, the TOGAF certification not only tells us nothing about professional competence, but can be actively misleading: a high score may well indicate that someone is not competent to do the work, whereas a low score indicates either high competence or complete failure, with no apparent means to distinguish between those two extremes.
All of which adds up to a serious problem.
It does not, however, mean that TOGAF training is wrong. Quite the opposite: many of the trainers I talked with at the conference made it clear that their training-courses emphasise the importance of adaptation of the ADM, development and use of reference-models, and all the other skills needed to assess and adapt to the enterprise context, and how to extend EA beyond the IT domain itself. To develop those professional skills, we’re likely to need more training, not less; and much of that training needs to be context-specific, too. The catch is that almost none of this material is in the current TOGAF specification, and none at all is assessed in the current TOGAF certification. So yes, whilst to my mind the TOGAF specification is still annoyingly limited and limiting, that’s not the real problem in this case. The point here is that, as it stands, the TOGAF certification is not only meaningless but actively misleading: and right now that is a real, genuine, active, in-your-face, fundamental problem for the profession.
This is a problem that’s being addressed: as I said in the previous post, Len Fehskens is specifically tasked with this on behalf of the Open Group, and others are tackling it in other ways with other groups. But we must first acknowledge that it is a real problem, and one that won’t go away simply by ignoring it, which is all that had been happening to date. So, yes, whilst it’s an uncomfortable fact to face, one of the key signs that the EA profession is maturing as a profession is that it is now willing to face up to such uncomfortable facts.
‘Bad’ news that’s good news all round, in fact. 🙂