Big-consultancies and bridging the chasm

Like all small independents in just about any industry, my relationship with ‘the big boys’ is ambivalent at best. All those big analyst-consultancies like Forrester or Gartner, the ratings-agencies like Moody’s or S&P, the big IT- or process-consultancies like IBM or KMPG or PWC: sometimes it can feel, as independents, that we do all the real work of innovation and the like, but they get all the money – which can get more than a bit galling at times…

Yet what’s the truth in that? Is there something about the way we each position ourselves, the markets we each serve, that brings on this difference in perspective and scale – and income too?

What got me thinking on this was, on one side, the series of posts I’ve just done on ‘seven sins of dubious discipline‘, on sensemaking and OODA, and the links between that and the ‘Hype-Cycle’; and on the other side, re-reading a Gartner article from about a year ago, Future of EA 2025 – Evolving From Enterprise To Ecosystem, by Marcus Blosch and Betsy Burton (Gartner ID:G00269850 [summary only]). From that, I’d have to say that that sense of ambivalence about ‘the big-cons’ – not just Gartner as there, but all of them in general – is even stronger than ever…

Why?

Tricky to say, really… Mostly that there are several things that are just ‘not right’ – perhaps more in terms of feel than fact – and in some ways those things seem to add up to something that’s very worrying indeed. For independents like me, anyway, if not necessarily for them.

Some of it is… – well, let’s just say that I would not and could not do some aspects of the business the way that they do. To give one definitely-worrying example, a few of the big-consultancies are so infamous now about some of their practices – such as placing half-competent junior trainees as ‘senior consultants’, at senior-consultant rates – that at times it really does look dangerously close to an all-too-literal ‘big-con’.

(I’ve had painfully-expensive first-hand experience of that, too, a few decades back, when our then small business – one of the first pioneers of desktop-publishing – hired one of ‘the big-boys’ to do our accounting, prior to a major expansion. Yet although they charged us full ‘senior-consultant’ rates, they actually put on an unsupervised junior to do the job – and then messed him around so much that he left the company, the country and the industry, leaving our expansion-programme stranded in mid-flight. The big-accountants’ response? – they denied everything, did nothing to help at all, and instead set off on a whole swathe of underhand back-door tricks to try to get us bankrupted, so as to cover up their mistakes. We did recover from their mess, eventually, despite their continual ‘anti-help’: but it cost us enormous stress, a lot of hard work, a lot of extra expense, and an opportunity-cost that would have been measured in the millions at today’s rates – so yeah, not happy at all about that…)

As for more general aspects of business-ethics, I could perhaps best illustrate this by describing what happened some years back, when I’d had a formal recruitment-invitation to join one of the big consultancy-groups, at a senior level. (I won’t say which consultancy it was, other than that it wasn’t Gartner.) During the second-interview, with their lead-consultant for that entire field, I admitted that my aim would always to be to tell the truth as I saw it, even if it might be politically unpalatable for some. At that point, the interview effectively stopped. He didn’t get in touch again: in fact no-one at the company ever bothered to tell me that I hadn’t got the job. What I did find out later was that in that consultancy, honest truth-telling was a definite no-no – or, at the least, a much lower priority than keeping the client in sweet ignorance of reality, regardless of how much damage that might cause to the client. What the…?

So yeah, not happy about that either. Makes my skin crawl, to be honest. But it’s evident that a lot of people in business are perfectly okay with that. No matter how much damage it does to everyone. Oh well.

Best not to go into any more specifics on that on a public site, I guess. But there are other aspects of the ‘big-cons’ world that we should be able to critique without running into lawsuit-land. In particular:

• ‘big-cons’ often position themselves as innovators – which they mostly aren’t
• they offer a ‘safe’ solution to the problem of hype – but they themselves are often key participants in creating that hype in the first place
• they offer to simplify complex ideas – but often do so in dangerously-simplistic ways
• they offer supposed certainty – but with ideas that may be just plain wrong

The positioning and hype-problem are closely-related, and likewise the over-simplification and pseudo-certainty, so it’s probably best to treat them as pairs – the first in this post, the second pair in the post that follows.

So here goes…

Positioning and hype

Probably the simplest way to understand what’s going on here is to start with Rogers et al’s model of the technology-adoption lifecycle:

For which we could summarise each of the groups or stages as follows:

• Innovators: creating new ideas, materials, processes and more
• Early Adopters: applying new ideas in real-world practice, improving them iteratively
• Early Majority: working with ‘new’ ideas etc that seem to be already proven in real-world practice
• Late Majority: apply new ideas etc only when they’re fully established and routine
• Laggards: apply new ideas etc only when ‘forced’ to do so – such as when the old ones cease to be available any more

Then, in terms of the SCAN model of sensemaking and decision-making, and the respective tactics for each – as also crossmapped to other models such as OODA, as described in the post ‘Seven sins, sensemaking and OODA‘…

…the Innovators are the ones most comfortable with uncertainty, working towards the right-hand edge of the frame, and the Laggards all the way up against the left-hand edge, holding hard onto the known certainties that they have.

(The SCAN transition we most need to note here is the ‘boundary of effective-certainty’ – the point at which we transition from ‘control’-based world in which doing the same thing should bring the same result, to a more ambiguous world in which doing the same thing may bring different results, or we may need to do different things in order to arrive at the same effective result.)

For our purposes here, it’s probably easiest if we flip the SCAN frame from left-to-right, and then crossmap it behind the adoption-lifecycle model:

The crucial point here is that transition at the ‘boundary of effective certainty’. On the Unorder side, the Innovators and Early Adopters accept that the world is inherently-uncertain, and work with change to make the best of that uncertainty. On that side of the boundary, there’s also often a strong focus on ‘finding the right question‘ rather than ‘finding The Answer’.

But on the Order side of that transition, what people seem to need, desire, demand is “a notional future that is totally proven” – to paraphrase one big-firm consultant – which happens to be an inherent impossibility. In contrast to the Late Majority – and, even more, the Laggards – the Early Majority are happy enough to acknowledge that change of some kind or another is all but inevitable, and even desirable in some ways: and they want to benefit from that change as soon as possible. But in large corporations especially, uncertainty is close to taboo: and that push-pull around any kind of change – especially with the Early Majority – can make things very tricky, for everyone.

The best way out of this is to tune the expectations carefully to match the opportunity/risk-appetite at each of those stages in the adoption-curve. What we might call the ‘natural’ or un-hyped mix for each would be roughly as follows:

• Innovators: expectations: low (but always hopeful!); opportunity-focus: low to medium (focus often more on the innovation itself than its use); risk-aversion: low
• Early Adopters: expectations: medium (but working on it); opportunity-focus: medium to high (focus is on usefulness); risk-aversion: low to medium (but always willing to experiment)
• Early Majority: expectations: high to very high; opportunity-focus: high to very high; risk-aversion: unstable, often high to very high (expect a ‘proven solution’ with guaranteed outcomes), but unrealistically-low at (un)certain moments
• Late Majority: expectations: high to medium; opportunity-focus: medium (‘everyone’s doing it’); risk-aversion: high
• Laggards: expectations: medium to low (often not even hopeful…); opportunity-focus: low (often a sense of being forced into change); risk-aversion: high

We could also note from this that the Early Majority group are also the most unrealistic: the highest expectations of the returns from any change, always the most urgent in seeking for any new opportunity, yet also the most risk-averse. It’s not a good mix – hence why they’re so often afflicted by (or even addicted to?) sensemaking-mistakes such as the Hype Hubris, the Golden Age Game and the Newage Nuisance, but also why they’re perhaps the most difficult, even dangerous group to market to. Which kind of explains why the big-consultancies – for whom Early-Majority corporates are their main market – perhaps do need to make some of the choices that they do. It’s just that we wish they didn’t, that’s all…

Anyway, if we were to do the innovation-development properly, keeping all of the sensemaking-failure ‘sins’ at bay – as described in the post ‘Seven sins and the Hype-Cycle‘, and as most of the small independent-consultancies I know are careful to do – then the expectations about the value or usefulness of an innovation would map quite closely to the front edge of that adoption-curve:

In this ‘un-hype curve’ – with the ‘sins’ kept at bay – the expectations, and validity of those expectations, would rise gently from the start, typically peaking somewhere around the boundary between Early Majority and Late Majority. Beyond that point – with the applicability, usefulness and value of the innovation fully established – expectations would level out, fading slowly over time as the innovation is supplanted by newer approaches coming downstream.

But if we don’t keep the ‘sins’ at bay, what we get is the infamous Hype Cycle. It starts off with a increasing crescendo of wildly-inflated expectations – much of it driven by the big-consultancies, with their Trend Reports and suchlike:

The Hype-Cycle’s ‘Peak of Inflated Expectations’ occurs just about at the point where the Early Adopters come in to the picture. And then, because the Early Adopters test out whether things do actually work in the real-world, that punctures the hype-balloon – which leads to a crash all the way back down again, at first often to well below a realistic level of expectation. (The Early Adopters are often blamed for this, which is somewhat unfair…) Only then do the expectations start to climb again, aligning expectation and reality, traversing upward on what the Hype Cycle describes as ‘the Slope Of Enlightenment’, and reconnecting with would have happened without all of the hype.

That collapse of expectations – in effect, an over-hyping of doubt – is what spooks the risk-averse and certainty-hungry Early Majority. Which not only shows how and why Geoffrey Moore’s ‘the Chasm‘ exists, but also how the hype itself creates that ‘the Chasm’:

The point here is that the Early Majority will back off until someone shows them that this innovation is indeed ‘safe to use’, that it represents “a notional future that is fully proven”. And they’re willing to pay a lot of money for that illusory sense of certainty – often more than they would pay for the innovation itself.

Which, we note, is where the big-consultancies come back into the picture again, offering a purported bridge across the Chasm:

(We might also note that since they’re the some of those who’d most over-hyped each new innovation in the first place, it’s arguable that, somewhat as per the Shirky Principle, they’re ‘solving’ a problem for others that they themselves have caused… But best we quietly skip over that point, shall we?)

In effect, the ‘bridge’ across the Chasm consists of a new round of hype, driving expectations upward again – but this time grounded in the realism wrought at such cost by the Early Adopters. That it’s still full-on hype, of course, is a fact that’s all too evident at any of the big-consultancies’ events. But it’s also real enough to get the Early Majority to believe that the innovation does indeed represent “a notional future that is fully proven” – or, at least, proven-enough to allow them to gain access to the purse-strings of the corporates’ very deep pockets. And that is an important point in itself – something of genuine business-value to both sides.

But there are two groups of people who’ll usually get shut out of that shared self-congratulatory money-fest: the Innovators who came up with the idea in the first place, and the Early Adopters who did all the work to make it workable. Almost without exception, the big-consultancies claim that what they present to the Early Majority is ‘all their own work’ – which in most cases is just not true at all.

Sure, there are real innovators there, often hidden away in the big-consultancies’ backrooms: Henry Peyret at Forrester, for example, or Bard Papegaaij at Gartner, or Marlies van Steenbergen at Sogeti, to name just three out of a fair few whose work I greatly respect. But to be blunt, from this side of the fence it seems all too often that the supposed ‘innovation’ by a big-consultancy consists of a small amount of condensing and simplifying – the kind of work that a good technical-writer or editor would do, not a so-called ‘consultant’. And the people whose hard work was seemingly used as the inputs to that process are not only not acknowledged, but unless they’re a member of that specific consultancy, they’re not even paid by anyone, either – a fact which frequently galls…

(Okay, it’s not plagiarism as such – though some of what we see does look suspiciously close to that at times. It’s more that, in a strict technical sense, each innovator is “standing on the shoulders of giants”, each idea floating around in the aether, ripe for the picking by anyone – especially as those ideas are likely to have been around for quite a while by the time the ‘big-cons’ take them up, and also likely to have been watched by the consultancies’ analysts for much of that time, too. In that sense, fair enough, sort-of – but not fair that those before them get none of the acknowledgement, and none of the money either. Hence not surprising that so many of us independents so often dismiss the ‘big-cons’ as little more than a bunch of parasites, always taking and taking but giving nothing back… – an unfair view of them, perhaps, but unsurprising.)

Big-consultancies as educators? – yes, sure. As salesfolk? – definitely yes. But innovators? – not so much. Not really. And the larger the consultancy, often the more extreme that dichotomy gets. We can see why that pretence of ‘innovator’ is sort-of necessary – not just to the consultancies themselves, but to their Early Majority clients, who often need this unlikely mix of ‘already-proven innovation’ before they’ll adopt anything new. Yet – to again be blunt – that claim is questionably-honest at best; and it does make things much harder for the real Innovators and Early Adopters who came before them.

In terms of the adoption-lifecycle, the big-consultancies’ real positioning is as ‘sort-of innovators for the Early Majority’ – which we could summarise visually like this:

We perhaps ought to note here that the Early Majority do need from the big-consultancies this pretence of being ‘the Innovators’. It’s a key part of how the Early Majority can resolve the inherent-uncertainty problem – the need for any change to seem already safe, predictable and certain in all of its potential outcomes – by having all supposed ‘innovations’ starting out on their own side of the ‘boundary of effective-certainty’, as “a notional future that is fully proven”, and suchlike. It’s a delusion, of course, but for those specific purposes, it’s a self-delusion that’s seemingly necessary if the adoption of that innovation is to progress beyond the less risk-averse Early Adopters.

Yet it’s also a delusion that’s quite fragile: hence, in some ways, the inordinate amount of hype needed in order to get adoption restarted after the deep-dive of the Trough of Disillusionment. So to make it work, in effect everyone needs to purport and pretend that innovation begins only at the leading-edge of the Early Majority: that’s what makes it possible to pretend that the innovation was always certain.

But to make that work, the ‘big-consultancy bridge’ has to be constructed in such a way that it blocks off the view to anything that happened before: that the big-consultancies purport to be not only ‘the fountains of all certainty’, but ‘the fountains of all innovation’ too, and even ‘the fountains of all knowledge’. Which, bluntly, is just not true – and everyone knows that. Worse, it really does hurt and insult every one of the Innovators and Early Adopters – and yeah, that’s a long way beyond merely galling at times…

Again, though, it seems to be a necessary constraint imposed by the nature of that transition across the boundary of effective-certainty, the edge-of-uncertainty that can otherwise so easily become the edge-of-panic. In terms of the ‘seven sins of dubious discipline’, in effect it’s an application of the Reality Risk – potentially valid, with very real advantages if done well, yet also very real risks if not. What worries me, from what I’ve seen of the big-consultancies over the years, is that I don’t they fully understand what they’re doing this regard – or the risks that they’re playing with, for everyone, in a world where any real form of certainty is fast becoming an almost unattainable myth.

But remember, the delusion of ‘innovators to the Early Majority’ is necessary only as a strategy to recover from the dysfunctions of excessive hype. If the ‘sins of dubious discipline’ that drive the Hype-Cycle had been properly managed in the first place, none of it would be necessary at all.

And for their Early Majority clients, there’s yet another deep downside of the big-consultancies’ ‘necessary’ pretence of being Innovators, whilst actually sitting at the front-edge of the Early Majority. This is that innovation actually started with the real Innovators – which can be a long way back in time relative to when the big-consultancies start to promote it in earnest as “all their own work”. The result is that many of their purported ‘innovations’ are nothing new at all.
In fact some of it is so much ‘not new’ that it can be just plain laughable. To give just one example from Gartner’s Future of EA 2025 paper, take a look at their description of the so-called ‘Vanguard Enterprise Architect’ – what we’re told will be the leading edge of enterprise architecture in 2025. In reality, barring a few minor technological tweaks and suchlike, it’s actually quite a good description of what we were already doing at Australia Post way back in 2005 – and to us that wasn’t even that much of leading-edge. In short, in that example, Gartner’s purported ‘future leading-edge’ is already decades behind the times, before it even starts – and that kind of time-gap is perhaps not all that unusual. For a self-styled ‘innovator’, that’s definitely ‘Oops…’

So to summarise everything above:

• the real value of the big-consultancies is that they provide a means to bridge across ‘the Chasm’ in the innovation-adoption lifecycle, such that the Early Majority can feel safe in taking up some existing innovation
• but the Chasm only exists because of the dysfunctions of the Hype-Cycle, creating the scary-looking Trough of Disillusionment
• and the inflation and crash of expectations in the early part of the Hype-Cycle occur primarily because big-consultancies and their allies – journalists, analysts and so on – manufacture that hype in the first place
• and also a side-effect of the ‘big-consultancy bridge’ is that it actively conceals from the Early Majority (and beyond) the very real work put in by the respective Innovators and Early Adopters
• and also another side-effect of the ‘big-consultancy bridge’ is that its purported ‘innovations’ can be a long way from innovative – in some cases years or even decades behind where the innovation actually started

Not only not-new, but in some cases also just plain wrong. We’ll look at that more in the next post in this brief series. For here, though, let’s stop for a while, and look at some of the implications of what we’ve seen so far.

Practical implications for enterprise-architecture

The above has huge implications both for enterprise-architecture itself, and for the practice of enterprise-architecture within organisations.

For enterprise-architecture itself, we have a very real problem of this type that sits right at the core of our discipline. Most of the EA frameworks started out as real innovations by small groups of people – in other words, the classic Innovators – and then tested out in practice in a wide variety of forms and experiments – the Early Adopters. But once they’d started to touch onto the needs of the Early Majority, the big-consultancies stepped in – with all of the dysfunctions that that implies, such as excessive hype, way behind the actual innovation-curve and innovation-need, and (as we’ll see in the next post) often hopelessly over-simplified or even just plain wrong.

Perhaps the most blatant example of this is TOGAF. It is, as Open Group rightly proclaim, a product of the Open Group’s membership – which, unsurprisingly, now includes a lot of the big-consultancies and consultancy-oriented big IT-vendors. Up until Version 7, TOGAF was actually an annual release, developed by real innovators, often from amongst the small niche-consultancies, and aimed towards a specific business-need, namely better management of big-IT infrastructures.

But by Version 8, it had started to get the attention of the larger corporates – and that’s when the big-consultancies moved in. Version 8.1 ‘TOGAF Enterprise’ was the first version focussed on the big-consultancy / big-corporation relationship, with a new emphasis on bulk-training and certification – and a huge amount of hype. More than a dozen years later, there’s only been one fully new version released – Version 9, in February 2009 – and yes, it showed the deadening influence of the big-consultancies, a huge missed-opportunity, an explicit rejection of much-needed change. Right now, even after the updates of Version 9.1, it’s so mis-structured and so out-of-date that it’s almost unusable even for current IT-oriented architectures, let alone anything with a broader whole-enterprise scope – and yet the TOGAF hype-machine just keeps on rolling on…

What can we do about this? Not much, is the honest answer: those frameworks are often worse-than-useless, but they’re often all we have. Just use them with care, and be aware of the limitations, I guess – and help others understand how to beware of the hype, and just how limited those frameworks really are.

For enterprise-architecture practice, the big-consultancies represent a huge problem for enterprise-architects practising within large-corporates (or anywhere, really), because their hype-fuelled ‘solutions’ are so pervasive and omnipresent, so often badly out-of-date, so often overly-simplistic and limited in context-adaptability, so often excessively Taylorist and IT-centric, and too often just plain wrong. Unfortunately for us, their undoubted skills with the hype-machine mean that at present they’re more likely than we are to win the attention of decision-makers across the organisation – and even more unfortunate for us because we’ll be the ones who’ll get landed with cleaning up the resultant near-inevitable large-scale mess…

What can we do about this? Quite a lot, actually. It’s true that we can’t fight the hype-machine – it’s just too big, too powerful and, like a tick, often too deeply embedded in its host to pull out without causing serious damage all round – to us if not to our organisation. What we can do, though, is study those ‘standard’ frameworks, to learn their limitations and actual applicabilities (if any); and then, as an overlay for that, build metaframeworks and metamethodologies to take any usable ideas from the source frameworks, and make them more self-adaptable to the complex ‘wild-problems‘ in our own organisations’ contexts. That’s pretty much where the leading-edge of theory for enterprise-architecture sits at present.

To deal with the problem that the big-consultancies’ frameworks and suchlike are often years or even decades out-of-date relative to current leading-edge theory and practice, it’s essential to develop situational-awareness and environmental-scanning on new developments in enterprise-architecture, and to find other approaches that might be useful in our business-contexts. In other words, look around and see what’s around – don’t just rely on ‘the usual suspects’.

For enterprise-architecture itself, I think it’s fair to say that there’s a lot here on this weblog – heading towards 1200 articles so far, in fact. A lot of it, though is more oriented towards theory and methodology, rather than down-in-the-weeds detailed practice – and, of course, it’s by no means ‘the fountain of all knowledge on enterprise-architecture’! Some other sources I’d recommend would include:

• Nigel Green on VPEC-T (Values, Policies, Events, Content, Trust) and Simplified Business Event Processing
• Simon Brown on ‘C4’ software-architecture and its links to enterprise-architecture
• Sinan Alhir on antifragility in enterprise-architecture and related fields
• Sally Bean with a broad-ranging Twitterstream on enterprise-architecture and related themes
• Chris Potts on investment-strategy and use of story to describe enterprise-architecture
• Kevin Smith on PEAF and POET and PF2 frameworks
• Doug McDavid on whole-of-enterprise services-architecture
• Ruth Malan and her must-read ‘Trace’ on software-architecture, enterprise-architecture and more
• Jörgen Dahlberg on templates and other tools to guide enterprise-architecture practice
• Patrick Stähler on business-model innovation
• Kim Parker on simple explanations of enterprise-architecture themes
• Ric Philips for being, well, Ric Philips, and for his must-read Twitterstream

It’s also wise to keep track of a wide variety of themes and practitioners that may at first seem distant from enterprise-architecture, but can be very useful indeed – the small but increasingly-global consultancy Anecdote, for example, or sole-practitioner Cynthia Kurtz, both doing important innovation in narrative-knowledge and business-storytelling. Be interested in everything, is perhaps the key watchword here – or, since that ideal is unachievable in practice, at least as broad a view as we can make it, across a wide range of industries, technologies, disciplines and more. Either way, a lot wider than the narrow diet of outdated dregs that are often all the big-consultancies can offer us…

Okay, I’d better stop there – hope it’s useful, and over to you for comment? See you shortly in the second of this pair of posts, anyway.