The toolset-ecosystem

This one extends the models-for-enterprise-architecture theme from the previous post. Although for me this theme goes back a long way, the start-point here was a Tweet from Dutch EA consultant Martin van den Berg (@bergmart) that triggered off a veritable flurry of replies:

• bergmart: I’m more and more convinced that it should be forbidden to show ArchiMate diagrams to business management #entarch #bizarch
• EAatWork: @bergmart why should it be forbidden to show archimate diagrams to business management?
• blomr: @bergmart @eaatwork Depends on level/type of business management + depends on complexity of the diagram(amount of (different) objects&rel’s)
• ArchiTool: @bergmart I’m intrigued to know why. Are they infrastructure models?
• bergmart: @ArchiTool I’m not talking about infrastructure models but business / information models
• ArchiTool: @bergmart @EAatWork Then maybe break them down into smaller focussed viewpoints?
• bergmart: I think we need to find other ways. Business Model Canvas is a much friendlier way or use the operating model stuff of Ross
• ArchiTool: @bergmart Right. Higher levels. And ArchiMate could be used in addition for more detail if needed? // Hmm….Business Model Canvas in Archi? Development of the Sketch View? #archimate #bmc
• bergmart: @ArchiTool Yes, in the architecture & engineering communities
• ArchiTool: @tetradian Synchronicity is nudging me toward possible Bus Model Canvas & Ent Canvas implementation in Archi. Reading up on it now…
• tetradian: @ArchiTool aiming to have 2 new posts for you soon: models as anchors for discussion/decision-records; roles of tools in toolset-spectrum
• tetradian: @bergmart ‘don’t show Archimate diags to biz mgmt’ – in my experience showing them BPMN diagrams is an even worse idea… 🙁
• bergmart: @tetradian Can imagine!
• EAatWork: @bergmart too complex from a modelling language point of view (archimate) or too complex because the diagrams are too complex?
• bergmart: @EAatWork Both
• EAatWork: @bergmart  I think that the concepts are ok but the  different relation types are causing the confusion (look all similar) for bizz mgmt
• tetradian: @bergmart @EAatWork with BPMN diags, prob was need to translate from abstract to concrete – so overlay Archimate rigour w visual images?
• ArchiTool: @bergmart But with Business Model Canvas approach maybe managers are concerned with $$$$$$ rather than architecture and internal workings?
• ceri: @tetradian @bergmart To quote @wilm on Tuesday “never ever show the entire model to ANYONE, only the view that is relevant to them” #jiscfsd
• bergmart: @EAatWork Yes, the arrows are the main problem. The concepts are ok.
• bergmart: @ArchiTool Nice  with Business Model Canvas is the combination of $$$$ with coherency.

As mentioned in that back-and-forth above, I’ve had painful first-hand experience of what happens when we show ‘formal’-type EA diagrams to executives, as described in this quoted from my book Real Enterprise-Architecture:

In our first attempt to show the true value of [a proposed] logistics early-warning system, we made the mistake of showing the process-flows [to the executive] in the form of a Business Process Modelling Notation diagram. BPMN is great for the formal modelling rigour needed by software engineers, but not for a board-level presentation: the blank stares and silence from round the table sent us away in shame.

For the next meeting, we redrew the diagram, with the exact same process-flows, but replacing the bland BPMN boxes with clip-art pictures of trucks, conveyor-belts, fork-lifts, sorting-machines, delivery staff. This time it clearly made sense: we gained our go-ahead.

These senior people weren’t ‘stupid’: when we explained it in their own terms, they understood straight away what we were aiming to do. The problem before was that they didn’t have time to learn an abstract language and translate it back into their concrete world. As architects, we did need that level of abstraction: but it was also our responsibility to each audience to do the translation from the abstract into the everyday.

This is going to be a long one… sorry…

In effect, what we need from our toolsets for enterprise-architecture and the like is strong coverage of several different spectra of requirements:

• from formal rigour to freeform, for capture, for display and (in some cases) for execution
• capture, edit, moderate, maintain, analyse, explore, search, cross-link, describe, display
• architecture- and project-governance, requirements tracking, version-control, architecture-dynamics (change over time)
• from binary-logic (true/false, linked/not-linked) to modal-logics (must, may, can, could, conditional/contextual, ‘known-to-work’ etc)
• broad range of model-types, including industry-standard models and metamodels and user-defined types (metamodel/metametamodel edit etc)
• single-user to meeting to team to group to business-unit to whole-organisation and beyond, with security and access-controls to match
• enterprise-wide repository to desktop to laptop to tablet to handheld, with different user-interfaces and user-experiences for each
• any and every possible stakeholder-group, in every discipline in the enterprise, and at every level from front-line operations to executive and external stakeholders
• support for full range of the order/unorder spectrum, and for the full range of the development lifecycle from ‘messy’ uncertainty to executable models
• full support for lifecycles ranging from milliseconds to decades or centuries

No existing toolset comes anywhere even remotely close to covering that full range of requirements, and I suspect it’s probable that none will ever do so: the scope is just too large to handle in one go – especially as that ‘one toolset’ would need to run within several fundamentally-different user-interface paradigms. Which leads us to another crucial requirement:

• able to share and ’round-trip’ information and models between toolsets

What we actually have at present we could perhaps summarise as follows:

• ‘enterprise’-scale toolsets [examples: IBM/Rational System Architect, Troux Metis, ARIS, MEGA]: typically based on a large repository, preferably though not always with strong version-control; usually based on a software-architecture/software-development metaphor, often capable of presenting ‘executable’ models (database, process etc); usually strong on rigour but with poor or non-existent support for the freeform experimental stages of architecture-development; broad range of model-types; some support for metamodel-edit, though usually proprietary/’own-standard’; limited to no support for governance, project-tracking and/or requirements-tracking; limited to poor support for architecture-dynamics; limited to no support for modal-logics; strong support for one-way publishing but weak to non-existent support for ‘social-media’ style multi-way collaboration
• ‘team’ toolsets [examples: Orbus, Sparx Enterprise Architect, Alfabet, Abacus Avolution, BizzDesign Architect, Essential]: often either a scaled-down ‘enterprise’-style toolset (using a simpler underlying repository) or scaled-up ‘desktop’-style toolset (expansion of repository, or addition of multi-user features to an otherwise single-use system); usually strong on rigour, poor on freeform; more limited range of model-types (sometimes only one); usually some metamodelling, with very strong support in some toolsets (e.g. Avolution, Essential); limited to no support for governance etc, for architecture-dynamics, or for modal-logics; variable publishing capabilities (sometimes only to desktop software), almost always one-way only
• ‘solo’ toolsets [examples: Archi (Archimate), Visual Paradigm, single-user versions of Sparx or BizzDesign]: basic single-user repository; usually only one or two model-types or sets (e.g. UML, Archimate); in some cases little more than repurposed or re-badged software-design tools; often very limited publishing facilities; otherwise similar to ‘team’-type toolsets
• non-repository toolset (office software) [examples: MS Powerpoint, MS Visio, MS Excel, Apple Keynote, Google Apps, Prezi]: still the most commonly-used ‘toolset’ applications for enterprise-architecture, yet in many ways the least appropriate toolset-type; usually single-user, sometimes sharable; no repository, hence no entity-linkage between diagrams (hence high probability of duplication, mis-versioning etc); emphasis is on diagrams rather than semantics; functionality is split between different applications (e.g. diagramming on Visio, governance or cataloguing on Excel); no metamodelling as such; awkward mix of pseudo-rigour (i.e. rigour in appearance only) yet only limited support for freeform (constructed slowly from uncontrolled image-objects)
• manual drawing [examples: whiteboard, flipchart-pad, sketchbook, ‘drawing’ application on tablet-computer or smartphone]: by far the most commonly used tools, though rarely acknowledged as such; allows true freeform exploration; no repository, no verification of standards-conformance, usually no linkage to any other model, usually no publishing as such, often no permanent record
• media record [examples: video recording, audio recording, photos]: record of live discussion-session, seminar or workshop; often paralleled with yet not actually linked to manual-drawing; captures decision-making process and resultant decisions in raw form (i.e. ‘raw data’ of the architecture); provides key component of ‘models as decision-records‘

Which, collectively, still only covers part of the overall requirements, in a very fragmented, disjointed way. Not good.

And, to anyone who’s done this kind of  work in practice, one of the more obvious yet painful points is that at present very little of this content can be transferred from one toolset to another… Some toolsets will allow import of some subsets of this information, but successful ’round-tripping’ between toolsets is almost unheard-of. Oh well.

Then there’s the different user-interface/user-experience types:

• repository, text-oriented [examples: Essential, DOORS, metamodel-editors within toolsets]: emphasis is on development of metamodels, ontologies and other rigorous metadata-structures, and capture of models, requirements etc via text-based forms
  • advantages: precision, formal rigour, often automatic verification and completeness/correctness-checking
  • disadvantages: often too much emphasis on ‘correctness’ and not enough on real-world modal-logics, also often accurately described as ‘user-hostile’…
• repository, diagram-oriented [examples: most ‘enterprise-architecture’ toolsets]: emphasis is on development of diagram-based models using entities captured and maintained within the repository; diagrams constructed by linking repository-entities, and relationships between those entities, in the terms specified by some formal notation (Archimate, UML, BPMN, Zachman etc)
  • advantages: as for text-oriented repository, though also often seemingly much easier to use and ‘read’
  • disadvantages: often no support for real-world non-binary logic (i.e. modal logics), leading to illusory ‘certainty’; modelling also often driven by assumptions from model-type (pictograms, visual notation etc) rather than by underlying entity (which in principle should be independent of its representation); still much harder to ‘drive’ than a simple diagramming tool, yet easily mistaken for the latter, leading to an unusable repository riddled with incompletions and thesaurus-errors (duplications, synonyms, heteronyms etc)
• ‘precision’ diagramming/text [examples: MS Visio, MS Powerpoint, Dia, Sketch, MS Excel]: emphasis is on visual and/or textual representation rather than underlying semantics; diagrams constructed from a ‘palette’ of visual objects, usually with no inherent underlying semantics; user-interface is ‘mouse’-type, with emphasis on precision placement; text is structured (e.g. spreadsheet) yet ‘standalone’, without linkage to controlled metadata-driven repository
  • advantages: looks good, relatively easy to ‘drive’, easy to ‘read’, allows for purported modal-logics and for any required exceptions; presentation may be in any required form; often the simplest way (or even the only viable way) to present technical information to a non-technical audience
  • disadvantages: no inherent underlying rigour, frequently delivers spurious sense of certainty; often (and easily) mistaken for proper repository-based diagramming etc
• freeform drawing/text [examples: bitmap and vector-graphics editors, text-editors, whiteboard, pen-and-paper]: emphasis is on speed of capture, without regard to rigour; user-interface is usually ‘touch’ type (stylus or fingers)
  • advantages: real-time or near-real-time; requires little to no prior technical knowledge to operate the tools
  • disadvantages: easy to do at a basic level (with mediocre communication at best), but requires real skill and experience to capture essence and meaning, especially at high speed; no inherent structure – rigour, metamodel etc depend entirely on knowledge and skill of user
• voice/gesture [examples: voice-recorders, voice/text recorders, video, Wii, Kinect]: emphasis is on capture of unstructured and often tacit cues and content in real-time
  • advantages: user-interface ‘disappears’, or is entirely automatic after setup; minimal knowledge required to operate
  • disadvantages: as for ‘freeform drawing/text’; content often requires separate transcription to enable re-use
• consumption-only [examples: web-browser, report-browser, smartphone app, text-to-voice]: emphasis is on search and reporting from existing information; variety of user-interfaces/user-metaphor types, but user-experience is similar in each case; interaction usually limited to query-creation and/or metadata-append (tagging etc)
  • advantages: can be configured to fit into almost any format – screen, e-book, paper, smartphone, media-player
  • disadvantages: no content-capture, and often no means to provide feedback or review

When we put all of this together, what we need is not so much ‘a toolset’ as a toolset ecosystem in which all of the parts and roles and user-experiences can mesh together as a unified whole, that allows and supports both structured and unstructured information, that can span the full truth/value spectrum from true/false logic to if-but-perhaps-maybe, and in which the best use is made of the strengths and limitations of each platform.

Is that such a big ask? I don’t think so: all that’s really standing in the way is a lack of imagination in how to tackle inherent uncertainty and non-binary forms of ‘truth’, and the determination to push through the shambles of proprietary pseudo-‘standards’ that stand in the way of proper information-interchange.

For example, consider for enterprise-architecture the following as one option for a bridge that could cut across just about all of the toolset-categories above:

• consistent underlying standards-based metametamodel (equivalent to MOF) or metametametamodel (equivalent to ORM) that supports modal-logics (models, model-types and metamodels built on this are exchangeable with any system that uses the same underlying core)
• lifecycle-managed repository for all entities
• full support for ‘unidentified’ entities that do not belong (or belong only partially) to any metamodel or model-type)
• explicit separation between entity and its representation (also allow any number of representations for any entity in addition to defined defaults)
• full support for Zachman-style concept of ‘primitives’ and ‘composites’ (‘primitive’ as associated with single cell in a given taxonomy-matrix, ‘composite’ may bridge multiple cells and/or aggregate multiple primitives, etc)
• full support for modal-logics (such as the MoSCoW set)  in reference-frameworks etc
• Prezi-style scaling for model-diagram development, to permit literal ‘drill-down’ into models
• capture, embed and linking of unstructured media, especially via tablets, handhelds and similar devices: freeform drawing (e.g. as per Adobe Ideas), video, photo, audio, synchronised audio/notetaking/drawing (e.g. as per AudioNote)
• consistent user-experience across all interface-metaphors (mouse, stylus, touch, gesture)

And to come back to the point where we first started: about not showing Archimate models to executives. What we need instead to show them is something is freeform, concrete, visually descriptive in everyday business terms – but that underneath that surface view actually is an Archimate model (or BPMN model, or UML model, or whatever), with full version-controlled, verified formal rigour. Far too many existing toolsets rigidly link their graphic modelling to the notation of a specific model-type: what we should be doing instead is sticking strictly to the principle of separation of concerns – that the representation of an entity is merely a transitory, optional attribute associated with that entity. And that’s doable if we build the metamodel in such a way as as to embody that separation of concerns – and not entrap each entity into a single predefined form.

All of that list above is doable with what exists right now: all that’s missing is the interchange-standards to make it work, and the willingness of enough people to sit down and do it. (My own coding skills are way out of date, unfortunately, but I’m happy to help in any way I can.)

I’ve written quite a lot on this over the past few years – for example:

• Meanderings on metamodels
• ‘Bindedness’ in metamodels
• More metamodel stuff (pointer to metamodel descriptions on ea.openfutures.org/OsEATools )
• A call for collaboration on enterprise-architecture
• Big EA, Little EA and Personal EA
• Executable enterprise-architecture
• Time for open-source enterprise-architecture?
• On EA tools and paper trials
• Next-generation toolsets for enterprise-architecture?
• Enabling enterprise-architecture conversations

So how about it, folks? Is anyone else actually interested in this?