Toolsets for associative modelling
Back on the toolsets theme, what’s the Why behind all this focus on a new type of toolset? Why won’t the existing toolsets do the job?
In practice, the core concern is as per the previous post ‘Toolsets, pinball and un-dotting the joins‘. What we need to work with, and on, is the development process as summarised so well by the Squiggle: yet whilst the existing tools are each often great at tackling one part of the puzzle, they give us no means to connect across the whole of the puzzle, the whole length of the Squiggle. Instead, they fragment the Squiggle into little fiefdoms – a whiteboard-sketch here, a spreadsheet there, and now a little process-model too – ‘dotting the joins’ into a mess of disjointed domains, across which we somehow have to rebuild some kind of continuity, sense, structure, story:
“So why is that a problem?”, quite a few people ask, “Isn’t that how we do our development-work already?” The short answer is yes, it is how we do it already – and it really doesn’t work well at all. It’s just that we’re so inured to working in a really bad, clunky way that we probably don’t even realise that there could be a better way to do it. Finding a way to do it in a better way is really what I’ve been on about in this series of posts on toolsets and metatools and the like.
Before we go on to a worked-example, let’s just do a quick refresh on the practical concerns here.
First, in any context – any context, at any scope or scale – ultimately everything depends on everything else. Which means that a single model-type for a single domain (such as a data-schema or a BPMN process-model), or even a cluster of linked model-types for a cluster of related domains (such as UML or Archimate), is unlikely to be enough for what we actually need: in fact we’re going to need model-types that could describe anything, in any way we need. Which, at first glance, probably doesn’t seem to be possible – which is why we put up with the current mess…
Let’s take this another step further. If everything depends on everything else, then everything is associated with everything else. Which means we’ll need to keep track of all those associations – at least to a level of Just Enough Detail, anyway. Which, again, most of our existing tools won’t allow us to do: they’ll each keep track of a subset of those associations, according to their own formal rules, but won’t allow us to connect beyond that point. Which, yet again, means that we’re left to do the rest of the linking-up on our own, without any toolset help at all – or, all too often, quite a lot of toolset hindrance instead. Yeah, it’s a mess…
The way those tools work is that they each declare a bunch of object-types that they ‘own’, with associations between them as a kind of afterthought. And yes, that does seem to be the ‘logical’ way to do it, if we centre everything around that model-type’s view of the world – which, yes, is actually what we do want to do as we move into solution-architectures and the like in the later stages of the Squiggle. Yet that ‘if‘ is really important – so important, and yet so often glossed-over, that most people don’t seem to realise that centring around a single model-type is exactly the wrong way to do it for the earlier parts of the Squiggle. If we think a bit more carefully about what that phrase “everything depends on everything else” really tells us, it’s that there is no ‘The Centre’ to the architecture: instead, everything and everywhere is ‘The Centre’, all at the same time. Which means that we need a fundamentally different approach for our toolsets, if they’re going help us link everything together across the whole of the Squiggle.
To make sense of this, we need to look more closely at how we work, at how we think as we work. What I’ve been describing in these posts – and as I’ll again summarise somewhat later in this post – is that to get this to work properly, we need to turn the model-thinking almost completely the other way round to how it’s usually done at present. But to illustrate why that’s so, let’s play ‘pinball-wizard‘ for a while, and look at the real enterprise-architecture of a small business near where I live – a café that I often frequent in one of the more upmarket streets in the old town. It’s recently changed ownership and undergone a makeover, and it’d be kinda interesting to look at the choices that the new owners have made in rethinking – and redoing – the architecture of this enterprise.
It’s true that in mainstream ‘enterprise’-architecture we’d probably look only at the information-systems for the business, whilst the process-modellers would look at only at their slice of the story, the compliance-guys at their own slice, and so on. But that’s yet another example of ‘dotting the joins’, isn’t it? Instead, for business-architecture, and even more so for a true ‘architecture of the enterprise’, we’d need to start much further back along the Squiggle – start right from scratch, in fact, and build outward from there.
Yet where do we start? Well, a nice corollary of “everything depends on everything else” is that we can start anywhere – anywhere that works, that gives us a place to start. So, sitting in the café, let’s start right where we are – look straight out in front of us, to the opposite wall:
Look at the wall. Look at the mood that that wall suggests; the clientele; the usage. Under the previous owner, the place was also a café/bar, but with a much stronger emphasis on ‘bar’ rather than ‘café’. Here, the sealed wooden floor, the light-grey tones, the lighter ceiling, and distribution of light in general, all help to shift the emphasis more to day-time work-oriented café rather than evening-oriented social bar. It’s an intended-usage that’s also emphasised by the pair of double power-sockets, for computer-users, on that wall alone – one socket clearly visible to the right, the other almost hidden by the leftmost chair.
(Notice how the associations jump from one theme to another to another – mood to clientele to usage to power-supply to lighting to colour to whatever. That’s exactly what happens – what needs to happen – throughout the earlier stages of the Squiggle. The one thing we don’t want to do at this stage is to put up arbitrary boundaries and partitions between architectural domains – otherwise the ability to associate ‘everything with everything else’ will break down.)
How else do we set the mood, the intent? Music is one part of it – note the loudspeaker mounted at upper-left. Furniture is another: different table-types for different groupings of clientele – solo, twos, threes, fours, more – yet continuity provided by the same type of chairs throughout. And there’s the wall-decoration, the cluster of ‘found-objects literally spelling-out the intended usage as ‘café’. Turns out that that decoration wasn’t assembled by the owners themselves, but bought direct from a company that specialises in providing prepackaged sets of exactly this type of material in accordance with a specified mood.
(Again notice the associative-modelling process here, and also the jumps back and forth from big-picture intent to the detailed practicalities of real-world implementation. Again, anything can usefully connect with anything else: and in the earlier stages of the Squiggle, we don’t – in fact often can’t – know beforehand what those useful-associations might be.)
Another place we could start would be as with Business Model Canvas and the like, though again also bouncing back-and-forth between big-picture and finer-detail. For example, we could start with the ‘front-end‘ of the business-model: if we’re offering a service of some kind, how would people connect with that service? What do our potential customers want to do? For a café, some of the intentions that come to mind would be:
- take-out food and drink
- quick sit-down eat and drink
- solo slow-time – coffee-break, lunch-break, sit and think
- business semi-formal – ‘doing deals’
- students talking loudly, tossing around ideas
- young-mums chatting together
- lounge around with friends
- sit and relax with a cigarette
If the café wants to support all of those client-stories (which they do, in fact – pretty much all apart from the ‘noisy students’ client-group, who are well served already by the franchise fast-food places just down the road), then that indicates a definite need for zoning – splitting the space into distinct zones.
(In Business Model Canvas terms, such zones will, in effect, also form distinct channels via which the respective value-proposition for each customer-group is served.)
And that kind of zoning is exactly what we can see here in this café: each zone demarcated by distinct boundaries of doorways, constrictions, changes of level – and different decor and lighting, too. From the ‘quiet-zone’ area where I am right now in the café, there’s the more ‘group-sitting-together’ space further towards the back of the café – often popular with the young-mum professionals talking babies and business, but here with just one woman working on her own – and then the lounge-area right at the back, with sofas and side-tables:
Looking towards the front of the café, there’s the bar-area for take-out orders; the quick-eat area with its small high-tables and high-chairs; the two bays for sitting solo in the window; and then the smokers’ steel tables out in the street:
Yet a business-model is not just the customer-facing ‘front-end’, but also the ‘back-end‘ that does the actual work – including much that’s intentionally hidden away from customer-view.
A value-proposition is a promise to deliver value – so we need to be able to deliver on that promise. Hence, for example, the all-important coffee-machine at this end of the bar; all the bottles behind the bar itself; the sandwiches and suchlike on display in the small serving-area at the front; and, for hot food, the kitchen-area upstairs. For the latter, a key piece of equipment is the ‘dumb-waiter’ food lift that connects between the kitchen and the bar – otherwise a real risk of spilled food and sprained ankles from traipsing up and down those stairs.
(Notice how the vertical separation of kitchen and serving-area makes that literal connection between them necessary – an association of ideas that arises because the direct association in another sense does not exist.)
To close the business-model loop, there’s the computer and cash-register – just about visible on the bar here, to the right of the balloon that’s advertising the current Valentines-Day special.
(We might note that that computer and cash-register would often be the only point within the whole café for which the classic concerns of mainstream ‘enterprise’-architecture will come into this overall picture – which itself should give some indication of just how limited and parochial that so-called ‘enterprise’-architecture really is…)
To make more sense of the business-model, we’d need also to look at other themes such as the workflow, the information-flow, the materials-flow – like the two delivery-guys coming in through the door right, one with a trolley-load of bottles that need to be stored somewhere, the other with the supplier’s bill that needs to be accounted for, in every sense of that term. We’d need to look at the customer-journey, the customer-story, the customer-experience: as Chris Potts put it, “customers do not appear in our processes, we appear in their experiences”. We’d need to explore how the café itself connects with the broader story of the street, of the town, of the region; how it connects with the overall shared-enterprise of ‘eat and drink away from home’. We’d need to address the often-thorny issues of licensing-laws, city-ordinances, regulations and compliance on health-and-safety, food-handling and all the rest. And so on and so on, more and more and more – all of which matters to the working of the whole, whether we like it or not.
And that’s the whole point here: at architectural level, everything really does depend on everything else, and we really do need always to see it as whole-as-whole, even – or perhaps especially – whenever we need to do a deep-dive down into the fine-detail.
Which means that the toolsets we use must be able to maintain connection across all of that space. Or, to put it the other way round, any tool or toolset that does not support continuity across the whole space will inherently cause architectural-fragmentation and increase overall architectural-risk: in other words, Not A Good Idea…
At present, just about every toolset assumes that it alone is or describes The Centre of the architecture. Yet there is no ‘The Centre’ to an architecture: everywhere and nowhere is ‘The Centre’, all at the same time.
Everything in an architecture depends on everything else in that architecture.
Everything associates with everything else in some way.
In a sense, we don’t need new tools as such: the tools we have are variously good, even excellent, at describing the ‘things’ within each architectural domain – the ‘thingness’ of those things.
What they’re not good at, though, is in describing and connecting between domains – the ‘betweenness’ of the associations between things, domains, levels, layers, implementations, whatever.
That’s why we need tools and toolsets that can support associative-modelling across the whole-as-whole of an architecture – that allow us to start anywhere, and model anything, anywhere, up, down, sideways, round, in any direction that we need.
So we don’t need ‘new tools’ as such: instead, we need the tools to connect with each other, ‘un-dotting the joins’ created by all those existing single-focus tools.
That’s what all this discussion of ‘new toolsets for enterprise-architecture’ is really about.
Brilliant! This is really great stuff Tom.
Many thanks, Stuart – much appreciated!
(More to come on this over the next few days.)
Hi Tom,
Your statement
inspired me to think differntly about my own attempts to automate (parts of) my workflow.
From now on I will focus on creating little input-output scripts which I can use to transport information through my (depending on the situation changing) tool-chain as easy, flexible and automated as possible.
So thanks for putting me on a better track!
Yes! Exactly the aim, exactly the point! Thank you! 🙂
(Thanks also for the reminder about the term ‘tool-chain’ – it’s the linkages between the tools, as a chain of associations and inferences, that make the tools useful. Will follow up on that later.)
Actually, thinking about it again, the real point is to not have to write a myriad of “little input-output scripts”, but instead to have a single common export-input format so that you don’t have to write those scripts…
I know that the real point is not to have a myriad of “little input-output scripts” 🙂
In my little very structured IT infrastructure world where I do my daily work I don’t need a myriad of “little input-output scripts”. I think <10 scripts will be more then sufficient, and they will give me the advantage that I can tweaks things as needed.
What I'm doing is making my personal work-life easier and enable myself (BTW is that good English?) to deliver more consistent and reliable output to my stakeholders. Your quest is much more ambitious and I follow that with interest and I learn a lot from it 🙂
I'm sometimes just commenting to let you know that your work and sharing of ideas is very much appreciated!
Thinking this through I end up at vocabularies, taxonomies and ontologies. This can be used as metadata for the transformations. Something like lov.okfn.org could be useful?
Sounds big I know and we’re looking for small steps that can be realised with existing tools. Maybe and inventory of tools used can help us to where easy results are possible.
I use:
Mindjet for ideas
Visio BPMN (iServer template) for processmodelling
Archi for archimate modelling
Visio and PowerPoint for business functions and/or responsibility mapping
Excel for relationship matrices
So transformations from mindjet to visio to archie to PowerPoint and vice versa vai excel would be usefull.
Mark – yes, that’s the right sort of direction (and thanks also to the Vocabularies site, I hadn’t come across it before, and should have 🙂 ).
And yes, you have a real tools-cluster there – Mindjet plus Visio plus Archi plus Powerpoint plus Excel – with real workflows that crosslink between them. If you’re like most of us, those workflows will pass through those various tools in all manner of different sequences – hence roundtripping will be important both in direct forms, and any number of indirect forms.
What we’re talking about here, though, is a exchange-format that can be used for round-tripping between any tools. Hence rather than thinking about exchanges between specific pairs of tools, or specific clusters of tools, but any combination not just of tools (as per eg. Mindjet) or tool-platforms (as per the Microsoft suite, or Archi as it breaks further out the Archimate-only space) but any analytics, any form of modelling, and so on. Hence we need to step back at least one level of abstraction from those individual tools, and explore instead what could be used to link all of them together with any workflow. That’s the real challenge here – because anything less than that would be another fragmentation of the workspace, another ‘dotting-the-joins’.
Once we do have a means for interchange across the whole of that space, then connecting individual tools into that shared-space becomes a relatively-trivial problem. But getting our thinking to move out of the single-solution habit and into a more deliberate and explicit habit of thinking always of whole-as-whole, is what’s most needed here.
Fascinating! Back in the day, the methodologists in my firm came up with an analogy for the problem-solving process that may have served in the same way as your Squiggle. “We arrive in the (client’s) hall and it’s just full of noise… we listen for voices, instruments, beats, melodies and discords to emerge… then we add our own and eventually we make music together. Or not!” References to this mental map lasted for years and generated a lot of laughs.
Thinking about your notion of associative tools, I was reminded of this TED talk: https://youtu.be/4c1lqFXHvqI. Apparently, each of us already has at our disposal the most powerful general purpose associative modeling tool around. It’s a fanciful notion, but if you take in the speaker’s idea of sensory extension, perhaps there is a way to avoid translating, transforming or integrating the outputs from many tools and simply consume them all at once, the idea being the more outputs and the more complex they are, the better to extract deeper and more powerful insights and patterns!
Thanks, Skip – and yes, I like that analogy of “we arrive in the (client’s) hall and it’s just full of noise”. Reminds me of the work of my colleague Michael Smith Castillo in Mexico – see http://orquestando.com
Further – hopefully more material – thoughts on associative modeling tools. I wonder: would you put domain-specific modeling (DSM) tools in that category? I’m thinking of MetaEdit+ from MetaCase but there are others. The point is that one actually becomes free to combine concepts, relationships, logic and operations from as many domains, and create as many analytic perspectives, as desired. Designing a new modeling tool with these ‘metatools’ is comparable to breadboarding digital circuits. There are even impedance-type mismatches to be overcome such as mixing passive (e.g. data) with active (e.g. object) concepts, needing the semantic equivalents of resistors and capacitors.
One other worth mentioning is IBM’s Rational System Architect, which is not billed as a DSM but is certainly a metatool. Out of the box it comes complete with several dozen metamodels from different methodologies (OMG’s Business Motivation Model, Open Group’s TOGAF, Zachman’s Framework, etc., etc.) already lashed together, with the expectation that customers will generally add their own relevant concepts and analytic algorithms.
I don’t know that I’d put DSM tools in the ‘associative-modelling’ category as such – more that associative-modelling is something that we might do within a domain, or (more usefully?) between domains.
The ‘meta-‘ concern is closer, of course, though I don’t know MetaEdit+. Very good point about “Designing a new modeling tool with these ‘metatools’ is comparable to breadboarding digital circuits”, and “There are even impedance-type mismatches to be overcome”.
Re Rational System Architect, we clearly don’t share the same experience. All I can say is that I hope it’s much improved than where it was when I last had the misfortune to have to use it intensively some not-quite-decade ago. Back then (in my experience, anyway), it was not only the most expensive tool on the market, but far and away the least-suited for enterprise-architecture: ludicrously-fragile (case-sensitive, no means to merge), inanely-constrained (no means to hold more than one time-horizon in any model, and no way to link between models), atrocious graphics (getting it to draw a simple connection between entities without it randomly changing everything else as it did so was a constant challenge), actively user-hostile (so much so that there was only one person in the entire 30,000-person company who ever managed to get to grips with it – and she’s just retired, leaving the company with no usable EA-toolset at all), and with the worst and most laughably-incompetent ‘metamodelling-tools’ that I’ve ever seen in an nominal EA tool (a plain-text file with no instructions and no debug). As I say, I sincerely hope that it’s improved a lot since then – because it desperately needed it. Oh well…
I used both Rational System Architect and MetaEdit in 2012 in a proof-of-concept project to design a modeling tool to merge goals and strategies defined by OMG’s Business Motivation Model, with public policy outputs and outcomes defined in a construct called a Program Logic Model used by governments. But I doubt you’d have had any reason to think System Architect was improved based on my experience with it.
However I don’t hold any special brief here for System Architect (or MetaEdit or any of the others for that matter) past the fact that they claim the label “DSM Tool” or domain-specific modeling tool. What strikes me is that in your “associative modeling” frame of reference, that label is misleading — it would be more accurate to call them “domainless” modeling tools (although that sounds like “brainless”, which they also are in a way).
That’s because their relevant and distinguishing property is that out of the box, there are no assumptions about the boundaries and structure of any conceptual space or domain built into them. I begin using one of these tools by creating a metamodel of my purpose-defined domain of interest with it. That declares the relationships and behavior of all the relevant elements in that domain. Then I use the metamodel to animate the tool to model occurrences of my domain, turning it into a ‘domain-specific’ modeling tool temporarily.
So it seems to me that tools of this sort could answer your fundamental challenge: they put within reach models that seamlessly join everything of interest within, across and between multiple conceptual spaces — for example research, concept, prototype and design — by defining one unified domain that encompasses everything of interest in those spaces.
Of course usable implementations of this approach are another matter entirely.
Yes, I’m familiar with Program Logic Models – we used them in a couple of Australian government contexts that I worked on. And yes, building crosslinks between Program Logic Model content and Business Motivation Model content is exactly the kind of ‘un-dotting the joins’ that I so much see as a need from our toolsets.
For the rest, mostly strong agree, though one important proviso. Yes, those (nominally)-adaptable-metamodel tools such as Rational System Architect or Troux or Abacus Avolution (the latter being the best of that bunch, from what I hear, though I haven’t used it myself) all do the modelling-with-predefined-entities bit variously-well. But they all have their own proprietary data-format, they don’t play nicely with other toolsets, and they don’t connect well or even at all with types of information that are not ‘modelling-with-predefined-entities’ – such as audio, video, photographs, freeform sketches and the like, all of which are very much part of the overall ‘the Squiggle’ of architecture-development. And we need to connect across the whole of the Squiggle – not just arbitrarily-selected subsets of it – otherwise that much-needed continuity becomes fragmented and lost. The practical problem is that, because the mainstream ‘EA’-toolsets are so bad at doing this, it’s sometimes hard to see that most of the other toolsets aren’t much good at it either. We need to do better…
As you say, we need to start from “one unified domain that encompasses everything of interest in those spaces”. Yet when we look at the full logic of ‘the architecture of the enterprise’, that domain is and must be always ‘the everything’ – in other words, several steps further out, or broader, than most concepts of ‘the enterprise as domain’. By definition, the Squiggle connects everything; but on top of that, there’s a kind of meta-Squiggle that connects all possible instances of the Squiggle – and it’s that that we need to be able to model and link across in any way that we need.
In some ways OMG’s MOF (Meta-Object Facility – the metametamodel behind UML, BPMN et al) gets quite a long way towards this – the main catch being that it models ‘Relation’ as a dependent second-class citizen, which causes breakdown in a lot of deeper places. The key here, though, is that when we create a metamodel for domain-specific modelling, we need to retain the ‘everything-is-an-equal-citizen’ connection right down to the core – rather than kind of throw it away, as happens in most domain-specific modelling at present. We must retain that connection across the whole of that “one unified [meta]-domain that encompasses everything of interest in those spaces” – which, given that anything could be “of interest” and relevant to an enterprise-architecture, by definition must be able to encompass ‘the everything’ in its broadest possible sense.
As you also say, “Of course usable implementations of this approach are another matter entirely.” Yep, I’d kinda noticed that one already… 😐 🙂 But that’s the whole point here, isn’t it? 🙂 – working towards something that we can use for this.
Thanks again for all your comments / help on this, anyway.