Back to the roots for EA toolset metamodels

Time to get back to the themes from the post ‘More on that enterprise-architecture ‘help wanted’‘, with a focus on toolsets and metamodels.

The usual approach to toolsets – just about any kind of toolset, as far as I can tell – is to describe the overall context, knock up a metamodel, and then build a toolset that works with that metamodel.

That’s fine as long as we don’t need to use the content for anything else, in any other way, or (in all too many cases) in any other toolset. If we do need to do anything like that – and frankly, most of us do – well, then, we have a problem…

What we actually need is a toolset that can do any kind of modelling and simulation that we could possibly want. Given the nature of Reality Department, we’re not likely to get it… 🙁 Oh well. But if we can’t have that One Toolset To Rule Them All, then perhaps a good second-best is a metamodel (or metametamodel, or whatever layer we need to go to) that underpins a file-format that can move anything that we need to share across all the disparate toolsets. And that, I believe, is doable. Hence this series of posts.

Let’s start right at the roots.

(And please watch for anything that I’ve missed, or that I seem to have got wrong – because that’s the way we’ll get this to work right, for everyone.)

Let’s start with the idea of a ‘thing’. This ‘thing’ could be or represent anything at all: an object, a piece of information, a perceived connection, an idea, a question, whatever. It’s just, well, something. Anything.

Given a collection of ‘things’, we then might want to describe perceived relationships between various of those ‘things’.

And we then might want to model those ‘things’ and relationships between ‘things’ in a structured or semi-structured way, to aid in sense-making and decision-making. (We then might also want to describe explicit viewpoints and views that determine the scope and role of models, as per IEEE1471 / ISO42010.)

This gives us the core of what we need to support here: entities (‘things’), relations, and model-types.

(I think that part is straightforward, but if not, please say so?)

Way back when, whilst we were designing an information-system for an aircraft tear-down, my colleague Graeme Burnett said that for anything of interest, anything in scope, we need to be able to ask of each ‘thing’:

  • Tell me about yourself, and about what happens to you, with you, by you?
  • Tell me what you’re associated with, and why?

Which in a sense leads us to the usual metamodel set, but with a few extra twists:

  • entities and their attributes
  • relationships and their attributes
  • lifecycles and other types of change of entities
  • flows and other types of change between entities
  • questions about entities, relationships, attributes, flows, lifecycles and other changes

(What else needs to be added to this list? What have I missed so far?)

We need these items to be able to be portrayed in any representation (in both the visual sense and technical sense), including plain-text. This means that any representation needs to be separate from any entity or relationship or other item, yet needs to be associatable with it. In other words, we can’t link a metamodel rigidly with a notation, or vice versa, as with UML or BPMN or Archimate, for example – we must be able to re-use the same entities and relations and so on in multiple notations.

Some examples of how we might want to re-use the same entities might include:

  • enterprise-architecture notation such as Archimate in Archi, or other notations such as UML, BPMN and so on
  • enterprise-architecture metamodels such as in Troux Semantics
  • systems models such as in Southbeach
  • narrative questions such as in Southbeach MyCreativity
  • context-maps such as in CMapTools
  • social-networks for shared-concepts such as in Cohere
  • personal sensemaking such as in Compendium
  • support ‘barely repeatable processes’ such as in Thingamy
  • partial-duplication for what-if experiments and for as-is versus to-be comparisons
  • modelling of flows between entities

(Any other items that we ought to add to this list?)

As indicated in that list, we need to be able to support a wide variety of views and model-types. But the key point here is that there’s actually only one type of entity – or rather, every different entity is based on and resolves to the same core entity-type. The same applies to relationships: although there are many different apparent types, ultimately they all resolve back to the same core relationship-entity. That’s what would enable their portability between views, notations, model-types and applications.

This also implies that there’s no fundamental distinction between a ‘type’ and an ‘instance’. The only difference is that a type is instantiable, and an instance isn’t: we convert an instance to a type by making it instantiable, and we create an instance by making a non-instantiable copy of a type.

(Is there any part of the above that seems unworkable or doesn’t make sense?)

We need to be able to support several different levels of model-validation:

  • free-form (no validation), as in Visio
  • partial or variable validation, as in Archimate (or most forms of architecture-development)
  • strict formal validation, as in BPMN to BPEL conversion

(Free-form and strict are relatively easy to implement; partial or variable validation can be a lot harder! Partial or variable-validation also means that we need to support null-entities or placeholders to indicate ‘dangling’ relations or items that have yet to be defined.)

We need to support to keep the information clean, including:

  • explicit identifiers (distinct from editable names)
  • standardised explicit change/lifecycle, as per CRUD or REST
  • versioning
  • deduplication, merge and split
  • change of base-type for entity or entity-instance
  • resolve of dangling links (i.e. links where the ‘far-end’ entity has been deleted)
  • entity owner(s) – or preferably a complete RACI set for each entity

(What have I missed here?)

If I’ve got this right – and I’ll admit it’s a big ‘if’ – then conceptually we need only one core entity-type, and one core-relation-type, to cover all potential uses.

If so, then that would open an enormous range of possibilities for enterprise-architecture and for many other disciplines as well.

There are a number of tweaks and tricks to make this work in practice – particularly the concept of ‘tags’, which I’ll explain in the next post, and likewise a fundamental reorientation of the relationship between entity-types, relation-types and model-types – but in essence that seems to be enough to get started.

Comments, anyone, please?

13 Comments on “Back to the roots for EA toolset metamodels

  1. Hello,

    Very interesting post. Just a comment about tools. I use Obeo Designer (http://www.obeodesigner.com), a tool based on Eclipse Modeling technologies. I defined entities, attributes and relations (aka metamodel) and viewpoints (a set of view (matrices, diagrams, tree representations) for each actors conforms to IEEE1471 approach. It’s similar to your approach.

    I designed the TOGAF 9 metamodel (the content framework) with this tool and parametrized all the standard catalogs, matrices and diagrams. I planned to publish an Open-Source version of my work in the next week. Tell me if you are interested.

    Regards,
    Jérôme.

  2. Tom,

    I’d like to comment on, what you suggest to be “the roots”, the, as you call it, “straightforward” part of your post. These roots are – in fact… a way to view the world… a paradigm on which you base your life/behaviour, all of your next thoughts/actions.

    In that worldview/paradigm:
    Things exist. Things exist separately from one another. Things exist by themselves. Things are first. Things can be grouped into collections. After things… there exist “perceived relationships” to relate things. And then, from the things-relationship structures, there emerges “sense-making” (meaning/interpretation?) followed by “decision-making” (behaviour?) – depending on views/viewpoints (interests/motives?).

    Questioning that paradigm:
    If one doesn’t relate to a thing at all… does that thing exist in ones realizations of reality?
    As one starts to relate to a thing… does that thing start to emerge into ones realizations of reality?
    As one develops to relate to a thing… does the emergence of that thing develop into ones realizations of reality as well?
    The way one relates to a thing over time… does that ever changing way on and on create ones situational meaning of and behaviour with that thing?
    Do things ‘really’ exist? Or do things continually emerge into/demerge from existence by the way one develops his/her relationships with that thing?

    It’s ‘rather’ important to explicitly choose the most productive roots. Perhaps… especially in your case – judging your last series of related posts that emerged because of the way you happen to relate to the things around you? Anyway… in my mind ‘it’ is not as straightforward as you put it. Today’s world is a world in which great changes take place (as you very well know, I’m sure). In such a world it is even more important to question ones ‘straightforward’ roots. One emerges from ones chosen realizations of reality.

    Best regards,
    Jan van Til

  3. @Jan – Jan, you cruel person you – going all philosophy on me… 🙂

    You’re entirely right, of course: the ‘straightforward’ only applies to a subject/verb/object metaphor and worldview. The reason it is sort-of ‘straightforward’ is that that’s the metaphor that almost all of our existing tools and systems work with: ‘straightforward’ in implementation-terms more than anything else.

    If we shift the emphasis to the ‘relations’ rather than the ‘entities’, it’s also ‘straightforward’ to support a ‘flow’-metaphor, in which the changes rather than the ‘things’ become the main focus of interest. Quite a few real-world languages work that way: everything is ‘being and becoming’, rather than ‘in state’.

    But you’re right: other metaphors can become a lot harder to implement or describe, even with the greater versatility that this metamodel would provide. For example, true contextual dynamics, in which everything is changing or transitory, is still impossible to describe, even with dynamic tags – hence the infamous misleading notion of ‘current state’ versus ‘future state’.

    I’d agree that it is a real problem in all sense-making: no doubt about that at all. In my defence, I would point out that I am aware of this, and do describe it a lot more than others seem to do. For example, you’ll see plenty of references on this weblog to ‘Gooch’s Paradox’, that “things have not only to be seen to be believed, but also have to be believed to be seen”. One of the aims of the metamodel is because of its ‘self-morphing’ nature, it shoulod tend to surface some of those more common embedded assumptions; but yes, there is a ‘bootstrap-problem’ here, in that whatever we do, the metamodel will always embed some assumptions. I just try to surface them as best I can, is all – and thanks for your help in that! 🙂

  4. Tom,

    Thank you for your constructive and inviting reply! Yes, indeed, “[t]he reason it is sort-of ‘straightforward’ is that that’s the metaphor that almost all of our existing tools and systems work with”. And a much larger part of our thinking – definitely not limited to tool-thinking/implementation-thinking so to speak – got, I’m afraid, imprisoned by it. The bad news is that we’re totally unaware of it (well, most of us). And that’s why we don’t experience the ‘deeper’ prison we’re in. We experience our prison at and limited to the tool-thinking/implementation-thinking level. But there is a ‘deeper’ prison; a prison that severely hinders us to escape from the insolvable problems we currently struggle with at the tool-thinking/implementation-thinking level.

    Please don’t “shift the emphasis to the ‘relations’”! It’s our ever evolving relationships that create the ever changing things we be-have with. At the same time it’s our ever ‘doing’ behaviour that changes our relationships towards things. In terms of a contragram: the relation of the thing is the thing of the relation. Totally intertwined. You stated something like this yourself in several of your previous posts. One example: “We can choose anywhere as ‘the centre’, for some given purpose, yet must also insist that everywhere and nowhere is ‘the centre’, all at the same time.” I agree. So… fundamentally there is no such thing as the primacy of the thing (Object Oriented). Or relation (Relation Oriented). Far more productive is, say, Situation Oriented. The relations of things reconciled with the things of relations.

    It’s ones set of assumptions (paradigm) that shapes ones particular prison. If one experiences problems, one, in fact, hits one (or more) of ones particular prison walls. At first these problems can be overcome easily. As time goes by newly arising problems tend to get nastier. In the end they become very hard or even impossible to solve within the boundaries of the current prison. One ‘simply’ escapes from a particular prison by adopting a new set of assumptions. That new set of assumptions opens up a bigger (far more productive) prison. A prison with a new language, new situations, new things, new relations, …. Only if one believes ‘all this’, one really starts to see it.

    Staying in the old prison is the easiest thing to do. Everybody speaks the same old language, recognises the old things, relations etc. One keeps on selling products based on the current set of assumptions. One keeps on bumping ones head against the walls. One keeps on surrounding oneself with familiar problems and, of course, ‘solutions’. One keeps on creating a world in which no ground breaking things happen. A world whose solution space is (almost) exhausted.

    Swapping the old prison for a bigger, more productive one… yields unexpected, unprecedented – far, far better results. For EA. For tooling. For metamodels. But… it’s the hard way: other language, things, relations, problems etc. In that new prison/world ground breaking things start to happen: unsolvable, stubborn problems suddenly become very, very easy. Solutions one has never thought of arise. But… only if one perseveringly believes ‘all this’, one really starts to see it. Is it worth a (your) try?

    Surfacing deep (subconscious) assumptions is one – important! – thing to do. I agree, but examining them, identifying (some of) them as unproductive and replacing these by more productive ones is quite another – ground breaking – step.

    For your convenience: “true contextual dynamics, in which everything is changing or transitory” can be modelled with, for instance, Metapattern. Be warned: it’s (quite) another language, other things, other relations etc. Just as an example, see here. In this model four governmental organisations are contextually modelled. Modelling the first organisation generated about 80% of it. The other organisations were added to it by juxtaposition. The fourth organisation appeared to be almost entirely present. Really enterprise-like. Really unprecedented. Isn’t it?

    Best regards,
    Jan van Til

    • Jan – I guess what I’m really aiming for here is not an academic tool, but one that is relatively easy to work with and to use, in an everyday context. The philosophical stuff is great, and important, yet it’s way outside of most people’s concerns. And that’s a real problem when it comes to actually doing something.

      In my opinion, we need all of the power I’ve been describing ‘under the hood’. We then need to cover most of it up. The hardest part of the work is to hide the underlying complexity: making something simple is darned hard work.

      The real point I’m on about is that I believe we need to be able to take rough sketches and slowly add rigour behind them as required, working our way towards a fully executable model if that’s what’s needed, or else show clearly where the automatically-executable parts are and where they are not, so that we can model a true end-to-end traverse of a value-network (aka ‘process’, though in less constrained form than the usual usage of the term).

      We could do a mind-map or concept-map, for example, and move it towards a UML or BPMN model. We could take a BPMN model, and re-map parts of it in Archimate – without losing the original links and relations from the original BPMN model. We could take a strict BPMN model and re-display it in more pictorial form to show at an executive meeting or at a change-programme roadshow. We could add an audio- or video-recording of a review-meeting. People can add comments about usage, or about work-arounds that they’ve found useful in relation to some part of a process or some older item of technology. In other words, a general-purpose repository that quietly evolves as a key part of the organisation’s knowledge of itself.

      That’s what I’m aiming for here: a notation-agnostic ‘holograph’ of the organisation, reusable in any appropriate way. Formal philosophy is a real nice-to-have, but it’s the practical aspects that I’m really aiming for here.

    • Hi Jan – thanks for the link to ‘Just an example’ (‘Beyond central registers’ MetaPattern). Will have to admit I don’t quite see your point here: to me it’s just another systems-model – i.e. an instance of usage of a model-type, rather than a metamodel. What have I missed?

  5. In “The Matrix” Morpeus explained to Neo: “I can only show ÿou the door, you must open it yourself”. In the same way I can only point you to your roots and invite you to – really – examine them. But if you choose to not open the door… you’re not going to catch even a glimpse of what’s behind it.

    You state: “it’s the practical aspects that I’m really aiming for here”. Me to. But I opened the door. You very much seem to leave it untouched. That’s a choice. And that choice makes the difference. “What have I missed?” Well, you ‘forgot’ to open the door.

  6. @Jan – Hmm… y’know, that could easily be taken as a deeply personal insult? Or an assertion of truly remarkable arrogance? 😐

    There is such a thing as context, y’know. For the purpose of this exercise – and, for that matter, the exercise of this purpose – there are in practice some explicit constraints about what doors can or should be opened, about which doors it is wise to open, and others about which it is not wise. I make my choices on that, with considerable care and forethought. Do you?

    There are consequences to opening doors, and sometimes – as in this case – the consequences can be very significant indeed. After some sixty years of intense, deliberative practice I believe I’m open enough in my thinking/sensing/whatever to be aware of some of the consequences of opening certain doors in this context. Unlike some others, I don’t just open doors at random: I listen outside the door first, look the through the window in the door, scent the air coming through and round it. There are also some interesting tricks with doors themselves: to paraphrase another ‘Matrix’ character, “there is a door, there is no door”. And I’m also aware that there are others around me who may not be comfortable or safe about having certain doors opened in their presence, and hence I’m respectful enough not to open certain doors in public. In private, or in contexts with others who do know how to take the risks? – a different matter indeed…

    “What have I missed?” – perhaps I might ask you to reflect on the same question? We’ve never met in person: it’s clear that you know very little about me, who I am, what I do, what I’ve done, how I work, the depth or intensity to which I actually work. Yet you seem willing enough to make arbitrary pejorative judgments about me or my competence, based on that lack of knowledge. A strange behaviour… but your choice, I guess. Oh well.

    Going back to the topic of this post, I’m going to be blunt here: that model you pointed to was a complete red-herring, completely off the track. It’s a model. Gosh. Wow. Interesting to some, I guess, but in practice just another over-complicated systems-context model. It describes one interpretation of one social-context. Sure it could purport to be ‘the truth’ about that context – but there are dozens, hundreds of others just like it, each of which could also claim to be ‘The Truth’, ‘The MetaPattern’. That kind of thing is great if you like religious-wars about defining ‘The Truth’, but I’ll have to admit that I’m boring in that respect, because I’m much more interested in doing something useful.

    Yes, in a strict technical sense, that diagram can also be used as a first-order metamodel for certain social contexts: in formal terms, an M0 model that can be re-purposed as an M1 metamodel. Yet what we’re talking about in these posts here is something that is structurally very different: an M2/M3 (metametamodel repurposed as metametametamodel) with some reference to and assumptions about the M3/M4 on which it’s based. And yes, that M4 and everything derived from it is subject to certain philosophy-layer constraints and assumptions such as the nature of words and and subject-verb-object relations and the rest: and we know and are aware of the implications and consequences of those assumptions.

    Those constraints are imposed on us by practicalities such as IT-based toolsets and the practical difficulties that most people have of holding in mind at any one time the entirety of very-large n-dimensional polymorphic naturally-ambiguous self-mutating dynamic structures. (You may be able to do that, but most people can’t, don’t or aren’t willing to inflict that kind of insanity on themselves – and we do need to respect that.) Hence, yes, boring, for this purpose, it’s wiser to stick with the kind of constraints that a relatively-conventional M4 will impose. What we’re not doing in the concept I’ve described here, you may note, is going straight to a M1/M2 like UML or BPMN: instead, we’re retaining the flexibility of an M3 all the way through to the M2 level, and hence creating a framework for M1 metamodels that on the surface looks as constrained as the usual types, but actually retains (almost) all of the underlying flexibility that we need. That’s different from the usual metamodels – and although it is a ‘meta-pattern’ of some kind – a pattern that repeats in ‘self-similar’ ways at every level, as a good pattern should – it’s a very different concept of pattern from the extraordinarily constraining assumptions of the ‘MetaPattern’ model to which you pointed.

    The whole point of the model you pointed to, it seems, is to try to define everything. The result – and I’ll be blunt here – is a huge, unwieldy, almost unusable mess. (The fact that the purported relationships between entities weren’t defined or explained in any way didn’t exactly help in that, either…) The focus here is the opposite: the aim is to support emergent modelling of emergent complexity through simplicity. At its core (M4), there is just one element, in two forms: the singleton, and the collection. At the first practical (M3) level, we still only have four elements: entity, relation, tag, and model – and that same simplicity remains available all the way to the M0 level.

    In short, there’s a fundamental difference between the two ‘meta-patterns’: one is at an M0/M1 layer, whilst the other (the one actually in context here) is at an M3/M4 layer. If we’d been talking about the M0/M1 layer, then yes, your comment about “Well, you ‘forgot’ to open the door” might have some merit. Since we’re not, it probably doesn’t?

    Enough said, I think.

  7. Tom,

    Thank you for your friend-ly reply to my succinct contribution. That contribution, as I understand from your reply, caused you a, to say the least, Mood Change. Personal insult? Arrogance? Any of that was not – never was my intention. My apologies for that. Correct: we never met. So… in that (and other) ways I do not know you. I ‘only’ happen to know you from your Set of Very-Varied and Very-Various Posts. Are you trying to tell me that such knowing doesn’t say quite something about, say, the root-doors you (seem to) systematically live by?

    Nevertheless, I’m glad that you made yourself absolutely clear: You’ve made your choices “with considerable care and forethought”. Thank you. And you’re not going to reconsider the ‘root-doors’ you opened/left untouched. Thanks again. I don’t see any right/wrong, good/bad in that. It is just the way it is. It’s your (rooted) way. And, yes, I Fully agree with you that “[t]here are consequences to opening doors, and sometimes – as in this case – the consequences can be very significant indeed.”

    Of course I don’t (exactly) know what you’ve missed. How would I…? However, I, for myself, Know that roots (especially nowadays!) don’t last forever and need to be revisited and revised every now and then. It is from that very perpective that I suggested to you the forgetting, oooops, the ‘forgetting’ – please do mark the quotes – of a root-door. Such ‘forgetting’ (overlooking) is rather easy. For all of us; its human. Whether one has “some sixty years of intense, deliberative practice” or not. Therefore, I really don’t think that there is any need to conclude that I’m “mak[ing] arbitrary pejorative judgments about [you] or [your] competence”. On the contrary; I Am Not!

    But, as said, I’m glad you made yourself absolutely clear. From such carefully rooted clearness it becomes obvious that you neither understand nor appreciate Metapattern. Thank you for making that clear too. Different sets of opened/closed root-doors inescapably yield different worlds in which there is little to really share. Again: I don’t see any right/wrong, good/bad in that. It is just the way it is: rooted ways unite as well as separate. Thanks anyway for having a (quick) look at it.

    Jan

Leave a Reply to Tom G Cancel reply

Your email address will not be published. Required fields are marked *

*