More notes on toolsets for EA

Various notes following on from my recent post ‘New toolsets for enterprise-architecture‘. Most of this is about some great interactions on the ‘EA-toolset’ themes with Phil Beauvoir (@ArchiToolkit), Peter Bakker (@mapbakery) and Gene Hughson (@GeneHughson), and also, if somewhat peripherally, with Simon Brown (@simonbrown). Yet it’s also about some other themes that have come up somewhat in parallel, that in some ways interweave with those conversations. I’ll summarise all of this under four headings:

  • Semantics of connections
  • Structure and search
  • Model-types and object-ownership
  • Support for story

Before we start on those, let’s do a quick recap on some of the key themes and concerns that underpin this whole ‘new EA-toolsets’ concept.

First, the purpose of all of this is to provide better support for how we work, as enterprise-architects and the like. More specifically, it’s about toolsets to support how we work – ‘tools’ both in the sense of frameworks to help guide our thinking, and software-based applications and repositories that support those frameworks and help with recording, cross-referencing, linking, retrieving and reporting on the information and ideas that we collect and collate during that work. (For more on this, see, for example, the posts ‘How do you think?‘ and ‘The toolset ecosystem‘, and the further posts linked to from those.)

What we also need is full support for the whole of the development-process, as classically depicted in Damien Newman’s ‘the Squiggle’:

The catch, for us, is that just about all that most of our current ‘EA’ tools will support is the ‘easy bit’ over at the far right-hand end of the Squiggle, where, in essence, the models become little more than static ‘decision-records‘. One of the key problems that arises from this over-emphasis on completions is that whilst we do have a useful ‘decision-record’ in each case, we’ll have almost certainly lost any connection with the ‘why’ or ‘how’ via which we arrived at that decision – including trade-offs and choices that might have had little to no connection with any real-world needs. This not only forces us to start all over again, every time, whenever Reality Department indicates or demands some form of future change, but – by depending too much on the seeming certainties of those former decision-records – can all too easily lead us once again down pathways that may be misleading, downright dangerous or just plain wrong. Not A Good Idea… – we urgently need something better than that.

So how do we support all of the Squiggle? – not just the tail-end of ‘Design’, but ‘Concept / Prototype’ and ‘Research’ and ideation too? Although software-tools and the like will always be limited, more providing decision-support rather than decision-making, what I’ve suggested so far is that we could perhaps usefully build around what I’d termed the Burnett Query – that for every concept or idea or ‘thing’, we should be able to derive useful answers from three related sets of questions:

  • Tell me about yourself? – and why you’re the way that you are?
  • Tell me what you’re associated with? – and why?
  • Tell me how you have changed and are expected to change? – and why?

In that sense, frameworks and model-types such as UML, BPMN, Archimate and the like are potentially-useful ways to constrain how we describe and view various types of ‘things’ and their interrelationships and changes. But we need to remember that in essence Archimate and the like are just sets of views, associated with specific needs and choices: using those frameworks, we see things in terms of those views, but the views do not and, in most cases, cannot describe the totality of the things themselves – that distinction is, as usual, somewhat subtle, but extremely important. Hence we need our toolsets not only to be able to support just about any framework or view that we might need, yet also allow each of the ‘things’ – and, as we’ll see, the nominal links between them – to be somewhat independent of how we would view or model them within the terms of any one framework.

And we also need this toolset to support the ‘messiness’ in the earlier stages of the Squiggle – which is exactly what most of those frameworks are designed to prevent. (And the latter for good reasons, of course, because we do usually need to trim out the uncertainties as we move towards any real-world implementation. But attempting to squash uncertainty is exactly what not to do during the earlier period of inherent-uncertainty – and that’s the point that we need to emphasise here.)

Which brings us to the first topic here…

Semantics of connections

The main set of responses that I had around the previous ‘New toolsets’ post were around how to implement describing the connections between things, as can be seen in the following tweets:

  • mapbakery: what about a Semantic Mediawiki in combination with RDF?
  • tetradian: thanks – useful themes to follow up (though does RDF require a predefined vocabulary? – if so, may not work for this…)
  • mapbakery: I’m not sure I understand the question correctly but “RDF is a framework for modeling all forms of data” (see Bergman, ‘Advantages and Myths of RDF‘)
  • ArchiToolkit: RDF is the implementation of a semantic model. Think in terms of an abstract semantic model, not its representation.
  • tetradian: specific-implementation vs abstract model – exactly the distinction I need – thanks Phil!
  • ArchiToolkit: Semantic model is about triples. Could be RDF, OWL, Turtle. See for example Stackoverflow (on the difference between RDF and OWL) but I’m no expert!
  • mapbakery: Me neither, but I’ve worked with a Semantic Mediawiki  >good test platform for Tom’s ideas…

And yes, a semantic-wiki or somesuch would be one way to make a start on this: I’ve long since thought that a wiki-type would be the best way to model a glossary and thesaurus, for examples, and in a sense everything in context is a kind of glossary-item in its own right.

But there’s a catch – a show-stopper of a catch, in fact. Current semantic-frameworks seem to be geared primarily around static perceived-relationships: and that seems especially to apply to RDF/OWL (from my somewhat-cursory reading so far). Which puts us, again, right over at the right-edge of the Squiggle, with nothing to support exploration and experimentation around concerns around inherent-uncertainty, such as requisite-fuzziness and ‘variety-weather‘ – the variability of variety itself – which is what we must be able to work with during the earlier stages of the process, over towards the left-side of the Squiggle.

We could perhaps illustrate this with a cross-map to SCAN:

(Okay, we need to note that the layout of SCAN is the opposite way round to the Squiggle – certainty on the left, versus to the right on the Squiggle – but we can live with that, I presume?)

The Squiggle says that, to arrive at a usable design, we first need to go through a period of uncertainty. I’d agree with that. Yet the deeper catch – and the point that SCAN does help us to see more easily – is that all we’ve done during the process through the Squiggle is filter out from ‘the Everything’ various useful bits of sort-of seeming-certainty, enough to actually build something that might support and (maybe) resolve some specific need. What SCAN shows us is the whole context, certainty and uncertainty blended together in recursive, fractal form – and that all of it is always there, whether or not we exclude it from view. We’ve used the Squiggle to help us filter out and select out what we need – or, in SCAN terms, move the focus steadily further and further towards the left-hand side of the frame. Yet the uncertainty itself never goes away: it’s still there, always, whether we like it or not. All we’ve done is selectively exclude out from view – which is very different from actually removing it from the real-world. If we give ourselves the illusion that everything is certain when it in fact is not, we can design ourselves into serious trouble.

And that’s especially true if we allow ourselves to settle too early into pseudo-certainty – which is precisely what tends to happen with all of those static ‘has-a’ and ‘is-a’ and suchlike associations in RDF and the existing ‘EA’-tools. Once again, it’s Not A Good Idea… By contrast, the kind of relationships we actually need to work with in the left-hand and central stages of the Squiggle are much more fluid, such as:

  • “this model might use that type of server”
  • “some of the options we could use are A, B, C and possibly D (if it’s available)”
  • “the vendor suggests that version 2.1 should be released by that time – otherwise we might have to go with the other vendor’s option, which may need us to change these parts of our infrastructure”
  • “David quite likes that option, Charley and Hazel don’t like it at all, and Andy is still sitting on the fence but won’t say why”
  • “there’s a big hole right there marked ‘Don’t Know’ – we’ve no idea what to do about it as yet”

At the minimum, we need an equivalent of the MoSCoW set – Must-have, Should-have, Could-have, can-Wait. (Or, for that matter, the SCAN framing: Simple, Complicated, Ambiguous, Not-known.) For semantics, the only frame that I know (thank you Erik Proper!) that comes even somewhere close to supporting this kind of need is ORM (Object Role Modelling): perhaps we could usefully focus some of our attention there for a while?

Structure and search

In another separate set of tweets, Simon Brown talked about software-architecture in terms of what he called ‘Shneiderman’s Mantra‘ – “Overview first; zoom and filter; then details on demand”. Simon mapped this to his ‘4C’ software-architecture hierarchy-model, as follows:

And as Gene Hughson noted, the same obviously also applies to the way in which we’d look for, and at, ‘things’ and collections of ‘things’, within the toolset itself:

  • GeneHughson: indeed…potentially lots of recursion, so that’s a given

Yet once again there’s a really important catch: within our kind of context, almost all ‘containers’ are virtual. Or, to be pedantic, a nominal container is itself just another kind of ‘thing’ that happens to contain links to other things – links that we’d describe as representing a ‘membership-of’ relationship for the respective referenced-thing. A ‘thing’ is also arguably a container for its attributes (though even that assertion can be questioned, as we’ll see in a moment); but beyond that, ‘containment’ is little more than a synonym for ‘included in something’s list somewhere’ – a very different concept compared to the kind of absolute ‘is-owned-by’ relationship that we’d see in, say, the classic organisational-silo structures, or the ‘Business / Application / Technology’ layering of Archimate and the like.

Anyone with a reasonable experience of enterprise-architecture should recognise straight away the problem here: if our structure mandates that all we have is things and containers, with every ‘thing’ belonging to a single container, how do we describe ‘things’ that naturally sit between containers, that link all those other things together into a (possibly)-unified whole? How do we describe how and why and when such ‘things’ move between containers? If something is ‘owned’ by a project, how does it stop being part of that project, and become part of operations instead – where it might itself be used and re-used in different configurations, each itself a kind of container? If we’re not careful, concepts of ‘containers’ in a toolset can quickly become more of a hindrance than a help…

In practice, containers represent potentially-intersecting sets that could be organised in whatever way we choose. A query-result is actually a type of container, though one that exists only for as long as we maintain that particular query – and its effective-content might vary over time, of course, as is probably true for many other kinds of containers.

We also need to be wary of arbitrary assumptions about set-membership, and, in particular, arbitrary conflations that can cause huge problems as soon as we move outside of the nominal scope within which those conflations might seem to be valid. As described back in the post ‘On layers in enterprise-architecture‘, the unquestioned-assumptions embedded in the layering in TOGAF and Archimate results in a set of arbitrary-conflations that are truly horrible:

  • ‘Business’ = anything-human + anything-relational + principle-based (‘vision/values’) decisions + higher-order abstraction + intent
  • ‘Application’ = anything-computational + anything-informational + ‘truth-based’ (algorithmic) decisions + virtual (lower-order ‘logical’ abstraction)
  • ‘Technology’ = anything-mechanical + anything-physical + physical-law (‘rule-based’) decisions + concrete (‘physical’/tangible abstraction)

Yes, it’s just-about possible to describe a meaningful architecture for classic ‘big-IT’ with that kind of mess. But it soon becomes clear how much of a disaster-area it really is when we try to describe routine real-world concerns such as human-based ‘applications’ (such as Amazon’s ‘Mechanical Turk‘) or physical-based information-systems (kanban-board, or ‘block’ system in single-track rail-working). And what about entirely new kinds of technology, such as programmable-hardware or shape-memory materials? – we cannot get away with the TOGAF/Archimate conflations there.

“Overview first; zoom and filter; then details on demand” – that’s a really useful pattern to support search and suchlike. And in turn, virtual-containers also represent another really-useful organisational-pattern to support a framing of that overview, zoom and detail-view, as ways to conceptualise ‘pre-package’ certain routine types of queries: membership of a project, for example, or the items associated with a particular work-session, with ‘the project’ or ‘the session’ providing the frame for the overview. But the moment we think that those containers are ‘real’, that they somehow ‘own’ other items in a supposedly absolute-exclusive sense? Or, even more, that all ‘things’ must have a single owning container, in a hierarchy-tree of containers with ‘things’ as the final leaf-nodes in the tree? – well, that’s where we’re likely to find ourselves in trouble, even just within the confines of a data-repository, let alone anywhere else. Whilst developing towards a more-useful type of toolset for our work, we do need to be careful about this point – because misplaced assumptions about ‘containers’, categories, classes and ‘owners’ still seem all too common in most mainstream thinking about enterprise-architecture and the like.

Which brings us to our next topic…

Model-types and object-ownership

It’s easy enough to talk about all of these ideas – but if it’s ever to have a chance of becoming something more than mere talk, at some point we do need to work out a way to implement it for real. I’m really pleased, and really grateful, that that part of the discussion is starting to happen – or, at least, that it’s now more than just me that’s talking about it. 🙂 Hence, for example, a trio of tweets respectively from Phil Beauvoir, Peter Bakker and Gene Hughson:

  • ArchiToolkit: I’m still thinking RDF/OWL. But the model can manifest in many ways depending on where it is implemented
  • mapbakery: I’m trying to implement some of @tetradian’s ideas on a personal (phone) scale with NoteLynx Pro http://t.co/KTI3kZVuaj
  • GeneHughson: for discussion-will add comment on “New toolsets for enterprise-architecture”

The tail-end of that last tweet from Gene (which unfortunately I’ve managed to lose…) pointed to this entity-relationship diagram:

And I’ll admit that that’s close to the kind of model I roughed out when I first started on this. Yet as I discovered the hard way – and I clearly haven’t explained well enough so far – there are two crucial points we need to address. The first is as in the section just above, the point about ‘containers’ always being virtual, not ‘real’. The second is what I’d sort-of hinted at in the tweet below, before Gene’s reply:

  • tetradian: (for now, crucial point to note is that, by definition, links are between ‘things’ – they don’t belong to one ‘thing’ only)
  • GeneHughson: @tetradian Right…used ‘Owner’ & ‘Target’ but definitely open to better terminology

What Gene’s shown here is a one-to-many link from Thing both to Attribute and to Link – in other words, that both Attribute and Link are contained within and ‘owned’ by Thing. (He’s also showing that Links can have their own Attributes too – with which I strongly agree.) On top of that, Links and Attributes can each be associated with one or more Profiles – the term he’s using for what I’d described as ‘editors’ in the previous ‘New Toolsets’ post.

To me, all of this is fair enough, as far it goes – and yes, it is possible to implement it this way, and make it work. Sort of – enough to use it as a start-point, anyway. Yet it doesn’t go far enough, in terms of breaking away from risks of inappropriate ‘ownership’, in several different senses – and I do need to explain those points better than I have done to date.

First point is that, in principle at least, everything is an ‘equal citizen’Everything is a Thing – and that not only includes the more-obvious Things and container-like Things as well, but also Links, and, technically, Attributes too. For the latter, it’s probably more as an abstract ideal rather than feasible in practice: every possible Attribute, and even every possible value, for example, could technically be a Thing in its own right – but it would be impossible to implement it as such, because by definition there could or would be an infinity of them. Yet it’s certainly true that we should be able to create links to specific Attributes as much as to their respective parent Thing – and links to Links, too. And that requirement makes things a lot more complex – yet also a lot more simple, if we can think creatively enough about that complexity…

In the first attempts I’d made at this, several years ago now, I developed the idea of what I called a ‘mote‘:

A mote is tiny – right down at what we might describe as an atomic scale. It consists solely of an item-identifier, a role-identifier, one parameter, and any number of links. These links point to other motes, which in turn point to other motes, which together build up towards even bigger molecule-like clusters of motes… – well, you get the idea. It’s a nice idea, and it actually does describe what’s needed. The catch is that it just isn’t implementable – not with current data-technologies, anyway. (As a table-structure it’s actually very easy to implement – it’s just one SQL table – but the scale of recursion it would need in search, retrieval and update, and the probable complexities of polymorphic mapping for different data-types, would make it horrendously inefficient and slow.) Even purpose-built graph-databases would struggle with this. Somehow, we do need to preserve that concept of ‘everything is an equal-citizen’: but in practice, we’re going to need a different trade-off than that too-theoretical ideal.

After a lot of research and exploration and heartache and thought-experiments, the best trade-off I’ve found so far could be summarised like this:

  • every Thing is an equal-citizen
  • every Thing will share an identical set of mandatory parameters – in particular, an item-ID, role-ID and access-rights controls
  • every Thing may carry any number (0..n) of non-editable, searchable tags
  • every Thing may carry any number (0..n) of optional parameters
  • every Thing may carry any number (0..n) of pointers to other Things
  • every parameter will be defined from an editor-module, identified by searchable tag
  • every pointer may be associated with an optional parameter defining the reason for the link

All of that is implementable quite easily in a NoSQL document-database such as MongoDB, and (I believe?) also possible, if with some serious table-juggling, to implement in SQL as well. It’s doable, anyway. But with various combinations of the above, what this gives us is as follows:

  • a basic tag-Thing with no editable parameters and no links, but can be pointed to by other Things’ links
  • standard-Thing that carries its own editable-parameters
  • a parameter-Thing that exists primarily to carry some parameter for another Thing, but for various reasons it’s useful to treat as a Thing in its own right – for example, a Thing that carries the content and edit-history of a wiki-page
    (an alternative implementation would be for the parameter-Thing to have one pointer that points back to its ‘parent’ Thing)
  • an extended-Thing that has its own parameters (like a standard-Thing) but also has pointers to ‘external’ parameter-Things
  • link-Thing that has pointers to (usually) two other Things, and that therefore describes an identifiable association between those Things
  • container-Thing that carries any number of pointers to other Things that are conceptually ‘contained’ by it in some way

There might perhaps be a few other variations on that overall theme, but the point is that the data-structures that we need do all seem to be implementable just through that one small set of elements: mandatory-parameters, non-editable tags, editable-parameters, and pointers.

The tricky part is in how those Things are defined, and their content edited and amended – because to make it work, we have to break away from one of the most common assumptions of all in current toolset-design: that an object is defined and ‘owned’ by some predefined model-type. Instead, we shift priority from the model-type to the object-instance: a Thing is ‘owned’ by nothing other than itself.

Let’s take a practical example, and imagine we want to implement support for Archimate in this toolset. (Which we do, of course, but it may take us a while to get that far – we’d start implementation with something simpler first!) Archimate defines a bunch of object-types, and the relationship-types that are permitted between each possible pairing of objects. The usual way to do this is to define everything in a formal metamodel, starting with the object-types and their parameters, and then distinct pairings for each relationship-type permitted for that object-type. It’s a complex piece of work – and it grows almost exponentially each time we add either a new object-type or relationship-type, so we’d always want to keep both of those to a minimum. Also, and importantly, every object created from this Archimate metamodel is ‘owned’ as an ‘Archimate-only’ item: in general, it can’t be used for anything else.

(That’s the way it’s usually implemented, anyway, although some of the existing toolsets – most notably Abacus Avolution – do allow us to add and edit further parameters to objects, if usually at a class-level only.)

What we do here instead is all but turn the whole thing upside-down. There’s still the same concept of a metamodel that defines objects and relationships: but it’s done in a crucially-different way. The object-definition is used as a template, either to create a new Thing, or to specify parameters and tags that can be applied, edited and used when an existing Thing is explored in context of this specific editor. (There’s a quite close analogy here with the ‘Open with…’ functionality in Windows or OS-X, though it applies at instance-level rather than at class-level – think in terms of individual-file rather than file-type.) And linking-rules relate to tags assigned to and associated with object-types – rather than object-types themselves – which greatly simplifies how linking works, and also obviates that near-exponential expansion of complexity as new object-types or link-types are added.

Hence, for example, we might find via a search-query a Thing that was originally created via a BPMN editor to represent an Activity called ‘Accept Room-Booking’, in a process-model for a hotel system. We decide we want to represent this in Archimate as a Business-Service. So having found this Thing with all of its embedded BPMN-parameters, we open it in the Archimate editor. We tag it as a ‘Business-Service’, and for the moment, keep its name of ‘Accept Room-Booking’. But having tagged it as a Business-Service, we can now edit parameters appropriate to an Archimate Business-Service; and we can also link it to other Archimate-tagged things in terms of Archimate’s linking-rules. Those Archimate-compatible parameters and tags then become a permanent part of the data of and for that Thing. Yet the respective Thing is not ‘owned’ by Archimate, any more than it is owned by BPMN, or any other editor. We work on it with the Archimate editor (in this case), to treat it as-if it’s an Archimate object, but that’s not the same as saying that it ‘is‘ an Archimate-only object – a subtle yet crucial distinction.

In that sense, each Thing may have a data-structure that is different from that of every other Thing – in effect reflecting its own distinct edit-history, as worked-on by any number of different editors, each of which assign their own parameters and tags to that Thing as required and appropriate. Yet because the core-structure and internal framing – mandatory-IDs, tags, editable-parameters, link-pointers – is the same for every Thing, they’re still searchable and suchlike as if they do have the same data-structure. Think for a while of the power and flexibility that comes from that – and yet it can all still be maintained under whatever formal-rigour may be required by any editor that applies.

There’s quite a lot more I need to write about this – particularly about the structure and content for editor-specifications – but that’s the basic idea, anyway.

And that theme of flexibility of content and ‘ownership’ and linking for Thing-entities leads us on to the final topic for this post…

Support for story

The start-point here is a tweet from Peter Bakker with a quote from the section ‘It’s all about story’ in my post ‘Enterprise Canvas as service-viability checklist‘:

  • mapbakery: “So whenever we look at an architecture, whenever we look at structure, we need also to look at, and look for, the stories”

This is a common theme in other forms of architecture too. For example, in the post ‘Two points of view on (enterprise) architecture‘, I pointed to a quote from Matthew Frederick’s 101 Things I Learned In Architecture School:

Two points of view on architecture

ARCHITECTURE IS AN EXERCISE IN TRUTH. A proper building is responsible to universal knowledge and is wholly honest in the expression of its functions and materials.

ARCHITECTURE IS AN EXERCISE IN NARRATIVE. Architecture is a vehicle for the telling of stories, a canvas for relaying societal myths, a stage for the theater of everyday life.

Or, to put it another way round, architecture is about structure, but also is about story: we need both of these in place if we’re to create an architecture that works and ‘makes sense’ in real-world practice and real-world use.

(There’s more on this in other posts here, such as ‘The enterprise is the story‘ and the NOTES series, and also the slidedecks ‘The enterprise is the story‘ and ‘Staging the story‘ on Slideshare).

Hence, as Peter says in another tweet:

I don’t know DesignVue, but I do know the story-builder in the Southbeach Modeller toolset – and yes, that’s something that we’re going to need. In Southbeach, it takes the embedded content of a model – the identifiers and parameters of the objects and the links between them – and builds a textual description that can be used in a report. (I believe that some of the existing ‘EA’-toolsets can also do something like that, though I don’t have any details as yet.) Although in Southbeach it’s made simpler by the fact that that toolset only allows a limited range of object-types and relations between them, it’s still something that should be doable with the data-structure I’ve described in the section above.

In Southbeach, we can generate a textual ‘story’ for a single object at a time, or the set of objects displayed in the current view. For the earlier stages of the Squiggle, though, we’d want to be able to build a story about any set of objects, that could be of nominally-disparate types, created by different editors, quite possibly at different levels of abstraction or aggregation or composition or whatever, that we choose to link together in some way.  In other words, not just a simple:

A > B > C > D > E

But something that could weave in all manner of different directions, and even loop back through itself to show how different insights have arisen from the process of connection:

A > B > C > J > Q > A > C > K > B

And so on – we choose the story, rather than having the story-structure imposed on us by the built-in rules of a single editor-specification.

Another theme to explore, I’d say – but definitely something that we’re going to need, anyway.

Best leave it at that for now: over to you for comments, perhaps?

18 Comments on “More notes on toolsets for EA

  1. Tom,

    I’d recommend having a closer look at OWL. Web ontologies can be an important part of the EA tool-set. And the that includes creating human and machine-readable descriptions, getting new knowledge through inferencing, and executing ontologies (the model is the application, unlike any other formal language where you need transformations between design- and run-time description). It can even be used for fuzzy logic.

    I’ve been very happy using RDF and OWL for EA in the last few years. But it takes some investment.

    Of course OWL cannot be sufficient as I already explained here. But it helps in what people are not good at – deductive reasoning. Having that capability supported, people can use their capabilities for inductive and abductive reasoning to complement in dealing with the messier areas.

    • Okay, okay, okay, I’ll look at OWL, honest, honest! 🙂

      @Ivo: “But it helps in what people are not good at – deductive reasoning.”

      Agreed – though my experience is that people are often even worse at inductive and abductive reasoning… Hence the reason why we probably need even better support for those than we do for straightforward deductive-reasoning. :wry-grin:

      Thanks again, anyway.

  2. It really depends on how you would use and implement OWL/RDF. Are we talking about it as a persistence format? At the end of the day web sites and services simply throw their data into a SQL or NoSQL DB or similar. What would you do with an OWL data model? How does “using OWL” specifically manifest itself? Are Tom’s “motes” expressed as triples, for example?

    • Phil,

      Re: “It really depends on how you would use and implement OWL/RDF”
      -> Indeed it depends. It is quite often that I see people use it as UML or even use UML class diagrams to represent OWL.

      RDF is great for storing simple formal descriptions and simple inferencing (e.g. transitivity), while OWL (esp OWL2) is capable for much more sophisticated description (class restrictions, property taxonomies), inferencing and ontology execution.

      Re: “At the end of the day web sites and services simply throw their data into a SQL or NoSQL DB or similar.”
      -> Yes, or other graph DBs such as MongoDB, or in triple stores
      What would you do with an OWL data model?

      Re: How does “using OWL” specifically manifest itself?
      -> building ontologies in a distributed way, include many domain experts, having interoperability by linking and not by alignment, ensuring logical consistency along the way and learning new things and in many other ways. It really needs to be experienced to be appreciated.

      Re: “Are Tom’s “motes” expressed as triples, for example?”
      -> everything in RDF and OWL is expressed as triples.

    • “At the end of the day web sites and services simply throw their data into a SQL or NoSQL DB or similar.”

      I’d argue that the repository needs to be concerned with search as well as CRUD behavior. Unless the document’s structure was compatible with the underlying DB (whether SQL or NoSQL), dumping the raw document would mean combining parsing with searching which would murder performance.

      • @Gene: “I’d argue that the repository needs to be concerned with search as well as CRUD behavior.”

        Very strongly agree.

        @Gene: “the document’s structure was compatible with the underlying DB…”

        I’m trying to find a way to avoid forcing that that requirement, with some kind of abstraction-layer. At the least, I want to build queries such that the underlying database-structure can be as transparent as possible. Again, it’s something I’ll have to blog on… – sorry! 😐

        @Gene: “…dumping the raw document would mean combining parsing with searching which would murder performance.”

        Yeah, I’m acutely aware of that one, too. I’ve been looking at some tricks in MongoDB that allow me to avoid much of the parsing-problem, but that’s just one example – I don’t know how it would work if some implementation needed to use a SQL database instead, for example.

    • @Phil: “Are we talking about [OWL/RDF] as a persistence format?”

      I hope not – it may be readable by machines, but it’s almost unreadable by humans. 😐 JSON ain’t wonderful, but it’s a heck of a lot more readable.

      @Phil: “How does “using OWL” specifically manifest itself?”

      I don’t know: I must admit a lot of this discussion of RDF/OWL does somewhat feel like ‘solutioneering’ for an entirely different problem than the one I’ve been describing.

      @Phil: “Are Tom’s “motes” expressed as triples, for example?”

      Probably, but again I need to get my head more solidly around the questions and data-structures first – I’m wary that there might again be a bit too much of force-fitting a problem to fit a predefined ‘solution’.

  3. Hi Tom,

    Re “So having found this Thing with all of its embedded BPMN-parameters, we open it in the Archimate editor. We tag it as a ‘Business-Service’, and for the moment, keep its name of ‘Accept Room-Booking’. But having tagged it as a Business-Service, we can now edit parameters appropriate to an Archimate Business-Service; and we can also link it to other Archimate-tagged things in terms of Archimate’s linking-rules. Those Archimate-compatible parameters and tags then become a permanent part of the data of and for that Thing.”

    This remembers me IBM Lotus Domino/Notes technology where all applications (including the eMail client) are based on document databases. The heart of everything is the document (ie. a free form attribute/values list) that you can edit through any kind of form (which is used to key in defined list of fields through an associated UI). The key point (related to your post) is that all these documents are accessed through views (ie. kind of queries) and each view can offer to view/edit a document through a defined set of forms.

    Example: the mail client define a defaut email form, but it is possible to open a mail through another form (e.g. FAX form, application log entry…) without loosing any information (just hide some to not be distracted).

    How does this applies to this discussion? I’d say that this is a good advocate for a document centric storage (and so are most NoSQL engines) associated with some good “forms” technologies and query engine. The key seems to be document polymorphism…

    (I now have to look for some already existing technologies to help do a POC…)

    • @JB: “How does this applies to this discussion? I’d say that this is a good advocate for a document centric storage … associated with some good “forms” technologies and query engine”

      Yeah, that’s essentially what I’m talking about here, though the ‘editor’ structure probably covers a bit more, defining not just the data-structure and forms and queries but a fair bit of the business-logic and context-based help and suchlike (yet another theme I need to write a blog-post on Real Soon Now… 😐 )

      @JB: “(I now have to look for some already existing technologies to help do a POC…)”

      Yes please yes please! 🙂

  4. Tom,

    Can we really say every Thing is a first class citizen? A link must have two other things that it connects (perhaps you could get away with one temporarily, but ultimately a link without things to connect seems to be problematic).

    Likewise, an attribute of thing A might be reference to thing B, in which case that attribute is truly independent of what it is linked to. By the same token, Name of A, Access Rights of A, etc. would cease to have meaning without A.

    Does this make sense?

    The “profile” concept was intended as a specialized tag. It definitely would not convey ownership. The Name attribute would most likely be associated with all profiles (which is the equivalent of being owned by none).

    Needless to say, coming up with a very abstract structure that can be made to represent almost anything is tough. Some thoughts I had in mind was coming up with a structure that supported recursion, but could be accessed by “shaving” those layers one at a time (thus enabling ‘Shneiderman’s Mantra’ for both functional and performance reasons).

    • @Gene: “Can we really say every Thing is a first class citizen?”

      Yes, from what I’ve seen so far, everything other than an embedded-parameter or tag really does need to be a first-class citizen. For example, we may well want to link to a Link (because the Link is the ‘between’ item, that’s itself often of real importance in its own right in enterprise-architectures).

      The way we make everything a first-class citizen is by using the exact same base-structure for everything. A Link is just another Thing that has its own attributes, just like any other Thing: it places a bit more emphasis on its links than other Things probably would, but otherwise no real difference as such.

      @Gene: “A link must have two other things”

      A Link would typically connect to two other Things, yes – but could in some cases start with just one (as you say), and in some cases might well have more than two. (A ‘Link’ that has no connections is otherwise known as a Thing. 🙂 )

      @Gene: “By the same token, Name of A, Access Rights of A, etc. would cease to have meaning without A”

      Yeah, agreed, there are some juggling-acts here that need to be better thought-through than I’ve so far described. (I’ve thought about them a lot, just have been able to find a way to write it all down… 🙁 ) What I can see is that there are a few special-cases where a nominal attribute ends up being referenced or ‘part of’ two or more Things, rather than always solely one. There are also quite a few cases where we might want to point a Link to a specific attribute of a Thing (e.g. via something like the HTML ‘#anchor’ mechanism), rather than always at the parent Thing itself. Hence at the abstract level I’d prefer to be able to keep my options open for as long as possible, and not constrain them too early if we don’t need to do so.

      @Gene: “The “profile” concept was intended as a specialized tag.”

      Ah, right – apologies that I misunderstood. So yeah, I really do need to do some kind of entity-model Real Soon Now, don’t I? 😐

      @Gene: “Needless to say, coming up with a very abstract structure that can be made to represent almost anything is tough.”

      Yeah, I did kinda know that? 🙁 🙂 🙂 It’s taken me about five years so far, since I really started working hard on it, but I do believe I’m getting somewhere a bit closer now.

      @Gene: “Some thoughts I had in mind was coming up with a structure that supported recursion, but could be accessed by “shaving” those layers one at a time”

      Expand on that, if you would, please? Perhaps in a blog-post on your own site, so that others can see it more easily (and so that you get the credit for it, too? 🙂 )

      Thanks very much for all the help so far, anyways.

      • “Expand on that, if you would, please?”

        No need for a full post, the easiest example I can give is a very simple html file with paragraph and anchor tags – the paragraph tags contain the primitive data and the anchor tags are a placeholder linking to a thing that requires another call to the repository (web server) to pull down the full depiction of what the placeholder is pointing to.

  5. Hi Tom,

    Your article inspired me to make this https://a8028076f2f562597256b6fac48c2c3eb7153d9c.googledrive.com/host/0B_UpD4GjuyxZRmF0SmNIaVh1dkE/On%20EA%20tools%20-%20narrative%20and%20dialogue%20mapping.pdf map to show my thinking about the relationships between the things that happen in a messy way in the early stages of the squiggle: dialogues/discussions & story building.

    I’m not sure if one tool can/should support all stages but for me Scapple, which I used to make this map, is a great and very simple paper-replacing tool to support my personal sensemaking in the first stages 🙂

    BTW: the map contains links to articles which are perhaps useful for your further thinking about the EA tool that rules them all 🙂

  6. I wonder if using an underlying Graph-Database would be a way to achieve “every-object-is-a-first-class-citizen” ?

    • Thanks, Graham. From what I’ve seen so far, graph-databases would support “every object connects as an equal citizen”, which is not quite the same as “is an equal citizen” – for that, we need a schema-less model, such as in a document-database. ArangoDB seems to combine both graph- and document-database, but I haven’t seem any common hosting that supports it. And graph-databases seem to use truly horrible query-languages – see Neo4J, for example. So it’s all a bit stuck at the moment. Again… 😐 Keeping moving pn the basic ideas, though.

  7. I think while a link could just be another (document) object like any other, we are a way from having an efficient platform to handle that. ArangoDB does look interesting.

    Do you think it would be fair to say that we are looking for a tool that allows every object to be, connect as, and be queryable as, an equal citizen?

Leave a Reply to Tom G Cancel reply

Your email address will not be published. Required fields are marked *

*