On human ‘applications’ in EA models

In enterprise-architecture, how should we model a human-based ‘application’ such as a customer-service line or reCaptcha or Amazon‘s ‘Mechanical Turk‘?

[Note: on a first glance, this all looks really simple. As soon as we delve anywhere beneath the surface, though, it’s definitely non-simple and non-trivial, and that most of the terminology we need is either non-existent or at best highly misleading. Expect confusion here… if you still think it’s simple by the time we’ve reached the end of this post, you’re probably missing something.]

If we don’t have a clear means to describe something, it becomes very difficult to explain it to others, or engage others in creating it, or identify the problems and challenges that arise from it. Yet in this specific case – a ‘human application’ – most of our existing techniques and toolsets actively get in the way of describing what actually happens in those applications. What I’m interested in here is how we can develop usable workarounds for this problem.

Identify the problem

One of the aims of enterprise-architecture is to assist in modelling the linkages all the way from strategy to execution. A key part of this is modelling the relationships between a ‘customer-view’ of a business-process, the internal execution of that process, and the technologies and infrastructures that underly and support the process.

In enterprise-architecture and related fields, the usual means to model this would be notations such as UML, BPMN and Archimate, as enacted and displayed through some appropriate toolset.

UML and BPMN in essence focus primarily or solely on the execution – the ‘application’ itself – without much or any linkage to the customer-view and/or underlying technology. Archimate is probably the main notation that purports to cover linkages across the entire space.

Most enterprise-architecture frameworks describe the context-space in terms of ‘layers’. The exact numbers and labels of these layers vary from one framework to another, but at present almost all use some variant of the layering built-in to Archimate and TOGAF:

Business (the focus of TOGAF ADM Phase B: ‘Business Architecture’)
Application (the focus of Phase C: ‘Information-Systems Architecture’)
Technology (the focus of PHase D: ‘Technology-Infrastructure Architecture’)

In principle, these should line up well with the modelling-requirements above:

Business: ‘customer-view’ of a business-process
Application: execution of the business-process
Technology: infrastructure supporting the business-process

And in practice they do line up well – but only if the application is an automated process running on some form of computer-hardware. In neither TOGAF nor Archimate is there any means to describe an application that is executed by people or by non-IT infrastructure, other than by kludges that in essence force anything human (‘manual’) into the Business layer, and anything mechanical into the Technology layer.

[In my opinion the underlying definitions don’t hold together correctly in Archimate either, as can be seen from the fact that we suddenly have ‘Process’ in place of ‘Function’ in the Business-layer. If we instead use the definitions explained in my posts ‘Service, function and capability (again)‘ and ‘Service, function and capability – an addendum‘, Archimate’s ‘Function’ or ‘Process’ would actually be capability, and ‘Interface’ would actually be function. The linkage between ‘Business role’ and ‘Business process’ doesn’t make sense in real-world practice, because a process is actually a chaining-together of services; using ‘Business process’ as an arbitrary label for a subset or superset of ‘Business service’ is guaranteed to cause confusion, especially anywhere outside of IT. If you’re interested, there’s more detail on structural problems in the Archimate metamodel – and, in parallel, the TOGAF metamodel – in my post ‘Unravelling the anatomy of Archimate‘. But again, I emphasise, that’s just my professional opinion – nothing more than that.]

Similarly, there is almost no means in either TOGAF or Archimate to describe execution of a process (‘application’) that does not centre primarily or solely around the manipulation of information: relational-processes are implicitly forced into the Business layer, and anything physical into the Technology layer.

In service-oriented terms, there’s a very clear difference between the ‘external’ provider or consumer of a service – what Archimate would describe as a Business-role – and the ‘internal’ agent that enacts the service – which may be any combination of IT, human or machine, but which Archimate and the like assume can only be IT-software.

The end-result is that we’re forced into an arbitrarily-constrained conflation:

‘Business’ = anything-human + anything-relational + principle-based decisions + higher-order abstraction + intent
‘Application’ = anything-computational + anything-informational + ‘truth-based’ decisions + virtual (lower-order ‘logical’ abstraction)
‘Technology’ = anything-mechanical + anything-physical + physical-law (‘rule-based’) decisions + concrete (‘physical’/tangible abstraction)

In short, it’s a jumbled-up mess…

In practice, we can just about get away with this type of conflation if and only if our ‘enterprise’-architecture is concerned solely with IT-systems and information-based processes, and as viewed from an IT-oriented or even fully-IT-centric perspective. For any other type of processes, though, or when viewed from any other perspective – such as an ‘outside-in’ customer-journey – that conflation can be wildly misleading, and may well render it all but impossible to describe what needs to happen within the overall architecture, or to do like-for-like comparisons of process-architectures and process-designs.

This mess will in turn make it very difficult to describe, assess or compare business-models where viability of the business-model depends on processes that cross the relational / informational / physical boundaries, or can or may or must be implemented in multiple forms. Most business-models and complete business-processes do straddle across all of those boundaries: constraining the architectural view to a single subset of the context, or arbitrarily partitioning-off different segments of the context to separate ‘architectures’, is guaranteed to cause architectural problems or even architectural failure.

There are three key areas where these problems will be highlighted in architectural practice:

substitution – themes such as business-continuity, disaster-recovery, load-balancing and resource-dependent implementation, where alternate implementations may need to be used for the same nominal application
emulation – themes where human, IT and/or machine capabilities are hidden behind a ‘black-box’ interface (‘function’) that implies a different type of capability: for example, a ‘manual’ capability is used with the same interface as an automated one
escalation – themes where capability must be escalated across the ‘Inverse-Einstein boundary‘, from rule-based or algorithmic decision-making (which can be handled by machines or IT) to guideline- or principle-based decision-making (which generally can’t be handled by machines or IT)

(These themes will often occur in combination with each other, as we’ll see in a moment.)

Once again, there’s also the practical point that those ‘standard’ modelling-notations only work well from an IT-oriented perspective: when we change the perspective, we can sometimes end up either with a radically-different model-structure, or in some cases actively blocked from modelling at all.

So let’s look at three real-world examples: reCaptcha, Mechanical Turk, and customer-service technical support.

Example: reCaptcha

reCaptcha is an important extension of the concept of CAPTCHA – ‘Completely Automated Public Turing Test To Tell Computers and Humans Apart’. A Captcha is typically used as a defence against automated spam-bots and suchlike: for example, it would be embedded within a web-form, and users would be required to complete the Captcha before their comment or whatever would be accepted for posting to the site. The whole point is that it uses some test which humans are likely to find relatively simple, but which programs would be find very difficult or even impossible. A simple (if often too-simple) example is to solve an arithmetic sum; a more realistic challenge is to read distorted text from a blurry background.

The purpose of reCaptcha is to link this activity, with minimal extra effort, to solving another type of human-interpretation task: making sense of poorly-scanned text that OCR programs have been unable to resolve. A reCaptcha interface – see the active example on the Google reCaptcha website – presents two items to solve:

the conventional distorted-text Captcha – used for the actual Turing-test, returning a ‘true/false’ response to the host website
a single-word cleaned-up text-string from an unresolved OCR – used to improve OCR-interpretation of some other (unknown) document, and returned to the institution with the original source-document

Comparison of multiple reCaptchas with the same OCR-text can also be used to ‘vote’ on the most probable interpretations for particularly problematic OCR text-chunks.

The result is that, in reCaptcha, a site-specific Captcha is also used to improve the digitisation of old books and other documents in the public domain – a tiny bit of extra effort on site-users’ part that ends up helping everyone.

Business-oriented model

There are two distinct business-views here: that of the website-owner and website-user, connected via the web-form; and that of the reCaptcha provider and the digitising-institution, connected via the reCaptcha interface.

From the perspective of the website-user, the reCaptcha is an embedded part of an automated application. The web-form is merely one item within a (possibly multi-part) transaction that takes place between the website-owner and website-user.

From the perspective of the digitising-institution, the interpretation of OCR-chunks will likewise appear to be an automated application – but beneath the surface, the actual interpretation is done by a human, not an automated system. A more complete view of the digitisation-process would include a mix of automated and human interpretation, with including mechanisms to switch transparently between the different capabilities, behind an external ‘black-box’ interface which on the surface would probably appear to be entirely automated.

(The latter is similar to the process of sorting mail, which can likewise switch between several types of automation and human-interpretation, according to resource-availability, workflow, mix of preprinted versus hand-addressed mail, and so on.)

IT-oriented model (as per Archimate)

The focus throughout will be on the reCaptcha software-application and the underlying IT-infrastructure. All human actors – including the website-user doing the OCR-reinterpretation for the digitising-institution – will be modelled as Business-actor, and each assigned a Business-role linked to a Business-process.

Although the automated part of OCR-interpretation would be described in terms of Application-layer entities (Data-object, Application-service, Application-function, Application-interface, Application-component), the human part of OCR-reinterpretation via reCaptcha can only be classed as a Business-process, in the Business-layer.

Comparison

The website-owner / website-user relationship is fairly straightforward, and would probably be shown in much the same way from either a business-oriented perspective (customer-journey etc) or IT-oriented perspective. The only difference would be that an IT-oriented model in Archimate would probably show more detail in the Application and Technology layers.

The website-user / digitising-institution relationship is not straightforward: the views are very different from the website-user’s view (which could be presented well enough in an Archimate-type layering) and from the digitising-institution’s view (in which the capability provided by website-user would need to be ‘black-boxed’ as an ‘application’).

For symmetry, all the executing capabilities and functions should be shown in the same layer – in effect, as Application entities. A Business-role should be able to link to the respective Application-layer entities for ‘human’-applications, and likewise Device (the only Archimate entity that could represent a physical machine) link to Application-layer entities for physical/mechanical applications. However, such linkages are non-legal in Archimate: any toolset that performs strict syntax-checking on Archimate will not permit any such links to be created – making it impossible to model the application-relationships correctly.

Example: Mechanical Turk

Amazon Mechanical Turk (‘MTurk’) is like a more general-purpose version of reCaptcha: the human-based OCR-reinterpretation task would merely be one amongst a near-infinity of task-types that could be partitioned into MTurk ‘HITs’ (Human Intelligence Tasks). Its primary purpose is to provide an apparently-automated means to partition, manage and execute tasks which are partly or wholly on the ‘far side’ (non-automatable side) of the Inverse-Einstein boundary.

(In the SCAN sensemaking/decision-making framework, the Inverse-Einstein boundary is represented by the red-line on the horizontal-axis. In essence, anything to the left-side of that line can be handled by machines or IT-systems; anything to the right-hand side can usually only handled by real people, with increasing levels of skill and experience the further ‘rightward’ the context-needs go.)

Although all HITs are intended to be done by humans, the effective interface is explicitly designed to resemble an IT-type application, and – via a published API – is accessible directly by other IT-applications. Note, though, that there is nothing that inherently prevents HITs from being done by a software or even mechanical application: several viable implementations (i.e. substitution and/or emulation of the human element, as per above) have been made created and tested, although their actual use would be in breach of Amazon’s service-agreement.

There are two core roles: the Requester, who defines HITs; and the Provider, who executes HITs. There is no direct person-to-person relationship between Requester and Provider: all relationship is mediated and ‘black-boxed’ through MTurk.

Business-oriented model

From the perspective of the Requester, MTurk appears to be a conventional computer interface, which accepts HITs and returns the results of completed HITs. To the Requester, the Provider will seem to be solely a software-application (i.e. emulates software).

From the perspective of the Provider, MTurk appears to be a conventional computer interface, which provides HITs and accepts completed HITs. To the Provider, the Requester will seem to be solely a data-source software-application (i.e. emulates a structured database and task-management/workflow application).

IT-oriented model (as per Archimate etc)

The modelling will be almost exactly the same as for reCaptcha: all humans and human roles will be placed solely in the Business layer. Only the software components will be placed in the Application layer. There is be no means to model the actual ‘application-like’ relative relationships of Requester to Provider, and Provider to Requester.

Comparison

As for reCaptcha, the Business-oriented model and IT-oriented model are radically different. In Archimate, there is no means to show the effective symmetry (substitution and emulation) of software and human-intelligence in HITs, or the API or API-like interfaces in each direction between Requester and Provider: the relationships can only be shown in terms of how they are mediated by software.

Example: Customer-support

Customer-support is the means by which ‘external’ service-consumers can access information, advice and resolution on issues arising from the proposed, in-process or delivered service or product.

In practice, customer-support can take many forms, from self-service (brochures, FAQs etc) to pro-active delivery of advice (pre-sales, training and mentoring etc) to real-time guidance (interactive troubleshooting, help-line etc) to off-line maintenance and repair.

From an ‘outside-in’ customer-journey perspective, the ideal is that all these forms of service should interweave seamlessly from one type to another, with clear paths of action and escalation as required. The reality, however, is often far from that ideal: the various forms of customer-support service are functionally and operationally separate from each other, with the end-customer often forced to ‘fill in the gaps’ for action and escalation. This is usually the result of taking an ‘inside-out’ (organisation-centric) rather than ‘outside-in’ (customer-centric) perspective on customer-support: “customers appear in our processes”, rather than “we appear in customers’ experiences” (to paraphrase Chris Potts’ dictum).

Since consistent, reliable and seamless customer-service provides distinct competitive-advantage, and is desirable in its own right for many other operational reasons, it will be useful to model not just the various forms of customer-support, but also the inter-relationships between them, so as to improve the overall provision of customer-support.

Business-oriented model

This model would need to show all types of customer-support, and their substitution and escalation relationships with each other.

A service-oriented form of modelling is probably the only approach that would make consistent sense. The core ‘external’ business-role is the Customer, in relationship to the various forms of customer-support service-delivery. This should be the only role shown at the ‘Business’ layer: all other human roles are at the ‘Application’ layer. In effect, each form of service-delivery is an ‘application’, in which the actual underlying method of delivery – human, IT or physical-object – should be largely transparent, behind a consistent ‘black-box’ type interface.

The customer seeking support should not be forced to identify which type of service-delivery best fits their needs, and especially should not be forced to exit and re-enter a different part of the overall customer-support system just to obtain a different type or escalation of advice – as is very commonly the case in many existing ‘support’-services. One of the key roles of modelling is to find ways to eliminate support-problems such as:

FAQ with no means to escalate other than a useless (to the customer) ‘Did this answer your question’ form
array of links to different parts of the customer-‘service’ system, with no means to identify which item applies to the customer’s need
multi-layered ‘Press 1 / Press 2’ call-option hierarchies that take far longer than a human operator to direct to the required need, and that force the customer to memorise the entire hierarchy in order to work down to an alternate branch
different phone-numbers for different escalation-levels or service-types, with no means to switch between them within the system, or to pass needed context-information to the next agent
high ‘failure-demand’ created by lack of crosslinking between

The practical problem is that almost all existing business-systems – especially IT-related systems – are built on a Taylorist model, which assumes repetitive processes delivered in linear or ‘controllable’ manner by distinct specialist functions in response to predictable events. The reality is that almost none of those criteria apply to customer-service: each enquiry is in some way unique, it rarely follows a linear path within or between ‘processes’, there is more emphasis on cross-functional generalist knowledge to link between specialists, and many of the events are inherently unpredictable. To resolve problems with customer-support, it’s therefore essential to break out of that ‘linear’ mindset – and to model accordingly, to keep track of substitution and escalation, and to bring those transitions inside the organisation’s overall customer-service system.

IT-oriented model (as per Archimate etc)

A conventional IT-oriented of a customer-support system will show only the parts of customer-service that are delivered and/or mediated via IT. Non-IT support facilities such as a direct (unmediated) person-to-person phone-line or a ‘Quick Start’ brochure would be excluded from view.

All human roles such as Customer’ and ‘Call-Centre Service-Agent’ would be placed in the ‘Business’ layer. Relationships between them would be described solely in terms of the IT-applications and IT-infrastructures that mediate between them, as described in the ‘Application’ and ‘Technology’ layers.

Since only IT-based applications can be portrayed in the Application layer, there is no direct means to describe substitution or escalation of service. Escalation to human-delivered service – as becomes almost inevitable in any transition of the problem-context to the ‘far-side’ of the Inverse-Einstein boundary – can only be portrayed as a transition to the Business layer; likewise any de-escalation such as a return to a predefined FAQ can only be described in terms of a transition to the Application layer. Delivery of person-to-person service (‘human-application’) can only be described in terms of any intermediary IT-systems, not the service-delivery itself; substitution (delivering the same nominal service via different agents – human, IT or physical-object) can only be described as a form of escalation or de-escalation, with human-applications in the Business layer, IT-applications in the Application layer, and physical objects (brochures etc) describable only via a kludge as ‘Business-object’ entities in the Business layer or ‘Artifact’ entities in the Technology layer.

Comparison

A conventional IT-oriented view of customer-service aligns with a business-oriented of the same service only in a very thin sense of ‘align’. The arbitrary constraints on its view make it impossible to see the actual relationships between the various components of the overall customer-support system. Many of the components of the overall system end up in the ‘wrong’ places – i.e. portrayed in an inappropriate layer or with an inappropriate entity – or cannot even be portrayed at all.

In short, an IT-oriented view of customer-service will inevitably be incomplete and, for most business purposes, seriously misleading. In practice, an IT-oriented view on its own will make it almost impossible to make sense of what actually happens in overall delivery of customer-service: so problematic and so misleading that it can easily be worse than useless. An IT-oriented view should only be used with – and explicitly linked with – a broader-scope business-oriented view of customer-service.

Questions and suggestions

The obvious question is: what do we do about this? Most the tools and notations that we have at present in ‘enterprise’-architecture will support only an IT-oriented view: and it should be obvious from the above that such tools are often problematic – to say the least. But if those tools are all that we have available to us, then we can’t simply sit back and complain about it: we need workarounds that work.

In Archimate and suchlike

The practical problem here is that Archimate is essentially IT-oriented, and does not really recognise anything that exists independent of and/or has no direct interaction with IT. Of other common notations, UML in principle is capable of being used at a broader scope, but in practice only at an abstract level that makes it very difficult to describe anything than an IT-implementation; and whilst BPMN can describe substitution, it can do so only via labelling of ‘swimlanes’, again making it difficult to compare implementations.

As mentioned above, Archimate’s formal syntax explicitly prevents us from doing some needed entity-relationships, such as positioning Business-actor or Business-role in the Application layer. Hence one option would be to disable formal syntax-checking in Archimate, or to use a simpler diagramming-tool such as Visio that ignores or doesn’t even know about proper repository-based modelling. Whilst in some cases there may be no other option, the disadvantages of the Visio-style route are such that for any kind of disciplined modelling it’s an option we should avoid wherever practicable.

However, many toolsets such as BizzDesign and Sparx Enterprise Architect do support a ‘profile’ mechanism, to extend the nominal capabilities of Archimate and other notations. It’s therefore possible to add new entities and relations to those notations, which would resolve some of the problems above. The catch is that, by definition, all such extensions are ‘non-standard’, which means that it becomes to exchange such models with other toolset-users; and it still doesn’t resolve the spurious pseudo-layering of Business, Application and Technology that, in Archimate’s case, is built into the very core of the language.

Personally, I don’t know how to use those ‘profile’-mechanisms, but there are plenty of other people who do: Bas van Gils at Bizzdesign, for example, or Alex Yakovlev or (I think?) Adrian Campbell for Sparx. It’d still be somewhat of a kludge, but at least it is doable, anyway.

Beyond Archimate

Thinking beyond Archimate, probably the best option would be to take a more service-oriented approach to modelling.

Everything outside of the blue region (‘this organisation’) is the equivalent of the overall ‘Business’ layer in Archimate; the blue region itself is Business-service; ‘Supplier’ and ‘Customer’ (service-provider and service-consumer) are example Business-roles. The outer edge of the blue region is the Business-interface, or, in more service-oriented terms, the Business-function. (See the note above earlier about apparent inconsistency of definitions and entity-roles in Archimate.) A Business-process encapsulates the sequence of activities and events across the supply-chain; a Business-object is something of business-value that is exchanged over the supply-chain.

(Note that Business-actors and Business-roles connect to the service not only across the supply-chain, but also via vision and values, and thence to the service’s value-proposition – hence the fundamental importance of the Motivation extension in Archimate.)

In Enterprise Canvas service-modelling, the internals of the service itself would be described in terms of the service-content checklist:

To align with the way Archimate does things, the closest equivalent of the content for the ‘Application’ layer would be the functions and capabilities columns in the checklist – for Application-interface and Application-function respectively; and the closest equivalent for the ‘Technology’ layer would be the content of the assets column. The Archimate Location entity would align with (and be expanded to) the content for the location column; and Business-event would be generalised to all layers to align with (and be expanded to) the content for the events column. Other than some aspects of the Motivation extension, there is no direct equivalent in Archimate for the content of the decisions column.

(By the way, note that the rows represent dimensions, not isolated primitives: almost all real-world entities will incorporate composites of multiple dimensions down the columns. For example, a printed-book is both a physical-asset – the book itself – and a virtual-asset – the information in the book – and must be managed in accordance with the effective rules for both asset-dimensions.)

So far so good, except that it’s essential to note the three distinct sub-columns for capabilities:

action – what type of assets the capability acts on
agent – the type of asset through which the capability is enacted
skill-level – the decision-types that can be applied within the capability

In Archimate and other IT-oriented notations, the action can only act on virtual-assets (information etc); the agent can only be virtual (IT-application); and in most cases, the skill-level can only be either rule-based or algorithmic, over on the left-side of the Inverse-Einstein boundary. To make sense of the broader contexts we’ve explored here – such as reCaptcha, Mechanical Turk and customer-support – we must be able to describe the full range of capability-types, asset-types, function-types and so on. A kind of much-expanded Archimate should be able to cover the required scope.

Anyway, stop there for now: over to you for your suggestions as to how best to handle this?