Short Description

The Workflow Building Block drives efficiency within GovStack by providing automation and orchestration capabilities for specified business processes within and across Building Blocks. The Workflow Building Block provides design-time mapping & modeling of business processes based on mature open standards like Business Process Model and Notation (BPMN) and facilitates the run-time execution of deployed workflows in order to orchestrate process flows from initiation to completion.

Full Description

The Workflow Building Block helps automate and control the flow of information and activities within and across various services based on predefined protocols. Workflow “processes” involve “steps” (things to be done), “gateways” (conditional logic), and “events” (things that happen). These processes may guide software systems to automate data exchange based on certain events or conditions. In addition to accelerating and automating information flow, it can be used as a mechanism to encourage or enforce best practices such as adherence to data standards, clinical guidelines, and policy.

It is important to note that from the perspective of the Workflow Building Block, any process may be defined and may make use of APIs across several applications and ministries (so long as it has access to those APIs via the Information Mediator).

From the high-level view, we should recognize that some instances of a Workflow Building Block may be deployed, for example, in the Ministry of Health (MoH) and only access services in the MoH, but another may be deployed in the MoH and access services provided by the Ministry of Insurance (MoI). So long as MoH’s Workflow Building Block is authorized to access the service at MoI (via the Information Mediator) then it will work just fine.

When to use a workflow engine

It should be noted that candidate applications playing the role of the Workflow Building Block are not always responsible for controlling the logical flow of data or automation of business processes.

There are many cases where a Building Block application will make a request to another GovStack service (via the Information Mediator) or to an external service (via an API gateway, for example) which does not involve workflow at all.

There are also plenty of cases where some slightly more complex automation will be a natural, already existing, in-built feature of some other application that's playing the role of, e.g. the Messaging specification. Let's consider an example.

1. The messaging spec might require that candidates be able to send SMS messages.

2. The Messaging Building Block might not require that all candidates be able to send a message, wait for a reply, and then send a canned response based on the reply.

3. But the logic defined in point 2 is so common in the messaging world, it would not be surprising to find that many Messaging Building Block candidates provide this type of business logic automation "out-of-the-box".

The only caveat to point 4 is that certain "built-in" business logic automation provided by other applications might be avoided if you're opting for a GovStack implementation style in which a very large portion of interactions are catalogued in the audit trails generated by independent security servers. Then, even though the "built-in" logic is more convenient and affordable, you'd opt to activate a workflow process stored in a workflow engine candidate to generate a more robust audit trail.

Example

The “Process Visit & Request Child Counselor” process is implemented using a Workflow Building Block installed at the Ministry of Health.

1. Receive webhook event from clinic system with a new patient visit.

2. Make GET request to another Ministry of Health API to retrieve full patient data.

3. Find patient date_of_birth in the response to the above request, calculate age from date of birth.

5.1 Requirements

The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the architecture specification document.

There are no required internal workflows for candidate applications that implement the Workflow Building Block specification. Below is an example of how an internal workflow might run when responding to an API request.

9.1 Start a workflow process via API

To satisfy the initial 6.2 functional requirements a candidate application may implement the following internal workflow.

The following Functional Requirements are grouped by the Key Digital Functionality that they support.

6.1 Workflow Process Building

• Must allow the design of business processes via User Interface, i.e. support creation of diagrams as it is depicted in the in section 7.3 (“BPMN Diagram Types”) with the subsequent possibility to generate from diagrams XML and/or JSON version of process definition (REQUIRED)

• Must allow users to define/execute processes using BPMN v.2.0 specification basic categories of elements such as Flow Objects, Data, Connecting Objects, Swimlanes, and Artifacts (REQUIRED)

• Should allow users to define/execute processes using BPMN v.2.0 specification Extended Modeling Elements as it is depicted in the specification section 7.2.2 (RECOMMENDED)

• Should allow 5 types of decision gateways: Exclusive Gateway, Event-Based Gateway, Inclusive Gateway, and Parallel Gateway (RECOMMENDED)

• Must allow the configuration and execution of an arbitrary block/script/program/component during a business process (REQUIRED)

• Must allow facilitating the development of an arbitrary script (to be used for things like message translation) (REQUIRED)

• Must allow “embedded sub-processes” which depend completely on the parent process and are not reusable; however, all process data used in the parent process is directly accessible by the embedded sub-process (REQUIRED)

• Must allow “reusable sub-processes” which do not depend on the parent process; they are typically modeled or designed separately so that they may be used in multiple contexts; this type does not necessarily have access to the parent's data by default; explicit mapping of attributes is often required to achieve data sharing between parent and sub-process (REQUIRED)

6.2 Workflow Process Execution

• Must be able to execute a business process (REQUIRED)

• Should be able to execute a process when given a valid BPMN-compliant workflow specification (RECOMMENDED)

• Must allow the initiation of an asynchronous business process via API and respond with an appropriate HTTP response code, e.g., 202/Accepted (REQUIRED)

6.3 Status Monitoring

• Must allow an administrator to view and manage all workflow processes (both active and inactive) from their interface (REQUIRED)

• Must provide a monitoring dashboard that allows an administrator user to view the existing processes and instances, the status of all instances, and the health of the system generally (REQUIRED)

6.4 Programmatic Process Discovery

• Must allow an administrator to “enable” or “disable” specific processes which will then appear in the BB API spec: “/api/process” LIST for discovery (REQUIRED)

6.5 Other Functionality

• Must have an OpenIAM-compatible API for user/privilege provisioning (REQUIRED)

Internal Function Blocks

The following are the internal key functionalities/capabilities that orchestrate the functionality of the implementation of the Workflow Building Block:

Flow Repository is the place where flow definitions are stored. This repository can be linked to related components, like that validate/parse flow definitions comply with open standards like Business Process Model and Notation. The repository is a general term that refers to the storage place of flow-related components of a flow definition including tasks, processes, decision gateways, etc.

Metadata Store refers to the storage of data that an Execution Engine needs to associate with a defined workflow to facilitate successful execution. This data includes but is not limited to input variables, constants, mathematical and/or logical functions, data objects, or any important data that gives context to a given workflow.

Trigger Manager facilitates the bootstrapping or initiates the instantiation of a workflow for execution by the Execution Engine. This component should be aware of different ways that a workflow can be triggered, e.g. through an API call, a timer, etc.

Execution Engine is responsible for running/executing a workflow and providing the results of the run. This component may at the very minimum execute valid atomic logical expressions defined in each workflow task or in an advanced scenario may be linked to intelligent machine learning-related capabilities that may provide recommendations or suggestions for improving workflow design.

Timer is responsible for the scheduling and initiation of time-based workflows. This component may also be responsible for the monitoring of execution time of given workflows and handling of time-out scenarios for workflows that may exceed the set maximum execution time.

Workflow Designer is a graphical user interface or editor that may be used by Process Designers to build workflows and have them saved in the Flow Repository. This interface must be intuitive, easy to use, and able to facilitate common CRUD-like functionality which includes: creating new workflows, listing saved workflows, updating existing workflows, and deletion of workflows. The designer may also facilitate imports of workflows from external sources, supporting open formats like Business Process Model and Notation.

Web API is an important component that should be designed to manage all interoperability requirements for the Workflow Building Block. This API layer should facilitate the remote initiation of workflow processes from external systems and also the exposure of workflow-related data (design time and execution time) to facilitate reporting by external systems when needed.

Batch Queue Manager is responsible for the queuing and batch processing of workflows. It is assumed that at any given time, multiple workflows may be active and may be in execution mode. Hence a queue manager that may have multi-threading capabilities may support the execution of a large number of workflow processes or tasks in real-time yet providing the required execution isolation to ensure the integrity of each workflow.

Developed by Khaled Ben Driss (Wevioo), Farai Mutero (HISP South Africa), Comfort Mankga (HISP South Africa), Dr. P. S. Ramkumar (ITU), Aare Lapõnin (Independent), and Taylor Downs (OpenFn)

7.1 Resource Model

The full resource model is described in detail in the API Specification, but the following is a high-level diagram of the key entities.

7.2 Data Elements

The data elements listed below provide detail for the resource model defined by the swagger specification. This section lists the required fields for each resource. The data elements listed are extensible in order to respond to specific use cases, however, the data elements listed below are a minimum requirement.

7.2.1 Process

Description: Represents a process that has been successfully created in a Workflow engine. This object contains important reference details like the generated process UUID (universally unique identifier), which is required to perform further operations on the process.

7.2.2 ProcessDefinition

Description: Reference to a blueprint of a process. Contains all the fields required to create a new process for a workflow engine.

7.2.3 ProcessInstance

Description: Used to list a single or list of process instances that are in various states like active, suspended or stopped, in a Workflow engine.

7.2.4 ProcessInstancePayload

Description: Initiates a process execution or creates a process instance. Contains the ability to capture input values that may be required to start a new process instance.

7.2.5 ListOfProperties

The Workflow Building Block enables the creation and automatic execution of business processes. Any application used to implement the Workflow Building Block specification must provide the following Key Digital Functionalities.

4.1 Workflow Process Building

Workflow Building Block candidates must allow business users to create arbitrary workflow processes. These processes must be able to perform calculations, make API requests (e.g. Hypertext Transfer Protocol Secure, HTTPS) make HTTP requests, and execute scripts.

This section provides a reference for APIs that should be implemented by this Building Block. The APIs defined here establish a blueprint for how the Building Block will interact with other Building Blocks. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block.

The GovStack non-functional requirements document provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here.

All APIs will be defined using the OpenAPI (Swagger) standard. The API definitions will be hosted outside of this document. This section may provide a brief description of the required APIs. This section will primarily contain links to the GitHub repository for OpenAPI definition (YAML) files as well as to a website hosted by GovStack that provides a live API documentation portal.

This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here.

The tests for the Workflow Building Block can be found in .

8.1 Workflow Processes

List processes

• Retrieves the list of workflow processes deployed on the workflow engine.

• Provides for each process the information: definition ID, version, status (active/suspended).

Get process definition

Instantiate (start) a process instance

• Instantiates a given process definition Id. Responds with instance Universal Unique Identifier.

• Process variables (just a PAYLOAD object - process, should be able to extract from the payload JSON object attributes) may be supplied in the request body.

• If the start event has mandatory variables, the workflow engine will perform backend validation.

8.2 Workflow Instances

List process instances

• Retrieves the list of running process instances for a given workflow process definition ID.

• Get Instances workflow process by ID (GET) /instances?processId=123

Get the status of an existing process instance

• Retrieves the status of a single process instance given an instance ID.

Stop a running process instance

• Stops execution of a running process instance

Service APIs List

1. List processes. (GET) /processes

   • Retrieves the list of workflow processes deployed on the workflow engine.

   • Provides for each process the information: definition ID, version, status (active/suspended).

Overview

The Workflow Building Block is generic, flexible, and un-opinionated. These aspects are crucial, as they enable many different kinds of automated processes to be designed and executed. Fundamentally, every activity in a process may itself be another process—which includes multiple activities, and so on ad infinitum.

Bearing this in mind, it is still useful to have shorthand terminology to refer to a collection of activities, or a single activity relative to some linguistic scope that’s being applied in conversation.

To achieve infinite “nestability” and enable conversational coherence for human beings, we have settled on two terms to refer to:

• activity: a step, relative to some multi-step workflow.

• process: a multi-step workflow itself, or a collection of steps.

Process

A business process is defined as a set of one or more linked activities that collectively realize a business objective. A single process may have branching logic based around “gateways” which automate decisions. Alternatively, whichever actor initiates a process may decide to initiate different (related) processes depending on different conditions or inputs.

Asynchronous Process

An asynchronous process is the default process type for the Workflow Building Block, and once instantiated will return a 202/Accepted but then continue to execute the instance. Note that many asynchronous processes may include requests back to the initially requesting application, but these are new requests, not responses to the initial request.

Synchronous Process

A synchronous process is a process that, once instantiated, will complete before returning a response to the request. The process instance is blocking and should be used sparingly.

Example Processes

Ex. Process “A”

1. Activity 1: jump in water

2. Activity 2: swim 1 mile east

3. Activity 3: get out of water

Ex. Process “Registration”

1. Activity 1: turn off light

2. Activity 2: execute “Process A”

3. Activity 3: multiply 7 * 5

Activity

An “activity” is an atomic task or step in a process. An activity is created when the task cannot be broken down to a finer level of detail. Generally, an application (or person) will perform the activity when executed. There may be many different types of activities.

For example, note that an activity may be another “Process” in its entirety (N.B., in Camunda this is an “activity”, in OpenFn this is an “operation”, elsewhere it is a “step” in a process).

Activity Types

User-driven

A “user-driven activity” is an activity that represents the completion of some unit of offline work, done by a human being. If a user-driven activity is used in a process, the _next _activity in the process will not be initiated until the activity is marked as completed, potentially with some output data captured to be used in subsequent activities.

Service-driven

A “service-driven activity” is an activity that is completed by a machine.

Example Activities

Ex. User-Driven Activity 1:

Ex. Service-driven Activity 2:

Instance

An instance, or “process instance” is the unique thread of execution of a process. It has input data, a start time, an end time, a log, an exit code, and other attributes. There may be N number of instances for a given process.

Consider a “Register Newborn Child” process and consider that 50 babies are born today. We may see 50 instances of this 1 “Register Newborn Child” process (some instances may still be running, some may have been completed, some may have failed).

Ex. Instance “061816db-221b-7ddf-a69c-d2b3e53f6094”

Ex. Instance “061816d9-f59c-7ae5-a2e5-a2a2b4b3bc0e”

Process: “Registration Process”

Gateway

Decision gateways are flow-control elements, which are used to control how activities interact as they converge and diverge within a process. Entered (or “triggered”) by activities, a gateway acts as a function that decides which (outgoing) path to follow based on the result of the evaluation of the given set of conditions. A gateway enables the implementation of branching, forking, merging, and joining of paths in a business process diagram.

There are different types of Gateways. The most used is the “exclusive gateway”, which is used for creating alternative paths within a process flow. For example, exclusive decisions, or sequence flow looping.

Another type is the “inclusive gateway” where all condition expressions are evaluated. Since each alternative path is independent, any combination of the outgoing paths may be taken.

The Gateways do not represent work that is done and they are not expected to have any effect on the operational measures of the process to be executed (costs, time, etc.).

Exclusive Gateway

An exclusive gateway is a point of diversion for a business process flow. For a given instance of the process, only one of the paths can be taken. An exclusive gateway may be used to achieve “looping” or “iteration” because if a condition is not met, the “next step” in the process may be another part of the process that provides a “way back” to the gateway in question after other activities are completed.

Inclusive Gateway

Unlike the exclusive gateway, an inclusive gateway may trigger the execution of more than one out-going flow. Since an inclusive gateway may trigger more than one out-going flow, the condition-checking process is a bit different from the exclusive gateway. Under an inclusive gateway, all the out-going conditions will be evaluated.

Event-based gateway

The Event-Based Gateway represents a branching point in the Process where the alternative paths that follow the Gateway are based on Events that occur, rather than the evaluation of Expressions using Process data (as with an Exclusive or Inclusive Gateway).

A specific Event, usually the receipt of a Message, determines the path that will be taken. Basically, the decision is made by another Participant, based on data that is not visible to Process, thus, requiring the use of the Event-Based Gateway.

Parallel Gateway

A parallel gateway is used to visualize the concurrent execution of activities and is used in pairs. When the process arrives at the “parallel gateway node”, all the outgoing flows exhibited from the gateway will be executed simultaneously. The flow will be merged at the “joining parallel gateway”.

State

State is a JSON object which contains the required data (or input values) for a process to begin execution. When a process is instantiated (we create a new “instance”) in many cases, the initial “state” will be provided to the instance. State should be a JSON object, and activities in the process should be able to make use of this initial state—i.e., to work with “variables” that may change across each instance for the process.

Consider a process that contains an activity to “send an SMS to a patient”. The process definition may dictate “send SMS to state.patient.ssid” and when the process is instantiated with the initial state from patient A’s clinical visit, the recipient of that SMS will be different from when the same process is instantiated with the initial state from patient B’s clinic visit (i.e., one process with two instances and SMSs sent to two different people).

Below, the same process (Process 1) sends an SMS to two different people depending on the initial state it receives from in the triggering webhook event.

E.g.: Start the beneficiary scoring for beneficiary X:

Make POST to /API/workflow/beneficiary-scoring-process with the following body:

The process should be built with knowledge (in so far as it’s needed) of the initial state that will be provided when it is called/a new instance is created.

The final activity in this process might be:

MAKE A POST to `${state.callbackUrl}/${state.id} with BODY { “score”: state.finalScore }

Event

An event is something that “happens” during the course of a process. Events affect the flow of the process and usually have a cause or an impact and in general require or allow for a reaction. The term “event” is generally enough to cover many things in a process. The start of an activity, the end of an activity, the change of state of a document, a Message that arrives, etc., all could be considered Events.

Types:

• Start event.

• Intermediate events.

• End event.

Business Process Model & Notation (BPMN)

At multiple points in this document we refer to BPMN and many elements of the terminology section are terms borrowed directly from BPMN. See a list of all standards we use in the “Standards” section, and view the .

Other terms to define

These terms may be defined in later versions of the specification. Proposals are welcome.

Sequence flow

A sequence flow is used to connect flow objects in a process or choreography to show the flow. A sequence flow is used to connect flow elements. It is shown in a solid line with an arrowhead. It shows the order of flow elements.

Message flow

Message flow is used to show the flow of messages between separate pools/lanes. You cannot use message flow to connect flow objects within the same participant.

Process model

A process model, often depicted using Business Process Mapping Notation (BPMN) is a design output that depicts the steps of a business process from end to end.

Token

A token is an abstract concept in BPMN. A token, formally referred to as a process token, refers to the current activity being executed within a process instance. A business process can have multiple tokens that indicate that the process is running in multiple paths. For example, gateways are often used to split the path of a process. Splitting a process path creates multiple process tokens.

Note that a single token will have at least three attributes: id, processInstanceId, and activityId. If a gateway is used to “split” a process, there may exist multiple tokens per process instance.

Pool

A pool in Business Process Model & Notation (BPMN) represents a participant in business collaboration. In a BPMN model (or diagram) a pool can represent an entity like a company, a role (e.g a buyer, seller, or customer), or even a system (e.g., OpenMRS API).

Lane

A BPMN Lane is a sub-partition within a Pool (see above) which extends the entire length of the Pool, either horizontally or vertically. Lanes are used to organize and categorize Activities within a Pool. In practice, lanes are commonly used for allocating activities to roles, systems, or the organization’s departments.

User Interfaces Examples

Example user interfaces provide illustrative context for common functionalities.

Building a process

Adding arbitrary scripts to a process

Adding new activities to existing processes

Adding credentials for use in a process

Configuring an HTTP request as an activity in a process

Adding a conditional gateway to a process

These user interface examples are meant to aid reviewers and developers in understanding the high-level requirements of the Workflow Building Block, not to specify certain design elements or guidelines.

1. Link to architecture requirements document (and specific sections within that document, such as cross-functional requirements, and general recommendations).

2. Link to use cases document – this document may be a valuable resource while developing workflows to ensure that a variety of different use cases are covered by the Building Block definition.

3. Link to the Building Block criteria and maturity metrics document created by Tanvir.

10.1 Key Decision Log

. ​

10.2 Future Considerations

.

10.3 Out-of-Scope Assumptions

.

User Journeys and Use Cases

The WFbb is present in multiple use cases. Use cases can be found in the and a subset of those that we focus on below can be found . These user journeys and use cases have been referenced by the authors of the specification to ensure that the functional requirements for the spec suit the needs of the proposed implementations.

Postpartum and Infant Care

Unconditional Social Cash Transfer