Tag Archives: OCR

Using Robotic Process Automation wisely – “If you are a hammer, everything looks like a nail” … but probably isn’t!

Recently Robotic Process Automation (RPA) was embedded in MS Windows 11. Although I’m happy to see such a great capability being added, I fear the incorrect application of this technology. Back in 2016 when the first RPA tooling came to market, I made an overview of technology capable of automating business processes. RPA is a solution in this area, but not the only solution. Out of fear of “if you a hammer, everything is a nail” or in other words “it is not because we can do it with RPA, we should do it with RPA”, some insights in the alternatives to RPA and guidelines around when to apply RPA and when not.

You can download the deck here:

End-to-End Business Process Automation (BPA) is nothing new. It has been around decades and comes in different flavors. The focus of this article is on automating a flow of activities across multiple systems. This to distinguish BPA from solutions that focus on one activity or that stay within one system. The latter are typical shortcuts or macro’s embedded in office tools or an integration API’s where one system invokes an activity in another system. The difference lays in the fact that a process by definition has a state. It knows about the sequence of activities that make up the flow, it knows the current activity under execution and it knows about the execution of previous activities that led-up to the current activity.

In the group of BPA systems when can identify two approaches with different characteristics and applications:

  • The Business Process Management Systems (BPMS)
  • The Robotic Process Automation Systems (RPAS)

What adds to the confusion is that most commercial products have become a hybrid between BPMS and RPAS but still it is good to understand the different approaches to business process automation.

Below a comparison table of typical use-cases and characteristics of RPAS-es and BPMS-es:

 RPAS
BPMS  
Products– BluePrism
– Automation Anywhere
– Power Automate
– K2/NinTex
– AgilePoint
– Windows Workflow Manager
Typical Use-Case– To integrate Line of Business (LOB) systems through the UI when there is no means to get information from the LOB system through a system-to-system interface .– The process is composed of activities executed in LOB system (CRM, ERP) and they can be targeted through system-to-system integration.
– The process requires human intervention and can be targeted to a human-to-system integration: writing a custom UI to handle the human input and link the UI through normal integration strategies with the process (web-service, messaging, DB). => Important: this human-process interaction is not UI-integration => SEE RPA.
Characteristics– Typically less building blocks and out-of-the-box integration components to build the full business process A-Z.
– Process complexity is limited to flow chart like flows. Not a lot of support for hierarchical or nested processes.
Processes are executed atomically from begin to end. Limited or no support for long-running processes that can be interrupted mid execution. The duration of an activity is at the level of magnitude of seconds.
Elaborate components to integrate LOB system at screen level: so screen scraping (visual pixel level) or screen spying (API widget ID level).
– Typically used when there are limited and simple activities in the process to automate, the process is organized to overcome the UI integration. => Important this UI integration is not human-process interaction => SEE BPMS.
Hard to make abstraction of the workflow and the systems it integrated with as UI is used for integration. This tight coupling through the UI requires a new flow per system that is integrated.
– Typically used when there are a lot of activities in the process and some complex logic to drive the process.
Processes can be interrupted mid execution (long running) waiting for an activity to complete. This interruption can be at the level of magnitude of hours, days, months, years.
– Typically have no components to integrate LOB system at screen level: so screen scraping (visual pixel level) or screen spying (API widget ID level).
Supports abstraction of the workflow from the systems it integrates with as technical interfaces are used for the integration (API/Web services). Loose coupling is used and can reuse flows as long as the system respect the same technical interface. This abstraction (partner management) is typically done by using middleware like a service bus (BizTalk).  

To summarize: RPAS’ weaknesses are BPMS’ strengths and vice versa

  • RPAS:

– Gaps in supporting all types of business processes and all levels of complexity.

+ Good components for UI integration.

  • BPMS:

– Gaps in UI integration.

+ Good support for all complex long running business processes.

So the solution is using a RPAS – BPMS combination?

Pro’s and Con’s:

  • Disadvantage: two licenses and two products
  • Advantages: optimizing the advantages of RPA and BPMS

Trade-off factors to make decision. Is UI-integration required and can no alternatives be found (through web-services, files, messages, DB)? Go for RPA. Is the business process complex? Go for BPMS

RPAS-es and BPMS-es also have touchpoints with other technologies. Some points of attention and advice here:

  • Optical Character Recognition (OCR) vs. RPA: scanning vs screen scraping, do not abuse RPA systems for OCR scanning!
  • Orchestrations vs. BPMS: atomic processes vs interruptible processes, do not use orchestrations for human-process interaction
  • Communication Bus (Messaging) vs. Orchestrator; atomic-requests vs. unit-of-work requests, do not use communication busses when collaboration between multiple systems is required to handle a request.

The distinction between BPMS-es and Orchestration Engines and the different approaches to system-to-system integration was not covered in this article but contains a lot of food for thought as well.

The Document Life Cycle in the Confusing World of Document Management and Record Management

In the digitalization race, companies struggle to correctly organize Document Management. Most of the time the struggle has to do with an inability to correctly identify what a document means in their business context, let alone define the management part of it and understand the difference with Record Management.

The goal of this article is to remove some of the confusion, to better define the roles of document and record management and to explain how these are related in managing the document life cycle.

The struggle has two main causes. First, the digitalization has turned documents from a paper carrier (physical form) into a digital equivalent. Companies fail to adapt and optimize their business processes. Most of time only the entry and exit activity are adapted: incoming paper documents are replaced by digital documents through scanning and outgoing digital documents are converted into a paper document through printing. An optimized business process should not care or be sensitive to the carrier but focus on the content or what is communicated within the document.

The content is the second cause. Companies struggle to distinguish Document Management from Record Management. The fact that both are intertwined enhances the confusion. Documents are used to carry business information in a way that it is consumable by people and systems. Records register business events to create evidence of a business activity. We often need documents to convey and exchange business events which makes things confusing.

What adds to the confusion is people seeing the physical document flow as an equivalent of the business flow and seeing the document as the synonym of the business event. Think for example about a purchase order. This happens because humans need business events being presented in an understandable way in order to interact with them. In business this is achieved through paper documents or, if things have evolved, electronic forms that represent structured data. Piling on top of the confusion, when the interaction is removed and the information is exchanged between systems, in an automated way without human intervention, business often keeps calling it the electronic document exchange. For example, a Purchase Order is called a document in Electronic Data Exchange. The confusion comes to an absolute high, when people understand that also unstructured data is communicated through documents. For example, the general conditions added along a Purchase Order

As shown below Record Management (RM) and Document Management (DM) have distinct characteristics but also a large area of joined characteristics. These are often combined into Data Management (DaM).

The disambiguation can start by saying that DM has an external focus i.e. how to communicate information between a company and its environment i.e. client, suppliers, government and other stakeholder. Whereas RM has an internal focus i.e. representing business events and storing these in and exchanging these between, Line of Business (LOB) systems. Mapping these concepts on a prototypical architecture diagram, the RM is mostly contained in what is depicted as the core services in the diagram and DM we can found in the satellite services around the core.

The exchange of data internally requires the introduction of another element of confusion. Systems that store business events are called Systems of Record. On the opposite site, there are the systems that use business events, these are called Systems of Reference. The organization and management of the data and data flows between Systems of Record and Systems of Reference is what Master Data Management (MDM) entails.

To link RM and DM, we created the following a high-level view on the Document Life Cycle as shown below.

This high-level view has a predominant DM focus. In the detailed view the interaction between DM and RM becomes clear as this shows the interaction points between DM and RM flow and activities. The RM related activities are encapsulated in the dashed rectangle and the interaction between DM and RM takes place in the Manage and Utilize phase of the document life cycle.

We have added a detailed explanation of all the phases and activities in the overview presentation you can download here:

The presentation focusses on resolving a lot of the confusion:

  • By defining concepts RM, DM, DaM, MDM, business record and business event
  • By distinguishing data carrier and content
  • By distinguishing paper carrier and digital communication
  • By distinguishing electronic and manual flows