A Process Mining Software Comparison

07/28/2020
by   Daniel Viner, et al.
FAU
0

www.processmining-software.com is a dedicated website for process mining software comparison and was developed to give practitioners and researchers an overview of commercial tools available on the market. Based on literature review and experimental tool testing, a set of criteria was developed in order to assess the tools' functional capabilities in an objective manner. With our publicly accessible website, we intend to increase the transparency of tool functionality. Being an academic endeavour, the non-commercial nature of the study ensures a less biased assessment as compared with reports from analyst firms.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/15/2019

Process Mining for Python (PM4Py): Bridging the Gap Between Process- and Data Science

Process mining, i.e., a sub-field of data science focusing on the analys...
06/04/2019

A Comparative Study of Milestones for Featuring GUI Prototyping Tools

Prototyping is one of the core activities of User-Centered Design (UCD) ...
07/09/2021

Can Commercial Testing Automation Tools Work for IoT? A Case Study of Selenium and Node-Red

Background: Testing IoT software is challenging due to large scale, volu...
01/28/2021

Peptipedia: a comprehensive database for peptide research supported by Assembled predictive models and Data Mining approaches

Motivation: Peptides have attracted the attention in this century due to...
02/09/2019

An Expert Module Design and Implementation that Evaluation the C&K Metric Suite in Terms of Object Oriented Software Quality

Object-oriented software has some features such as encapsulation, inheri...
09/07/2018

Dynamic Mutant Subsumption Analysis using LittleDarwin

Many academic studies in the field of software testing rely on mutation ...
07/02/2019

GTIRB: Intermediate Representation for Binaries

GTIRB is an intermediate representation for binary analysis and transfor...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Intro

Starting in the late nineties as an academic research project, the discipline of process mining enjoys an increasing penetration in various industries over the past few years [1]. Process mining helps organisations leverage event log data stored in databases or IT systems with the objective to discover, monitor and enhance processes [2].

The diversity of real-world applications is exemplified by the use of process mining software in banking, manufacturing, online gaming, healthcare, public service and many more industries [3]. Use cases include compliance checks, continuous process improvement (CIP) and the assessment of robotic process automation (RPA) initiatives, to name a few. With the rise of use cases and continuous adoption of process mining in various industries, various commercial tools have emerged on the process mining software market. The changing dynamics of the software market is marked by the increasing number of solutions, continuous releases of new features and acquisitions. Looking forward, the global process analytics market, which includes the discipline of process mining, is expected to grow at a rate of around 50% annually from 2018 to 2023 to reach USD 1.42 billion by 2023 [4].

Depending on the scope and intended scalability, process mining initiatives may require high investments in terms of cost and stakeholder involvement, thus underscoring the danger of selecting the wrong tool. Considering the academic context, process mining researchers are often not fully aware of practitioners’ needs and the developments in the software market. An overview of available tools and their capabilities is important to address these issues. While several analyst firms such as Gartner [5] published market studies that deliver an overview of the software landscape, we provide a more detailed analysis of process mining software with tangible criteria that examines functional capabilities. We conducted a non-commercial process mining software analysis and published the results on www.processmining-software.com. Besides serving as an independent software selection support for practitioners, the website also intends to help researchers understand state-of-the-art in practice, allowing to evaluate the usefulness of their work in regards to practical utility.

Based on literature review and experimental software testing, a set of criteria was derived in order to compare the features and functional capabilities of process mining software. This paper describes the underlying methodology and presents nine criteria categories with a brief description for each category and criterion.

Ii Methodology

Ii-a Software Selection

In order to ensure a comprehensive and representative listing, the most recent process mining-related reports of three analyst firms were taken as a basis to identify relevant software. Reports from Gartner [5], Everest Group [6, 7] and Forrester [8]

were analysed accordingly. Taking into consideration all software vendors stated in the commercial reports, a list of 34 potential tools was derived and further refined in three steps. First, three vendors not granting access to a demo environment were excluded from the study as we did not want to rely on information provided by the vendors. As the reports do not exclusively cover process mining software but also software of related disciplines such as task mining or documentation, the respective tools were identified and excluded from the analysis in the second step, reducing the number of relevant vendors to 19. Third, three open source tools were neglected. The ProM framework offers a comprehensible library of scientific techniques and algorithms, but is geared towards academic scholars. PM4Py is an open-source Python library that currently does not provide a graphical user interface, making the solution difficult to use in the organisational context. Similarly, we excluded Apromore but due to the recent release of a commercial edition, we will consider it in the second testing cycle. Finally, 16 tools were tested, see Table

I.

Ii-B Evaluation Criteria

In order to create a list of relevant criteria, a two-sided approach was followed. First, a literature review was undertaken to identify potential criteria from previous studies. The academic search engine sites WorldCat, SpringerLink and Google Scholar were searched by the following terms: “process mining software”, “process mining software comparison”, “process mining tools”, “process mining tool comparison” and “process mining criteria”. Second, the software was tested upfront in an experimental manner to better understand what features and capabilities the vendors offer. Vendors were asked to grant access to all features in the demo environment to ensure all available features can be explored. The experimental approach also included the screening of all available knowledge bases and product documentations made accessible by the vendor. The derived criteria set was applied in three steps. In Phase 1, a test scenario was conducted for every tool using the same logs and files. In Phase 2, the results were compared with each other to identify inconsistent terminology and discrepancy in the level of detail. The final assessment was conducted in Phase 3. After testing, follow-up workshops were conducted with every vendor to clarify open questions and to get additional context for features. The exchange with the vendors also served as a quality gate for the correctness of the test results.

Ii-C Testing Setup

The software testing was conducted primarily using event logs of Purchase-to-Pay (P2P) processes with their respective “happy path” reference models in BPMN format.

Iii Software Analysis

Iii-a Analysed Tools

In the course of the study, 16 tools capable of mining event log files were analysed, see Table I. The study was carried out in spring 2020.

Tool Name (Vendor)
ABBYY Timeline (ABBYY) MEHRWERK ProcessMining (Mehrwerk GmbH)
ARIS Process Mining
(Software AG)
Minit (Minit j.s.a.)
BusinessOptix (BusinessOptix) myInvenio (myInvenio Srl)
Celonis Process Mining
(Celonis SE)
PAFnow (Process Analytics
Factory GmbH)
Disco (Fluxicon BV) ProDiscovery
(Puzzle Data Co., Ltd.)
EverFlow (EverFlow) QPR ProcessAnalyzer
(QPR Software Plc)
LANA Process Mining
(Lana Labs GmbH)
Signavio Process Intelligence (Signavio GmbH)
Logpickr Process Explorer 360 (Logpickr) UiPath Process Mining
(UiPath Inc.)
TABLE I: Commercial process mining tools

Iii-B Website

The website is mainly built on three layers. While the homepage (first layer) introduces the discipline of process mining, typical use cases and our criteria overview, the “Tools” page (second layer) lists brief profiles of all tools which are linked to the detailed tool profile pages (third layer). An introductory paragraph briefly describes the vendor and the strengths of its software. Eight criteria categories examine the availability and extent of tested functionality while one criteria category provides general information. The “Distinctive Focus and Features” section provides more context by highlighting outstanding functionality. In order to offer users visual impressions of a tool, every profile is enriched with a “featured video” provided by the vendor and up to seven screenshots, whereof five are defined and two undefined (proprietary). In addition, any two selected tool profiles can be contrasted with each other through a side-by-side comparison.

Iii-C Software Criteria

The software criteria derived from literature review and experimental software testing represents the core of this study. The criteria were grouped into nine categories depicted in Tables II - X in the appendix.

Category General gives a brief overview of the vendor and key aspects of the tool. Data Management examines functionalities and factors related to the extraction, transformation and loading (ETL) of process data into the process mining tool. The Process Discovery category examines process graph capabilities and process analysis features such as benchmarking and rework analysis. Conformance Checking is a fundamental process mining feature with the objective to identify deviations between the actual “as is” process and an “a-priori” reference model. This category considers all relevant factors pertaining to conformance checking. The Operational Support criteria examine the availability of forward-looking capabilities to help users anticipate the outcome of running cases and facilitate decision making with the help of intelligent recommendations. Views, Monitoring and Reporting addresses the ability to monitor processes with the help of metrics and visualisations to support decision making. Additional criteria examine available languages and means of collaboration to share insights with other users. While process enhancement functionality such as performance metrics in the process graph are partly covered in the aforementioned criteria categories, Advanced Enhancement Capabilities investigates further capabilities that add a new perspective to the graph or the overall process. Lastly, Security & Compliance addresses role-based access control and the availability of audit logs.

Iv Contributions, Limitations and Outlook

The study of 16 process mining solutions with commercial licenses showed that the maturity level of the investigated software is highly varying. While some vendors offer basic discovery functionality without conformance checking in some cases, other vendors offer more elaborate features such as process simulation, predictive analytics and decision rule mining. We observe a potential trend: The boundaries between mere process mining functionality and other disciplines such as process modelling (BPMN), business intelligence and Machine Learning become more and more blurred.

The software selection is based on software listed in commercial reports and hence reflects a non-exhaustive picture of the market. Further, open source software was not analysed. It is important to note that the software listing represents only a snapshot of the tools’ capabilities and features in terms of information timeliness. Vendors are continuously improving their products and extend the functionalities with periodic releases.

A follow-up study could examine the perspective of organisations on the relevance of the suggested criteria. Interviews may be conducted with organisations interested in process mining as well as organisations with already implemented process mining software.

Acknowledgement

The authors would like to express their gratitude to all vendors that participated in the study for their time and effort.

References