Hybrid Approach to Automation, RPA and Machine Learning: a Method for the Human-centered Design of Software Robots

11/06/2018 ∙ by Wiesław Kopeć, et al. ∙ Uniwersytet Warszawski National Information Processing Institute 0

One of the more prominent trends within Industry 4.0 is the drive to employ Robotic Process Automation (RPA), especially as one of the elements of the Lean approach. The full implementation of RPA is riddled with challenges relating both to the reality of everyday business operations, from SMEs to SSCs and beyond, and the social effects of the changing job market. To successfully address these points there is a need to develop a solution that would adjust to the existing business operations and at the same time lower the negative social impact of the automation process. To achieve these goals we propose a hybrid, human-centered approach to the development of software robots. This design and implementation method combines the Living Lab approach with empowerment through participatory design to kick-start the co-development and co-maintenance of hybrid software robots which, supported by variety of AI methods and tools, including interactive and collaborative ML in the cloud, transform menial job posts into higher-skilled positions, allowing former employees to stay on as robot co-designers and maintainers, i.e. as co-programmers who supervise the machine learning processes with the use of tailored high-level RPA Domain Specific Languages (DSLs) to adjust the functioning of the robots and maintain operational flexibility.



There are no comments yet.


This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

As Economy 4.0 is firmly upon us the digitization and mechanization of business processes is increasing in all industries. Thus, to stay competitive it is no longer enough to optimize and simplify processes but there is a need to support them with automation, especially if they are repetitive, predictable and mostly digital.

Within Industry 4.0 there exists a prominent trend to employ Robotic Process Automation (RPA) especially as one of the elements of the Lean approach. To follow it companies often need to adjust the industry processes to the limitations of bulky, automated, often rule-based, IT systems and the format and structure of the input and output data required. Apart from the need to transform the business operations, there is also the negative impact this shift may have on the job market within a given industry, as the target job posts are fully reduced due to automation.

To achieve these goals we propose a hybrid, human-centered approach to the development of software robots. This design and implementation method combines the Living Lab approach[8, 7] with empowerment through participatory design to kick-start the co-development of hybrid software robots which, supported by variety of AI methods and tools, including interactive and collaborative ML[9], transform menial job posts into higher-skilled positions, allowing some of the former employees to stay on as robot co-programmers and co-maintainers who supervise machine learning and use pseudo-code to adjust the functioning of the robots to maintain operational flexibility. We believe that this approach will provide a relatively low-cost and user-friendly RPA implementation and sustainable maintenance solution.

2 Challenges to the Implementation of Automated Processes

The use of Robotic Process Automation is on the rise throughout the Western economies as the automation it offers can solve one of the largest business challenges of today: the need to process increasingly larger amount of data.

However, this method of automation itself since its inception has faced multiple challenges which still remain relevant. They are concentrated in three key areas depicted in the sidebar note related to technical, organizational and socioeconomic aspects.

2.1 Technical challenge

Many rule-based robots are difficult to scale because the rules are written by hand. Moreover, they are difficult to maintain to remain flexible, given the varied format and structure of the data to be processed, which often includes e-mails, web forms, faxes, scans of paper documents, phone calls or even financial or sensor data. Moreover, some output documents still need to be produced in paper and sent out to clients. Data already present in the current systems is often of poor quality in general, or for automation as it lacks tags and division by categories; on top of this it may be outdated and based on old regulations, checks and processes which makes it difficult to use machine learning to properly train neural networks.

margin: Benefits of the Hybrid Approach 1. Flexibility and Participatory Maintenance : AI and ML-Powered design allows for automation is derived from the patterns taught by employees in a Living Lab environment, who in turn learn how to facilitate this process. 2. Adjustment to Existing Business Processes: This ensures lower entry barrier than classic BPA with process re-engineering. The businesses can retain their current practices, supplementing human work with RPAs where possible, which means that the entry barrier is lower, as this solution is cheaper than BPA with process re-engineering and seamless, allowing for continuity of operation. 3. Empowerment: Some employees who used to perform low-skilled jobs instead of being displaced, become co-creators and co-maintainers of the software robots.

2.2 Organizational challenge

In multiple companies the existing business processes rely on complex chains of manufacturing involving multiple approval steps, contractors, clients and convoluted internal procedures. Such companies often use legacy software, which was developed with the rise of new business needs. These custom solutions are often rule-based and maintained by different external contractors. Such software can be fragmentary in nature as different processes are segmented into different software solutions, which often are not compatible with either the current standards, or even one another and can only be maintained through constant patching. When automation is delivered with RPA solutions, they are often neither intuitive nor user-friendly, and prone to errors. At the same time, multiple organizations lack sufficient knowledge about their own business processes, especially on handling exceptions and allowing for shortcuts and bypasses to effectively build such robots on their own.

2.3 Socioeconomic challenge

The implementation of automation with software robots is connected to ethical dilemmas as it is often followed by restructuring and massive loss of job posts[4]. This general awareness is one of the barriers to automation, as the managing staff and employees are aware of the associated risks and benefits, but have limited knowledge of how to mitigate them.

3 Discussion of the Proposed Solutions

The three problem areas discussed above can be directly addressed by a shift in thinking about Robotic Process Automation from a purely algorithmic IT perspective to the HCI one. Ultimately, RPA can become a human-centered endeavor as software robots relieve employees of tedious repetitive tasks, allowing them to increase their competences and build value in other areas. Thus, below we discuss the proposed solutions.

3.1 Distributed and Crowdsourced Machine Learning Approach

This postulate coresponds with the novel interactive and collaborative apprach to machine learning[9]. In particular we think that the use of neural networks to expand the functionality of software robots can ensure that various input and output data is efficiently handled, including audio and images. This process ought to be supported by empowered employees, who can verify the quality of the input and output data retrived and analysed by various ML tools and techniques [11, 12], including correct OCR-tagging, handling exceptions and rare cases. This process should be supported by state-of-the-art technology based on our advanced research on eyetracking methods and techniques[3, 2]. Finally, the employees from co-maintainers of the solution can become co-designers and co-programmers, as they learn to modify the pseudo-code responsible for the functioning of the software robot they oversee.

3.2 Supplementation of Existing Solutions

The use of software robots which directly emulate the jobs of specific human employees allows the companies to continue to use their time-tested business processes and legacy software. Moreover, the central platform for RPA and the high level language tailored to the crafted RPA Domain Specific Language are easier to maintain, and will remain up to date, including the cloud-based Machine Learning and Neural Network components employed in the abovementined interactive and collaborative mode[9]. The 1:1 mapping of the human tasks and the involvement of current employees as co-programmers ensures the continuity of business operations as exceptions can be handled on the fly by internal staff.

3.3 Employee Empowerment through Participation

The implementation of software robots need not be followed by massive layoffs. Based on our previous advances in participatory design and co-design[6, 5], Living Lab activities[6] and higher level crowdsourcing solutions coupled with cloud-based collaborative solutions for quality assurance, also in ML[10], we postulate that, according to the principles of Participatory Design, employees who work in the target capacity of the robots can be the best co-designers and co-maintainers. Through their empowerment via participation in Living Labs and training they are motivated to increase their competences and learn how to efficiently perform their job of supervisors of machine learning and hybrid RPA high-level DLS programmers. Through this their jobs are transformed from menial to skilled, and their time is freed to work on more challenging aspects of the business.

4 A Holistic Solution: The Hybrid Approach

Thus, we propose the creation of a software platform and hardware setup that would allow employees to participate in the development and maintenance of software robots supported by various AI-powered solutions, including neural networks and interactive and collaborative machine learning. This solution ensures business continuity as the learning process and software robot operations are overseen by employees possessing the internal know-how of the company. This platform would allow businesses to retain the most motivated of their current employees while at the same time empowering them to learn new skills of co-maintainers and co-programmers. The implementation process of this Hybrid Approach can consist of the following stages:

  1. Analysis of RPA penetration and potential within the company

  2. Workshops with employees to identify opportunity areas

  3. Living Lab approach to process analysis with data-collection workstations

  4. Participatory design of specific software robots

  5. Supervised Training of AI-based solutions

  6. Employee empowerment training sessions

  7. Co-programming and co-maintenance of software robots in RPA DSLs

5 Conclusions

As the trends in Industry 4.0 are leaning towards automation there the need to develop positive human-centered methods. Thus, in this abstract we proposed a design approach that addresses key challenges of RPA. The method relies on participatory design of software robots, facilitated by a Living Lab environment, interactive and collaborative AI solutions, including machine learning and neural networks whereby menial tasks are turned into high-skilled jobs, increasing employee satisfaction and lowering turnover usable in multiple contexts, such as for example automatic tests, code deployment, customer service and machine and device control.


  • [1]
  • [2] Cezary Biele and Pawel Kobylinski. 2018. Surface Recalibration as a New Method Improving Gaze-Based Human-Computer Interaction. In International Conference on Intelligent Human Systems Integration. Springer, 197–202.
  • [3] Andrew T. Duchowski, Krzysztof Krejtz, Izabela Krejtz, Cezary Biele, Anna Niedzielska, Peter Kiefer, Martin Raubal, and Ioannis Giannopoulos. 2018. The Index of Pupillary Activity: Measuring Cognitive Load Vis-à-vis Task Difficulty with Pupil Oscillation. In Proc. of the 2018 CHI Conf. on Human Factors in Computing Systems (CHI ’18). ACM, NY, USA, Article 282.
  • [4] Carl Benedikt Frey and Michael A Osborne. 2017. The future of employment: how susceptible are jobs to computerisation? Technological forecasting and social change 114 (2017), 254–280.
  • [5] Wiesław Kopeć, Radosław Nielek, and Adam Wierzbicki. 2018. Guidelines Towards Better Participation of Older Adults in Software Development Processes using a new SPIRAL Method and Participatory Approach. In Proceedings of the CHASE’18: International Workshop on Cooperative and Human Aspects of Software (ICSE ’18). ACM, New York, NY, USA. DOI:http://dx.doi.org/10.1145/3195836.3195840 
  • [6] Wiesław Kopeć, Kinga Skorupska, Anna Jaskulska, Katarzyna Abramczuk, Radoslaw Nielek, and Adam Wierzbicki. 2017. LivingLab PJAIT: Towards Better Urban Participation of Seniors. In Proceedings of the International Conference on Web Intelligence (WI ’17). ACM, New York, NY, USA, 1085–1092. DOI:http://dx.doi.org/10.1145/3106426.3109040 
  • [7] Veli-Pekka Niitamo, Seija Kulkki, Mats Eriksson, and Karl A Hribernik. 2006. State-of-the-art and good practice in the field of living labs. In Technology Management Conference (ICE), 2006 IEEE International. IEEE, 1–8.
  • [8] Corinna Ogonowski, Benedikt Ley, Jan Hess, Lin Wan, and Volker Wulf. 2013. Designing for the living room: long-term user involvement in a living lab. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1539–1548.
  • [9] Sebastian Robert, Sebastian Büttner, Carsten Röcker, and Andreas Holzinger. 2016. Reasoning under uncertainty: Towards collaborative interactive machine learning. In Machine learning for health informatics. Springer, 357–376.
  • [10] Kinga Skorupska, Manuel Núñez, Wiesław Kopeć, and Radoslaw Nielek. 2018. Older Adults and Crowdsourcing: Android TV App for Evaluating TEDx Subtitle Quality. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 159 (Nov. 2018), 23 pages. DOI:http://dx.doi.org/10.1145/3274428 
  • [11] Dominika Tkaczyk, Paweł Szostek, Mateusz Fedoryszak, Piotr Jan Dendek, and Łukasz Bolikowski. 2015a. CERMINE: automatic extraction of structured metadata from scientific literature. International Journal on Document Analysis and Recognition (IJDAR) 18, 4 (01 Dec 2015), 317–335. DOI:http://dx.doi.org/10.1007/s10032-015-0249-8 
  • [12] Dominika Tkaczyk, Bartosz Tarnawski, and Lukasz Bolikowski. 2015b. Structured affiliations extraction from scientific literature. D-Lib Magazine 21, 11/12 (2015). DOI:http://dx.doi.org/10.1045/november2015-tkaczyk