Skip to main content
Springer Nature Link
Log in

Analyzing Variable Human Actions for Robotic Process Automation

  • Conference paper
  • First Online:
Business Process Management (BPM 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13420))

Included in the following conference series:

Abstract

Robotic Process Automation (RPA) provides a means to automate mundane and repetitive human tasks. Task Mining approaches can be used to discover the actions that humans take to carry out a particular task. A weakness of such approaches, however, is that they cannot deal well with humans who carry out the same task differently for different cases according to some hidden rule. The logs that are used for Task Mining generally do not contain sufficient data to distinguish the exact drivers behind this variability. In this paper, we propose a new Task Mining framework that has been designed to support engineers who wish to apply RPA to a task that is subject to variable human actions. This framework extracts features from User Interface (UI) Logs that are extended with a new source of data, namely screen captures. The framework invokes Supervised Machine Learning algorithms to generate decision models, which characterize the decisions behind variable human actions in a machine-and-human-readable form. We evaluated the proposed Task Mining framework with a set of synthetic UI Logs. Despite the use of only relatively small logs, our results demonstrate that a high accuracy is generally achieved.

This research has been supported by the Spanish Ministry of Science, Innovation and Universities under the NICO project (PID2019-105455GB-C31) and the Centro para el Desarrollo Tecnológico Industrial (CDTI) of Spain under the CODICE project (EXP 00130458/IDI-20210319 - P018-20/E09) and by the FPU scholarship program, granted by the Spanish Ministry of Education and Vocational Training (FPU20/05984).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Availabe at: www.spyrix.com and bestxsoftware.com/es/.

  2. 2.

    We trained the model with this dataset: https://doi.org/10.5281/zenodo.2530277.

  3. 3.

    The set of problems are available at: https://doi.org/10.5281/zenodo.5734323.

References

  1. van der Aalst, W.M.P.: Process Mining: Data Science in Action. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49851-4

    Book  Google Scholar 

  2. Scheppler, B., Weber, C.: Robotic process automation. Informatik Spektrum 43(2), 152–156 (2020). https://doi.org/10.1007/s00287-020-01263-6

    Article  Google Scholar 

  3. Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: Automated generation of executable RPA scripts from user interface logs. In: BPM, pp. 116–131 (2020)

    Google Scholar 

  4. Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: SmartRPA: a tool to reactively synthesize software robots from user interface logs. In: CAiSE, pp. 137–145 (2021)

    Google Scholar 

  5. Augusto, A., et al.: Automated discovery of process models from event logs: review and benchmark. IEEE Trans. Knowl. Data Eng. 31(4), 686–705 (2018)

    Article  MathSciNet  Google Scholar 

  6. Bazhenova, E., Bülow, S., Weske, M.: Discovering decision models from event logs. In: BIS, pp. 237–251 (2016)

    Google Scholar 

  7. Brahmbhatt, S.: Shapes in Images, pp. 67–93. Apress, Berkeley, CA (2013)

    Google Scholar 

  8. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks, Monterey, CA (1984)

    MATH  Google Scholar 

  9. Capgemini, C.: Robotic Process Automation - Robots conquer business processes in back offices (2017)

    Google Scholar 

  10. Denagama Vitharanage, I.M., Bandara, W., Syed, R., Toman, D.: An empirically supported conceptualisation of robotic process automation (RPA) benefits. In: ECIS (2020)

    Google Scholar 

  11. Egger, A., ter Hofstede, A.H., Kratsch, W., Leemans, S.J., Röglinger, M., Wynn, M.T.: Bot log mining: using logs from robotic process automation for process mining. In: ER, pp. 51–61 (2020)

    Google Scholar 

  12. Gao, J., van Zelst, S.J., Lu, X., van der Aalst, W.M.: Automated robotic process automation: a self-learning approach. In: OTM, pp. 95–112 (2019)

    Google Scholar 

  13. Geyer-Klingeberg, J., Nakladal, J., Baldauf, F., Veit, F.: Process mining and robotic process automation: a perfect match. In: BPM, pp. 124–131 (2018)

    Google Scholar 

  14. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th DSAA, pp. 80–89 (2018)

    Google Scholar 

  15. Jimenez-Ramirez, A., Reijers, H.A., Barba, I., Del Valle, C.: A method to improve the early stages of the robotic process automation lifecycle. In: CAiSE, pp. 446–461 (2019)

    Google Scholar 

  16. Kass, G.V.: An exploratory technique for investigating large quantities of categorical data. J. R. Statist. Soc. Ser. C (Appl. Statist.) 29(2), 119–127 (1980)

    Google Scholar 

  17. Keras OCR. https://github.com/faustomorales/keras-ocr. Accessed Mar 2022

  18. Lacity, M., Willcocks, L.: What knowledge workers stand to gain from automation. Harv. Bus. Rev. 19, 1–6 (2015)

    Google Scholar 

  19. Leno, V., Armas-Cervantes, A., Dumas, M., La Rosa, M., Maggi, F.M.: Discovering process maps from event streams. In: ICSSP, pp. 86–95 (2018)

    Google Scholar 

  20. Leno, V., Augusto, A., Dumas, M., La Rosa, M., Maggi, F.M., Polyvyanyy, A.: Identifying candidate routines for robotic process automation from unsegmented UI logs. In: ICPM, pp. 153–160 (2020)

    Google Scholar 

  21. Leno, V., Dumas, M., La Rosa, M., Maggi, F.M., Polyvyanyy, A.: Automated discovery of data transformations for robotic process automation. arXiv preprint arXiv:2001.01007 (2020)

  22. Leno, V., Polyvyanyy, A., Dumas, M., La Rosa, M., Maggi, F.M.: Robotic process mining: vision and challenges. Bus. Inf. Syst. Eng. 63, 1–14 (2020)

    Google Scholar 

  23. Leno, V., Polyvyanyy, A., La Rosa, M., Dumas, M., Maggi, F.M.: Action logger: enabling process mining for robotic process automation. In: CEUR Workshop Proceedings (2019)

    Google Scholar 

  24. Leshob, A., Bourgouin, A., Renard, L.: Towards a process analysis approach to adopt robotic process automation. In: ICEBE, pp. 46–53 (2018)

    Google Scholar 

  25. López-Carnicer, J.M., del Valle, C., Enríquez, J.G.: Towards an opensource logger for the analysis of RPA projects. In: BPM, pp. 176–184 (2020)

    Google Scholar 

  26. Majumder, B.P., Potti, N., Tata, S., Wendt, J.B., Zhao, Q., Najork, M.: Representation learning for information extraction from form-like documents. In: ACL, pp. 6495–6504 (2020)

    Google Scholar 

  27. Mitchell, T.: Machine Learning. McGraw Hill, New York (1997)

    MATH  Google Scholar 

  28. Moran, K., Bernal-Cárdenas, C., Curcio, M., Bonett, R., Poshyvanyk, D.: Machine learning-based prototyping of graphical user interfaces for mobile apps. IEEE Trans. Softw. Eng. 46(2), 196–221 (2018)

    Article  Google Scholar 

  29. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)

    Google Scholar 

  30. Quinlan, J.R.: C4. 5: Programs For Machine Learning. Elsevier, Amsterdam (2014)

    Google Scholar 

  31. Reinkemeyer, L.: Process Mining in Action: Principles, Use Cases and Outlook. Springer, Switzerland (2020). https://doi.org/10.1007/978-3-030-40172-6

    Book  Google Scholar 

  32. Rozinat, A., van der Aalst, W.M.P.: Decision mining in ProM. In: Dustdar, S., Fiadeiro, J.L., Sheth, A.P. (eds.) BPM 2006. LNCS, vol. 4102, pp. 420–425. Springer, Heidelberg (2006). https://doi.org/10.1007/11841760_33

    Chapter  Google Scholar 

  33. Willcocks, L., Lacity, M.: A new approach to automating services. MIT Sloan Manage. Rev. 58(1), 40–49 (2016)

    Google Scholar 

  34. Xu, Z., Baojie, X., Guoxin, W.: Canny edge detection based on open CV. In: 2017 13th ICEMI, pp. 53–56 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Lenguajes y Sistemas Informáticos, Escuela Técnica Superior de Ingeniería Informática, Avenida Reina Mercedes, s/n, 410121, Sevilla, Spain

    A. Martínez-Rojas, A. Jiménez-Ramírez & J. G. Enríquez

  2. Department of Information and Computing Sciences, Utrecht University, Princetonplein 5, Utrecht, 3584 CC, The Netherlands

    H. A. Reijers

Authors
  1. A. Martínez-Rojas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Jiménez-Ramírez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. G. Enríquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. H. A. Reijers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Martínez-Rojas .

Editor information

Editors and Affiliations

  1. Sapienza University of Rome, Rome, Italy

    Claudio Di Ciccio

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Remco Dijkman

  3. Universidad de Sevilla, Seville, Spain

    Adela del Río Ortega

  4. Technical University of Munich, Garching, Germany

    Stefanie Rinderle-Ma

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Martínez-Rojas, A., Jiménez-Ramírez, A., Enríquez, J.G., Reijers, H.A. (2022). Analyzing Variable Human Actions for Robotic Process Automation. In: Di Ciccio, C., Dijkman, R., del Río Ortega, A., Rinderle-Ma, S. (eds) Business Process Management. BPM 2022. Lecture Notes in Computer Science, vol 13420. Springer, Cham. https://doi.org/10.1007/978-3-031-16103-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16103-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16102-5

  • Online ISBN: 978-3-031-16103-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics