Skip to main content
SpringerLink
Log in

Enriching Process Models with Relevant Process Details for Flexible Human-Robot Teaming

  • Conference paper
  • First Online:
Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2023)

  • 104 Accesses

Abstract

Human-robot teaming is crucial for future automation in small and medium enterprises. In that context, domain-specific process models are used as an intuitive description of work to share between two agents. Process designers usually introduce a certain degree of abstraction into the models. This way, models are better to trace for humans, and a single model can moreover enable flexibility by capturing several process variations. However, abstraction can lead to unintentional omission of information (e.g., experience of skilled workers). This may impair the quality of process results. To balance the trade-off between model readability and flexibility, we contribute a novel human-robot teaming approach with incremental learning of relevant process details (RPDs). RPDs are extracted from imagery during process execution and used to enrich an integrated process model which unifies human worker instruction and robot programming. Experiments based on two use cases demonstrate the practical feasibility and scalability of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.tensorflow.org/ (Accessed: 02 May 2023).

  2. 2.

    https://github.com/marcotcr/lime (Accessed: 30 April 2023).

  3. 3.

    https://scikit-image.org/docs/dev/api/skimage.segmentation.html (Accessed: 08 May 2023).

References

  1. Ahmadikatouli, A., Aboutalebi, M.: New evolutionary approach to business process model optimization. In: Proceedings of the International MultiConference of Engineers and Computer Scientists, vol. 2 (2011)

    Google Scholar 

  2. Andersen, R.H., Solund, T., Hallam, J.: Definition and initial case-based evaluation of hardware-independent robot skills for industrial robotic co-workers. In: 41st International Symposium on Robotics (ISR), pp. 1–7. VDE (2014)

    Google Scholar 

  3. Becker, J., Rosemann, M., Von Uthmann, C.: Guidelines of business process modeling. In: BPM: Models, Techniques, and Empirical Studies, pp. 30–49 (2002)

    Google Scholar 

  4. Beumelburg, K.: Fähigkeitsorientierte Montageablaufplanung in der direkten Mensch-Roboter-Kooperation (2005)

    Google Scholar 

  5. Bobrik, R., Reichert, M., Bauer, T.: View-based process visualization. In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 88–95. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75183-0_7

    Chapter  Google Scholar 

  6. Chinosi, M., Trombetta, A.: BPMN: an introduction to the standard. Comput. Stand. Interfaces 34(1), 124–134 (2012)

    Article  Google Scholar 

  7. Coradeschi, S., Saffiotti, A.: An introduction to the anchoring problem. Robot. Auton. Syst. 43(2–3), 85–96 (2003)

    Article  Google Scholar 

  8. Darvish, K., et al.: A hierarchical architecture for human-robot cooperation processes. IEEE Trans. Rob. 37(2), 567–586 (2021)

    Article  Google Scholar 

  9. Dietz, T., et al.: Programming system for efficient use of industrial robots for deburring in SME environments. In: 7th German Conference on Robotics, pp. 1–6 (2012)

    Google Scholar 

  10. Fichtner, M., Fichtner, U.A., Jablonski, S.: An experimental study of intuitive representations of process task annotations. In: Sellami, M., Ceravolo, P., Reijers, H.A., Gaaloul, W., Panetto, H. (eds.) CoopIS 2022. LNCS, vol. 13591, pp. 311–321. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17834-4_19

    Chapter  Google Scholar 

  11. Fichtner., M., Schönig., S., Jablonski., S.: How lime explanation models can be used to extend business process models by relevant process details. In: Proceedings of the 24th International Conference on Enterprise Information Systems. vol. 2, pp. 527–534 (2022)

    Google Scholar 

  12. Gounaris, A.: Towards automated performance optimization of BPMN business processes. In: New Trends in Database and Information Systems (ADBIS), pp. 19–28 (2016)

    Google Scholar 

  13. Hasegawa, T., Suehiro, T., Takase, K.: A model-based manipulation system with skill-based execution. IEEE Trans. Robot. Autom. 8(5), 535–544 (1992)

    Article  Google Scholar 

  14. Koschmider, A., et al.: Business process modeling support by depictive and descriptive diagrams. In: Enterprise Modelling and Information Systems Architectures, pp. 31–44 (2015)

    Google Scholar 

  15. Matheson, E., Minto, R., Zampieri, E.G., Faccio, M., Rosati, G.: Human-robot collaboration in manufacturing applications: a review. Robotics 8(4), 100 (2019)

    Article  Google Scholar 

  16. Mendling, J., Reijers, H.A., van der Aalst, W.M.: Seven process modeling guidelines (7pmg). Inf. Softw. Technol. 52(2), 127–136 (2010)

    Article  Google Scholar 

  17. Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)

    Article  MathSciNet  Google Scholar 

  18. Niedermann, F., Radeschütz, S., Mitschang, B.: Deep business optimization: a platform for automated process optimization. INFORMATIK - Business Process and Service Science - Proceedings of ISSS and BPSC (2010)

    Google Scholar 

  19. Nottensteiner, K., et al.: A complete automated chain for flexible assembly using recognition, planning and sensor-based execution. In: Proceedings of 47st International Symposium on Robotics (ISR), pp. 1–8 (2016)

    Google Scholar 

  20. Paxton, C., et al.: Costar: instructing collaborative robots with behavior trees and vision. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 564–571 (2017)

    Google Scholar 

  21. Pedersen, M.R., et al.: Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput. Integr. Manuf. 37, 282–291 (2016)

    Article  Google Scholar 

  22. Polyvyanyy, A., Smirnov, S., Weske, M.: Process model abstraction: a slider approach. In: 12th International IEEE Enterprise Distributed Object Computing Conference (2008)

    Google Scholar 

  23. Reichert, M., et al.: Enabling personalized visualization of large business processes through parameterizable views. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp. 1653–1660 (2012)

    Google Scholar 

  24. Ribeiro, M.T., Singh, S., Guestrin, C.: “why should i trust you?” explaining the predictions of any classifier. In: Proceedings of 22nd ACM SIGKDD, pp. 1135–1144 (2016)

    Google Scholar 

  25. Riedelbauch, D.: Dynamic Task Sharing for Flexible Human-Robot Teaming under Partial Workspace Observability. Ph.D. thesis, University of Bayreuth (2020)

    Google Scholar 

  26. Riedelbauch, D., Henrich, D.: Fast graphical task modelling for flexible human-robot teaming. In: 50th International Symposium on Robotics (ISR), pp. 1–6. VDE (2018)

    Google Scholar 

  27. Riedelbauch, D., Hümmer, J.: A benchmark toolkit for collaborative human-robot interaction. In: 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 806–813. IEEE (2022)

    Google Scholar 

  28. Riedelbauch, D., Sucker., S.: Visual programming of robot tasks with product and process variety. In: Annals of Scientific Society for Assembly, Handling and Industrial Robotics (to appear) (2022)

    Google Scholar 

  29. Scheer, A.W., Thomas, O., Adam, O.: Process modeling using event-driven process chains. In: Process-Aware Information Systems: Bridging People and Software through Process Technology, pp. 119–145 (2005)

    Google Scholar 

  30. Selic, B., et al.: Omg unified modeling language (version 2.5). Technical Report (2015)

    Google Scholar 

  31. Senft, E., et al.: Situated live programming for human-robot collaboration. In: ACM Symposium on User Interface Software and Technology, pp. 613–625 (2021)

    Google Scholar 

  32. Steinmetz, F., Wollschläger, A., Weitschat, R.: Razer - a HRI for visual task-level programming and intuitive skill parameterization. IEEE Robot. Autom. Lett. 3(3), 1362–1369 (2018)

    Article  Google Scholar 

  33. Thomas, U., et al.: A new skill based robot programming language using UML/P statecharts. In: IEEE International Conference on Robotics and Automation (2013)

    Google Scholar 

  34. Van Der Aalst, W.M., Ter Hofstede, A.H., Weske, M.: Business process management: a survey. Bus. Process Manage. 2678(1019), 1–12 (2003)

    Google Scholar 

  35. Wiedmann, P.C.K.: Agiles Geschäftsprozessmanagement auf Basis gebrauchssprachlicher Modellierung. Universitaet Bayreuth (Germany) (2017)

    Google Scholar 

Download references

Acknowledgements

We thank Philipp Jahn and Carsten Scholle for their valuable work supporting the implementation and evaluation of our approach.

Author information

Authors and Affiliations

  1. University of Bayreuth, Universitätsstrasse 30, 95447, Bayreuth, Germany

    Myriel Fichtner, Sascha Sucker, Dominik Riedelbauch, Stefan Jablonski & Dominik Henrich

Authors
  1. Myriel Fichtner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sascha Sucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dominik Riedelbauch
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Jablonski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dominik Henrich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Myriel Fichtner .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. Xi’an Jiaotong-Liverpool, Suzhou, China

    Xinheng Wang

  3. University of Peloponnese, Patra, Greece

    Nikolaos Voros

Rights and permissions

Reprints and permissions

Copyright information

© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fichtner, M., Sucker, S., Riedelbauch, D., Jablonski, S., Henrich, D. (2024). Enriching Process Models with Relevant Process Details for Flexible Human-Robot Teaming. In: Gao, H., Wang, X., Voros, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 563. Springer, Cham. https://doi.org/10.1007/978-3-031-54531-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54531-3_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54530-6

  • Online ISBN: 978-3-031-54531-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation