Skip to main content
SpringerLink
Log in

Activity Recommendation for Business Process Modeling with Pre-trained Language Models

  • Conference paper
  • First Online:
The Semantic Web (ESWC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13870))

Included in the following conference series:

Abstract

Activity recommendation in business process modeling is concerned with suggesting suitable labels for a new activity inserted by a modeler in a process model under development. Recently, it has been proposed to represent process model repositories as knowledge graphs, which makes it possible to address the activity-recommendation problem as a knowledge graph completion task. However, existing recommendation approaches are entirely dependent on the knowledge contained in the model repository used for training. This makes them rigid in general and even inapplicable in situations where a process model consists of unseen activities, which were not part of the repository used for training. In this paper, we avoid these issues by recognizing that the semantics contained in process models can be used to instead pose the activity-recommendation problem as a set of textual sequence-to-sequence tasks. This enables the application of transfer-learning techniques from natural language processing, which allows for recommendations that go beyond the activities contained in an available repository. We operationalize this with an activity-recommendation approach that employs a pre-trained language model at its core, and uses the representations of process knowledge as structured graphs combined with the natural-language-based semantics of process models. In an experimental evaluation, we show that our approach considerably outperforms the state of the art in terms of semantic accuracy of the recommendations and that it is able to recommend and handle activity labels that go beyond the vocabulary of the model repository used during training.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that process model nodes may have empty labels (\(\lambda (n) = \epsilon \)), such as the XOR-join in Fig. 1, which is different from a node being unlabeled (\(\lambda (n) = \bot \)).

  2. 2.

    We provide the source code of the employed implementation under this link: https://github.com/disola/bpart5.

  3. 3.

    SAP-SAM contains a high number of vendor-provided example models. The publishers of the dataset recommend sorting them out as they negatively affect the diversity of the dataset.

  4. 4.

    Compared to T5-Base with its 220 million parameters, T5-Small is a model checkpoint that has only 60 million parameters.

  5. 5.

    Note that BLEU and METEOR are designed for the comparison of (long) sentences or text corpora. Penalties in the definitions of the metrics can thus cause the metrics to be (close to) zero for short activity recommendations, even if ground truth and recommendation match. Therefore, we manually set the BLEU and METEOR scores to 1 if a recommended activity and the ground-truth activity are an exact match.

  6. 6.

    We performed t-tests for all reported differences between the evaluated approaches, which showed that the differences are statistically significant (\(p<0.001\)).

References

  1. Abran, A., Moore, J.W., Bourque, P., Dupuis, R., Tripp, L.: Software Engineering Body of Knowledge. IEEE Computer Society, Angela Burgess (2004)

    Google Scholar 

  2. Annane, A., Aussenac-Gilles, N., Kamel, M.: BBO: BPMN 2.0 based ontology for business process representation. In: 20th European Conference on Knowledge Management (ECKM 2019), vol. 1, pp. 49–59 (2019)

    Google Scholar 

  3. Bachhofner, S., Kiesling, E., Revoredo, K., Waibel, P., Polleres, A.: Automated process knowledge graph construction from BPMN models. In: Strauss, C., Cuzzocrea, A., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) Database and Expert Systems Applications. DEXA 2022. LNCS, vol. 13426, pp. 32–47. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-12423-5_3

  4. Banerjee, S., Lavie, A.: Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72 (2005)

    Google Scholar 

  5. Boehm, B.W., Papaccio, P.N.: Understanding and controlling software costs. IEEE Trans. Softw. Eng. 14(10), 1462–1477 (1988)

    Article  Google Scholar 

  6. Cao, B., Yin, J., Deng, S., Wang, D., Wu, Z.: Graph-based workflow recommendation: on improving business process modeling. In: CIKM, pp. 1527–1531. ACM (2012)

    Google Scholar 

  7. Cer, D., et al.: Universal sentence encoder (2018)

    Google Scholar 

  8. Davies, I., Green, P., Rosemann, M., Indulska, M., Gallo, S.: How do practitioners use conceptual modeling in practice? Data Knowl. Eng. 58(3), 358–380 (2006)

    Article  Google Scholar 

  9. Deng, S., et al.: A recommendation system to facilitate business process modeling. IEEE Trans. Cybern. 47(6), 1380–1394 (2017)

    Article  Google Scholar 

  10. Fundamentals of Business Process Management. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-662-56509-4_9

  11. Ehrig, M., Koschmider, A., Oberweis, A.: Measuring similarity between semantic business process models. In: APCCM, vol. 7, pp. 71–80 (2007)

    Google Scholar 

  12. Fahland, D., Favre, C., Koehler, J., Lohmann, N., Völzer, H., Wolf, K.: Analysis on demand: instantaneous soundness checking of industrial business process models. Data Knowl. Eng. 70(5), 448–466 (2011)

    Article  Google Scholar 

  13. Fellmann, M., Delfmann, P., Koschmider, A., Laue, R., Leopold, H., Schoknecht, A.: Semantic technology in business process modeling and analysis. part 1: matching, modeling support, correctness and compliance. EMISA Forum 35, 15–31 (2015)

    Google Scholar 

  14. Fellmann, M., Delfmann, P., Koschmider, A., Laue, R., Leopold, H., Schoknecht, A.: Semantic technology in business process modeling and analysis. part 2: Domain patterns and (semantic) process model elicitation. EMISA Forum 35(2), 12–23 (2015)

    Google Scholar 

  15. Fellmann, M., Zarvic, N., Metzger, D., Koschmider, A.: Requirements catalog for business process modeling recommender systems. In: WI, pp. 393–407 (2015)

    Google Scholar 

  16. Frederiks, P.J., Van der Weide, T.P.: Information modeling: the process and the required competencies of its participants. DKE 58(1), 4–20 (2006)

    Article  Google Scholar 

  17. Friedrich, F., Mendling, J., Puhlmann, F.: Process model generation from natural language text. In: Mouratidis, H., Rolland, C. (eds.) CAiSE 2011. LNCS, vol. 6741, pp. 482–496. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21640-4_36

    Chapter  Google Scholar 

  18. Goldstein, M., González-Álvarez, C.: Augmenting modelers with semantic autocompletion of processes. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNBIP, vol. 427, pp. 20–36. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85440-9_2

    Chapter  Google Scholar 

  19. Goldstein, M., González-Álvarez, C.: Evaluating semantic autocompletion of business processes with domain experts. In: ASE, pp. 1116–1120 (2021)

    Google Scholar 

  20. Graves, A.: Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711 (2012)

  21. Gunawardana, A., Shani, G., Yogev, S.: Evaluating Recommender Systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 547–601. Springer, New York, NY (2022). https://doi.org/10.1007/978-1-0716-2197-4_15

  22. Jannach, D., Fischer, S.: Recommendation-based modeling support for data mining processes. In: RecSys, pp. 337–340 (2014)

    Google Scholar 

  23. Jannach, D., Jugovac, M., Lerche, L.: Supporting the design of machine learning workflows with a recommendation system. ACM TiiS 6(1), 1–35 (2016)

    Article  Google Scholar 

  24. Kampik, T., et al.: Sap signavio academic models (2022). https://doi.org/10.5281/zenodo.7012043

  25. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar 

  26. Klein, G., Kim, Y., Deng, Y., Senellart, J., Rush, A.M.: Opennmt: open-source toolkit for neural machine translation. In: Proceedings of ACL 2017, System Demonstrations, pp. 67–72 (2017)

    Google Scholar 

  27. Kudo, T.: Subword regularization: improving neural network translation models with multiple subword candidates. In: Gurevych, I., Miyao, Y. (eds.) ACL, no. 1, pp. 66–75. Association for Computational Linguistics (2018)

    Google Scholar 

  28. Kudo, T., Richardson, J.: Sentencepiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. CoRR abs/1808.06226 (2018)

    Google Scholar 

  29. de Leoni, M., Felli, P., Montali, M.: A holistic approach for soundness verification of decision-aware process models. In: Trujillo, J.C., et al. (eds.) ER 2018. LNCS, vol. 11157, pp. 219–235. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00847-5_17

    Chapter  Google Scholar 

  30. Leopold, H., Niepert, M., Weidlich, M., Mendling, J., Dijkman, R., Stuckenschmidt, H.: Probabilistic optimization of semantic process model matching. In: Barros, A., Gal, A., Kindler, E. (eds.) BPM 2012. LNCS, vol. 7481, pp. 319–334. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32885-5_25

    Chapter  Google Scholar 

  31. Li, B., Han, L.: Distance weighted cosine similarity measure for text classification. In: Yin, H., et al. (eds.) IDEAL 2013. LNCS, vol. 8206, pp. 611–618. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41278-3_74

    Chapter  Google Scholar 

  32. Li, Y., et al.: An efficient recommendation method for improving business process modeling. IEEE Trans. Ind. Inf. 10(1), 502–513 (2014)

    Article  Google Scholar 

  33. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2018)

    Google Scholar 

  34. Meilicke, C., Betz, P., Stuckenschmidt, H.: Why a naive way to combine symbolic and latent knowledge base completion works surprisingly well. In: 3rd Conference on Automated Knowledge Base Construction (2021)

    Google Scholar 

  35. Meilicke, C., Chekol, M.W., Ruffinelli, D., Stuckenschmidt, H.: Anytime bottom-up rule learning for knowledge graph completion. In: IJCAI, pp. 3137–3143. AAAI Press (2019)

    Google Scholar 

  36. Mendling, J., Reijers, H.A., van der Aalst, W.M.: Seven process modeling guidelines (7pmg). Inf. Softw. Technol. 52(2), 127–136 (2010)

    Article  Google Scholar 

  37. Mendling, J., Reijers, H.A., Recker, J.: Activity labeling in process modeling: empirical insights and recommendations. Inf. Syst. 35(4), 467–482 (2010)

    Article  Google Scholar 

  38. Ott, S., Meilicke, C., Samwald, M.: SAFRAN: an interpretable, rule-based link prediction method outperforming embedding models. In: 3rd Conference on Automated Knowledge Base Construction (2021)

    Google Scholar 

  39. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)

    Google Scholar 

  40. Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization. In: International Conference on Learning Representations (2018)

    Google Scholar 

  41. Pfeiffer, P., Lahann, J., Fettke, P.: Multivariate business process representation learning utilizing Gramian angular fields and convolutional neural networks. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 327–344. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_21

    Chapter  Google Scholar 

  42. Pittke, F., Leopold, H., Mendling, J.: Automatic detection and resolution of lexical ambiguity in process models. IEEE Trans. Softw. Eng. 41(6), 526–544 (2015)

    Article  Google Scholar 

  43. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)

    MathSciNet  MATH  Google Scholar 

  44. Rosemann, M.: Potential pitfalls of process modeling: part a. Bus. Process. Manag. J. 12(2), 249–254 (2006)

    Google Scholar 

  45. Sola, D., Meilicke, C., van der Aa, H., Stuckenschmidt, H.: On the use of knowledge graph completion methods for activity recommendation in business process modeling. In: Marrella, A., Weber, B. (eds.) BPM 2021. LNBIP, vol. 436, pp. 5–17. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-94343-1_1

    Chapter  Google Scholar 

  46. Sola, D., Meilicke, C., van der Aa, H., Stuckenschmidt, H.: A rule-based recommendation approach for business process modeling. In: La Rosa, M., Sadiq, S., Teniente, E. (eds.) CAiSE 2021. LNCS, vol. 12751, pp. 328–343. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79382-1_20

    Chapter  Google Scholar 

  47. Sola, D., Van der Aa, H., Meilicke, C., Stuckenschmidt, H.: Exploiting label semantics for rule-based activity recommendation in business process modeling. Inf. Syst. 108, 102049 (2022)

    Google Scholar 

  48. Sola, D., Warmuth, C., Schäfer, B., Badakhshan, P., Rehse, J.R., Kampik, T.: Sap signavio academic models: a large process model dataset. arXiv e-prints pp. arXiv-2208 (2022)

    Google Scholar 

  49. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS (2014)

    Google Scholar 

  50. Thomas, O., Fellmann, M.: Semantic process modeling - design and implementation of an ontology-based representation of business processes. Bus. Inf. Syst. Eng. 1(6), 438–451 (2009)

    Article  Google Scholar 

  51. Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  52. Wang, H., Wen, L., Lin, L., Wang, J.: RLRecommender: a representation-learning-based recommendation method for business process modeling. In: Pahl, C., Vukovic, M., Yin, J., Yu, Q. (eds.) ICSOC 2018. LNCS, vol. 11236, pp. 478–486. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03596-9_34

    Chapter  Google Scholar 

  53. Wolf, T., et al.: Huggingface’s transformers: state-of-the-art natural language processing. CoRR abs/1910.03771 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SAP Signavio, Walldorf, Germany

    Diana Sola

  2. Data and Web Science Group, University of Mannheim, Mannheim, Germany

    Diana Sola, Han van der Aa, Christian Meilicke & Heiner Stuckenschmidt

Authors
  1. Diana Sola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han van der Aa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Meilicke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heiner Stuckenschmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Diana Sola .

Editor information

Editors and Affiliations

  1. Universidade de Lisboa, Lisbon, Portugal

    Catia Pesquita

  2. University of London, London, UK

    Ernesto Jimenez-Ruiz

  3. Rensselaer Polytechnic Institute, Troy, MI, USA

    Jamie McCusker

  4. Universidade de Lisboa, Lisbon, Portugal

    Daniel Faria

  5. Fondazione Bruno Kessler, Povo, Trento, Italy

    Mauro Dragoni

  6. KU Leuven, Sint-Katelijne-Waver, Belgium

    Anastasia Dimou

  7. EURECOM, Biot, France

    Raphael Troncy

  8. University of Mannheim, Mannheim, Germany

    Sven Hertling

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sola, D., van der Aa, H., Meilicke, C., Stuckenschmidt, H. (2023). Activity Recommendation for Business Process Modeling with Pre-trained Language Models. In: Pesquita, C., et al. The Semantic Web. ESWC 2023. Lecture Notes in Computer Science, vol 13870. Springer, Cham. https://doi.org/10.1007/978-3-031-33455-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33455-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33454-2

  • Online ISBN: 978-3-031-33455-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation