Skip to main content
SpringerLink
Log in

A Consolidated View on Specification Languages for Data Analysis Workflows

  • Conference paper
  • First Online:
Book cover Leveraging Applications of Formal Methods, Verification and Validation. Software Engineering (ISoLA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13702))

Included in the following conference series:

Abstract

Data analysis workflows (DAWs) are widely used in the scientific world. However, different communities created a plethora of domain-specific languages (DSLs) to specify their DAWs. Consequently, across DSLs, it is hard to perform operations on DAWs such as share, port, compare, re-use, adapt, or even merge. Thus, we have analyzed DAW specification languages and created a unified DAW metamodel. Given an instance of a DAW specification that can be matched to our metamodel, we are now able to apply CRUD operations (create, read, update, delete), and can potentially translate between different DAW specification languages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://kubernetes.io/.

  2. 2.

    This is only a very basic example meant to provide an intuition. Using tasks that inherently support different kinds of input data is just one other possible solution. Additionally, the output data type must also be considered in a real world example.

References

  1. Stoudt, S., Vásquez, V.N., Martinez, C.C.: Principles for data analysis workflows. PLoS Comput. Biol. 17(3), e1008770 (2021)

    Article  Google Scholar 

  2. Fowler, M.: Domain-Specific Languages. Pearson Education, London (2010)

    Google Scholar 

  3. Tracz, W.: DSSA (Domain-Specific Software Architecture): pedagogical example. SIGSOFT Softw. Eng. Notes 20(3), 49–62 (1995)

    Article  Google Scholar 

  4. Apache airflow project (2022). https://airflow.apache.org/

  5. Di Tommaso, P., Chatzou, M., Floden, E.W., Barja, P.P., Palumbo, E., Notredame, C.: Nextflow enables reproducible computational workflows. Nat. Biotechnol. 35(4), 316–319 (2017)

    Article  Google Scholar 

  6. Rak, R., Rowley, A., Black, W., Ananiadou, S.: Argo: an integrative, interactive, text mining-based workbench supporting curation. Database 1–7, 2012 (2012)

    Google Scholar 

  7. Stackstorm (2022). https://stackstorm.com/

  8. Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache flink: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Comm. Data Eng. 36(4), 28–38 (2015)

    Google Scholar 

  9. Jain, A., et al.: Fireworks: a dynamic workflow system designed for high-throughput applications. Concurr. Comput. Practic. Exp. 27(17), 5037–5059 (2015)

    Article  Google Scholar 

  10. Prefect (2022). https://docs.prefect.io/

  11. Köster, J., Rahmann, S.: Snakemake-a scalable bioinformatics workflow engine. Bioinformatics 28(19), 2520–2522 (2012)

    Article  Google Scholar 

  12. Luigi (2022). https://github.com/spotify/luigi

  13. Chinosi, M., Trombetta, A.: BPMN: an introduction to the standard. Comput. Stand. Interfaces 34(1), 124–134 (2012)

    Article  Google Scholar 

  14. Camunda (2022). https://camunda.com/bpmn/

  15. Orchestra (2022). https://orchestra.b12.io/

  16. Activiti (2022). https://www.activiti.org/

  17. Atkinson, C., Kühne, T.: Model-driven development: a metamodeling foundation. IEEE Softw. 20(5), 36–41 (2003)

    Article  Google Scholar 

  18. Amstutz, P., et al.: Common workflow language, v1. 0 (2016)

    Google Scholar 

  19. Perkel, J.M.: Workflow systems turn raw data into scientific knowledge. Nature 573, 149–150 (2019)

    Article  Google Scholar 

  20. Szyperski, C., Gruntz, D., Murer, S.: Component Software: Beyond Object-Oriented Programming, 2nd edn. ACM Press/Addison-Wesley, New York (2002)

    Google Scholar 

  21. Schiefer, C., et al.: Portability of scientific workflows in NGS data analysis: a case study. CoRR, abs/2006.03104 (2020)

    Google Scholar 

  22. Brambilla, M., Cabot, J., Wimmer, M.: Model-Driven Software Engineering in Practice. Synthesis Lectures on Software Engineering, 2nd edn. Morgan & Claypool, Vermont (2017)

    Book  Google Scholar 

  23. Grunske, L., Geiger, L., Zündorf, A., Van Eetvelde, N., Van Gorp, P., Varró, D.: Using graph transformation for practical model-driven software engineering. In: Beydeda, S., Book, M., Gruhn, V. (eds.) Model-Driven Software Development, pp. 91–117. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-28554-7_5

    Chapter  Google Scholar 

  24. Object Management Group (OMG). OMG Unified Modeling Language (OMG UML®), Version 2.5.1 (2017). https://www.omg.org/spec/UML/2.5.1/PDF

  25. Rupp, C., Queins, S., Zengler, B.: UML 2 glasklar: Praxiswissen für die UML-Modellierung. Hanser (2007)

    Google Scholar 

  26. Metamodel and mappings of specification languages for data analysis workflows (2022). https://github.com/CRC-FONDA/T1-DAW-Metamodel

  27. Boulakia, S.C., Leser, U.: Search, adapt, and reuse: the future of scientific workflows. SIGMOD Rec. 40(2), 6–16 (2011)

    Article  Google Scholar 

  28. Fernandez, R.C., Culhane, W., Watcharapichat, P., Weidlich, M., Morales, V.L., Pietzuch, P.R.: Meta-dataflows: efficient exploratory dataflow jobs. In: Das, G., Jermaine, C.M., Bernstein, P.A. (eds.) Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, 10–15 June 2018, pp. 1157–1172. ACM (2018)

    Google Scholar 

  29. Leser, U., et al.: The Collaborative Research Center FONDA. Datenbank-Spektrum, (1610–1995), November 2021

    Google Scholar 

Download references

Aknowledgements

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project-ID 414984028 - SFB 1404 FONDA [29].

Author information

Authors and Affiliations

  1. Humboldt-Universität zu Berlin, Berlin, Germany

    Marcus Hilbrich, Sebastian Müller, Svetlana Kulagina, Christopher Lazik, Ninon De Mecquenem & Lars Grunske

Authors
  1. Marcus Hilbrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sebastian Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Svetlana Kulagina
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher Lazik
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ninon De Mecquenem
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lars Grunske
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher Lazik .

Editor information

Editors and Affiliations

  1. University of Limerick, CSIS and Lero, Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hilbrich, M., Müller, S., Kulagina, S., Lazik, C., De Mecquenem, N., Grunske, L. (2022). A Consolidated View on Specification Languages for Data Analysis Workflows. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Software Engineering. ISoLA 2022. Lecture Notes in Computer Science, vol 13702. Springer, Cham. https://doi.org/10.1007/978-3-031-19756-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19756-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19755-0

  • Online ISBN: 978-3-031-19756-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation