Skip to main content
SpringerLink
Log in

MTCL: A Multi-transport Communication Library

  • Conference paper
  • First Online:
Euro-Par 2023: Parallel Processing Workshops (Euro-Par 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14351))

Included in the following conference series:

  • 45 Accesses

Abstract

To pave the way toward adopting the Compute Continuum paradigm, there is the need to support highly distributed heterogeneous application workflows that require the simultaneous use of multiple communication protocols in different parts of the application. In this work, we present for the first time the MTCL C++ communication library. It aims to abstract multiple transport protocols (e.g., MQTT, MPI, TCP) and related implementations under a single connection-oriented API, offering point-to-point and collective communication patterns to the programmers. We discuss the main design choices and preliminary performance results measured using the OSU micro-benchmarks. Finally, through a simple Federated Learning application, we showcase the flexibility of the MTCL library.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    MTCL repository: https://github.com/ParaGroup/MTCL.

  2. 2.

    Hosted by the Green Data Center of the University of Pisa.

  3. 3.

    GALILEO100: https://www.hpc.cineca.it/hardware/galileo100 Tier-1 supercomputer hosted by the CINECA supercomputing center.

References

  1. gRPC. https://grpc.io/

  2. OSU Microbenchmarks. https://mvapich.cse.ohio-state.edu/benchmarks/

  3. Beckman, P., et al.: Harnessing the computing continuum for programming our world, pp. 215–230 (2020). https://doi.org/10.1002/9781119551713.ch7

  4. Belcastro, L., Marozzo, F., Orsino, A., et al.: Edge-cloud continuum solutions for urban mobility prediction and planning. IEEE Access 11, 38864–38874 (2023). https://doi.org/10.1109/ACCESS.2023.3267471

    Article  Google Scholar 

  5. Carlson, J.L.: Redis in Action. Manning Publications Co. (2013)

    Google Scholar 

  6. Godoy, W.F., Podhorszki, N., Wang, R., et al.: ADIOS 2: the adaptable input output system. A framework for high-performance data management. SoftwareX 12, 100561 (2020). https://doi.org/10.1016/j.softx.2020.100561

  7. Gropp, W., Lusk, E.: Dynamic process management in an MPI setting. In: Proceedings, Seventh IEEE Symposium on Parallel and Distributed Processing, pp. 530–533 (1995). https://doi.org/10.1109/SPDP.1995.530729

  8. Grun, P., Hefty, S., Sur, S., et al.: A brief introduction to the OpenFabrics interfaces - a new network API for maximizing high performance application efficiency. In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, pp. 34–39 (2015). https://doi.org/10.1109/HOTI.2015.19

  9. Hintjens, P.: ZeroMQ: Messaging for Many Applications. O’Reilly Media (2013)

    Google Scholar 

  10. Kamburugamuve, S., Wickramasinghe, P., Govindarajan, K., et al.: Twister: net-communication library for big data processing in HPC and cloud environments. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), pp. 383–391. IEEE (2018). https://doi.org/10.1109/CLOUD.2018.00055

  11. Message Passing Interface Forum: MPI: A Message-Passing Interface Standard Version 4.0, June 2021. https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf

  12. Mittone, G., Tonci, N., Birke, R., et al.: Experimenting with emerging arm and RISC-V systems for decentralised machine learning. arXiv preprint arXiv:2302.07946 (2023). https://doi.org/10.48550/arXiv.2302.07946

  13. Panayiotou, K., Tsardoulias, E., Symeonidis, A.: Commlib: an easy-to-use communication library for cyber-physical systems. SoftwareX 19, 101180 (2022). https://doi.org/10.1016/j.softx.2022.101180

    Article  Google Scholar 

  14. Perera, C., Qin, Y., Estrella, J.C., et al.: Fog computing for sustainable smart cities: a survey. ACM Comput. Surv. 50(3) (2017). https://doi.org/10.1145/3057266

  15. Ramon-Cortes, C., Alvarez, P., Lordan, F., et al.: A survey on the distributed computing stack. Comput. Sci. Rev. 42, 100422 (2021). https://doi.org/10.1016/j.cosrev.2021.100422

    Article  MathSciNet  Google Scholar 

  16. Reed, D.A., Dongarra, J.: Exascale computing and big data. Commun. ACM 58(7), 56–68 (2015). https://doi.org/10.1145/2699414

  17. Sellami, R., Bhiri, S., Defude, B.: ODBAPI: a unified rest API for relational and NoSQL data stores. In: 2014 IEEE International Congress on Big Data, pp. 653–660 (2014). https://doi.org/10.1109/BigData.Congress.2014.98

  18. Shamis, P., Venkata, M.G., Lopez, M.G., et al.: UCX: an open source framework for HPC network APIs and beyond. In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, pp. 40–43 (2015). https://doi.org/10.1109/HOTI.2015.13

  19. Soumagne, J., Kimpe, D., Zounmevo, J., et al.: Mercury: enabling remote procedure call for high-performance computing. In: 2013 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1–8 (2013). https://doi.org/10.1109/CLUSTER.2013.6702617

  20. Vinoski, S.: Advanced message queuing protocol. IEEE Internet Comput. 10(6), 87–89 (2006). https://doi.org/10.1109/MIC.2006.116

    Article  Google Scholar 

  21. Yassein, M.B., Shatnawi, M.Q., Aljwarneh, S., Al-Hatmi, R.: Internet of things: survey and open issues of MQTT protocol. In: 2017 International Conference on Engineering & MIS (ICEMIS), pp. 1–6 (2017). https://doi.org/10.1109/ICEMIS.2017.8273112

Download references

Acknowledgements

This work was partially funded by Spoke 1 “FutureHPC & BigData” of the Italian Research Center on High-Performance Computing, Big Data and Quantum Computing (ICSC) funded by MUR Missione 4 Componente 2 Investimento 1.4: Potenziamento strutture di ricerca e creazione di “campioni nazionali di R&S (M4C2-19 )” - Next Generation EU (NGEU), and by the European Union’s Horizon 2020 under the ADMIRE project, grant Agreement number 956748. We acknowledge the CINECA award under the ISCRA initiative, for the availability of high-performance computing resources and support.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Pisa, Pisa, Italy

    Federico Finocchio, Nicoló Tonci & Massimo Torquati

Authors
  1. Federico Finocchio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicoló Tonci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Massimo Torquati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Massimo Torquati .

Editor information

Editors and Affiliations

  1. University of Cyprus, Nicosia, Cyprus

    Demetris Zeinalipour

  2. University of Santiago de Compostela, Santiago de Compostela, Spain

    Dora Blanco Heras

  3. University of Cyprus, Nicosia, Cyprus

    George Pallis

  4. Cyprus University of Technology, Limassol, Cyprus

    Herodotos Herodotou

  5. University of Nicosia, Nicosia, Cyprus

    Demetris Trihinas

  6. Inria, Nantes, France

    Daniel Balouek

  7. Louisiana State University, Baton Rouge, LA, USA

    Patrick Diehl

  8. Karlsruhe Institute of Technology, Karlsruhe, Germany

    Terry Cojean

  9. Ludwig-Maximilians-Universität, Munich, Germany

    Karl Fürlinger

  10. Roskilde University, Roskilde, Denmark

    Maja Hanne Kirkeby

  11. Bank of Italy, Rome, Italy

    Matteo Nardelli

  12. Roma Tre University, Rome, Italy

    Pierangelo Di Sanzo

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Finocchio, F., Tonci, N., Torquati, M. (2024). MTCL: A Multi-transport Communication Library. In: Zeinalipour, D., et al. Euro-Par 2023: Parallel Processing Workshops. Euro-Par 2023. Lecture Notes in Computer Science, vol 14351. Springer, Cham. https://doi.org/10.1007/978-3-031-50684-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50684-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50683-3

  • Online ISBN: 978-3-031-50684-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation