Skip to main content
SpringerLink
Log in

An Efficient Scalable Runtime System for Macro Data Flow Processing Using S-Net

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

S-Net is a declarative coordination language and component technology aimed at radically facilitating software engineering for modern parallel compute systems by near-complete separation of concerns between application (component) engineering and concurrency orchestration. S-Net builds on the concept of stream processing to structure networks of communicating asynchronous components implemented in a conventional (sequential) language. In this paper we present the design, implementation and evaluation of a new and innovative runtime system for S-Net streaming networks. The Front runtime system outperforms the existing implementations of S-Net by orders of magnitude for stress-test benchmarks, significantly reduces runtimes of fully-fledged parallel applications with compute-intensive components and achieves good scalability on our 48-core test system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Available at https://github.com/snetdev/snet-rts/blob/master/examples/fibonacci/fibonacci3.snet.

  2. Available at https://github.com/snetdev/snet-rts/blob/master/examples/poweroftwo/poweroftwo2.snet.

  3. Available at https://github.com/snetdev/snet-rts/blob/master/examples/sieve/sieve.snet.

  4. Available at https://github.com/snetdev/snet-rts/blob/master/examples/cholesky/cholesky.snet.

References

  1. Blumofe, R., Leiserson, C.: Scheduling multithreaded computations by work stealing. In: Foundations of Computer Science, 1994 Proceedings, 35th Annual Symposium on, pp. 356–368 (1994)

  2. Burton, F.W., Sleep, M.R.: Executing functional programs on a virtual tree of processors. In: Proceedings of the 1981 Conference on Functional Programming Languages and Computer Architecture, FPCA ’81, pp. 187–194. ACM, New York, NY, USA (1981)

  3. Buttari, A., Langou, J., Kurzak, J., Dongarra, J.: A class of parallel tiled linear algebra algorithms for multicore architectures. Parallel Comput. 35(1), 38–53 (2009)

    Article  MathSciNet  Google Scholar 

  4. Gijsbers, B.: An efficient scalable work-stealing runtime system for the S-Net coordination language. Master’s thesis, University of Amsterdam, Amsterdam, Netherlands (2013)

  5. Grelck, C.: The essence of synchronisation in asynchronous data flow. In: 25th IEEE International Parallel and Distributed Processing Symposium (IPDPS’11), Anchorage, USA. IEEE Computer Society Press (2011)

  6. Grelck, C., Julku, J., Penczek, F.: Distributed S-Net: cluster and grid computing without the hassle. In: Cluster, Cloud and Grid Computing (CCGrid’12), 12th IEEE/ACM International Conference, Ottawa, Canada. IEEE Computer Society (2012)

  7. Grelck, C., Penczek, F.: Implementation Architecture and Multithreaded Runtime System of S-Net. In: Scholz, S., Chitil, O. (eds.) Implementation and Application of Functional Languages, 20th International Symposium, IFL’08, Hatfield, United Kingdom, Revised Selected Papers, Lecture Notes in Computer Science, vol. 5836, pp. 60–79. Springer (2011)

  8. Grelck, C., Scholz, S., Shafarenko, A.: Asynchronous stream processing with S-Net. Int. J. Parallel Program. 38(1), 38–67 (2010). doi:10.1007/s10766-009-0121-x

    Article  MATH  Google Scholar 

  9. Grelck, C., Scholz, S.B., Shafarenko, A.: Coordinating data parallel SAC programs with S-Net. In: Proceedings of the 21st IEEE International Parallel and Distributed Processing Symposium (IPDPS’07), Long Beach, California, USA. IEEE Computer Society Press, Los Alamitos, CA, USA (2007)

  10. Kahn, G., MacQueen, D.: Coroutines and Networks of Parallel Processes. Technical report, Laboria, University of Edinburgh (1976)

  11. Le Chevalier, F., Maria, S.: Stap processing without noise-only reference: requirements and solutions. Radar, 2006. CIE ’06. International Conference on pp. 1–4 (2006)

  12. Mattheis, S., Schuele, T., Raabe, A., Henties, T., Gleim, U.: Work stealing strategies for parallel stream processing in soft real-time systems. In: Herkersdorf, A., Römer, K., Brinkschulte, U. (eds.) Architecture of Computing Systems: ARCS 2012, Lecture Notes in Computer Science, vol. 7179, pp. 172–183. Springer, Berlin (2012)

  13. Neill, D., Wierman, A.: On the Benefits of Work Stealing in Shared-Memory Multiprocessors. Technical report, Department of Computer Science, Carnegie Mellon University (2009)

  14. Penczek, F., Cheng, W., Grelck, C., Kirner, R., Scheuermann, B., Shafarenko, A.: A data-flow based coordination approach to concurrent software engineering. In: 2nd Workshop on Data-Flow Execution Models for Extreme Scale Computing (DFM 2012). IEEE Computer Society, Minneapolis, USA (2012)

  15. Penczek, F., Grelck, C., Cai, H., Julku, J., Hölzenspies, P., Scholz, S., Shafarenko, A.: S-Net Language Report 2.0. Technical Report 499. School of Computer Science, University of Hertfordshire, Hatfield, England, UK (2010)

  16. Penczek, F., Grelck, C., Scholz, S.B.: An operational semantics for S-Net. In: Chapman, B., Desprez, F., Joubert, G., Lichnewsky, A., Peters, F., Priol, T. (eds.) Parallel Computing: From Multicores and GPU’s to Petascale, Advances in Parallel Computing, vol. 19, pp. 467–474. IOS Press, Amsterdam, Netherlands (2010). doi:10.3233/978-1-60750-530-3-467

  17. Penczek, F., Herhut, S., Grelck, C., Scholz, S.B., Shafarenko, A., Barrière, R., Lenormand, E.: Parallel signal processing with S-Net. In: 10th International Conference on Computational Science, Amsterdam, Netherlands. Procedia Comput. Sci. 1(1), 2079–2088 (2010)

  18. Prokesch, D.: A light-weight parallel execution layer for shared-memory stream processing. Master’s thesis, Technical University of Vienna, Vienna, Austria (2011)

  19. Sanchez, D., Lo, D., Yoo, R., Sugerman, J., Kozyrakis, C.: Dynamic fine-grain scheduling of pipeline parallelism. In: 20th International Conference on Parallel Architectures and Compilation Techniques (PACT 2011), Galveston Island, TX, USA, pp. 22–32 (2011)

  20. Stephens, R.: A survey of stream processing. Acta Inform. 34(7), 491–541 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  21. S-Net runtime system software distribution (2013). URL:https://github.com/snetdev/snet-rts

  22. Thies, W., Karczmarek, M., Amarasinghe, S.: StreamIt: a language for streaming applications. In: Nigel Horspool, R. (ed.) Compiler Construction, 11th International Conference, CC 2002, Grenoble, France. Lecture Notes in Computer Science, vol. 2304, pp. 179–196. Springer, Berlin (2002)

Download references

Acknowledgments

We thank the anonymous reviewers for their valuable comments that very much helped us to shape the paper. We further thank Pāvels Zaičenkovs for the S-Net implementation of Cholesky factorization.

Author information

Authors and Affiliations

  1. Institute of Informatics, University of Amsterdam, Science Park 904, Amsterdam, The Netherlands

    Bert Gijsbers & Clemens Grelck

Authors
  1. Bert Gijsbers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Clemens Grelck
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clemens Grelck.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gijsbers, B., Grelck, C. An Efficient Scalable Runtime System for Macro Data Flow Processing Using S-Net . Int J Parallel Prog 42, 988–1011 (2014). https://doi.org/10.1007/s10766-013-0271-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-013-0271-8

Keywords

Search

Navigation