Skip to main content
Log in

Distributed execution of communicating sequential process-style concurrency: Golang case study

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In the last decade, the majority of new central processing units (CPU) have become multicore. To take advantage of these new architectures, we need programming languages that can express parallelisms. The programming language Golang is well known for providing developers with an easy programming model for communicating sequential process-style concurrency enabling programmers to easily write functions that will execute on the different cores of a modern multicore CPUs. Unfortunately, Golang does not support distributed execution of goroutines on clusters or distributed systems. In this paper, we extend the concurrency capabilities of Golang to a distributed cluster by providing a library called Gluster that is simple and easy to use. We developed a programming model that allows users to easily distribute work between machines, in a similar way as workloads are distributed in multicore CPUs by using operating system threads with libraries similar to Pthreads and OpenMP. Our Gluster solution is based on a single master node that connects to peers over a network and distributes work to these peers. The master node is able to send function arguments over the network to worker nodes as well as receive return values. Results using matrix multiplication show that our distributed implementation can speed up the goroutines by 5\(\times \) in a small 16 nodes cluster, but more importantly, it shows that the results are scalable to cluster size.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. GoLang (2018) Go -Lang reference. https://golang.org/doc/. Accessed 1 May 2018

  2. Hoare CAR (1978) Communicating sequential processes. Commun. ACM 21(8):666–677. https://doi.org/10.1145/359576.359585

    Article  MATH  Google Scholar 

  3. Go usage (2018) https://research.swtch.com/gophercount. Accessed 1 May 2018

  4. Go Usage (2018) https://www.zdnet.com/article/googles-go-beats-java-c-python-to-programming-language-of-the-year-crown/. Accessed 5 May 2018

  5. Jones S (2018) CUDA—new features and beyond, developer talk. GTC 2018—ID S8278

  6. Prasertsang A, Pradubsuwun D (2016) Formal verification of concurrency in go. In: 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), Khon Kaen, pp 1–4

  7. Togashi N, Klyuev V (2014) Concurrency in Go and Java: performance analysis. In: 2014 4th IEEE International Conference on Information Science and Technology, Shenzhen, pp 213–216

  8. Ueda Y, Ohara M (2017) Performance competitiveness of a statically compiled language for server-side Web applications. In: 2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp 13–22

  9. Lange J, Ng N, Toninho B, Yoshida N (2018) A static verification framework for message passing in Go using behavioural types. Accepted draft at ICSE 2018

  10. Midtgaard J, Nielson F, Nielson H (2018) Process-local static analysis of synchronous processes. In: 25th Static Analysis Symposium

  11. Jenkins L, Zhou T, Spear M (2017) Redesigning Gos built-in map to support concurrent operations. In: 2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT), Portland, OR, pp 14–26

  12. Pasarella E, Vidal ME, Zoltan C (2016) Comparing Mapreduce and pipeline implementations for counting triangles. In: Proceedings XVI Jornadas sobre Programación y Lenguajes, PROLE 2016, Salamanca, Spain, 14–16th September 2016

  13. Binet S (2018) Go-HEP: writing concurrent software with ease and Go. arXiV:1808.06529 https://go-hep.org

  14. Togashi N, Klyuev V (2015) A novel approach for web development: a schedule management system using GAE/Go. In: 2015 IEEE 7th International Conference on Awareness Science and Technology (iCAST), Qinhuangdao, pp 55–59

  15. Fang Z, Luo M, Anwar FM, Zhuang H, Gupta RK (2018) Go-realtime: a lightweight framework for multiprocessor real-time system in user space. SIGBED Rev 14(4):46–52

    Article  Google Scholar 

  16. Scionti A, Mazumdar S (2017) Let’s go: a data-driven multi-threading support. In: Proceedings of the Computing Frontiers Conference (CF’17). ACM, New York, NY, USA, pp 287–290

  17. Anurag VN (2018) Distributed computing with Go: practical concurrency and parallelism for Go applications. Packt Publishing, Birmingham

    Google Scholar 

  18. Proto-actor (2018) http://proto.actor/. Accessed 1 May 2018

  19. Hewitt C, Bishop P, Steiger R (1973) A universal modular actor formalism for artificial intelligence. IJCAI

  20. Lu C (2017) Gleam. https://github.com/chrislusf. Accessed 1 Jan 2018

  21. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: OSDI04 Proceedings of the 6th Conference on Symposium on Operating Systems Design and Implementation, vol 6

  22. OpenMPI forum website. http://www.mpi-forum.org/. Accessed 1 Feb 2018

  23. Dagum I, Menon R (1998) OpenMP: an industry standard API for shared-memory programming. IEEE Comput Sci Eng 5(1):46–55

    Article  Google Scholar 

  24. Lewis B, Berg D (1998) Multithreaded programming with Pthreads. Prentice-Hall, Inc., Upper Saddle River

    Google Scholar 

  25. Pike R (2011) Gobs of data. The Go blog. https://blog.golang.org/gobs-of-data. Accessed 24 Mar 2011

  26. Algee-Hewitt M (2016) Counting words in HathiTrust with Python and MPI. Stanford Literary Lab, Stanford

    Google Scholar 

  27. Github Gluster. https://github.com/James-Whitney/gluster. Accessed 1 Oct 2018

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maria Pantoja.

Additional information

James Whitney and Chandler Gifford have contributed equally to this work.

Appendices

Appendices

Appendix Gluster exposes the following functions to the user:

  • AddRunner(ip string)—Adds a runner with given ip on the default port.

  • AddRunnerPort(ip string, port int) Adds runner with specified port.

  • ImportFunctionFile(filename string) -imports .go file containing functions to be run on distributed machines.

  • RunDist(funct string, reply reflect.Type, args ...interface) int—runs a function on a runner and returns an id for the job.

  • JobDone(id int) bool—returns true if a job with the given id is finished and the result is set.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Whitney, J., Gifford, C. & Pantoja, M. Distributed execution of communicating sequential process-style concurrency: Golang case study. J Supercomput 75, 1396–1409 (2019). https://doi.org/10.1007/s11227-018-2649-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2649-2

Keywords

Navigation