Skip to main content
SpringerLink
Log in

A domain decomposition strategy for hybrid parallelization of moving particle semi-implicit (MPS) method for computer cluster

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The MPS is a particle method developed to simulate incompressible flows with free surfaces that has several applications in nonlinear hydrodynamics. Much effort has been done to handle the large amount of particles required for simulating practical problems with desired refinement. However, the efficient use of the currently available computational resources, such as computer cluster, remains as a challenge. The present paper proposes a new strategy to parallelize the MPS method for fully distributed computing in cluster, which enables to simulate models with hundreds of millions of particles and keeps the required runtime within reasonable limits, as shown by the analysis of scalability and performance. The proposed strategy uses a non-geometric dynamic domain decomposition method that provides homogeneous load balancing and for very large models the scalability is supra-linear. Also, the domain decomposition (DD) is carried out only in the initial setup. As a result, the DD method is based on renumbering of particles using an original fully distributed sorting algorithm. Moreover, unlike the usual strategies, none of the processors require access to global data of the particles on any time step. Therefore, the limit for the maximum size of the model depends more on the total memory of the allocated nodes than the quantity of the local memory of each node. Thus, by extending the application of MPS method to very large models, this study contributes to consolidating the method as a practical tool to investigate complex engineering problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. The difference between the amount of particles in the subdomains is at most 1.

References

  1. Koshizuka, S., Tamako, H., Oka, Y.: A particle method for incompressible viscous flow with fluid fragmentation. CFD J. 4(1), 29–46 (1995)

    Google Scholar 

  2. Iwashita, T., Shimasaki, M.: Parallel processing of 3-D eddy current analysis with moving conductor using parallelized ICCG solver with renumbering process. IEEE Trans. Magn. 36(4), 1504–1509 (2000)

    Article  Google Scholar 

  3. Motezuki, F.K., Cheng, L.Y., Tsukamoto, M.M.: A study on parallelized solvers for moving particle semi-implicit method (MPS). In: Proceedings of the 20th International Congress of Mechanical Engineering (COBEM2009). Gramado, Brazil (2009)

  4. Yao, Z., Wang, J.S., Cheng, M.: Improved o(n) neighbor list method using domain decomposition and data sorting. High Perform. Comput. Eng. Syst. (HPCES) 161(1–2), 27–35 (2004)

    Google Scholar 

  5. Nyland, L., Prins, J., Yun, R.H., Hermans, J., Kum, H., Wang, L.: Achieving scalable parallel molecular dynamics using dynamic spatial domain decomposition techniques. J. Parallel Distrib. Comput. 47, 125–138 (1997)

    Article  Google Scholar 

  6. Gotoh, H., Khayyer, A., Ikari, H., Chiemi, H.: 3D-CMPS method for improvement of water surface tracking in breaking waves. In: 4th SPHERIC Workshop, pp. 265–272. Nantes, France (2009)

  7. Iribe, T., Fujisawa, T., Koshizuka, S.: Reduction of communication in parallel computing of particle method for flow simulation of seaside areas. Cost Eng. 52(4), 287–304 (2010)

    Article  Google Scholar 

  8. Gotoh, H., Sakai, T.: Key issues in the particle method for computation of wave breaking. Cost Eng. 53(2–3), 171–179 (2006)

    Article  Google Scholar 

  9. Ikeda, H., Koshizuka, S., Oka, Y.: Numerical analysis of jet injection behavior for fuel-coolant interaction using particle method. J. Nucl. Sci. Tech. 38(3), 174–182 (2001)

    Article  Google Scholar 

  10. Formaggia, L., Sala, M., Saleri, F.: Domain decomposition techniques. Tech. rep, Sandia National Laboratories, Albuquerque, USA (2005)

  11. Knuth, D.E.: The Art of Computer Programming. Sorting and Searching, vol. 3. Addison Wesley Longman Publishing Co., Inc., Redwood City (1998)

    Google Scholar 

  12. Meloni, S., Rosati, M., Colombo, L.: Efficiently particle labeling in atomistic simulations. J. Chem. Phys. 126, 121102 (2007)

    Article  Google Scholar 

  13. The OpenMP API specification for parallel programming. http://openmp.org/wp/

  14. Message passing interface (MPI). https://computing.llnl.gov/tutorials/mpi/

  15. VTK, Visualization toolkit. http://www.vtk.org/

  16. Issa, R., Violeau, D.: Test case 2: 3D-dambreaking. Tech. rep, SPH European Research Interest Community—SPHERIC ERCOFTAC, Rome, Italy (2006)

  17. Issa, R., Violeau, D.: SPHERIC Test2, 3D schematic dam break and evolution of the free surface. https://wiki.manchester.ac.uk/spheric/index.php/Test2

  18. Ganglia monitoring system—scalable distributed monitoring system for high-performance computing systems. http://ganglia.sourceforge.net/

  19. Marr, D.T., Binns, F., Hill, D.L., Hinton, G., Koufaty, D.A., Miller, J.A., Upton, M.: Hyper-threading technology architecture and microarchitecture. Intel Technol. J. 6–1, 4–15 (2002)

    Google Scholar 

  20. Grama, A., Gupta, A., Karypis, G., Kumar, V.: Introduction to Parallel Computing, 2nd edn. Addison Wesley, Boston (2003)

    Google Scholar 

  21. Ogino, M., Shioya, R., Kawai, H., Yoshimura, S.: Seismic response analysis of nuclear pressure vessel model with ADVENTRUE system on the Earth Simulator. J. Earth Simul. 2, 41–54 (2005)

    Google Scholar 

  22. Ihmsen, M., Akinci, N., Becker, M., Teschner, M.: A parallel SPH implementation on multi-core CPUs. Comput. Graph. Forum. 30(1), 99–112 (2011)

    Article  Google Scholar 

  23. Hori, C., Gotoh, H., Ikari, H., Khayyer, A.: GPU-acceleration for Moving Particle Semi-implicit Method. Comput. Fluids 51, 174 (2011)

  24. Ovaysi, S., Piri, M.: Multi-GPU acceleration of direct pore-scale modeling of fluid flow in natural porous media. Comput. Phys. Commun. 183, 1890–1898 (2012)

    Article  MathSciNet  Google Scholar 

  25. Bellezi, C.A., Cheng, L.Y.: Particle based numerical analysis of green water on FSOP deck. In: 32nd International Conference on Ocean, Offshore and Arctic Engineering, OMAE2013-11553, Nantes, France (2013)

  26. Murotani, K., Koshizuka, S., Tamai, T., Shibata, K., Mitsume, N., Yoshimura, S., Tanaka, S., Hasegawa, K., Nagai, E., Fujisawa, T.: Development of hierarchical domain decomposition explicit MPS method and application to large-scale tsunami analysis with floating objects. J. Adv. Simul. Sci. Eng. (JASSE) 1(1), 16–35 (2014)

    Article  Google Scholar 

  27. Duan, G., Chen, B.: Comparison of parallel solvers for Moving Particle Semi-Implicit method. Eng. Comput. 32(3), 834–862 (2015)

    Article  Google Scholar 

  28. Taniguchi, D., Sato, L.M., Cheng, L.Y.: Explicit moving particle simulation method on GPU clusters. Blucher Mech. Eng. Proc. 1(1), 1155 (2014)

    Google Scholar 

Download references

Acknowledgments

The authors would like to express their gratitude to PETROBRAS S.A for the financial support to present research.

Author information

Authors and Affiliations

  1. Numerical Offshore Tank of Escola Politécnica, University of São Paulo, São Paulo, 05508-03, Brazil

    Davi Teodoro Fernandes, Eric Henrique Favero & Kazuo Nishimoto

  2. Cidade Universitária, Avenue Professor Mello Moraes, 2231, São Paulo, SP, 05508-900, Brazil

    Davi Teodoro Fernandes

  3. Department of Construction Engineering, Escola Politécnica, University of São Paulo, São Paulo, 05508-03, Brazil

    Liang-Yee Cheng

Authors
  1. Davi Teodoro Fernandes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang-Yee Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Henrique Favero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuo Nishimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Davi Teodoro Fernandes.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernandes, D.T., Cheng, LY., Favero, E.H. et al. A domain decomposition strategy for hybrid parallelization of moving particle semi-implicit (MPS) method for computer cluster. Cluster Comput 18, 1363–1377 (2015). https://doi.org/10.1007/s10586-015-0483-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-015-0483-3

Keywords

Search

Navigation