Skip to main content

Advertisement

Springer Nature Link
Log in

A Parallel Approach for the Generation of Unstructured Meshes with Billions of Elements on Distributed-Memory Supercomputers

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

This paper describes a parallel approach for the rapid generation of ultra-large-scale unstructured meshes on distributed-memory supercomputers. A medium-sized initial mesh is prepared first. Afterwards, a two-level domain decomposition (DD) strategy is used to split and distribute the initial mesh to different cores. Finally, the parallel mesh generation, comprising a recursive procedure which includes parallel surface recovery, parallel boundary updating, and parallel mesh multiplication, is performed. The two-level DD differentiates the intra-node and inter-node communication to reduce communication overheads. A global indexing and updating scheme is used to make the mesh multiplication devoid of communication. A new parallel surface recovery algorithm without communication is developed to maintain the fidelity of the resulting mesh model to the original geometric model. Tests of the parallel approach for some real-life problems on supercomputers (Dawning-5000A and Tianhe-2) are presented. Issues regarding the speedup, parallel efficiency, and mesh quality are discussed. Results show that the proposed parallel approach has a reasonably good scalability, that the quality of the resulting mesh is improved, and that ultra-large-scale meshes with billions of elements can be generated quickly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28

Similar content being viewed by others

References

  1. Aliabadi, S., Johnson, A., Abedi, J., Zellars, B.: High performance computing of fluid–structure interactions in hydrodynamics applications using unstructured meshes with more than one billion elements. In: High Performance Computing—HiPC 2002, Lecture Notes in Computer Science, vol. 2552, pp. 519–533 (2002)

  2. Komatitsch, D., Tsuboi, S., Ji, C., Tromp, J.: A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth Simulator. In: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, pp. 1–8 (2003)

  3. Tautges, T.J., Jain, R.: Creating geometry and mesh models for nuclear reactor core geometries using a lattice hierarchy-based approach. Eng. Comput. 28(4), 319–329 (2012)

    Article  Google Scholar 

  4. Ivanov, E., Andrä, H., Kudryavtsev, A.: Domain decomposition approach for automatic parallel generation of tetrahedral grids. Comput. Methods Appl. Math. 6(2), 178–193 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. Andrä, H., Gluchshenko, O., Ivanov, E.G., Kudryavtsev, A.N.: Automatic parallel generation of tetrahedral grids by using a domain decomposition approach. Comput. Math. Math. Phys. 48(8), 1367–1375 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Yilmaz, Y., Ozturan, C.: Using sequential NETGEN as a component for a parallel mesh generator. Adv. Eng. Softw. 84, 3–12 (2015)

    Article  Google Scholar 

  7. Chrisochoides, N.P.: A survey of parallel mesh generation methods. Technical Report BrownSC-2005-09

  8. Chrisochoides, N., Chernikov, A., Fedorov, A., Kot, A., Linardakis, L., Foteinos, P.: Towards exascale parallel delaunay mesh generation. In: Proceedings of the 18th International Meshing Roundtable, pp. 319–336 (2009)

  9. Houzeaux, G., de la Cruz, R., Owen, H., Vázquez, M.: Parallel uniform mesh multiplication applied to a Navier–Stokes solver. Comput. Fluids 80, 142–151 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  10. Casoni, E., Jérusalem, A., Samaniego, C., Eguzkitza, B., Lafortune, P., Tjahjanto, D., Sáez, X., Houzeaux, G., Vázquez, M.: Alya: computational solid mechanics for supercomputers. Arch. Comput. Methods Eng. 22(4), 557–576 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Yilmaz, E., Aliabadi, S., Sahu, J.: Variable intensity computational mesh and data technique: a practical approach for large-scale flow simulation and visualization. AIAA Pap. 5100, 1–9 (2010)

    Google Scholar 

  12. Yilmaz, E., Aliabadi, S.: Surface conformed linear mesh and data subdivision technique for large-scale flow simulation and visualization in Variable Intensity Computational Environment. Comput. Fluids 80, 388–402 (2013)

    Article  Google Scholar 

  13. Yilmaz, E., Aliabadi, S.: Application of surface conformed linear mesh and data subdivision technique to a spinning projectile. Comput. Fluids 88, 773–781 (2013)

    Article  Google Scholar 

  14. Johnson, A.A., Aliabadi, S.K.: Application of automatic mesh generation and mesh multiplication techniques to very large scale free-surface flow simulations. In: Proceedings of the 7th International Conference on Numerical Grid Generation in Computational Field Simulations, pp. 25–28 (2000)

  15. Moulinec, C.: HPC evolution of the TELEMAC system. In: E-Proceedings of the 36th IAHR World Congress (2003)

  16. Yilmaz, Y., Özturan C., Tosun O., Özer A.H., Soner S.: Parallel mesh generation, migration and partitioning for the Elmer application. http://www.prace-project.eu/IMG/pdf/Parallel_Mesh_Generation_Migration_and_Partitioning_for_the_Elmer_Application-2.pdf (2010). Accessed 10 Oct 2015

  17. Elias, R.N., Camata, J.J., Coutinho, A.L.: Fast edge-based tetrahedral mesh refinement for large scale CFD simulations. In: 27th International Conference on Parallel Computational Fluid Dynamics (2015)

  18. Coutinho, L.G.A.: Parallel mesh multiplication and generation: towards petascale simulations. In: Proceedings of the VI International Conference on Adaptive Modeling and Simulation (2013)

  19. Said, R., Weatherill, N., Morgan, K., Verhoeven, N.: Distributed parallel Delaunay mesh generation. Comput. Methods Appl. Mech. Eng. 177(1), 109–125 (1999)

    Article  MATH  Google Scholar 

  20. Larwood, B., Weatherill, N.P., Hassan, O., Morgan, K.: Domain decomposition approach for parallel unstructured mesh generation. Int. J. Numer. Methods Eng. 58(2), 177–188 (2003)

    Article  MATH  Google Scholar 

  21. Ito, Y., Shih, A.M., Erukala, A.K., Soni, B.K., Chernikov, A., Chrisochoides, N.P., Nakahashi, K.: Parallel unstructured mesh generation by an advancing front method. Math. Comput. Simul. 75(5), 200–209 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chai, L.: High performance and scalable MPI Intra-node Communication middleware for multi-core clusters. The Ohio State University (2009)

  23. Tendulkar, S., Beall, M., Shephard, M.S., Jansen, K.: Parallel mesh generation and adaptation for CAD geometries. Technical Report SCOREC Report 2011-2

  24. Dongarra, J., Hittinger, J., Bell, J., Chacon, L., Falgout, R., Heroux, M., Hovland, P., Ng, E., Webster, C., Wild, S.: Applied mathematics research for exascale computing. Lawrence Livermore National Laboratory (LLNL) Report LLNLTR-651000

  25. Ito, Y., Nakahashi, K.: Direct surface triangulation using stereolithography data. AIAA J. 40(3), 490–496 (2002)

    Article  Google Scholar 

  26. Löhner, R.: Regridding surface triangulations. J. Comput. Phys. 126(1), 1–10 (1996)

    Article  MATH  Google Scholar 

  27. Metis. http://glaros.dtc.umn.edu/gkhome/metis/metis/overview

  28. ParMetis. http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview

  29. Zhou, M., Xie, T., Seol, S., Shephard, M.S., Sahni, O., Jansen, K.E.: Tools to support mesh adaptation on massively parallel computers. Eng. Comput. 28(3), 287–301 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This work is financially supported by the National High Technology Research and Development Program (863 Program) of China (2012AA01A307) and the Natural Science Foundation of China (11272214 and 51475287). The authors would like to thank Shanghai Supercomputer Center and Guangzhou Supercomputer Center for help and useful advice. The authors are also indebted to khytti Whitaker, an English teacher from America; an anonymous Ph.D. candidate from Australia; an anonymous research associate from Europe; and Nicholas Allen, an exchange student from England, for sacrificing their precious time to proofread and polish the English of our paper.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Xiao-qing Wang & Xian-long Jin

  2. State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, China

    Xiao-qing Wang & Xian-long Jin

  3. Shanghai Supercomputer Center, Shanghai, China

    Da-zhi Kou & Jia-hui Chen

Authors
  1. Xiao-qing Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xian-long Jin
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Da-zhi Kou
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jia-hui Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiao-qing Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Xq., Jin, Xl., Kou, Dz. et al. A Parallel Approach for the Generation of Unstructured Meshes with Billions of Elements on Distributed-Memory Supercomputers. Int J Parallel Prog 45, 680–710 (2017). https://doi.org/10.1007/s10766-016-0452-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-016-0452-3

Keywords