Abstract
We present a holistic optimization of the ADER-DG finite element software SeisSol targeting the Intel\(^{\textregistered }\) Xeon Phi\(^\mathrm{TM}\) x200 processor, codenamed Knights Landing (KNL). SeisSol is a multi-physics software package performing earthquake simulations by coupling seismic wave propagation and the rupture process. The code was shown to scale beyond 1.5 million cores and achieved petascale performance when using local time stepping for the computationally heavy seismic wave propagation. Advancing further along these lines, we discuss the utilization of KNL’s core features, the exploitation of its two-level memory subsystem (which allows for efficient out-of-core implementations), and optimizations targeting at KNL’s 2D mesh on-die interconnect. Our performance comparisons demonstrate that KNL is able to outperform its previous generation, the Intel\(^{\textregistered }\) Xeon Phi™ coprocessor x100 family, by more than 2.9\(\times \) in time-to-solution. Additionally, our results show a 3.4\(\times \) speedup compared to latest Intel\(^{\textregistered }\) Xeon\(^{\textregistered }\) E5v3 CPUs.
Keywords
Optimization Notice: Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.
Intel, Xeon, and Intel Xeon Phi are trademarks of Intel Corporation in the U.S. and/or other countries.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
https://github.com/SeisSol/SeisSol, git-tag 201511 was used in this paper.
- 2.
TSX instructions, however, are not considered to be legacy x86 instructions.
- 3.
- 4.
References
Benjemaa, M., et al.: 3-D dynamic rupture simulations by a finite volume method. Geophys. J. Int. 178, 541–560 (2009)
Bielak, J., et al.: Parallel octree-based finite element method for large-scale earthquake ground motion simulation. Comput. Model. Eng. Sci. 10(2), 99 (2005)
Bielak, J., et al.: The shakeout earthquake scenario: verification of three simulation sets. Geophys. J. Int. 180(1), 375–404 (2010)
Borstnik, U., et al.: Sparse matrix multiplication: the distributed block-compressed sparse row library. Parallel Comput. 40(5–6), 47–58 (2014)
Breuer, A., et al.: High-order ADER-DG minimizes energy- and time-to-solution of SeisSol. In: Kunkel, J.M., Ludwig, T. (eds.) ISC High Performance 2015. LNCS, vol. 9137, pp. 340–357. Springer, Heidelberg (2015)
Breuer, A., et al.: Petascale local time stepping for the ADER-DG finite element method. In: Proceedings of IPDPS 2016 (2016). To appear
Breuer, A., et al.: Sustained petascale performance of seismic simulations with SeisSol on SuperMUC. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2014. LNCS, vol. 8488, pp. 1–18. Springer, Heidelberg (2014)
Carrington, L., et al.: High-frequency simulations of global seismic wave propagation using SPECFEM3D_GLOBE on 62K processors. In: Proceedings of SC 2008 (2008)
Cui, Y., et al.: Physics-based seismic hazard analysis on petascale heterogeneous supercomputers. In: Proceedings of SC 2013 (2013)
Cui, Y., et al.: Scalable earthquake simulation on petascale supercomputers. In: Proceedings of SC 2010 (2010)
Day, S.M., et al.: Tests of 3D elastodynamic codes: final report for lifelines project 1A02. Pacific Earthquake Engineering Research Center (2003)
de la Puente, J., et al.: An arbitrary high-order discontinuous galerkin method for elastic waves on unstructured meshes-IV. Anisotropy. Geophys. J. Int. 169(3), 1210–1228 (2007)
de la Puente, L., et al.: Dynamic rupture modeling on unstructured meshes using a discontinuous Galerkin method. J. Geophys. Res. 114, B10302 (2009)
Dumbser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-II. The three-dimensional isotropic case. Geophys. J. Int. 167(1), 319–336 (2006)
Harris, R.A., et al.: The SCEC/USGS dynamic earthquake rupture code verification exercise. Seismol. Res. Lett. 80(1), 119–126 (2009)
Heinecke, A., et al.: Petascale high order dynamic rupture earthquake simulations on heterogeneous supercomputers. In: Proceedings of SC 2014. Gordon Bell Finalist (2014)
Ichimura, T., et al.: Implicit nonlinear wave simulation with 1.08 T DOF and 0.270 T unstructured finite elements to enhance comprehensive earthquake simulation. In: Proceedings of SC 2015 (2015)
Ichimura, I., et al.: Physics-based urban earthquake simulation enhanced by 10.7 BLNDOF \(\times \, 30\) K time-step unstructured fe non-linear seismic wave simulation. In: Proceedings of SC 2014 (2014)
Intel Corporation: Intel(R) 64 and IA-32 Architectures Optimization Reference Manual, January 2016
Käser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-III. Viscoelastic attenuation. Geophys. J. Int. 168(1), 224–242 (2007)
Komatitsch, D., et al.: High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster. J. Comput. Phys. 229(20), 7692–7714 (2010)
Pelties, C., et al.: Three-dimensional dynamic rupture simulation with a high-order discontinuous Galerkin method on unstructured tetrahedral meshes. J. Geophys. Res. 117, B02309 (2012)
Pelties, C., et al.: Verification of an ADER-DG method for complex dynamic rupture problems. Geosci. Model Dev. Discuss. 6, 5981–6034 (2013)
Shin, J., et al.: Speeding up Nek5000 with autotuning and specialization. In: Proceedings of the 24th ACM International Conference on Supercomputing (ICS 2010), pp. 253–262. ACM, New York (2010)
Sodani, A.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. In: Hotchips-2015 (2015)
Sodani, A., et al.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. IEEE Micro, Hot Chips Special Issue, (2016, to appear)
Tago, J., et al.: A 3D hp-adaptive discontinuous Galerkin method for modeling earthquake dynamics. J. Geophys. Res. 117, B09312 (2012)
Tu, T., et al.: From mesh generation to scientific visualization: an end-to-end approach to parallel supercomputing. In: Proceedings of SC 2006 (2006)
Wilcox, L.C., et al.: A high-order discontinuous Galerkin method for wave propagation through coupled elastic-acoustic media. J. Comput. Phys. 229(24), 9373–9396 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Heinecke, A., Breuer, A., Bader, M., Dubey, P. (2016). High Order Seismic Simulations on the Intel Xeon Phi Processor (Knights Landing). In: Kunkel, J., Balaji, P., Dongarra, J. (eds) High Performance Computing. ISC High Performance 2016. Lecture Notes in Computer Science(), vol 9697. Springer, Cham. https://doi.org/10.1007/978-3-319-41321-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-41321-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41320-4
Online ISBN: 978-3-319-41321-1
eBook Packages: Computer ScienceComputer Science (R0)