Abstract
The growing computational capabilities of nowadays supercomputers have made highly resolved turbulence simulations possible. The large data-sets and tremendous amount of required compute resources create serious new challenges when attempting to share the data between different research groups. But even more difficult to solve is the incompatibility of the data formats and numerical approaches used for turbulence simulations, which in detail are often only known to the simulation code developer. In this paper a framework for sharing data of large scale simulations is presented, which simplifies the access and further post-processing even beyond a single supercomputing center. It combines established services to provide an easy to manage-and-extend software setup without the need to standardize a database or -format. Beside other advantages, it enables the use of direct file outputs from simulation runs which are often archived anyway.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Boschung, J., Schaefer, P., Peters, N., Meneveau, C.: The local topology of stream-and vortex lines in turbulent flows. Phys. Fluids (1994-present) 26(4), 045107 (2014)
DataTurb: Direct simulation data of turbulent flows. https://eudat.eu/communities/dataturb-direct-simulation-data-of-turbulent-flows. [Accessed 30 Nov 2016]
Frings, W., Wolf, F., Petkov, V.: Scalable massively parallel i/o to task-local files. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 1–11. IEEE (2009)
Gampert, M., Goebbert, J.H., Schaefer, P., Gauding, M., Peters, N., Aldudak, F., Oberlack, M.: Extensive strain along gradient trajectories in the turbulent kinetic energy field. New J. Phys. 13(4), 043012 (2011)
Gauding, M., Goebbert, J.H., Hasse, C., Peters, N.: Line segments in homogeneous scalar turbulence. Physics of Fluids (1994-present) 27(9), 095102 (2015)
Gauding, M., Wick, A., Peters, N., Pitsch, H.: Generalized scale-by-scale energy budget equations for large-eddy simulations of scalar turbulence at various schmidt numbers. J. Turbul. (2014)
Goebbert, J.H., Gauding, M., Gampert, M., Schaefer, P., Peters, N.: A new view on geometry and conditional statistics in turbulence. Inside: Innovatives Supercomputing in Deutschland (2011)
JHTDB: Johns hopkins turbulence database. http://turbulence.pha.jhu.edu (2008–2016). [Accessed 30 Nov 2016]
JUQUEEN: Jülich blue gene/q. http://www.fz-juelich.de/ias/juqueen (2012–2015). [Accessed 01 Aug 2015]
Kellner, A.: Typo3/powermail. https://docs.typo3.org/typo3cms/extensions/powermail (2005–2016). [Accessed 30 Nov 2016]
Lecarpentier, D., Wittenburg, P., Elbers, W., Michelini, A., Kanso, R., Coveney, P., Baxter, R.: Eudat: a new cross-disciplinary data infrastructure for science. Int. J. Digit. Curation 8(1), 279–287 (2013)
Li, Y., Perlman, E., Wan, M., Yang, Y., Meneveau, C., Burns, R., Chen, S., Szalay, A., Eyink, G.: A public turbulence database cluster and applications to study lagrangian evolution of velocity increments in turbulence. J. Turbul. (9), N31 (2008)
Shraiman, B.I., Siggia, E.D.: Scalar turbulence. Nature 405(6787), 639–646 (2000)
The HDF Group: Hierarchical data format version 5. http://www.hdfgroup.org/HDF5 (2000–2016). [Accessed 30 Nov 2016]
UCAR/Unidata: Network common data format. http://www.unidata.ucar.edu/software/netcdf (1989–2016). [Accessed 30 Nov 2016]
Acknowledgements
The authors gratefully acknowledge the computing time granted by the John von Neumann Institute for Computing (NIC) and provided on the supercomputer JURECA at Jülich Supercomputing Centre (JSC) in the context of the Scientific Big Data Analytics (SBDA) project No. 006.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Tweddell, B., Göbbert, J.H., Gauding, M., Weyers, B., Hagemeier, B. (2017). Framework for Sharing of Highly Resolved Turbulence Simulation Data. In: Di Napoli, E., Hermanns, MA., Iliev, H., Lintermann, A., Peyser, A. (eds) High-Performance Scientific Computing. JHPCS 2016. Lecture Notes in Computer Science(), vol 10164. Springer, Cham. https://doi.org/10.1007/978-3-319-53862-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-53862-4_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53861-7
Online ISBN: 978-3-319-53862-4
eBook Packages: Computer ScienceComputer Science (R0)