Skip to main content
SpringerLink
Log in

Performance optimization and evaluation for parallel processing of big data in earth system models

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Big data and high performance computing in Earth System Models (ESMs) are receiving increased attention in earth science research. When scaling to large-scale multi-core computing, efficient parallelization of an ESM, which demands fast parallel computing for long-term integration or climate simulation, becomes extremely challenging because of time-consuming internal big data communication. In this paper, an optimization algorithm for the massive data communication between the Weather Research and Forecasting model and Coupler version 7 in the Chinese Academy of Sciences-Earth System Model (CAS-ESM) is proposed. The optimization strategy is to transmit data from a small packet into a larger packet. Through experiments on a multi-core cluster, the efficiency of the algorithm is confirmed. Then, the parallel performance of the CAS-ESM is evaluated fully. Results show that the parallel efficiency of the CAS-ESM on 1024 CPU cores reaches nearly 70%, indicating that the CAS-ESM has desirable parallel performance and strong scalability. In addition, a generic performance evaluation method for ESMs from perspectives of optimal load balance and efficiency is proposed. Results show that the computing speed is the fastest and computational efficiency is the highest when the CAS-ESM runs on a certain number of cores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Wang, L., Geng, H., Liu, P., et al.: Particle swarm optimization based dictionary learning for remote sensing big data. Knowl.-Based Syst. 79, 43–50 (2015)

    Article  Google Scholar 

  2. Wang, L., Lu, K., Liu, P., et al.: IK-SVD: dictionary learning for spatial big data via incremental atom update. Comput. Sci. Eng. 16(4), 41–52 (2014)

    Article  Google Scholar 

  3. Song, W., Liu, P., Wang, L.: Sparse representation-based correlation analysis of non-stationary spatiotemporal big data. Int. J. Digit. Earth 9(9), 892–913 (2016)

    Article  Google Scholar 

  4. Wang, L., Song, W., Liu, P.: Link the remote sensing big data to the image features via wavelet transformation. Clust. Comput. 19(2), 793–810 (2016)

    Article  Google Scholar 

  5. He, Z., Wu, C., Liu, G., Zheng, Z., Tian, Y.: Decomposition tree: a spatio-temporal indexing method for movement big data. Clust. Comput. 18(4), 1481–1492 (2015)

    Article  Google Scholar 

  6. Wang, Y., Liu, Z., Liao, H., Li, C.: Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing. Clust. Comput. 18(2), 507–516 (2015)

    Article  Google Scholar 

  7. Deng, Z., Hu, Y., Zhu, M., Huang, X., Du, B.: A scalable and fast OPTICS for clustering trajectory big data. Clust. Comput. 18(2), 549–562 (2015)

    Article  Google Scholar 

  8. Chen, Y., Li, F., Fan, J.: Mining association rules in big data with NGEP. Clust. Comput. 18(2), 577–585 (2015)

    Article  Google Scholar 

  9. Song, W., Deng, Z., Wang, L., Du, B., Liu, P., Lu, K.: G-IK-SVD: parallel IK-SVD on GPUs for sparse representation of spatial big data. J. Supercomput. 73(8), 3433–3450 (2017)

    Article  Google Scholar 

  10. Xue, W., Yang, C., Fu, H., Wang, X., Xu, Y., Liao, J., Gan, L., Lu, Y., Ranjan, R., Wang, L.: Ultra-scalable CPU-MIC acceleration of mesoscale atmospheric modeling on Tianhe-2. IEEE Trans. Comput. 64(8), 2382–2393 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Wang, L., Ma, Y., Yan, J., Chang, V., Zomaya, A.Y.: pipsCloud: high performance cloud computing for remote sensing big data management and processing. Future Gener. Comput. Syst. 78, 353–368 (2018)

    Article  Google Scholar 

  12. Hurrell, J.W., Holland, M.M., Gent, P.R., et al.: The community earth system model: a framework for collaborative research. Bull. Am. Meteorol. Soc. 94(9), 1339–1360 (2013)

    Article  Google Scholar 

  13. Vertenstein, M., Craig, T., Middleton, A., Feddema, D., Fischer, C.: CESM1. 0.4 Users Guide. Technical report, Community Earth System Model, NCAR, USA (2011)

  14. Sun, H., Zhou, G., Zeng, Q.: Assessments of the climate system model (CAS-ESM-C) using IAP AGCM4 as its atmospheric component. Chin. J. Atmos. Sci. 36(2), 215–233 (2012). in Chinese

    Google Scholar 

  15. Dong, X., Su, T., Wang, J., Lin, R.: Decadal variation of the Aleutian low-icelandic low seesaw simulated by a climate system model (CAS-ESM-C). Atmos. Ocean. Sci. Lett. 7(2), 110–114 (2014)

    Article  Google Scholar 

  16. Taylor, K.E., Stouffer, R.J., Meehl, G.A.: An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 93(4), 485–498 (2012)

    Article  Google Scholar 

  17. Montoya, M., Griesel, A., Levermann, A., Mignot, J., Hofmann, M., Ganopolski, A., Rahmstorf, S.: The earth system model of intermediate complexity CLIMBER-3\(\alpha \). Part I: description and performance for present-day conditions. Clim. Dyn. 25(2–3), 237–263 (2005)

    Article  Google Scholar 

  18. Duffy, P.B., Govindasamy, B., Iorio, J.P., et al.: High-resolution simulations of global climate, part 1: present climate. Clim. Dyn. 21(5–6), 371–390 (2003)

    Article  Google Scholar 

  19. Khairoutdinov, M.F., Randall, D.A.: A cloud resolving model as a cloud parameterization in the NCAR community climate system model: preliminary results. Geophys. Res. Lett. 28(18), 3617–3620 (2001)

    Article  Google Scholar 

  20. Washington, W.M., Buja, L., Craig, A.: The computational future for climate and earth system models: on the path to petaflop and beyond. Philos. Trans. R. Soc. Lond. A 367(1890), 833–846 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wehner, M.F., Reed, K.A., Li, F., et al.: The effect of horizontal resolution on simulation quality in the Community Atmospheric Model, CAM5.1. J. Adv. Model. Earth Syst. 6(4), 980–997 (2014)

    Article  Google Scholar 

  22. Nakaegawa, T., Kitoh, A., Ishizaki, Y., Kusunoki, S., Murakami, H.: Caribbean low-level jets and accompanying moisture fluxes in a global warming climate projected with CMIP3 multi-model ensemble and fine-mesh atmospheric general circulation models. Int. J. Climatol. 34(4), 964–977 (2014)

    Article  Google Scholar 

  23. Miyamoto, Y., Kajikawa, Y., Yoshida, R., Yamaura, T., Yashiro, H., Tomita, H.: Deep moist atmospheric convection in a subkilometer global simulation. Geophys. Res. Lett. 40(18), 4922–4926 (2013)

    Article  Google Scholar 

  24. Craig, A.P., Vertenstein, M., Jacob, R.: A new flexible coupler for earth system modeling developed for CCSM4 and CESM1. Int. J. High Perform. Comput. Appl. 26(1), 31–42 (2012)

    Article  Google Scholar 

  25. Dennis, J.M., Edwards, J., Evans, K.J., et al.: CAM-SE: a scalable spectral element dynamical core for the Community Atmosphere Model. Int. J. High Perform. Comput. Appl. 26(1), 74–89 (2012)

    Article  Google Scholar 

  26. Dennis, J.M., Vertenstein, M., Worley, P.H., Mirin, A.A., Craig, A.P., Jacob, R., Mickelson, S.: Computational performance of ultra-high-resolution capability in the Community Earth System Model. Int. J. High Perform. Comput. Appl. 26(1), 5–16 (2012)

    Article  Google Scholar 

  27. Wehner, M.F., Ambrosiano, J.J., Brown, J.C., et al.: Toward a high performance distributed memory climate model. In: High Performance Distributed Computing, 1993. IEEE Proceedings the 2nd International Symposium, pp. 102–113 (1993)

  28. Mechoso, C.R., Drummond, L.A., Farrara, J.D., Spahr, J.A.: The UCLA AGCM in high performance computing environments. In: Proceedings of the 1998 ACM/IEEE Conference on Supercomputing, pp. 1–7. IEEE Computer Society (1998)

  29. Drake, J., Foster, I., Michalakes, J., Toonen, B., Worley, P.: Design and performance of a scalable parallel community climate model. Parallel Comput. 21(10), 1571–1591 (1995)

    Article  MATH  Google Scholar 

  30. Mirin, A.A., Sawyer, W.B.: A scalable implementation of a finite-volume dynamical core in the Community Atmosphere Model. Int. J. High Perform. Comput. Appl. 19(3), 203–212 (2005)

    Article  Google Scholar 

  31. Zou, Y., Xue, W., Liu, S.: A case study of large-scale parallel I/O analysis and optimization for numerical weather prediction system. Future Gener. Comput. Syst. 37, 378–389 (2014)

    Article  Google Scholar 

  32. Li, L., Xue, W., Ranjan, R., Jin, Z.: A scalable Helmholtz solver in GRAPES over large-scale multicore cluster. Concurr. Comput. 25(12), 1722–1737 (2013)

    Article  Google Scholar 

  33. Zhang, T., Sun, X., Xue, W., Qiao, N., Huang, H., Shu, J., Zheng, W.: ParSA: high-throughput scientific data analysis framework with distributed file system. Future Gener. Comput. Syst. 51, 111–119 (2015)

    Article  Google Scholar 

  34. Zhang, T., Li, L., Lin, Y., Xue, W., Xie, F., Xu, H., Huang, X.: An automatic and effective parameter optimization method for model tuning. Geosci. Model Dev. 8(11), 3579–3591 (2015)

    Article  Google Scholar 

  35. Wang, Y., Jiang, J., Ye, H., He, J.: A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster. Concurr. Comput. 28(15), 4144–4160 (2016)

    Article  Google Scholar 

  36. Zhang, H., Zhang, M., Zeng, Q.: Sensitivity of simulated climate to two atmospheric models: interpretation of differences between dry models and moist models. Mon. Weather Rev. 141(5), 1558–1576 (2013)

    Article  Google Scholar 

  37. Wang, Y., Jiang, J., Zhang, H., et al.: A scalable parallel algorithm for atmospheric general circulation models on a multi-core cluster. Future Gener. Comput. Syst. 72, 1–10 (2017)

    Article  Google Scholar 

  38. Skamarock, W.C., Klemp, J.B., Dudhia, J., et al.: A description of the advanced research WRF version 3. NCAR technical note, TN-475+STR (2008)

  39. Johnsen, P., Straka, M., Shapiro, M., Norton, A., Galarneau, T.: Petascale WRF simulation of hurricane sandy: Deployment of NCSA’s cray XE6 blue waters. In: High Performance Computing, Networking, Storage and Analysis (SC’13), pp. 1–7. IEEE (2013)

  40. Xie, S., Zhang, M., Branson, M., et al.: Simulations of midlatitude frontal clouds by single-column and cloud-resolving models during the atmospheric radiation measurement March 2000 cloud intensive operational period. J. Geophys. Res. 110(D15) (2005)

  41. He, J., Zhang, M., Lin, W., Colle, B., Liu, P., Vogelmann, A.M.: The WRF nested within the CESM: simulations of a midlatitude cyclone over the Southern Great Plains. J. Adv. Model. Earth Syst. 5(3), 611–622 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No. 2016YFB0200800), National Natural Science Foundation of China (No. 61602477, No. 41401512), China Postdoctoral Science Foundation (No. 2016M601158), Youth Innovation Promotion Association of CAS (No. Y6YR0300QM), and the Fundamental Research Funds for the Central Universities (No. 2652017113).

Author information

Authors and Affiliations

  1. School of Information Engineering, China University of Geosciences, Beijing, People’s Republic of China

    Yuzhu Wang

  2. Computer Network Information Center, Chinese Academy of Sciences, Beijing, People’s Republic of China

    Huiqun Hao & Jinrong Jiang

  3. University of Chinese Academy of Sciences, Beijing, People’s Republic of China

    Huiqun Hao

  4. School of Computer Science, China University of Geosciences, Wuhan, People’s Republic of China

    Junqiang Zhang

  5. Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, People’s Republic of China

    Juanxiong He

  6. Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing, People’s Republic of China

    Yan Ma

Authors
  1. Yuzhu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqun Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinrong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Juanxiong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Junqiang Zhang or Yan Ma.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Hao, H., Zhang, J. et al. Performance optimization and evaluation for parallel processing of big data in earth system models. Cluster Comput 22 (Suppl 1), 2371–2381 (2019). https://doi.org/10.1007/s10586-017-1477-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-1477-0

Keywords

Search

Navigation