Skip to main content
SpringerLink
Log in

Accelerating the cryo-EM structure determination in RELION on GPU cluster

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

The cryo-electron microscopy (cryo-EM) is one of the most powerful technologies available today for structural biology. The RELION (Regularized Likelihood Optimization) implements a Bayesian algorithm for cryo-EM structure determination, which is one of the most widely used software in this field. Many researchers have devoted effort to improve the performance of RELION to satisfy the analysis for the ever-increasing volume of datasets. In this paper, we focus on performance analysis of the most time-consuming computation steps in RELION and identify their performance bottlenecks for specific optimizations. We propose several performance optimization strategies to improve the overall performance of RELION, including optimization of expectation step, parallelization of maximization step, accelerating the computation of symmetries, and memory affinity optimization. The experiment results show that our proposed optimizations achieve significant speedups of RELION across representative datasets. In addition, we perform roofline model analysis to understand the effectiveness of our optimizations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Frank J, Shimkin B, Dowse H. Spider—a modular software system for electron image processing. Ultramicroscopy, 1981, 6(4): 343–357

    Article  Google Scholar 

  2. Grigorieff N. Frealign: high-resolution refinement of single particle structures. Journal of Structural Biology, 2007, 157(1): 117–125

    Article  Google Scholar 

  3. Tang G, Peng L, Baldwin P R, Mann D S, Jiang W, Rees I, Ludtke S J. Eman2: an extensible image processing suite for electron microscopy. Journal of Structural Biology, 2007, 157(1): 38–46

    Article  Google Scholar 

  4. Elmlund D, Elmlund H. Simple: software for ab initio reconstruction of heterogeneous single-particles. Journal of Structural Biology, 2012, 180(3): 420–427

    Article  Google Scholar 

  5. Schrers S HW. Relion: implementation of a bayesian approach to cryoem structure determination. Journal of Structural Biology, 2012, 180(3): 519–530

    Article  Google Scholar 

  6. Punjani A, Rubinstein J L, Fleet D J, Brubaker M A. Cryosparc: algorithms for rapid unsupervised cryo-EM structure determination. Nature Methods, 2017, 14(3): 290

    Article  Google Scholar 

  7. Hu M, Yu H, Gu K, Wang Z, Ruan H, et al. A particle-filter framework for robust cryo-em 3d reconstruction. Nature Methods, 2018, 15(12): 1083

    Article  Google Scholar 

  8. Khoshouer M, Radjainia M, Baumeister W, Danev R. Cryo-em structure of haemoglobin at 3.2 å determined with the volta phase plate. Nature Communications, 2017, 8: 16099

    Article  Google Scholar 

  9. Paulino C, Kalienkova V, Lam A KM, Neldner Y, Dutzler R. Activation mechanism of the calciumactivated chloride channel tmem16a revealed by cryo-EM. Nature, 2017, 552(7685): 421

    Article  Google Scholar 

  10. Bai X, Yan C, Yang G, Lu P, Ma D, et al. An atomic structure of human γ-secretase. Nature, 2015, 525(7568): 212

    Article  Google Scholar 

  11. Fernandez-Leiro R, Scheres S HW. A pipeline approach to single-particle processing in relion. Acta Crystallographica Section D: Structural Biology, 2017, 73(6): 496–502

    Article  Google Scholar 

  12. Su H, Wen W, Du X, Lu X, Liao M, et al. Gerelion: Gpu-enhanced parallel implementation of single particle cryo-EM image processing. bioRxiv, 2016, 075887

  13. Kimanius D, Forsberg B O, Scheres S HW, Lindahl E. Accelerated cryo-EM structure determination with parallelisation using GPUs in relion-2. Elife, 2016, 5: e18722

    Article  Google Scholar 

  14. You X, Yang H, Luan Z, Qian D. Performance analysis and optimization of cyro-em structure determination in relion-2. In: Proceedings of Conference on Advanced Computer Architecture. 2018: 195–209

  15. Relion version 2.1 stable, 2017.

  16. Li X, Grigorieff N, Cheng Y. GPU-enabled frealign: accelerating single particle 3d reconstruction and refinement in fourier space on graphics processors. Journal of Structural Biology, 2010, 172(3): 407–412

    Article  Google Scholar 

  17. Wang K, Xu S, Yu H, Fu H, Yang G. GPU-based 3d cryo-EM reconstruction with key-value streams: poster. In: Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming. 2019: 421–422

  18. Wang W, Duan B, Tang W, Zhang C, Tang G, Zhang P, Sun N. A coarse-grained stream architecture for cryo-electron microscopy images 3d reconstruction. In: Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 2012: 143–152

  19. Zivanov J, Nakane T, Forsberg B O, Kimanius D, Hagen W JH, Lindahle E, Scheres S HW. New tools for automated high-resolution cryo-EM structure determination in relion-3. Elife, 2018, 7: e42166

    Article  Google Scholar 

  20. Pipe J G, Menon P. Sampling density compensation in mri: rationale and an iterative numerical solution. Magnetic Resonance in Medicine, 1999, 41(1): 179–186

    Article  Google Scholar 

  21. Reinders J. VTune (TM) Performance Analyzer Essentials: Measurement and Tuning Techniques for Software Developers. 1st ed. California: Intel Press, 2004.

    Google Scholar 

  22. Wang E, Zhang Q, Shen B, Zhang G, Lu X, Wu Q, Wang Y. High-Performance Computing on the Intel® Xeon Phi™. 1st ed. New York: Springer, 2014: 167–188.

    Google Scholar 

  23. CUDA Nvidia. Cufft library, 2010.

  24. Frigo M, Johnson S G. Fftw user’s manual. Massachusetts Institute of Technology, 1999

  25. Hursey J, Mallove E, Squyres J M, Lumsdaine A. An extensible framework for distributed testing of mpi implementations. In: Proceedings of Euro PVM/MPI. 2007.

  26. Williams S, Waterman A, Patterson D. Roofline: an insightful visual performance model for multicore architectures. Communications of the ACM, 2009, 52(4): 65–76

    Article  Google Scholar 

  27. NVIDIA. Nvidia tesla v100 performance, 2019.

  28. Intel. Intel® xeon® gold 6148 processor, 2019.

  29. Sodani A. Knights landing (knl): 2nd generation intel® xeon phi processor. In: Proceedings of 2015 IEEE Hot Chips 27 Symposium (HCS). 2015: 1–24

  30. David H, Gorbatoy E, Hanebutte U R, Khanna R, Le C. Rapl: memory power estimation and capping. In: Proceedings of 2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED). 2010: 189–194

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China (2020YFB1506703), the National Natural Science Foundation of China (Grant No. 62072018), and the Open Project Program of the State Key Laboratory of Mathematical Engineering and Advanced Computing (2019A12). Hailong Yang is the corresponding author.

Author information

Authors and Affiliations

  1. Sino-German Joint Software Institute, School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Xin You, Hailong Yang, Zhongzhi Luan & Depei Qian

  2. State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, 214125, China

    Hailong Yang

Authors
  1. Xin You
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hailong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongzhi Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Depei Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hailong Yang.

Additional information

Xin You is a PhD student in School of Computer Science and Engineering, Beihang University, China. He is currently working on identifying performance opportunities for both scientific and AI applications. His research interests include HPC, performance optimization and performance analysis tools.

Hailong Yang is an associate professor in School of Computer Science and Engineering, Beihang University, China. He received the PhD degree in the School of Computer Science and Engineering, Beihang University in 2014. His research interests include parallel and distributed computing, HPC, performance optimization, and energy efficiency. He is a member of China Computer Federation (CCF).

Zhongzhi Luan received the PhD degree in the School of Computer Science in Xi’an Jiaotong University, China. He is an associate professor of computer science and engineering, and assistant director of the Sino-German Joint Software Institute (JSI) Laboratory at Beihang University, China. Since 2003, His research interests include distriparallel computing, grid computing, HPC and the network technology.

Depei Qian is a professor at the Department of Computer Science and Engineering, Beihang University, China. He received his masters degree from University of North Texas, USA in 1984. He is currently serving as the chief scientist of China National High Technology Program (863 Program) on high productivity computer and service environment. He is also a fellow of China Computer Federation (CCF). His research interests include innovative technologies in distributed computing, high performance computing and computer architecture.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, X., Yang, H., Luan, Z. et al. Accelerating the cryo-EM structure determination in RELION on GPU cluster. Front. Comput. Sci. 16, 163102 (2022). https://doi.org/10.1007/s11704-020-0169-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-0169-8

Keywords

Search

Navigation