Abstract
Reconstructing 3D volumes from optical microscopic images is useful in important areas such as cellular analysis, cancer research, and drug development. However, existing techniques either require specialized hardware or extensive sample preprocessing. Recently, Yamaguchi et al. [20] proposed to solve this problem by just using a single stack of optical microscopic images with different focus settings and reconstructing a voxel-based representation of the observation using the classical iterative optimization method. Inspired by this result, this work aims to explore this method further using new state-of-the-art optimization techniques such as Deep Image Prior (DIP). Our analysis showcases the superiority of this approach over Yamaguchi et al. [20] in reconstruction quality, hard metrics, and robustness to noise on the synthetic data. Finally, we also demonstrate the effectiveness of our approach on real data, producing excellent reconstruction quality. Code available at: https://github.com/caiocj1/multifocus-3d-reconstruction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abrahamsson, S., Chen, J., Hajj, B., Stallinga, S., Katsov, A.Y., Wisniewski, J., Mizuguchi, G., Soule, P., Mueller, F., Darzacq, C.D., et al.: Fast multicolor 3D imaging using aberration-corrected multifocus microscopy. Nature Methods 10(1), 60–63 (2013)
Barbano, R., Leuschner, J., Schmidt, M., Denker, A., Hauptmann, A., Maass, P., Jin, B.: An Educated Warm Start for Deep Image Prior-Based Micro CT Reconstruction. IEEE Transactions on Computational Imaging 8, 1210–1222 (2022)
Calì, C., Agus, M., Kare, K., Boges, D.J., Lehväslaiho, H., Hadwiger, M., Magistretti, P.J.: 3d cellular reconstruction of cortical glia and parenchymal morphometric analysis from serial block-face electron microscopy of juvenile rat. Progress in neurobiology 183, 101696 (2019)
Chen, Y., Shen, Q., White, S.L., Gokmen-Polar, Y., Badve, S., Goodman, L.J.: Three-dimensional imaging and quantitative analysis in clarity processed breast cancer tissues. Scientific Reports 9(1), 5624 (2019)
Gong, K., Catana, C., Qi, J., Li, Q.: Pet image reconstruction using deep image prior. IEEE transactions on medical imaging 38(7), 1655–1665 (2018)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge university press (2003)
Hashimoto, F., Onishi, Y., Ote, K., Tashima, H., Yamaya, T.: Fully 3d implementation of the end-to-end deep image prior-based pet image reconstruction using block iterative algorithm. arXiv preprint arXiv:2212.11844 (2022)
Horé, A., Ziou, D.: Image quality metrics: Psnr vs. ssim. In: 2010 20th International Conference on Pattern Recognition. pp. 2366–2369 (2010). https://doi.org/10.1109/ICPR.2010.579
Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2017)
Klacansky, P.: Open SciVis Datasets (December 2017), https://klacansky.com/open-scivis-datasets/, https://klacansky.com/open-scivis-datasets/
Mertan, A., Duff, D.J., Unal, G.: Single image depth estimation: An overview. Digital Signal Processing p. 103441 (2022)
Nayar, S.K., Nakagawa, Y.: Shape from focus. IEEE Transactions on Pattern analysis and machine intelligence 16(8), 824–831 (1994)
Peddie, C.J., Collinson, L.M.: Exploring the third dimension: Volume electron microscopy comes of age. Micron 61, 9–19 (2014).https://doi.org/10.1016/j.micron.2014.01.009, https://www.sciencedirect.com/science/article/pii/S0968432814000250
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. pp. 234–241. Springer (2015)
Suwajanakorn, S., Hernandez, C., Seitz, S.M.: Depth from focus with your mobile phone. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3497–3506 (2015)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep Image Prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018)
Upla, P., Kim, S.J., Sampathkumar, P., Dutta, K., Cahill, S.M., Chemmama, I.E., Williams, R., Bonanno, J.B., Rice, W.J., Stokes, D.L., Cowburn, D., Almo, S.C., Sali, A., Rout, M.P., Fernandez-Martinez, J.: Molecular architecture of the major membrane ring component of the nuclear pore complex. Structure 25(3), 434–445 (2017).https://doi.org/10.1016/j.str.2017.01.006, https://www.sciencedirect.com/science/article/pii/S0969212617300060
Wang, H., Li, T., Zhuang, Z., Chen, T., Liang, H., Sun, J.: Early stopping for deep image prior (2022)
Wenzel, C., Riefke, B., Gründemann, S., Krebs, A., Christian, S., Prinz, F., Osterland, M., Golfier, S., Räse, S., Ansari, N., Esner, M., Bickle, M., Pampaloni, F., Mattheyer, C., Stelzer, E.H., Parczyk, K., Prechtl, S., Steigemann, P.: 3d high-content screening for the identification of compounds that target cells in dormant tumor spheroid regions. Experimental Cell Research 323(1), 131–143 (2014).https://doi.org/10.1016/j.yexcr.2014.01.017, https://www.sciencedirect.com/science/article/pii/S0014482714000263
Yamaguchi, T., Nagahara, H., Morooka, K., Nakashima, Y., Uranishi, Y., Miyauchi, S., Kurazume, R.: 3D Image Reconstruction from Multi-focus Microscopic Images. In: Dabrowski, J.J., Rahman, A., Paul, M. (eds.) Image and Video Technology. pp. 73–85. Springer International Publishing, Cham (2020)
Yoo, S., Ruiz, P., Huang, X., He, K., Ferrier, N.J., Hereld, M., Selewa, A., Daddysman, M., Scherer, N., Cossairt, O., Katsaggelos, A.K.: 3D Image Reconstruction from Multi-Focus Microscope: Axial Super-Resolution and Multiple-Frame Processing. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1453–1457 (2018).https://doi.org/10.1109/ICASSP.2018.8462234
Zhou, K.C., Horstmeyer, R.: Diffraction tomography with a deep image prior. Optics express 28(9), 12872–12896 (2020)
Acknowledgments
The part of this project were supported by JST AIP Acceleration Research JPMJCR23U4 and JSPS KAKENHI Grant Numbers JP23H05490.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Azevedo, C., Santra, S., Kumawat, S., Nagahara, H., Morooka, K. (2024). Deep Volume Reconstruction from Multi-focus Microscopic Images. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15004. Springer, Cham. https://doi.org/10.1007/978-3-031-72083-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-72083-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72082-6
Online ISBN: 978-3-031-72083-3
eBook Packages: Computer ScienceComputer Science (R0)