Abstract
Technological advancements in modern scientific instruments, such as scanning electron microscopes (SEMs), have significantly increased data acquisition rates and image resolutions enabling new questions to be explored; however, the resulting data volumes and velocities, combined with automated experiments, are quickly overwhelming scientists as there remain crucial steps that require human intervention, for example reviewing image focus. We present a fast out-of-focus detection algorithm for electron microscopy images collected serially and demonstrate that it can be used to provide near-real-time quality control for neuroscience workflows. Our technique, Multi-scale Histologic Feature Detection, adapts classical computer vision techniques and is based on detecting various fine-grained histologic features. We exploit the inherent parallelism in the technique to employ GPU primitives in order to accelerate characterization. We show that our method can detect out-of-focus conditions within just 20 ms. To make these capabilities generally available, we deploy our feature detector as an on-demand service and show that it can be used to determine the degree of focus in approximately 230 ms, enabling near-real-time use.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abbott, L.F., et al.: The mind of a mouse. Cell 182(6), 1372–1376 (2020)
Ananthakrishnan, R., et al.: Globus platform services for data publication. In: Practice and Experience on Advanced Research Computing, pp. 14:1–14:7 (2018)
Bian, Z., et al.: Autofocusing technologies for whole slide imaging and automated microscopy. J. Biophoton. 13(12), e202000227 (2020)
Bicer, T., et al.: Real-time data analysis and autonomous steering of synchrotron light source experiments. In: 13th International Conference on e-Science (e-Science), pp. 59–68. IEEE (2017)
Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)
Carl Zeiss AG: ZEISS MultiSEM Research Partner Program (10 2018), the World’s Fastest Scanning Electron Microscopes, October 2018
Chard, R., et al.: funcX: a federated function serving fabric for science. In: Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing, pp. 65–76 (2020)
Derpanis, K.G.: The gaussian pyramid (2005)
Duits, R., Florack, L., De Graaf, J., ter Haar Romeny, B.: On the axioms of scale space theory. J. Math. Imaging Vis. 20(3), 267–298 (2004)
Herculano-Houzel, S., Watson, C.R., Paxinos, G.: Distribution of neurons in functional areas of the mouse cerebral cortex reveals quantitatively different cortical zones. Front. Neuroanatom. 7 (2013)
Hua, Y., Laserstein, P., Helmstaedter, M.: Large-volume en-bloc staining for electron microscopy-based connectomics. Nat. Commun. 6(1), 1–7 (2015)
Kasthuri, N., et al.: Saturated reconstruction of a volume of neocortex. Cell 162(3), 648–661 (2015)
Koenderink, J.J.: The structure of images. Biol. Cybern. 50(5), 363–370 (1984)
von Laszeski, G., et al.: Real-time analysis, visualization, and steering of microtomography experiments at photon sources. Technical report, Argonne National Lab (2000)
Levental, M., et al.: Towards online steering of flame spray pyrolysis nanoparticle synthesis. In: IEEE/ACM 2nd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP), pp. 35–40. IEEE (2020)
Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30, 79–116 (2004)
Liu, Z., et al.: Bridging data center AI systems with edge computing for actionable information retrieval. In: 3rd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP), pp. 15–23. IEEE (2021)
Luo, Y., Huang, L., Rivenson, Y., Ozcan, A.: Single-shot autofocusing of microscopy images using deep learning. ACS Photon. 8(2), 625–638 (2021)
Marr, D., Hildreth, E.: Theory of edge detection. Proc. Royal Soc. London. Ser. B. Biol. Sci. 207(1167), 187–217 (1980)
Merrill, D.: Cuda unbound (cub). https://github.com/NVIDIA/cub (2021)
Neubeck, A., Van Gool, L.: Efficient non-maximum suppression. In: 18th International Conference on Pattern Recognition, vol. 3, pp. 850–855. IEEE (2006)
Pan, J., Libera, J.A., Paulson, N.H., Stan, M.: Flame stability analysis of flame spray pyrolysis by artificial intelligence. Int. J. Adv. Manuf. Technol. 2215–2228 (2021). https://doi.org/10.1007/s00170-021-06884-z
Potluri, S., Wang, H., Bureddy, D., Singh, A.K., Rosales, C., Panda, D.K.: Optimizing MPI communication on multi-GPU systems using CUDA inter-process communication. In: 26th International Parallel and Distributed Processing Symposium Workshops PhD Forum, pp. 1848–1857 (2012)
Redondo, R., et al.: Autofocus evaluation for brightfield microscopy pathology. J. Biomed. Opt. 17(3), 1–9 (2012)
Stevanovic, U., et al.: A control system and streaming DAQ platform with image-based trigger for x-ray imaging. IEEE Trans. Nucl. Sci. 62(3), 911–918 (2015)
Wildenberg, G.A., Rosen, M.R., Lundell, J., Paukner, D., Freedman, D.J., Kasthuri, N.: Primate neuronal connections are sparse in cortex as compared to mouse. Cell Rep. 36(11), 109709 (2021)
Yeo, T., Ong, S., Jayasooriah, Sinniah, R.: Autofocusing for tissue microscopy. Image Vis. Comput. 11(10), 629–639 (1993)
Sun, Y., Duthaler, S., Nelson, B.J.: Autofocusing algorithm selection in computer microscopy. In: International Conference on Intelligent Robots and Systems, pp. 70–76 (2005)
Acknowledgements
This work was supported by the U.S. Department of Energy, Office of Science, under contract DE-AC02-06CH11357.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Levental, M., Chard, R., Chard, K., Foster, I., Wildenberg, G. (2022). Ultrafast Focus Detection for Automated Microscopy. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13350. Springer, Cham. https://doi.org/10.1007/978-3-031-08751-6_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-08751-6_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08750-9
Online ISBN: 978-3-031-08751-6
eBook Packages: Computer ScienceComputer Science (R0)