Abstract
Corneal confocal laser scanning microscopy is a promising method for in vivo investigation of cellular structures, e. g., of nerve fibers in the sub-basal nerve plexus. During recording, even slight displacements of the focal plane lead to images of adjacent tissue layers. In this work, we propose a closed-loop control of the focal plane. To detect and evaluate the visible tissues, we utilize the Bag of Visual Words approach to implement a customizable image processing pipeline for real-time applications. Furthermore, we show that the proposed model can be trained with small classification datasets and can be applied as a segmentation method. The proposed control loop, including tissue detection, is implemented in a proof-of-concept setup and shows promising results in a first evaluation with a human subject.
Zusammenfassung
Die Konfokalmikroskopie der Kornea ist eine vielversprechende Methode zur in vivo Untersuchung von Zellstrukturen, z. B. von Nervenfasern im subbasalen Nervenplexus. Während der Aufnahme können schon geringe Verschiebungen der Fokusebene dazu führen, dass Bildern benachbarter Gewebeschichten aufgenommen werden. In dieser Arbeit schlagen wir eine Regelung der Fokusebene vor. Zum Erkennen und Auswerten der sichtbaren Gewebe verwenden wir die Bag of Visual Words Methode zur Implementierung einer anpassbaren Bildverarbeitungspipeline für Echtzeitanwendungen. Außerdem zeigen wir, dass das vorgeschlagene Modell mit kleinen Lerndatensätzen trainiert und als Segmentierungsmethode angewendet werden kann. Der vorgeschlagene Regelkreis, einschließlich der Gewebeerkennung, wurde im Rahmen einer Machbarkeitsstudie implementiert und zeigt vielversprechende Ergebnisse in einer ersten Auswertung mit einem menschlichen Probanden.
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: MI 1315/5-1
Award Identifier / Grant number: KO 5003/1-1
Award Identifier / Grant number: STA 543/6-1
Funding statement: This work was supported in part by the DFG (German Research Foundation) [MI 1315/5-1, KO 5003/1-1 and STA 543/6-1] and the Helmholtz Association.
About the authors
andreas.bartschat@kit.edu
Dr.-Ing. Stephan Allgeier works at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology. Research areas: Biomedical image processing, automated inspection of microstructures.
Tim Scherr works at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology. Research areas: Machine learning, image processing.
Jun.-Prof. Dr.-Ing. Johannes Stegmaier is heading the group for Biomedical Image Analysis at the Institute of Imaging and Computer Vision at RWTH Aachen University. Research areas: Automated analysis of large, time-resolved 3D microscopy image data.
Sebastian Bohn is PhD student at the Department of Ophthalmology at the Rostock University Medical Center. Research areas: Biophotonics, corneal confocal microscopy.
Klaus-Martin Reichert works at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology. Research areas: Biomedical image processing, automated inspection of microstructures.
Prof. Dr. Arjan Kuijper is research coach at Fraunhofer Institute for Computer Graphics Research IGD and professor at TU Darmstadt. Research areas: All aspects of mathematics-based methods for computer vision, graphics, imaging, pattern recognition, interaction, and visualization.
PD Dr.-Ing. Markus Reischl is project manager at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology. Research areas: Human-machine interfaces, image processing, machine learning, data mining.
apl. Prof. Dr. Oliver Stachs is the head of the Experimental Ophthalmology Group at the Department of Ophthalmology at the Rostock University Medical Center. Research areas: Medical Imaging, biophotonics, corneal confocal microscopy.
Dr.-Ing. Bernd Köhler is project manager at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology. Research areas: Biomedical image processing, automated inspection of microstructures.
apl. Prof. Dr.-Ing. Ralf Mikut is Head of the Research Area Automated Image and Data Analysis at the Institute for Automation and Applied Informatics at the Karlsruhe Institute of Technology and Speaker of the Helmholtz Information and Data Science School for Health (HIDSS4Health). Research areas: Computational intelligence, data analytics, modelling and image processing with applications in biology, chemistry, medical engineering, energy systems and robotics.
Acknowledgment
We would like to thank Katherine Quinlan-Flatter for proofreading.
References
1. A. Ahmed, V. Bril, A. Orszag, J. Paulson, E. Yeung, M. Ngo, S. Orlov and B. A. Perkins. Detection of Diabetic Sensorimotor Polyneuropathy by Corneal Confocal Microscopy in Type 1 Diabetes: A Concurrent Validity Study. Diabetes Care, 35(4):821–828, 2012.10.2337/dc11-1396Search in Google Scholar PubMed PubMed Central
2. S. Allgeier, A. Bartschat, S. Bohn, S. Peschel, K.-M. Reichert, K. Sperlich, M. Walckling, V. Hagenmeyer, R. Mikut, O. Stachs and B. Köhler. 3D Confocal Laser-Scanning Microscopy for Large-Area Imaging of the Corneal Subbasal Nerve Plexus. Scientific Reports, 8:7468, 2018.10.1038/s41598-018-25915-6Search in Google Scholar PubMed PubMed Central
3. S. Allgeier, S. Maier, R. Mikut, S. Peschel, K.-M. Reichert, O. Stachs and B. Köhler. Mosaicking the Subbasal Nerve Plexus by Guided Eye Movements. Investigative Ophthalmology & Visual Science, 55(9):6082–6089, 2014.10.1167/iovs.14-14698Search in Google Scholar PubMed
4. A. Bartschat, J. Stegmaier, S. Allgeier, K.-M. Reichert, S. Bohn, O. Stachs, B. Koehler and R. Mikut. Augmentations of the Bag of Visual Words Approach for Real-Time Fuzzy and Partial Image Classification. In Proc., 27. Workshop Computational Intelligence, Dortmund, 227–242, 2017.Search in Google Scholar
5. A. Bartschat, L. Toso, J. Stegmaier, A. Kuijper, R. Mikut, B. Koehler and S. Allgeier. Automatic corneal tissue classification using bag-of visual-words approaches. Forum Bildverarbeitung 2016, 84(7-8):245. KIT Scientific Publishing, 2016.Search in Google Scholar
6. H. Bay, T. Tuytelaars and L. Van Gool. SURF: Speeded Up Robust Features. In A. Leonardis, H. Bischof and A. Pinz, editors, 9th European Conference on Computer Vision (ECCV), 404–417. Springer Berlin Heidelberg, 2006.10.1007/11744023_32Search in Google Scholar
7. C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, NY, USA, 2006.Search in Google Scholar
8. G. Bitirgen, Z. Akpinar, R. A. Malik and A. Ozkagnici. Use of Corneal Confocal Microscopy to Detect Corneal Nerve Loss and Increased Dendritic Cells in Patients with Multiple Sclerosis. JAMA Ophthalmology, 135(7):777–782, 2017.10.1001/jamaophthalmol.2017.1590Search in Google Scholar PubMed PubMed Central
9. S. Bohn, K. Sperlich, S. Allgeier, A. Bartschat, R. Prakasam, K.-M. Reichert, H. Stolz, R. Guthoff, R. Mikut, B. Köhler and O. Stachs. Cellular in Vivo 3D Imaging of the Cornea by Confocal Laser Scanning Microscopy. Biomedical Optics Express, 9(6):2511–2525, 2018.10.1364/BOE.9.002511Search in Google Scholar PubMed PubMed Central
10. A. Bosch, A. Zisserman and X. Munoz. Representing Shape with a Spatial Pyramid Kernel. In Proc. of the 6th ACM International Conference on Image and Video Retrieval, 401–408. ACM, 2007.10.1145/1282280.1282340Search in Google Scholar
11. G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2236121, 2000.Search in Google Scholar
12. M. A. Bueno-Ibarra, J. Álvarez-Borrego, L. Acho and M. C. Chávez-Sánchez. Fast Autofocus Algorithm for Automated Microscopes. Optical Engineering, 44(6):063601, 2005.10.1117/1.1925119Search in Google Scholar
13. A. Cruzat, Y. Qazi and P. Hamrah. In Vivo Confocal Microscopy of Corneal Nerves in Health and Disease. The Ocular Surface, 15(1):15–47, 2017.10.1016/j.jtos.2016.09.004Search in Google Scholar PubMed PubMed Central
14. R. F. Guthoff, C. Baudouin and J. Stave. Atlas of Confocal Laser Scanning in-Vivo Microscopy in Ophthalmology. Springer Berlin Heidelberg, 2006.10.1007/3-540-32707-XSearch in Google Scholar
15. A. Kheirkhah, R. Muller, J. Mikolajczak, A. Ren, E. M. Kadas, H. Zimmermann, H. Pruess, F. Paul, A. U. Brandt and P. Hamrah. Comparison of Standard Versus Wide-Field Composite Images of the Corneal Subbasal Layer by in Vivo Confocal Microscopy. Investigative Ophthalmology & Visual Science, 56(10):5801–5807, 2015.10.1167/iovs.15-17434Search in Google Scholar PubMed PubMed Central
16. J. Kim and M. Markoulli. Automatic Analysis of Corneal Nerves Imaged Using in Vivo Confocal Microscopy. Clinical and Experimental Optometry, 101(2):147–161, 2018.10.1111/cxo.12640Search in Google Scholar PubMed
17. B. Köhler, S. Allgeier, A. Bartschat, R. F. Guthoff, S. Bohn, K.-M. Reichert, O. Stachs, K. Winter and R. Mikut. In-Vivo Bildgebung des Kornealen Nervenplexus. Der Ophthalmologe, 114(7):601–607, 2017.10.1007/s00347-017-0464-4Search in Google Scholar PubMed
18. N. S. Lagali, S. Allgeier, P. Guimarães, R. A. Badian, A. Ruggeri, B. Köhler, T. P. Utheim, B. Peebo, M. Peterson, L. B. Dahlin, et al. Wide-Field Corneal Subbasal Nerve Plexus Mosaics in Age-Controlled Healthy and Type 2 Diabetes Populations. Scientific data, 5, 2018.10.1038/sdata.2018.75Search in Google Scholar PubMed PubMed Central
19. G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken and C. I. Sánchez. A Survey on Deep Learning in Medical Image Analysis. Medical Image Analysis, 42:60–88, 2017.10.1016/j.media.2017.07.005Search in Google Scholar PubMed
20. R. A. Malik, P. Kallinikos, C. Abbott, C. H. van Schie, P. Morgan, N. Efron and A. J. Boulton. Corneal Confocal Microscopy: A Non-Invasive Surrogate of Nerve Fibre Damage and Repair in Diabetic Patients. Diabetologia, 46(5):683–688, 2003.10.1007/s00125-003-1086-8Search in Google Scholar PubMed
21. A. Oliva and A. Torralba. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. International Journal of Computer Vision, 42(3):145–175, 2001.10.1023/A:1011139631724Search in Google Scholar
22. B. A. Perkins, L. E. Lovblom, V. Bril, D. Scarr, I. Ostrovski, A. Orszag, K. Edwards, N. Pritchard, A. Russell, C. Dehghani, et al. Corneal Confocal Microscopy for Identification of Diabetic Sensorimotor Polyneuropathy: A Pooled Multinational Consortium Study. Diabetologia, 61(8):1856–1861, 2018.10.1007/s00125-018-4653-8Search in Google Scholar PubMed PubMed Central
23. W. M. Petroll and D. M. Robertson. In Vivo Confocal Microscopy of the Cornea: New Developments in Image Acquisition, Reconstruction, and Analysis Using the HRT-Rostock Corneal Module. The Ocular Surface, 13(3):187–203, 2015.10.1016/j.jtos.2015.05.002Search in Google Scholar PubMed PubMed Central
24. J. Platt. Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. Advances in Large Margin Classifiers, 10(3):61–74, 1999.Search in Google Scholar
25. N. Prodanova, J. Stegmaier, S. Allgeier, S. Bohn, O. Stachs, B. Köhler, R. Mikut and A. Bartschat. Transfer Learning with Human Corneal Tissues: An Analysis of Optimal Cut-Off Layer. arXiv preprint arXiv:1806.07073, 2018.Search in Google Scholar
26. D. Shen, G. Wu and H.-I. Suk. Deep Learning in Medical Image Analysis. Annual Review of Biomedical Engineering, 19:221–248, 2017.10.1146/annurev-bioeng-071516-044442Search in Google Scholar PubMed PubMed Central
27. J. Stave, G. Zinser, G. Grümmer and R. Guthoff. Modified Heidelberg Retinal Tomograph HRT. Initial Results of in Vivo Presentation of Corneal Structures. Der Ophthalmologe, 99(4):276–280, 2002.10.1007/s003470100535Search in Google Scholar PubMed
© 2019 Walter de Gruyter GmbH, Berlin/Boston
