Abstract
Recently, a lot of automated white blood cells (WBC) or leukocyte classification techniques have been developed. However, all of these methods only utilize a single modality microscopic image i.e. either blood smear or fluorescence based, thus missing the potential of a better learning from multimodal images. In this work, we develop an efficient multimodal architecture based on a first of its kind multimodal WBC dataset for the task of WBC classification. Specifically, our proposed idea is developed in two steps - 1) First, we learn modality specific independent subnetworks inside a single network only; 2) We further enhance the learning capability of the independent subnetworks by distilling knowledge from high complexity independent teacher networks. With this, our proposed framework can achieve a high performance while maintaining low complexity for a multimodal dataset. Our unique contribution is two-fold - 1) We present a first of its kind multimodal WBC dataset for WBC classification; 2) We develop a high performing multimodal architecture which is also efficient and low in complexity at the same time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Please note that the colors of the images are for representation only and not the actual colors. The technical details of excitation lights and actual colours will be released once the disclosure has been filed.
References
Baltrušaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 423–443 (2018)
Blumenreich, M.S.: The white blood cell and differential count. In: Clinical Methods: The History, Physical, and Laboratory Examinations. 3rd edn. (1990)
Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 702–703 (2020)
Damer, N., Dimitrov, K., Braun, A., Kuijper, A.: On learning joint multi-biometric representations by deep fusion. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2019)
Das, B., Bansal, S., Mohanta, G.C., Debnath, S.K., Bhatia, P.: Fluorescence imaging-based system for performing white blood cell counts. In: Singh, K., Gupta, A.K., Khare, S., Dixit, N., Pant, K. (eds.) ICOL-2019. SPP, vol. 258, pp. 617–620. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-9259-1_142
D’mello, S.K., Kory, J.: A review and meta-analysis of multimodal affect detection systems. ACM Comput. Surv. (CSUR). 47(3), 1–36 (2015)
Forcucci, A., Pawlowski, M.E., Majors, C., Richards-Kortum, R., Tkaczyk, T.S.: All-plastic, miniature, digital fluorescence microscope for three part white blood cell differential measurements at the point of care. Biomed. Opt. Express 6(11), 4433–4446 (2015)
Gadzicki, K., Khamsehashari, R., Zetzsche, C.: Early vs late fusion in multimodal convolutional neural networks. In: 2020 IEEE 23rd International Conference on Information Fusion (FUSION), pp. 1–6. IEEE (2020)
George-Gay, B., Parker, K.: Understanding the complete blood count with differential. J. Perianesth. Nurs. 18(2), 96–117 (2003)
Gou, J., Yu, B., Maybank, S.J., Tao, D.: Knowledge distillation: a survey. Int. J. Comput. Vision 129(6), 1789–1819 (2021)
Havasi, M., et al.: Training independent subnetworks for robust prediction. arXiv preprint arXiv:2010.06610 (2020)
Hinton, G., et al.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.025312(7) (2015)
Khamael, A.D., Banks, J., Nugyen, K., Al-Sabaawi, A., Tomeo-Reyes, I., Chandran, V.: Segmentation of white blood cell, nucleus and cytoplasm in digital haematology microscope images: a review-challenges, current and future potential techniques. IEEE Rev. Biomed. Eng. 14, 290–306 (2020)
Liu, J., Zhang, S., Wang, S., Metaxas, D.N.: Multispectral deep neural networks for pedestrian detection. arXiv preprint arXiv:1611.02644 (2016)
Ojaghi, A., et al.: Label-free hematology analysis using deep-ultraviolet microscopy. Proc. Natl. Acad. Sci. 117(26), 14779–14789 (2020)
Ramesh, N., Dangott, B., Salama, M.E., Tasdizen, T.: Isolation and two-step classification of normal white blood cells in peripheral blood smears. J. Pathol. Inform. 3, 13 (2012)
Snoek, C.G., Worring, M., Smeulders, A.W.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, pp. 399–402 (2005)
Theml, H., Diem, H., Haferlach, T.: Color Atlas of Hematology: Practical Microscopic and Clinical Diagnosis. Thieme, Leipzig (2004)
Tkachuk, D.C., Hirschmann, J.V., Wintrobe, M.M.: Wintrobe’s Atlas of Clinical Hematology. Lippincott Williams & Wilkins, Philadelphia (2007)
Yakimov, B.P., et al.: Label-free characterization of white blood cells using fluorescence lifetime imaging and flow-cytometry: molecular heterogeneity and erythrophagocytosis. Biomed. Opt. Express 10(8), 4220–4236 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, L., Mehta, D., Mahapatra, D., Ge, Z. (2022). Leukocyte Classification Using Multimodal Architecture Enhanced by Knowledge Distillation. In: Huo, Y., Millis, B.A., Zhou, Y., Wang, X., Harrison, A.P., Xu, Z. (eds) Medical Optical Imaging and Virtual Microscopy Image Analysis. MOVI 2022. Lecture Notes in Computer Science, vol 13578. Springer, Cham. https://doi.org/10.1007/978-3-031-16961-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-16961-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16960-1
Online ISBN: 978-3-031-16961-8
eBook Packages: Computer ScienceComputer Science (R0)
