Skip to main content
SpringerLink
Log in

Development and Validation of an Unsupervised Feature Learning System for Leukocyte Characterization and Classification: A Multi-Hospital Study

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The characterization and classification of white blood cells (WBC) are critical for the diagnosis of anemia, leukemia, and many other hematologic diseases. We developed WBC-Profiler, an unsupervised feature learning system for quantitative analysis of leukocytes. We demonstrate, through independent validation, that WBC-Profiler enables automatic extraction of complex and robust signatures from microscopic images without human-intervention and, thereafter, effective construction of interpretable leukocyte profiles, which decouples large scale complex leukocyte characterization from limitations in both human-based feature engineering/optimization and the end-to-end solutions provided by many modern deep neural networks. Further evaluation in a real-world clinical setting confirms that, compared with 23 clinicians from 8 hospitals (class-average-sensitivity, 0.798; class-average-specificity, 0.963; cell-average-timecost: 3.158  s), WBC-Profiler performs with significantly improved accuracy and speed (class-average-sensitivity, 0.890; class-average-specificity, 0.980; cell-average-timecost: 0.375  s). Our findings suggest that WBC-Profiler has the potential clinical implications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://bmihub.org/project/wbc-profiler.

References

  • Ahn, E., Kumar, A., Fulham, M., Feng, D., & Kim, J. (2019). Convolutional sparse kernel network for unsupervised medical image analysis. Medical Image Analysis, 56, 140–151.

    Article  Google Scholar 

  • Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828.

    Article  Google Scholar 

  • Chang, H., Zhou, Y., Borowsky, A., Barner, K. E., Spellman, P. T., & Parvin, B. (2015). Stacked predictive sparse decomposition for classification of histology sections. International Journal of Computer Vision, 113(1), 3–18.

    Article  MathSciNet  Google Scholar 

  • Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1800–1807).

  • Deng, J., Dong, W., Socher, R., Li, L., Li, K., & Li, F. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE computer society conference on computer vision and pattern recognition (CVPR 2009), 20–25 June 2009, Miami, Florida, USA (pp. 248–255).

  • Elsalamony, H. A. (2016). Detection of some anaemia types in human blood smears using neural networks. Measurement Science and Technology, 27(8), 085401.

    Article  Google Scholar 

  • Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118.

    Article  Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770–778).

  • Hinton, G. E., & Osindero, S. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.

    Article  MathSciNet  Google Scholar 

  • Huang, F. J., & LeCun, Y. (2006). Large-scale learning with SVM and convolutional for generic object categorization. In Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition—volume 1, CVPR ’06, (Washington, DC, USA) (pp. 284–291). IEEE Computer Society.

  • Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2010a). Fast inference in sparse coding algorithms with applications to object recognition.

  • Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2008). Fast inference in sparse coding algorithms with applications to object recognition. Tech. Rep. CBLL-TR-2008-12-01, Computational and Biological Learning Lab, Courant Institute, NYU.

  • Kavukcuoglu, K., Sermanet, P., Boureau, Y.-L., Gregor, K., Mathieu, M., & LeCun, Y. (2010b). Learning convolutional feature hierarchies for visual recognition. In Proceedings of the 23rd international conference on neural information processing systems—volume 1, NIPS’10 (pp. 1090–1098). Curran Associates Inc.

  • Kwolek, B. (2005). Face detection using convolutional neural networks and gabor filters. In Proceedings of the 15th international conference on artificial neural networks: Biological Inspirations—volume Part I, ICANN’05 (pp. 551–556). Springer.

  • Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 2278–2324.

  • Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007). Efficient sparse coding algorithms. In NIPS (pp. 801–808). NIPS.

  • Lee, H., Ekanadham, C., & Ng, A. Y.(2008). Sparse deep belief net model for visual area v2. In J. C. Platt, D. Koller, Y. Singer, & S. T. Roweis (Eds.) Advances in neural information processing systems 20 (pp. 873–880). Curran Associates, Inc.

  • Lee, T. S., & Mumford, D. (2003). Hierarchical Bayesian inference in the visual cortex. Journal of the Optical Society of America, 20, 1434–1448.

    Article  Google Scholar 

  • Lee, T. S., Mumford, D., Romero, R., & Lamme, V. A. (1998). The role of the primary visual cortex in higher level vision. Vision Research, 38(15/16), 2429–2454.

    Article  Google Scholar 

  • Lin, T., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In 2017 IEEE international conference on computer vision (ICCV) (pp. 2999–3007).

  • Liu, Y., Fu, Y., & Chen, P. (2019). Wbcaps: A capsule architecture-based classification model designed for white blood cells identification. In 2019 41st Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 7027–7030).

  • Lv, L., Zhao, D., & Deng, Q. (2016). A semi-supervised predictive sparse decomposition based on task-driven dictionary learning. Cognitive Computation, 9, 115–124.

    Article  Google Scholar 

  • Manik, S., Saini, L. M., & Vadera, N. (2016). Counting and classification of white blood cell using artificial neural network (ANN). In 2016 IEEE 1st international conference on power electronics, intelligent control and energy systems (ICPEICES) (pp. 1–5).

  • McKenna, S. J., Ricketts, I. W., Cairns, A. Y., & Hussein, K. A. (1993). A comparison of neural network architectures for cervical cell classification. In 1993 Third international conference on artificial neural networks (pp. 105–109).

  • Osadchy, M., Cun, Y. L., Miller, M. L., & Perona, P. (2005). Synergistic face detection and pose estimation with energy-based model. In In advances in neural information processing systems (NIPS (pp. 1017–1024). MIT Press.

  • Ouyang, W., Wang, X., Zhang, C., & Yang, X. (2016). Factors in finetuning deep model for object detection with long-tail distribution. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 864–873).

  • Poultney, C., Chopra, S., & Lecun, Y. (2006). Efficient learning of sparse representations with an energy-based model. In Advances in neural information processing systems (NIPS 2006 (pp. 1137–1144). MIT Press.

  • Preston, K. (1987). High-resolution leukocyte analyzers: Retrospective and prospective. Applied Optics, 26, 3258–3265.

    Article  Google Scholar 

  • Qian, L., & Shi, X. (2014). Denoising predictive sparse decomposition. In 2014 International conference on big data and smart computing (BIGCOMP) (pp. 223–228). IEEE Computer Society.

  • Qin, F., Gao, N., Peng, Y., Wu, Z., Shen, S., & Grudtsin, A. (2018). Fine-grained leukocyte classification with deep residual learning for microscopic images. Computer Methods and Programs in Biomedicine, 162, 243–252.

    Article  Google Scholar 

  • Rezatofighi, S. H., & Soltanian-Zadeh, H. (2011). Automatic recognition of five types of white blood cells in peripheral blood. Computerized Medical Imaging and Graphics, 35(4), 333–343.

    Article  Google Scholar 

  • Sahlol, A. T., Kollmannsberger, P., & Ewees, A. A. (2020). Efficient classification of white blood cell leukemia with improved swarm optimization of deep features. Scientific Reports, 10, 2536.

    Article  Google Scholar 

  • Shahin, A., Guo, Y., Amin, K., & Sharawi, A. (2017). White blood cells identification system based on convolutional deep neural learning networks. Computer Methods and Programs in Biomedicine, 168, 11.

    Google Scholar 

  • Simard, P. Y., Steinkraus, D., & Platt, J. C. (2003). Best practices for convolutional neural networks applied to visual document analysis. In Proceedings of the seventh international conference on document analysis and recognition—volume 2, ICDAR ’03, Washington, DC, USA (p. 958). IEEE Computer Society.

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. CoRR, arXiv:1409.1556.

  • Sobrevilla, P., Montseny, E., & Keller, J. (1999). White blood cell detection in bone marrow images. In 18th International conference of the north american fuzzy information processing society—NAFIPS (Cat. No.99TH8397) (pp. 403–407).

  • Sukittanon, S., Surendran, A. C., Platt, J. C., & Burges, C. J. C. (2004). Convolutional networks for speech detection. INTERSPEECH, 1533–1545.

  • Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015). Rethinking the inception architecture for computer vision. CoRR. arXiv:1512.00567.

  • Templeton, A. J., McNamara, M. G., Šeruga, B., Vera-Badillo, F. E., Aneja, P., Ocaña, A., et al. (2014). Prognostic role of neutrophil-to-lymphocyte ratio in solid tumors: A systematic review and meta-analysis. JNCI: Journal of the National Cancer Institute, 106(6), dju124.

    Article  Google Scholar 

  • Tropp, J., & Gilbert, A. (2007). Signal recovery from random measurements via orthogonal matching pursuit. Information Theory, IEEE Transactions on, 53(12), 4655–4666.

    Article  MathSciNet  Google Scholar 

  • Ushizima, D. M., Lorena, A. C., & de Carvalho, A. C. P. L. F. (2005). Support vector machines applied to white blood cell recognition. In Fifth international conference on hybrid intelligent systems (HIS’05) (p. 6).

  • Wang, S., & Wang, M. (2006). A new detection algorithm (NDA) based on fuzzy cellular neural networks for white blood cell detection. IEEE Transactions on Information Technology in Biomedicine, 10, 5–10.

  • Wang, J., Jia, Y., Wang, N., Zhang, X., Tan, B., Zhang, G., et al. (2014). The clinical significance of tumor-infiltrating neutrophils and neutrophil-to-cd8+ lymphocyte ratio in patients with resectable esophageal squamous cell carcinoma. Journal of Translational Medicine, 12, 7.

    Article  Google Scholar 

  • Wang, X., Zhang, G., Jiang, X., Zhu, H., Lu, Z., & Xu, L. (2014). Neutrophil to lymphocyte ratio in relation to risk of all-cause mortality and cardiovascular events among patients undergoing angiography or cardiac revascularization: A meta-analysis of observational studies. Atherosclerosis, 234(1), 206–213. Retrieved November 26, 2020, from https://app.dimensions.ai

  • Xue, P., Kanai, M., Mori, Y., Nishimura, T., Uza, N., Kodama, Y., et al. (2014). Neutrophil-to-lymphocyte ratio for predicting palliative chemotherapy outcomes in advanced pancreatic cancer patients. Cancer Medicine, 3, 406–415.

    Article  Google Scholar 

  • Yu, W., Chang, J., Yang, C., Zhang, L., Shen, H., Xia, Y., & Sha, J. (2017). Automatic classification of leukocytes using deep neural network. In 2017 IEEE 12th international conference on ASIC (ASICON) (pp. 1041–1044).

  • Yu, K., Zhang, T., & Gong, Y. (2009). Nonlinear learning using local coordinate coding. Advances in Neural Information Processing Systems, 22, 2223–2231.

    Google Scholar 

  • Zhang, J., Hu, H., Chen, S., Huang, Y., & Guan, Q. (2016). Cancer cells detection in phase-contrast microscopy images based on faster R-CNN. In 2016 9th International symposium on computational intelligence and design (ISCID), vol. 1 (pp. 363–367).

Download references

Acknowledgements

The authors would like to thank all the participating clinicians and collaborating hospitals involved in this multi-hospital study. This work was supported by the Medical Key Science and Technology Development Projects of Nanjing (ZKX18016), the Medical Science and Technology Development Projects of Nanjing (YKK18167).

Author information

Author notes

  1. Hong Yan and Xuanyu Mao contributed equally to this work.

Authors and Affiliations

  1. Laboratory Medicine Center, The Second Affiliated Hospital of Nanjing Medical University, 210011, Nanjing, China

    Hong Yan

  2. Department of Laboratory Medicine, Affiliated Brain Hospital of Nanjing Medical University, 210029, Nanjing, China

    Hong Yan, Rui Xia, Zhiqiang Wang, Yan Li & Guoye Liu

  3. Berkeley Biomedical Data Science Center, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, United States

    Xuanyu Mao & Hang Chang

  4. Biological Systems and Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, United States

    Xuanyu Mao & Hang Chang

  5. State Key Laboratory of Reproductive Medicine, School of Public Health, Nanjing Medical University, 211166 Nanjing, China

    Xu Yang

  6. Key Laboratory of Modern Toxicology of Ministry of Education, School of Public Health, Nanjing Medical University, 211166, Nanjing, China

    Xu Yang

  7. Department of Clinical Laboratory, The Affiliated Drum Tower Hospital of Nanjing University Medical School, 210008, Nanjing, China

    Yongquan Xia, Xuejing Xu, Zhiyang Li, Xie Zhao & Han Shen

  8. Department of Laboratory Medicine, 301 Hospital, 100085, Beijing, China

    Chengbin Wang

  9. Department of Clinical Laboratory, Jinling Hospital, School of Medical, Nanjing University, 210002, Nanjing, China

    Junjun Wang

  10. Department of Hematology, Zhongnan Hospital, Wuhan University, 430071, Wuhan, China

    Li He

  11. First Clinical School, Hubei University of Medicine, 442000, Shiyan, China

    Zhongyu Wang

  12. Department of Hematology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030 Wuhan, China

    Zhiqiong Wang

  13. Department of Neurosurgery, Zhongnan Hospital, Wuhan University, 430071, Wuhan, China

    Zhiqiang Li

  14. School of Computer Science, The University of Sydney, Sydney, NSW, 2006, Australia

    Weidong Cai

Authors
  1. Hong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuanyu Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongquan Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chengbin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rui Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xuejing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Zhiqiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Zhiyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Xie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Guoye Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Li He
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Zhongyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Zhiqiong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Zhiqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Weidong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Han Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Hang Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XY.M., X.Y and H.C. wrote the software, performed the experiments, and analyzed the data. X.Y and GY.L. performed statistical analysis. H.C., H.S., H.Y., JJ.W., ZQ.L and CB.W conceived and supervised the study. H.C., H.S., H.Y. and W.C wrote the manuscript. YQ.X., XJ.X., R.X., ZQ.W., ZY.L., Y.L, and X.Z created the leukocyte image databases. L.H., H.Y., ZQ.W., and ZY.W. coordinated the multi-hospital clinical evaluation. All the authors edited the manuscript and approved the final manuscript.

Corresponding authors

Correspondence to Han Shen or Hang Chang.

Ethics declarations

Conflict of interests

The authors declare no competing interests.

Data and Code availability

Matlab scripts for WBC-Profiler as well as example data are available online .The data that supports the findings of this study are available from the corresponding author upon request.

Additional information

Communicated by Jan Kybic.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, H., Mao, X., Yang, X. et al. Development and Validation of an Unsupervised Feature Learning System for Leukocyte Characterization and Classification: A Multi-Hospital Study. Int J Comput Vis 129, 1837–1856 (2021). https://doi.org/10.1007/s11263-021-01449-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-021-01449-9

Keywords

Search

Navigation