Skip to main content
SpringerLink
Log in

Local bit-plane decoded convolutional neural network features for biomedical image retrieval

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Biomedical image retrieval is a challenging problem due to the varying contrast and size of structures in the images. The approaches for biomedical image retrieval generally rely on the feature descriptors to characterize the images. The feature descriptor of query image is compared with the descriptors of images from the database, to find the best matches. Several hand-crafted feature descriptors have been proposed so far for biomedical image retrieval by exploiting the local relationship of neighboring image pixels. It is observed in the literature that the local bit-plane decoded features are well suited for this retrieval task. Moreover, in recent past, it is also observed that the convolutional neural network-based features such as AlexNet, Vgg16, GoogleNet and ResNet perform well in many computer vision-related tasks. Motivated by the success of the deep learning-based approaches, this paper proposes a local bit-plane decoding-based AlexNet descriptor (LBpDAD) for biomedical image retrieval. The proposed LBpDAD is computed by max-fusing the ReLU operated feature maps of pre-trained AlexNet at a particular layer, obtained from the original and local bit-plane decoded images. The proposed approach is also compared with Vgg16, GoogleNet and ResNet models. The experiments on the proposed method over three benchmark biomedical databases of different modalities such as MRI, CT and microscopic show the efficacy of the proposed descriptor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The trained AlexNet weights available in the MATLAB are considered.

References

  1. Boland MV, Murphy RF (2001) A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of hela cells. Bioinformatics 17(12):1213–1223

    Article  Google Scholar 

  2. Carneiro G, Nascimento J, Bradley AP (2015) Unregistered multiview mammogram analysis with pre-trained deep learning models. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 652–660

    Google Scholar 

  3. Chakraborty S, Singh S, Chakraborty P (2016) Local gradient hexa pattern: a descriptor for face recognition and retrieval. IEEE Trans Circuits Syst Video Technol 28:171–180

    Article  Google Scholar 

  4. Chakraborty S, Singh SK, Chakraborty P (2017) Local directional gradient pattern: a local descriptor for face recognition. Multimed Tools Appl 76(1):1201–1216

    Article  Google Scholar 

  5. Chang YY, Tai SC, Lin JS (2012) Segmentation of multispectral mr images through an annealed rough neural network. Neural Comput Appl 21(5):911–919

    Article  Google Scholar 

  6. Chowdhury M, Bulo SR, Moreno R, Kundu MK, Smedby Ö (2016) An efficient radiographic image retrieval system using convolutional neural network. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, New York, pp 3134–3139

  7. Chung YA, Weng WH (2017) Learning deep representations of medical images using siamese CNNs with application to content-based image retrieval. Preprint. arXiv:1711.08490

  8. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M et al (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Dig Imaging 26(6):1045–1057

    Article  Google Scholar 

  9. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, CVPR 2009. IEEE, New York, pp 248–255

  10. Dubey SR, Chakraborty S (2018) Average biased ReLU based CNN descriptor for improved face retrieval. Preprint. arXiv:1804.02051

  11. Dubey SR, Mukherjee S (2018) Ldop: local directional order pattern for robust face retrieval. Preprint. arXiv:1803.07441

  12. Dubey SR, Singh SK, Singh RK (2014) Rotation and illumination invariant interleaved intensity order-based local descriptor. IEEE Trans Image Process 23(12):5323–5333

    Article  MathSciNet  MATH  Google Scholar 

  13. Dubey SR, Singh SK, Singh RK (2015) Local diagonal extrema pattern: a new and efficient feature descriptor for CT image retrieval. IEEE Signal Process Lett 22(9):1215–1219

    Article  Google Scholar 

  14. Dubey SR, Singh SK, Singh RK (2015) Local wavelet pattern: a new feature descriptor for image retrieval in medical CT databases. IEEE Trans Image Process 24(12):5892–5903

    Article  MathSciNet  MATH  Google Scholar 

  15. Dubey SR, Singh SK, Singh RK (2016) Local bit-plane decoded pattern: a novel feature descriptor for biomedical image retrieval. IEEE J Biomed Health Inf 20(4):1139–1147

    Article  Google Scholar 

  16. Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032

    Article  MathSciNet  MATH  Google Scholar 

  17. Dubey SR, Singh SK, Singh RK (2016) Novel local bit-plane dissimilarity pattern for computed tomography image retrieval. Electron Lett 52(15):1290–1292

    Article  Google Scholar 

  18. Ge Y, Jiang S, Xu Q, Jiang C, Ye F (2017) Exploiting representations from pre-trained convolutional neural networks for high-resolution remote sensing image retrieval. Multimed Appl 77:1–27

    Google Scholar 

  19. Guo K, Duan G (2014) 3D image retrieval based on differential geometry and co-occurrence matrix. Neural Comput Appl 24(3–4):715–721

    Article  Google Scholar 

  20. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, New York, pp 2980–2988

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  22. Hermessi H, Mourali O, Zagrouba E (2018) Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput Appl 30:1–17

    Article  Google Scholar 

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems conference, pp 1097–1105

  24. Lan R, Zhou Y (2017) Medical image retrieval via histogram of compressed scattering coefficients. IEEE J Biomed Health Inf 21(5):1338–1346

    Article  MathSciNet  Google Scholar 

  25. Liu P, Guo JM, Wu CY, Cai D (2017) Fusion of deep learning and compressed domain features for content-based image retrieval. IEEE Trans Image Process 26(12):5706–5717

    Article  MathSciNet  MATH  Google Scholar 

  26. Lu K, He N, Xue J, Dong J, Shao L (2015) Learning view-model joint relevance for 3D object retrieval. IEEE Trans Image Process 24(5):1449–1459

    Article  MathSciNet  MATH  Google Scholar 

  27. Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL (2007) Open access series of imaging studies (OASIS): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci 19(9):1498–1507

    Article  Google Scholar 

  28. Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. Int J Med Inf 73(1):1–23

    Article  Google Scholar 

  29. Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

    Article  MathSciNet  MATH  Google Scholar 

  30. Murala S, Wu QJ (2013) Local ternary co-occurrence patterns: a new feature descriptor for MRI and CT image retrieval. Neurocomputing 119:399–412

    Article  Google Scholar 

  31. Murala S, Wu QJ (2014) Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval. IEEE J Biomed Health Inf 18(3):929–938

    Article  Google Scholar 

  32. Nanni L, Brahnam S, Ghidoni S, Lumini A (2018) Bioimage classification with handcrafted and learned features. IEEE/ACM Trans Comput Biol Bioinform. https://doi.org/10.1109/TCBB.2018.2821127

    Article  Google Scholar 

  33. Nanni L, Lumini A (2008) A reliable method for cell phenotype image classification. Artif Intell Med 43(2):87–97

    Article  Google Scholar 

  34. Nanni L, Lumini A, Brahnam S (2010) Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 49(2):117–125

    Article  MATH  Google Scholar 

  35. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  MATH  Google Scholar 

  36. Paci M, Nanni L, Lahti A, Aalto-Setala K, Hyttinen J, Severi S (2013) Non-binary coding for texture descriptors in sub-cellular and stem cell image classification. Curr Bioinform 8(2):208–219

    Article  Google Scholar 

  37. Pang S, Yu Z, Orgun MA (2017) A novel end-to-end classifier using domain transferred deep convolutional neural networks for biomedical images. Comput Methods Programs Biomed 140:283–293

    Article  Google Scholar 

  38. Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns, vol 40. Springer, Berlin

    Book  Google Scholar 

  39. Qayyum A, Anwar SM, Awais M, Majid M (2017) Medical image retrieval using deep convolutional neural network. Neurocomputing 266:8–20

    Article  Google Scholar 

  40. Qiu C, Cai Y, Gao X, Cui Y (2017) Medical image retrieval based on the deep convolution network and hash coding. In: 2017 10th international congress on image and signal processing, biomedical engineering and informatics (CISP-BMEI). IEEE, New York, pp 1–6

  41. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K et al (2017) Chexnet: radiologist-level pneumonia detection on chest X-rays with deep learning. Preprint. arXiv:1711.05225

  42. Ramírez I, Cuesta-Infante A, Pantrigo JJ, Montemayor AS, Moreno JL, Alonso V, Anguita G, Palombarani L (2018) Convolutional neural networks for computer vision-based detection and recognition of dumpsters. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3390-8

    Article  Google Scholar 

  43. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems conference, pp 91–99

  44. Roy SK, Chanda B, Chaudhuri B, Ghosh DK, Dubey SR (2017) A complete dual-cross pattern for unconstrained texture classification. In: 4th IAPR Asian conference on pattern recognition (ACPR 2017), Nanjing, pp 741–746

  45. Roy SK, Chanda B, Chaudhuri BB, Banerjee S, Ghosh DK, Dubey SR (2017) Local jet pattern: a robust descriptor for texture classification. Preprint. arXiv:1711.10921

  46. Roy SK, Chanda B, Chaudhuri BB, Banerjee S, Ghosh DK, Dubey SR (2018) Local directional ZigZag pattern: a rotation invariant descriptor for texture classification. Pattern Recognit Lett 108:23–30

    Article  Google Scholar 

  47. Sezer A, Sezer HB, Albayrak S (2017) Hermite-based texture feature extraction for classification of humeral head in proton density-weighted MR images. Neural Comput Appl 28(10):3021–3033

    Article  Google Scholar 

  48. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Preprint. arXiv:1409.1556

  49. Singh GAP, Gupta P (2018) Performance analysis of various machine learning-based approaches for detection and classification of lung cancer in humans. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3518-x

    Article  Google Scholar 

  50. Sirinukunwattana K, Raza SEA, Tsang YW, Snead DR, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35(5):1196–1206

    Article  Google Scholar 

  51. Sorensen L, Shaker SB, De Bruijne M (2010) Quantitative analysis of pulmonary emphysema using local binary patterns. IEEE Trans Med Imaging 29(2):559–569

    Article  Google Scholar 

  52. Srivastava D, Rajitha B, Agarwal S et al (2018) Pattern-based image retrieval using GLCM. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3611-1

    Article  Google Scholar 

  53. Suri JS, Wilson D, Laxminarayan S (2005) Handbook of biomedical image analysis, vol 2. Springer, Berlin

    Book  Google Scholar 

  54. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A et al (2015) Going deeper with convolutions. In: CVPR

  55. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  56. Wang DH, Conilione P (2012) Machine learning approach for face image retrieval. Neural Comput Appl 21(4):683–694

    Article  Google Scholar 

  57. Wang G, Xu X, Jiang X, Ding S (2016) Medical image registration based on self-adapting pulse-coupled neural networks and mutual information. Neural Comput Appl 27(7):1917–1926

    Article  Google Scholar 

  58. Wang Z, Fan B, Wu F (2011) Local intensity order pattern for feature description. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, New York, pp 603–610

  59. Wu Y, Wang L, Cui F, Zhai H, Dong B, Wang JY (2016) Cross-model convolutional neural network for multiple modality data representation. Neural Comput Appl 30:1–11

    Google Scholar 

  60. Yang Z, Yu W, Liang P, Guo H, Xia L, Zhang F, Ma Y, Ma J (2018) Deep transfer learning for military object recognition under small training set condition. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3468-3

    Article  Google Scholar 

  61. Yao J, Liu F, Geng Y (2017) Query-specific optimal convolutional neural ranker. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3257-4

    Article  Google Scholar 

  62. Zhang B, Gao Y, Zhao S, Liu J (2010) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544

    Article  MathSciNet  MATH  Google Scholar 

  63. Zhang L, Lu L, Nogues I, Summers RM, Liu S, Yao J (2017) Deeppap: deep convolutional networks for cervical cell classification. IEEE J Biomed Health Inf 21(6):1633–1643

    Article  Google Scholar 

Download references

Acknowledgements

This research is funded by Science and Engineering Research Board (SERB), Govt. of India through Project Sanction No. ECR/2017/000082. The authors would like to thank NVIDIA Corporation for the support of 2 GeForce Titan X Pascal GPU donated to Computer Vision Group, IIIT Sri City.

Author information

Authors and Affiliations

  1. Computer Vision Group, Indian Institute of Information Technology, Sri City, Chittoor, Andhra Pradesh, 517646, India

    Shiv Ram Dubey & Snehasis Mukherjee

  2. Computer Vision and Pattern Recognition Unit, Indian Statistical Institute, Kolkata, 700108, India

    Swalpa Kumar Roy & Bidyut Baran Chaudhuri

  3. Computer Science and Engineering Department, Indian Institute of Information Technology, Lucknow, Uttar Pradesh, India

    Soumendu Chakraborty

Authors
  1. Shiv Ram Dubey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Swalpa Kumar Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soumendu Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Snehasis Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bidyut Baran Chaudhuri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiv Ram Dubey.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dubey, S.R., Roy, S.K., Chakraborty, S. et al. Local bit-plane decoded convolutional neural network features for biomedical image retrieval. Neural Comput & Applic 32, 7539–7551 (2020). https://doi.org/10.1007/s00521-019-04279-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04279-6

Keywords

Search

Navigation