Skip to main content
SpringerLink
Log in

Gabor Filter-Embedded U-Net with Transformer-Based Encoding for Biomedical Image Segmentation

  • Conference paper
  • First Online:
Multiscale Multimodal Medical Imaging (MMMI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13594))

Included in the following conference series:

  • 663 Accesses

Abstract

Medical image segmentation involves a process of categorization of target regions that are typically varied in terms of shape, orientation and scales. This requires highly accurate algorithms as marginal segmentation errors in medical images may lead to inaccurate diagnosis in subsequent procedures. The U-Net framework has become one of the dominant deep neural network architectures for medical image segmentation. Due to complex and irregular shape of objects involved in medical images, robust feature representations that correspond to various spatial transformations are key to achieve successful results. Although U-Net-based deep architectures can perform feature extraction and localization, the design of specialized architectures or layer modifications is often an intricate task. In this paper, we propose an effective solution to this problem by introducing Gabor filter banks into the U-Net encoder, which has not yet been well explored in existing U-Net-based segmentation frameworks. In addition, global self-attention mechanisms and Transformer layers are also incorporated into the U-Net framework to capture global contexts. Through extensive testing on two benchmark datasets, we show that the Gabor filter-embedded U-Net with Transformer encoders can enhance the robustness of deep-learned features, and thus achieve a more competitive performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.isles-challenge.org/ISLES2018/.

  2. 2.

    http://atriaseg2018.cardiacatlas.org/.

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  2. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  3. Alom, M.Z., et al.: A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3), 292 (2019)

    Article  Google Scholar 

  4. Shen, H., Wang, R., Zhang, J., McKenna, S.J.: Boundary-aware fully convolutional network for brain tumor segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 433–441. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_49

    Chapter  Google Scholar 

  5. Nie, D., Wang, L., Gao, Y., Shen, D.: Fully convolutional networks for multi-modality isointense infant brain image segmentation. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1342–1345. IEEE (2016)

    Google Scholar 

  6. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  7. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  8. Oktay, O., et al.: Attention U-Net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)

  9. Paheding, S., Reyes, A.A., Alam, M., Asari, V.K.: Medical image segmentation using U-Net and progressive neuron expansion. In: Pattern Recognition and Tracking XXXIII, vol. 12101, p. 1210102. SPIE (2022)

    Google Scholar 

  10. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation. arXiv preprint arXiv:1802.06955 (2018)

  11. Siddique, N., Paheding, S., Alom, M.Z., Devabhaktuni, V.: Recurrent residual U-Net with efficientnet encoder for medical image segmentation. In: Pattern Recognition and Tracking XXXII, vol. 11735, pp. 134–142. SPIE (2021)

    Google Scholar 

  12. Siddique, N., Paheding, S., Elkin, C.P., Devabhaktuni, V.: U-Net and its variants for medical image segmentation: a review of theory and applications. IEEE Access 9, 82031–82057 (2021)

    Article  Google Scholar 

  13. Paheding, S., Reyes, A.A., Kasaragod, A., Oommen, T.: GAF-NAU: Gramian angular field encoded neighborhood attention U-Net for pixel-wise hyperspectral image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 409–417 (2022)

    Google Scholar 

  14. Soares, L.P., Dias, H.C., Grohmann, C.H.: Landslide segmentation with U-Net: evaluating different sampling methods and patch sizes. arXiv preprint arXiv:2007.06672 (2020)

  15. McGlinchy, J., Johnson, B., Muller, B., Joseph, M., Diaz, J.: Application of UNet fully convolutional neural network to impervious surface segmentation in urban environment from high resolution satellite imagery. In: 2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019, pp. 3915–3918. IEEE (2019)

    Google Scholar 

  16. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  17. Dosovitskiy, A., et al.: An image is worth \(16 \times 16\) words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  18. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning, pp. 10347–10357. PMLR (2021)

    Google Scholar 

  19. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13

    Chapter  Google Scholar 

  20. Chen, J., et al.: TransUNet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)

  21. Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., Li, J.: TransBTS: multimodal brain tumor segmentation using transformer. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 109–119. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_11

    Chapter  Google Scholar 

  22. Dai, J., et al.: Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 764–773 (2017)

    Google Scholar 

  23. Luan, S., Chen, C., Zhang, B., Han, J., Liu, J.: Gabor convolutional networks. IEEE Trans. Image Process. 27(9), 4357–4366 (2018)

    Article  MathSciNet  Google Scholar 

  24. Zhou, Y., Ye, Q., Qiu, Q., Jiao, J.: Oriented response networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2017)

    Google Scholar 

  25. Gabor, D.: Theory of communication. Part 1: The analysis of information. J. Inst. Electr. Eng.-Part III: Radio Commun. Eng. 93(26), 429–441 (1946)

    Google Scholar 

  26. Jain, A.K., Ratha, N.K., Lakshmanan, S.: Object detection using Gabor filters. Pattern Recogn. 30(2), 295–309 (1997)

    Article  Google Scholar 

  27. Kwolek, B.: Face detection using convolutional neural networks and Gabor filters. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3696, pp. 551–556. Springer, Heidelberg (2005). https://doi.org/10.1007/11550822_86

    Chapter  Google Scholar 

  28. Gong, X., Xia, X., Zhu, W., Zhang, B., Doermann, D., Zhuo, L.: Deformable Gabor feature networks for biomedical image classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4004–4012 (2021)

    Google Scholar 

  29. Ouyang, W., Wang, X.: Joint deep learning for pedestrian detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2056–2063 (2013)

    Google Scholar 

  30. Zhang, B., Yang, Y., Chen, C., Yang, L., Han, J., Shao, L.: Action recognition using 3D histograms of texture and a multi-class boosting classifier. IEEE Trans. Image Process. 26(10), 4648–4660 (2017)

    Article  MathSciNet  Google Scholar 

  31. Alekseev, A., Bobe, A.: GaborNet: Gabor filters with learnable parameters in deep convolutional neural network. In: 2019 International Conference on Engineering and Telecommunication (EnT), pp. 1–4. IEEE (2019)

    Google Scholar 

  32. Yuan, Y., et al.: Adaptive Gabor convolutional networks. Pattern Recogn. 124, 108495 (2022)

    Google Scholar 

  33. Yang, D., Myronenko, A., Wang, X., Xu, Z., Roth, H.R., Xu, D.: T-AutoML: automated machine learning for lesion segmentation using transformers in 3D medical imaging. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3962–3974 (2021)

    Google Scholar 

  34. Chen, Yu., Chen, J., Wei, D., Li, Y., Zheng, Y.: OctopusNet: a deep learning segmentation network for multi-modal medical images. In: Li, Q., Leahy, R., Dong, B., Li, X. (eds.) MMMI 2019. LNCS, vol. 11977, pp. 17–25. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37969-8_3

    Chapter  Google Scholar 

  35. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  36. Wang, B., Dong, P., et al.: Multiscale transunet++: dense hybrid U-Net with transformer for medical image segmentation. Signal Image Video Process. 16, 1607–1614 (2022). https://doi.org/10.1007/s11760-021-02115-w

    Article  Google Scholar 

  37. Wang, S., Li, L., Zhuang, X.: AttU-Net: attention U-Net for brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) BrainLes 2021, vol. 12963, pp. 302–311. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-09002-8_27

    Chapter  Google Scholar 

  38. Cereda, C.W., et al.: A benchmarking tool to evaluate computer tomography perfusion infarct core predictions against a DWI standard. J. Cereb. Blood Flow Metab. 36(10), 1780–1789 (2016)

    Article  Google Scholar 

  39. Hakim, A., et al.: Predicting infarct core from computed tomography perfusion in acute ischemia with machine learning: lessons from the isles challenge. Stroke 52(7), 2328–2337 (2021)

    Article  Google Scholar 

  40. Maier, O., et al.: ISLES 2015-a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Med. Image Anal. 35, 250–269 (2017)

    Article  Google Scholar 

  41. Xiong, Z.: A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med. Image Anal. 67, 101832 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Michigan Technological University, Houghton, MI, 49931, USA

    Abel A. Reyes & Sidike Paheding

  2. Norfolk State University, Norfolk, VA, 23504, USA

    Makarand Deo

  3. Old Dominion University, Norfolk, VA, 23529, USA

    Michel Audette

Authors
  1. Abel A. Reyes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sidike Paheding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Makarand Deo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michel Audette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sidike Paheding .

Editor information

Editors and Affiliations

  1. Massachusetts General Hospital, Boston, MA, USA

    Xiang Li

  2. University of Sydney, Sydney, Australia

    Jinglei Lv

  3. Vanderbilt University, Nashville, TN, USA

    Yuankai Huo

  4. Peking University, Beijing, China

    Bin Dong

  5. University of Southern California, Los Angeles, CA, USA

    Richard M. Leahy

  6. Massachusetts General Hospital, Boston, MA, USA

    Quanzheng Li

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Reyes, A.A., Paheding, S., Deo, M., Audette, M. (2022). Gabor Filter-Embedded U-Net with Transformer-Based Encoding for Biomedical Image Segmentation. In: Li, X., Lv, J., Huo, Y., Dong, B., Leahy, R.M., Li, Q. (eds) Multiscale Multimodal Medical Imaging. MMMI 2022. Lecture Notes in Computer Science, vol 13594. Springer, Cham. https://doi.org/10.1007/978-3-031-18814-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-18814-5_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-18813-8

  • Online ISBN: 978-3-031-18814-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation