Abstract
Recent neural networks have shown impressive performance in computer vision tasks. However, these models mainly focus on designing deep architectures and strongly depend on the architectures themselves. This paper proposes a simple yet effective deep equilibrium model (DEQ) that exploits the advantages of implicit deep learning and multi-scale self-attention. In particular, our approach reduces the need for simultaneously finding multiple fixed points at different scales in Multi-scale Deep Equilibrium Models (MDEQs) to finding a unique fixed point at the highest resolution. Therefore, our method is more memory efficient and requires less computational complexity. To the best of our knowledge, this is the first attempt toward building an effective DEQ for polyp segmentation, and thus, we call the model PolypDEQ. Experiments on five popular polyp segmentation benchmarks show that our proposed method yields superior performance compared to previous MDEQ and Transformers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
An, N.S., et al.: BlazeNeo: blazing fast polyp segmentation and neoplasm detection. IEEE Access 10, 43669–43684 (2022)
Bai, S., Kolter, J.Z., Koltun, V.: Deep equilibrium models. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Bai, S., Koltun, V., Kolter, J.Z.: Multiscale deep equilibrium models. In: Advances in Neural Information Processing Systems, vol. 33, pp. 5238–5250 (2020)
Bai, S., Koltun, V., Kolter, J.Z.: Neural deep equilibrium solvers. In: International Conference on Learning Representations (2021)
Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilariño, F.: WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput. Med. Imaging Graph. 43, 99–111 (2015)
Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Duc, N.T., Oanh, N.T., Thuy, N.T., Triet, T.M., Dinh, V.S.: ColonFormer: an efficient transformer based method for colon polyp segmentation. IEEE Access 10, 80575–80586 (2022)
Fan, D.-P., et al.: PraNet: parallel reverse attention network for polyp segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 263–273. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_26
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hung, N.B., Duc, N.T., Van Chien, T., Sang, D.V.: AG-ResUNet++: an improved encoder-decoder based method for polyp segmentation in colonoscopy images. In: 2021 RIVF International Conference on Computing and Communication Technologies (RIVF), pp. 1–6. IEEE (2021)
Jha, D., et al.: Kvasir-SEG: a segmented polyp dataset. In: Ro, Y.M., et al. (eds.) MMM 2020. LNCS, vol. 11962, pp. 451–462. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_37
Jha, D., et al.: ResUNet++: an advanced architecture for medical image segmentation. In: 2019 IEEE International Symposium on Multimedia (ISM), pp. 225–2255 (2019). https://doi.org/10.1109/ISM46123.2019.00049
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Ngoc Lan, P., et al.: NeoUNet: towards accurate colon polyp segmentation and neoplasm detection. In: Bebis, G., et al. (eds.) ISVC 2021. LNCS, vol. 13018, pp. 15–28. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90436-4_2
Pal, A., Edelman, A., Rackauckas, C.: Mixing implicit and explicit deep learning with skip DEQs and infinite time neural odes (continuous DEQs). arXiv preprint arXiv:2201.12240 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Silva, J., Histace, A., Romain, O., Dray, X., Granado, B.: Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int. J. Comput. Assist. Radiol. Surg. 9(2), 283–293 (2014)
Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans. Med. Imaging 35(2), 630–644 (2015)
Vázquez, D., et al.: A benchmark for endoluminal scene segmentation of colonoscopy images. J. Healthcare Eng. 2017 (2017)
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: SegFormer: simple and efficient design for semantic segmentation with transformers. In: Advances in Neural Information Processing Systems, vol. 34, pp. 12077–12090 (2021)
Zhang, Y., Liu, H., Hu, Q.: TransFuse: fusing transformers and CNNs for medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 14–24. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_2
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Acknowledgment
This work was funded by Vingroup Innovation Foundation (VINIF) under project code VINIF.2020.DA17.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chau, N.M., Giang, L.T., Sang, D.V. (2022). PolypDEQ: Towards Effective Transformer-Based Deep Equilibrium Models for Colon Polyp Segmentation. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13598. Springer, Cham. https://doi.org/10.1007/978-3-031-20713-6_35
Download citation
DOI: https://doi.org/10.1007/978-3-031-20713-6_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20712-9
Online ISBN: 978-3-031-20713-6
eBook Packages: Computer ScienceComputer Science (R0)