Skip to main content
Springer Nature Link
Log in

Imbalance multiclass problem: a robust feature enhancement-based framework for liver lesion classification

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

The classification of liver lesions in CT images is essential for the diagnosis and treatment of liver diseases. Since the characteristics of different classes of lesions are similar and the degree of differentiation is not obvious, it is difficult to accurately classify different classes of liver lesions, especially imbalanced multiclass distribution. To this end, we propose a novel feature enhancement-based framework for imbalanced liver lesion classification. Specifically, a liver lesion processing method is introduced to expand data based on Augmentor. Besides, based on Augmentor, we design dual feature enhancement, which is based on feature refinement extraction and feature global correlation, to enhance feature reconstruction ability. To further alleviate the problem of class imbalance, the improved loss function based on standard cross-entropy (CE) is also adopted to make the network pay more attention to the class with fewer samples. Experiments on 551 liver lesions in 120 patients showed that: (1) This framework improved the classification performance of liver tumors under imbalanced multiclass data distribution; (2) The dual feature enhancement was lightweight and efficient which enhanced the semantic expression of overall features without introducing additional parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The data that support the findings of this study are available on request from the corresponding author.

References

  1. Ryerson, A.B., Eheman, C.R., Altekruse, S.F., Ward, J.W., Jemal, A., Sherman, R.L., Henley, S.J., Holtzman, D., Lake, A., Noone, A.-M.: Annual report to the nation on the status of cancer, 1975–2012, featuring the increasing incidence of liver cancer. Cancer 122(9), 1312–1337 (2016)

    Article  Google Scholar 

  2. Romero, F.P., Diler, A., Bisson-Gregoire, G., Turcotte, S., Lapointe, R., Vandenbroucke-Menu, F., Tang, A., Kadoury, S.: End-to-end discriminative deep network for liver lesion classification. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1243–1246. IEEE (2019)

  3. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Gan-based synthetic medical image augmentation for increased cnn performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)

    Article  Google Scholar 

  4. Balasubramanian, P.K., Lai, W.-C., Seng, G.H., Selvaraj, J.: Apestnet with mask r-cnn for liver tumor segmentation and classification. Cancers 15(2), 330 (2023)

    Article  Google Scholar 

  5. Ma, C., Liu, Z., Song, Y., Qiu, C., Yu, A., Zhang, J.: Self-supervision adversarial learning network for liver lesion classification. In: Artificial Intelligence and Security: 7th International Conference, ICAIS 2021, Dublin, Ireland, July 19–23, 2021, Proceedings, Part I 7, pp. 540–552. Springer (2021)

  6. Ben-Cohen, A., Mechrez, R., Yedidia, N., Greenspan, H.: Improving cnn training using disentanglement for liver lesion classification in ct. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 886–889. IEEE (2019)

  7. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using gan for improved liver lesion classification. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 289–293. IEEE (2018)

  8. Saha Roy, S., Roy, S., Mukherjee, P., Halder Roy, A.: An automated liver tumor segmentation and classification model by deep learning based approaches. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2022, 1–13 (2022)

    Google Scholar 

  9. Heker, M., Ben-Cohen, A., Greenspan, H.: Hierarchical fine-tuning for joint liver lesion segmentation and lesion classification in ct. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 895–898. IEEE (2019)

  10. Lango, M., Stefanowski, J.: What makes multi-class imbalanced problems difficult? An experimental study. Expert Syst. Appl. 199, 116962 (2022)

    Article  Google Scholar 

  11. Cui, Y., Jia, M., Lin, T.-Y., Song, Y., Belongie, S.: Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9268–9277 (2019)

  12. Janicka, M., Lango, M., Stefanowski, J.: Using information on class interrelations to improve classification of multiclass imbalanced data: a new resampling algorithm. Int. J. Appl. Math. Comput. Sci. 29(4), 769–781 (2019)

    Article  Google Scholar 

  13. Dai, W., Ng, K., Severson, K., Huang, W., Anderson, F., Stultz, C.: Generative oversampling with a contrastive variational autoencoder. In: 2019 IEEE International Conference on Data Mining (ICDM), pp. 101–109. IEEE (2019)

  14. Bellinger, C., Corizzo, R., Japkowicz, N.: Calibrated resampling for imbalanced and long-tails in deep learning. In: Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11–13, 2021, Proceedings 24, pp. 242–252. Springer (2021)

  15. Khorshidi, H.A., Aickelin, U.: Constructing classifiers for imbalanced data using diversity optimisation. Inf. Sci. 565, 1–16 (2021)

    Article  MathSciNet  Google Scholar 

  16. Tang, K., Huang, J., Zhang, H.: Long-tailed classification by keeping the good and removing the bad momentum causal effect. Adv. Neural Inf. Process. Syst. 33, 1513–1524 (2020)

    Google Scholar 

  17. Wang, P., Han, K., Wei, X.-S., Zhang, L., Wang, L.: Contrastive learning based hybrid networks for long-tailed image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 943–952 (2021)

  18. Estabrooks, A., Jo, T., Japkowicz, N.: A multiple resampling method for learning from imbalanced data sets. Comput. Intell. 20(1), 18–36 (2004)

    Article  MathSciNet  Google Scholar 

  19. Li, B., Han, Z., Li, H., Fu, H., Zhang, C.: Trustworthy long-tailed classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6970–6979 (2022)

  20. Li, X., Wang, W., Hu, X., Yang, J.: Selective kernel networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 510–519 (2019)

  21. Lee, H., Kim, H.-E., Nam, H.: Srm: A style-based recalibration module for convolutional neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1854–1862 (2019)

  22. Ji, C., Huang, X., Cao, W., Zhu, Y., Zhang, Y.: Fusion of deep learning and global-local features of the image salient region calculation. J. Comput. Aided Des. Comput. Graph. 31(10), 1838–1846 (2019)

    Google Scholar 

  23. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  24. Park, J., Woo, S., Lee, J.-Y., Kweon, I.S.: Bam: Bottleneck attention module. arXiv preprint arXiv:1807.06514 (2018)

  25. Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: International Conference on Machine Learning, pp. 7354–7363. PMLR (2019)

  26. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  27. Zhu, Z., Xu, M., Bai, S., Huang, T., Bai, X.: Asymmetric non-local neural networks for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 593–602 (2019)

  28. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)

  29. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  30. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  31. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  32. Wang, J., Han, X.-H., Xu, Y., Lin, L., Hu, H., Jin, C., Chen, Y.-W.: Sparse codebook model of local structures for retrieval of focal liver lesions using multiphase medical images. Int. J. Biomed. Imaging 2017, 1413297 (2017)

    Article  Google Scholar 

  33. Yang, W., Lu, Z., Yu, M., Huang, M., Feng, Q., Chen, W.: Content-based retrieval of focal liver lesions using bag-of-visual-words representations of single-and multiphase contrast-enhanced ct images. J. Digit. Imaging 25, 708–719 (2012)

    Article  Google Scholar 

  34. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  35. Chen, P., Song, Y., Yuan, D., Liu, Z.: Feature fusion adversarial learning network for liver lesion classification. In: Proceedings of the ACM Multimedia Asia, pp. 1–7 (2019)

  36. Wang, X., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., Han, X.: Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022)

    Article  Google Scholar 

  37. Xing, X., Yuan, Y., Meng, M.Q.-H.: Zoom in lesions for better diagnosis: Attention guided deformation network for wce image classification. IEEE Trans. Med. Imaging 39(12), 4047–4059 (2020)

    Article  Google Scholar 

  38. Fang, L., Wang, C., Li, S., Rabbani, H., Chen, X., Liu, Z.: Attention to lesion: lesion-aware convolutional neural network for retinal optical coherence tomography image classification. IEEE Trans. Med. Imaging 38(8), 1959–1970 (2019)

    Article  Google Scholar 

  39. Cohen, J.P., Morrison, P., Dao, L., Roth, K., Duong, T.Q., Ghassemi, M.: Covid-19 image data collection: Prospective predictions are the future. arXiv preprint arXiv:2006.11988 (2020)

  40. Chen, K., Lei, W., Zhao, S., Zheng, W.-S., Wang, R.: Pcct: Progressive class-center triplet loss for imbalanced medical image classification. IEEE J. Biomed. Health Inform. 27(4), 2026–2036 (2023)

    Article  Google Scholar 

  41. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

Download references

Acknowledgements

The authors would like to thank radiologists of the Medical Imaging Department of Affiliated Hospital of Jiangsu University. This work was supported by the National Natural Science Foundation of China (62276116, 61976106, 61772242); Six talent peaks project in Jiangsu Province (DZXX-122); Key RESEARCH and development program for social development (SH2021056).

Author information

Authors and Affiliations

  1. School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, 212003, China

    Rui Hu, Yuqing Song, Yi Liu, Nuo Feng, Chengjian Qiu, Kai Han, Qiaoying Teng, Imran Ul Haq & Zhe Liu

  2. Department of Imaging, Affiliated Hospital of Jiangsu University, Zhenjiang, 210031, China

    Yan Zhu

Authors
  1. Rui Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yuqing Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yi Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yan Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Nuo Feng
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Chengjian Qiu
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Kai Han
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Qiaoying Teng
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Imran Ul Haq
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Zhe Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

These authors contributed equally to this work.

Corresponding author

Correspondence to Zhe Liu.

Ethics declarations

Conflict of interests

The authors declare no competing interests.

Additional information

Communicated by B. Bao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, R., Song, Y., Liu, Y. et al. Imbalance multiclass problem: a robust feature enhancement-based framework for liver lesion classification. Multimedia Systems 30, 104 (2024). https://doi.org/10.1007/s00530-024-01291-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01291-2

Keywords