Skip to main content
SpringerLink
Log in

M-GCN: Brain-inspired memory graph convolutional network for multi-label image recognition

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Traditional single-label classification methods can not be effectively applied in multi-label classification due to the semantic correlation. Conventional methods using the attention mechanism or prior knowledge, lacks deep semantic correlations, resulting in degradation for detection performance. Considering the hippocampal circuit and memory mechanism of human brain, a brain-inspired Memory Graph Convolutional Network (M-GCN) is proposed. M-GCN presents crucial short-term and long-term memory modules to interact attention and prior knowledge, learning complex semantic enhancement, and suppression. We evaluate the effectiveness of our method on public benchmarks (Microsoft COCO and PASCAL VOC). Extensive experiments demonstrate that M-GCN outperforms general state-of-the-art methods and shows the advantages in semantic correlation and complexity comparing with traditional memory models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Chen T, Wang Z, Li G, Lin L (2018) Recurrent attentional reinforcement learning for multi-label image recognition. In: Proceedings of the AAAI conference on artificial intelligence, 32

  2. Chen T, Xu M, Hui X, Wu H, Lin L (2019) Learning semantic-specific graph representation for multi-label image recognition. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 522–531

  3. Chen ZM, Wei XS, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5177–5186

  4. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Ieee, pp 248–255

  5. Dingledine R, Borges K, Bowie D, Traynelis SF (1999) The glutamate receptor ion channels. Pharmacol Rev 51(1):7–62

    Google Scholar 

  6. Eichenbaum H (2000) A cortical-hippocampal system for declarative memory. Nat Rev Neurosci 1(1):41–50

    Article  Google Scholar 

  7. Eichenbaum H (2004) Hippocampus: cognitive processes and neural representations that underlie declarative memory. Neuron 44(1):109–120

    Article  MathSciNet  Google Scholar 

  8. Eichenbaum H, Dudchenko P, Wood E, Shapiro M, Tanila H (1999) The hippocampus, memory, and place cells: is it spatial memory or a memory space? Neuron 23(2):209–226

    Article  Google Scholar 

  9. Engle RW, Tuholski SW, Laughlin JE, Conway AR (1999) Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. J Exp Psychol Gen 128(3):309

    Article  Google Scholar 

  10. Everingham M, Gool LV, Williams C, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

  11. Gao BB, Zhou HY (2020) Learning to discover multi-class attentional regions for multi-label image recognition. IEEE Trans Image Process 30:5920–32

    Article  Google Scholar 

  12. Ge W, Yang S, Yu Y (2018) Multi-evidence filtering and fusion for multi-label classification, object detection and semantic segmentation based on weakly supervised learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1277–1286

  13. Ghamrawi N, Mccallum A (2005) Collective multi-label classification. In: Proceedings of the 2005 ACM CIKM international conference on information and knowledge management, Bremen, Germany

  14. Grunwald T, Kurthen M (2006) Novelty detection and encoding for declarative memory within the human hippocampus. Clin EEG Neurosci 37(4):309–314

    Article  Google Scholar 

  15. Guo Y, Gu S (2011) Multi-label classification using conditional dependency networks. In: IJCAI Proceedings-international joint conference on artificial intelligence, Citeseer, 22: 1300

  16. Hao Y, Zhou JT, Yu Z, Gao BB, Wu J, Cai J (2016) Exploit bounding box annotations for multi-label object recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR)

  17. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  18. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  19. Jeneson A, Squire LR (2012) Working memory, long-term memory, and medial temporal lobe function. Learn Memory 19(1):15–25

    Article  Google Scholar 

  20. Jiang W, Yi Y, Mao J, Huang Z, Wei X (2016) Cnn-rnn: A unified framework for multi-label image classification. In: IEEE

  21. Kandel ER (2001) The molecular biology of memory storage: a dialogue between genes and synapses. Science 294(5544):1030–1038

    Article  Google Scholar 

  22. Li Y, Song Y, Luo J (2017) Improving pairwise ranking for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3617–3625

  23. Lin TY, Maire M, Belongie S, Hays J, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision

  24. Liu L, Wong TP, Pozza MF, Lingenhoehl K, Wang Y, Sheng M, Auberson YP, Wang YT (2004) Role of nmda receptor subtypes in governing the direction of hippocampal synaptic plasticity. Science 304(5673):1021–1024

    Article  Google Scholar 

  25. Martin SJ, Grimwood PD, Morris RG (2000) Synaptic plasticity and memory: an evaluation of the hypothesis. Annu Rev Neurosci 23(1):649–711

    Article  Google Scholar 

  26. McClelland JL, McNaughton BL, O’Reilly RC (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102(3):419

    Article  Google Scholar 

  27. O’Reilly RC, Rudy JW (2001) Conjunctive representations in learning and memory: principles of cortical and hippocampal function. Psychol Rev 108(2):311

    Article  Google Scholar 

  28. Qu X, Che H, Huang J, Xu L, Zheng X (2021) Multi-layered semantic representation network for multi-label image classification. arXiv preprint arXiv:210611596

  29. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Computer Science

  30. Tommasi T, Patricia N, Caputo B, Tuytelaars T (2017) A deeper look at dataset bias. In: Domain adaptation in computer vision applications, Springer, pp 37–55

  31. Torralba A, Efros AA (2011) Unbiased look at dataset bias. In: CVPR 2011, IEEE, pp 1521–1528

  32. Wang Y, Xie Y, Liu Y, Zhou K, Li X (2020) Fast graph convolution network based multi-label image recognition via cross-modal fusion. In: CIKM ’20: The 29th ACM international conference on information and knowledge management

  33. Wang Z, Chen T, Li G, Xu R, Lin L (2017) Multi-label image recognition by recurrently discovering attentional regions. In: Proceedings of the IEEE international conference on computer vision, pp 464–472

  34. Wei Y, Xia W, Lin M, Huang J, Ni B, Dong J, Zhao Y, Yan S (2015) Hcp: A flexible cnn framework for multi-label image classification. IEEE Trans Pattern Anal Mach Intell 38(9):1901–7

    Article  Google Scholar 

  35. Xue X, Zhang W, Zhang J, Wu B, Fan J, Lu Y (2011) Correlative multi-label multi-instance image annotation. In: 2011 international conference on computer vision, IEEE, pp 651–658

  36. You R, Guo Z, Cui L, Long X, Wen S (2020) Cross-modality attention with semantic graph embedding for multi-label classification. In: Proceedings of the AAAI conference on artificial intelligence 34(7):12709–12716

  37. Zhu F, Li H, Ouyang W, Yu N, Wang X (2017) Learning spatial regularization with image-level supervisions for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5513–5522

  38. Zola-Morgan S, Squire LR, Amaral DG (1986) Human amnesia and the medial temporal region: enduring memory impairment following a bilateral lesion limited to field ca1 of the hippocampus. J Neurosci 6(10):2950–2967

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the Fundamental Research Funds for the Central Universities B200202205, National Nature Science Foundation of China under grants (61501170, 41876097), Young Talent Development Plan of Changzhou Health Commission (2020-233), the Key Research and Development Program of Jiangsu under grants BK20192004, BE2018004-04, Guangdong Forestry Science and Technology Innovation Project under grant 2020KJCX005, International Cooperation and Exchanges of Changzhou under grant CZ20200035, and by the State Key Laboratory of Integrated Management of Pest Insects and Rodents(Grant No. IPM1914).

Author information

Authors and Affiliations

  1. The College of IoT Engineering, Hohai University, Changzhou, People’s Republic of China

    Xiao Yao & Feiyang Xu

  2. The First People’s Hospital of Changzhou, Changzhou, People’s Republic of China

    Min Gu & Peipei Wang

Authors
  1. Xiao Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feiyang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peipei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feiyang Xu.

Ethics declarations

Conflict of interest

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, X., Xu, F., Gu, M. et al. M-GCN: Brain-inspired memory graph convolutional network for multi-label image recognition. Neural Comput & Applic 34, 6489–6502 (2022). https://doi.org/10.1007/s00521-021-06803-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06803-z

Keywords

Search

Navigation