Abstract
The human visual system is a complex network of neurons that employs robust mechanisms to perceive and interpret the environment. Despite significant advancements in computer vision technologies in recent years, they still need to improve compared to human abilities, particularly in recognizing faces and interpreting scenes. As a result, there is a growing interest in understanding the underlying mechanisms of human vision. Artificial Intelligence (AI) systems, specifically computer vision models, represent the assigned task using a learned or systematically generated vector space known as the Latent Space. However, the field of brain representation space remains relatively less explored. Despite significant progress, a research gap exists in generating an optimal representation space for human visual processing. While graph-based representations have been proposed to better capture inter-region relationships in visual processing, learning an optimal graph representation from limited data remains a challenge, especially when there is no ground truth. Due to the lack of labeled data, supervised learning approaches are less preferred. The present study introduces a novel method for graph-based representation of the human visual processing system, utilizing Neural Combinatorial Optimization(NCO). We have obtained an accuracy of 60% from our proposed framework, which is comparable to other methods for eight class classification in Visual Modelling.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agrawal, P., Stansbury, D., Malik, J., Gallant, J.: Pixels to voxels: Modeling visual representation in the human brain (07 2014)
Bello, I., Pham, H., Le, Q., Norouzi, M., Bengio, S.: Neural combinatorial optimization with reinforcement learning (2016)
Chatterjee, S., Pain, S., Samanta, D.: Adversarial policy gradient for learning graph-based representation in human visual processing (2023). https://openreview.net/forum?id=5-ROmmBJKV
Cui, Y., Qiao, K., Zhang, C., Wang, L., Yan, B., Tong, l.: GaborNet visual encoding: a lightweight region-based visual encoding model with good expressiveness and biological interpretability. Front. Neurosci. 15, 614182 (2021). https://doi.org/10.3389/fnins.2021.614182
Deshpande, G., Wang, Y.: Noninvasive characterization of functional pathways in layer-specific microcircuits of the human brain using 7T fMRI. Brain Sci. 12, 1361 (2022). https://doi.org/10.3390/brainsci12101361
Dipasquale, O., et al.: Comparing resting state fMRI de-noising approaches using multi- and single-echo acquisitions. PLoS ONE 12, e0173289 (2017)
Gilson, M., et al.: Network analysis of whole-brain fMRI dynamics: a new framework based on dynamic communicability. NeuroImage 201, 116007 (2019). https://doi.org/10.1016/j.neuroimage.2019.116007
Güçlü, U., van Gerven, M.A.J.: Deep neural networks reveal a gradient in the complexity of neural representations across the brain’s ventral visual pathway (2014). https://doi.org/10.1523/JNEUROSCI.5023-14.2015. http://arxiv.org/abs/1411.6422https://doi.org/10.1523/JNEUROSCI.5023-14.2015
Han, K., et al.: Variational autoencoder: an unsupervised model for encoding and decoding fMRI activity in visual cortex. NeuroImage 198, 125–136 (2019). https://doi.org/10.1016/j.neuroimage.2019.05.039
Haxby, J.V.: Multivariate pattern analysis of fMRI: the early beginnings (2012). https://doi.org/10.1016/j.neuroimage.2012.03.016
Haynes, J.D., Rees, G.: Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8, 686–91 (2005). https://doi.org/10.1038/nn1445
Kay, K., Naselaris, T., Prenger, R., Gallant, J.: Identifying natural images from human brain activity. Nature 452, 352–5 (2008). https://doi.org/10.1038/nature06713
Kay, K.N.: Principles for models of neural information processing (2018). https://doi.org/10.1016/j.neuroimage.2017.08.016
Kriegeskorte, N.: Pattern-information analysis: from stimulus decoding to computational-model testing. NeuroImage 56, 411–421 (2011). https://doi.org/10.1016/j.neuroimage.2011.01.061
Li, X., et al.: BrainGNN: interpretable brain graph neural network for fMRI analysis. Medical Image Analysis 74, 102233 (2021). https://doi.org/10.1016/j.media.2021.102233
Li, Y., et al.: Brain connectivity based graph convolutional networks for infant age prediction. IEEE Trans. Med. Imaging, 1–1 (2022). https://doi.org/10.1109/TMI.2022.3171778
Meng, L., Ge, K.: Decoding visual fMRI stimuli from human brain based on graph convolutional neural network. Brain Sci. 12, 1394 (2022). https://doi.org/10.3390/brainsci12101394
Mohanty, R., Sethares, W., Nair, V., Prabhakaran, V.: Rethinking measures of functional connectivity via feature extraction. Sci. Rep. 10, 1298 (2020). https://doi.org/10.1038/s41598-020-57915-w
Naselaris, T., Kay, K.N., Nishimoto, S., Gallant, J.L.: Encoding and decoding in fMRI (2011). https://doi.org/10.1016/j.neuroimage.2010.07.073
Thirion, B., et al.: Inverse retinotopy: inferring the visual content of images from brain activation patterns. NeuroImage 33, 1104–16 (2007). https://doi.org/10.1016/j.neuroimage.2006.06.062
Wen, H., Shi, J., Chen, W., Liu, Z.: Deep residual network predicts cortical representation and organization of visual features for rapid categorization. Sci. Rep. 8, 3752 (2018). https://doi.org/10.1038/s41598-018-22160-9
Wen, H., Shi, J., Zhang, Y., Lu, K.H., Cao, J., Liu, Z.: Neural encoding and decoding with deep learning for dynamic natural vision. Cereb. Cortex 28, 4136–4160 (2018). https://doi.org/10.1093/cercor/bhx268
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chatterjee, S., Pain, S., Samanta, D. (2023). A Novel Graph Representation Learning Approach for Visual Modeling Using Neural Combinatorial Optimization. In: Maji, P., Huang, T., Pal, N.R., Chaudhury, S., De, R.K. (eds) Pattern Recognition and Machine Intelligence. PReMI 2023. Lecture Notes in Computer Science, vol 14301. Springer, Cham. https://doi.org/10.1007/978-3-031-45170-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-45170-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45169-0
Online ISBN: 978-3-031-45170-6
eBook Packages: Computer ScienceComputer Science (R0)