Skip to main content

Advertisement

SpringerLink
Log in

Toward Robot Self-Consciousness (II): Brain-Inspired Robot Bodily Self Model for Self-Recognition

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The neural correlates and nature of self-consciousness is an advanced topic in Cognitive Neuroscience. Only a few animal species have been testified to be with this cognitive ability. From artificial intelligence and robotics point of view, few efforts are deeply rooted in the neural correlates and brain mechanisms of biological self-consciousness. Despite the fact that the scientific understanding of biological self-consciousness is still in preliminary stage, we make our efforts to integrate and adopt known biological findings of self-consciousness to build a brain-inspired model for robot self-consciousness. In this paper, we propose a brain-inspired robot bodily self model based on extensions to primate mirror neuron system and apply it to humanoid robot for self recognition. In this model, the robot firstly learns the correlations between self-generated actions and visual feedbacks in motion by learning with spike timing dependent plasticity (STDP), and then learns the appearance of body part with the expectation that the visual feedback is consistent with its motion. Based on this model, the robot uses multisensory integration to learn its own body in real world and in mirror. Then it can distinguish itself from others. In a mirror test setting with three robots with the same appearance, with the proposed brain-inspired robot bodily self model, each of them can recognize itself in the mirror after these robots make random movements at the same time. The theoretic modeling and experimental validations indicate that the brain-inspired robot bodily self model is biologically inspired, and computationally feasible as a foundation for robot self recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Robot Self-Consciousness Project: http://bii.ia.ac.cn/robot-self

References

  1. Molnarszakacs I, Uddin LQ. Self-processing and the default mode network: Interactions with the mirror neuron system. Front Hum Neurosci 2013;7(571):571.

    Google Scholar 

  2. Uddin LQ, Kaplan JT, Molnar-Szakacs I, Zaidel E, Iacoboni M. Self-face recognition activates a frontoparietal mirror network in the right hemisphere: an event-related fmri study. Neuroimage 2005;25(3):926–35.

    Article  PubMed  Google Scholar 

  3. van Veluw SJ, Chance SA. Differentiating between self and others: an ale meta-analysis of fmri studies of self-recognition and theory of mind. Brain Imaging Behav 2014;8(1):24–38.

    Article  PubMed  Google Scholar 

  4. Sugiura M, Miyauchi CM, Kotozaki Y, Akimoto Y, Nozawa T, Yomogida Y, Hanawa S, Yamamoto Y, Sakuma A, Nakagawa S, et al. Neural mechanism for mirrored self-face recognition. Cereb Cortex 2014;25(9):2806–14.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Platek S, Wathne K, Tierney NG, Thomson JW. Neural correlates of self-face recognition: an effect-location meta-analysis. Brain Res 2008;1232:173–84.

    Article  CAS  PubMed  Google Scholar 

  6. Hu C, Di X, Eickhoff SB, Zhang M, Peng K, Guo H, Sui J. Distinct and common aspects of physical and psychological self-representation in the brain: a meta-analysis of self-bias in facial and self-referential judgements. Neurosci Biobehav Rev 2016;61:197–207.

    Article  PubMed  Google Scholar 

  7. Devue C, Collette F, Balteau E, Degueldre C, Luxen A, Maquet P, Brédart S. Here i am: the cortical correlates of visual self-recognition. Brain Res 2007;1143:169–82.

    Article  CAS  PubMed  Google Scholar 

  8. Kruse B, Bogler C, Haynes JD, Schütz-Bosbach S. Am i seeing myself, my friend or a stranger? The role of personal familiarity in visual distinction of body identities in the human brain. Cortex 2016;83:86–100.

    Article  PubMed  Google Scholar 

  9. Uddin LQ, Molnar-Szakacs I, Zaidel E, Iacoboni M. rtms to the right inferior parietal lobule disrupts self–other discrimination. Soc Cogn Affect Neurosci 2006;1(1):65.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Farrer C, Franck N, Georgieff N, Frith CD, Decety J, Jeannerod M. Modulating the experience of agency: a positron emission tomography study. Neuroimage 2003;18(2):324–33.

    Article  CAS  PubMed  Google Scholar 

  11. Macuga KL, Frey SH. Selective responses in right inferior frontal and supramarginal gyri differentiate between observed movements of oneself vs. another. Neuropsychologia 2011;49(5):1202–7.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Gold K, Scassellati B. Using probabilistic reasoning over time to self-recognize. Robot Auton Syst 2009;57(4):384–92.

    Article  Google Scholar 

  13. Takeno J, Inaba K, Suzuki T. Experiments and examination of mirror image cognition using a small robot. Proceedings of the 2005 IEEE International Symposium on Computational Intelligence in Robotics and Automation; 2005. p. 493–498.

  14. Broun A, Beck C, Pipe T, Mirmehdi M, Melhuish C. Bootstrapping a robots kinematic model. Robot Auton Syst 2014;62(3):330–9.

    Article  Google Scholar 

  15. Ciliberto C, Smeraldi F, Natale L, Metta G. Online multiple instance learning applied to hand detection in a humanoid robot. Ieee/rsj International Conference on Intelligent Robots and Systems; 2011. p. 1526–1532.

  16. Bongard J, Zykov V, Lipson H. Resilient machines through continuous self-modeling. Science 2006;314(5802):1118.

    Article  CAS  PubMed  Google Scholar 

  17. Anderson J, Gallup GG Jr. Mirror self-recognition: a review and critique of attempts to promote and engineer self-recognition in primates. Primates 2015;56(4):317.

    Article  PubMed  Google Scholar 

  18. Gallup G, Anderson J, Platek S. Self-recognition. The Oxford handbook of the self. Oxford: Oxford University Press; 2011. p. 80–110.

  19. Zeng Y, Zhao Y, Bai J. Toward robot self-consciousness (i): Brain-inspired robot mirror neuron system model and its application in mirror self-recognition. Proceedings of the 8th International Conference on Brain Inspired Cognitive Systems (BICS 2016); 2016. p. 11–21.

  20. Iacoboni M, Dapretto M. The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci 2006;7(12):942–951.

    Article  CAS  PubMed  Google Scholar 

  21. Zhao W, Luo L, Li Q, Kendrick KM. What can psychiatric disorders tell us about neural processing of the self? Front Hum Neurosci 2013;7:485.

    PubMed  PubMed Central  Google Scholar 

  22. Thakkar KN, Peterman JS, Park S. Altered brain activation during action imitation and observation in schizophrenia: a translational approach to investigating social dysfunction in schizophrenia. Am J Psychiatr 2014;171(5):539–48.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Haist F, Anzures G. Functional development of the brain’s face-processing system. Wiley Interdiscip Rev Cogn Sci 2017;8(1-2):1–11.

    Article  Google Scholar 

  24. Duchaine B, Yovel G. A revised neural framework for face processing. Ann Rev Vis Sci 2015;1:393–416.

    Article  Google Scholar 

  25. Yovel G, OToole AJ. Recognizing people in motion. Trends Cogn Sci 2016;20(5):383–95.

    Article  PubMed  Google Scholar 

  26. Peelen MV, Downing PE. The neural basis of visual body perception. Nat Rev Neurosci 2007;8(8):636–48.

    Article  CAS  PubMed  Google Scholar 

  27. Taylor JC, Wiggett AJ, Downing PE. Functional mri analysis of body and body part representations in the extrastriate and fusiform body areas. J Neurophysiol 2007;98(3):1626–33.

    Article  PubMed  Google Scholar 

  28. Gazzaniga MS, Ivry RB, Mangun GR, Steven MS. Cognitive neuroscience: the biology of the mind. New York: W. W. Norton & Company Inc.; 2013.

    Google Scholar 

  29. Perrone JA, Thiele A. Speed skills: measuring the visual speed analyzing properties of primate mt neurons. Nat Neurosci 2001;4(5):526–32.

    Article  CAS  PubMed  Google Scholar 

  30. Grossman ED, Blake R. Brain areas active during visual perception of biological motion. Neuron 2002;35(6):1167–75.

    Article  CAS  PubMed  Google Scholar 

  31. Hamzei F, Vry MS, Saur D, Glauche V, Hoeren M, Mader I, Weiller C, Rijntjes M. The dual-loop model and the human mirror neuron system: an exploratory combined fmri and dti study of the inferior frontal gyrus. Cereb Cortex 2016;26(5):2215–24.

    Article  PubMed  Google Scholar 

  32. Georgopoulos AP, Schwartz AB, Kettner RE, et al. Neuronal population coding of movement direction. Science 1986;233(4771):1416–9.

    Article  CAS  PubMed  Google Scholar 

  33. Craig AD. How do you feelłnow? The anterior insula and human awareness. Nat Rev Neurosci 2009;10(1):59–70.

    Article  CAS  PubMed  Google Scholar 

  34. Sasaki AT, Kochiyama T, Sugiura M, Tanabe HC, Sadato N. Neural networks for action representation: a functional magnetic-resonance imaging and dynamic causal modeling study. Front Hum Neurosci 2012;6(1):236.

    PubMed  PubMed Central  Google Scholar 

  35. Mehta UM, Thirthalli J, Aneelraj D, Jadhav P, Gangadhar BN, Keshavan MS. Mirror neuron dysfunction in schizophrenia and its functional implications: a systematic review. Schizophr Res 2014;160(1):9–19.

    Article  PubMed  Google Scholar 

  36. Yun JY, Hur JW, Jung WH, Jang JH, Youn T, Kang DH, Park S, Kwon JS. Dysfunctional role of parietal lobe during self-face recognition in schizophrenia. Schizophr Res 2014;152(1):81–8.

    Article  PubMed  Google Scholar 

  37. Beyeler M, Richert M, Dutt ND, Krichmar JL. Efficient spiking neural network model of pattern motion selectivity in visual cortex. Neuroinformatics 2014;12(3):435–54.

    Article  PubMed  Google Scholar 

  38. Escobar MJ, Wohrer A, Kornprobst P, Viéville T. Biological motion recognition using a mt-like model. Proceedings of the 3rd IEEE Latin American Robotics Symposium (LARS’06); 2006. p. 47–52.

  39. Bi GQ, Poo MM. Synaptic modification by correlated activity: Hebb’s postulate revisited. Ann Rev Neurosci 2001;24(1):139–66.

    Article  CAS  PubMed  Google Scholar 

  40. Song S, Miller KD, Abbott LF. Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 2000;3(9):919–26.

    Article  CAS  PubMed  Google Scholar 

  41. Izhikevich EM. Simple model of spiking neurons. IEEE Trans Neural Netw 2003;14(6):1569–72.

    Article  CAS  PubMed  Google Scholar 

  42. Girshick R. Fast r-cnn. Proceedings of the 2015 IEEE International Conference on Computer Vision; 2015. p. 1440– 1448.

  43. Rochat P. Five levels of self-awareness as they unfold early in life. Conscious Cogn 2003;12(4):717–31.

    Article  PubMed  Google Scholar 

  44. Henriksson L, Mur M, Kriegeskorte N. Faciotopyła face-feature map with face-like topology in the human occipital face area. Cortex 2015;72:156–67.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Liu H, Yu Y, Sun F, Gu J. Visualctactile fusion for object recognition. IEEE Trans Autom Sci Eng 2017;14(2):996–1008.

    Article  Google Scholar 

  46. Liu H, Guo D, Sun F. Object recognition using tactile measurements: Kernel sparse coding methods. IEEE Trans Instrum Meas 2016;65(3):656–65.

    Article  Google Scholar 

Download references

Author information

Author notes

    Authors and Affiliations

    1. Institute of Automation, Chinese Academy of Sciences, Beijing, China

      Yi Zeng, Yuxuan Zhao, Jun Bai & Bo Xu

    2. Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China

      Yi Zeng & Bo Xu

    3. University of Chinese Academy of Sciences, Beijing, China

      Yi Zeng & Bo Xu

    Authors
    1. Yi Zeng
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Yuxuan Zhao
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Jun Bai
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Bo Xu
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Yi Zeng.

    Ethics declarations

    Compliance with Ethical Standards

    This study was funded by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB02060007), and Beijing Municipal Commission of Science and Technology (Z161100000216124).

    Ethical approval

    This article does not contain any studies with human participants performed by any of the authors.

    Conflict of interests

    All authors declare that they have no conflict of interest.

    Additional information

    Yi Zeng and Yuxuan Zhao have equal contribution to this work and should be regarded as co-first authors

    Appendices

    Appendix

    Appendix A

    Take one movement cycle as an example, as shown in Table 1. N denotes that the related brain regions are not functionally activated. If the value is 0, it means the value is too small and can be omitted.

    Table 1 Time consumption of the experimental validation
    Full size table

    During the learning phase, the motion sequences, the detected moving parts, and the visual inputs are used. The actual motion sequences and the detected motion sequences from vision are used for motor-visual associative learning, while the detected moving parts and the acquired visual inputs are used for appearance learning. The inputs for Fast R-CNN are images for left arm, and right arm of the robot itself, head of the self, and head of others (1000 pieces for each class, with different backgrounds and random movements). The training was conducted on GPU K40 for approximately 16 h.

    Appendix B

    In the multi-robots mirror self-recognition test, three robots made random movements facing a mirror. Since the three robots are making random movements at the same time, the robots will acquire all the angles for associative learning. As the times of movements grow, the robot will gradually identify which angle from vision are from itself. As an illustrative example, robot A is assigned to have repetitive movement with the angle value 88°, while the other two robots make random movements for 100 times respectively. The associative weights for motor angles (88°) and visual angles (0°–360°) are shown in Fig. 10. Based on Eq. 12, the angle with the maximum weight is 88°. At this time, when robot A generates a motion with 88°, it expect to detect 88° from vision.

    Fig. 10
    figure 10

    Weight distribution

    Full size image

    There are errors for angle detection which are hard to be completely avoided. Hence, the detected visual angles may not always be the same when the motor angles are the same. We can identify from Fig. 10 that the detected visual angles are distributed in a small range. Hence, 𝜃 threshold are used to represent this range in Eq. 12 for self-recognition. Figure 11 presents the standard deviation between angles from vision and the expected angles, while the red line (with value 3.69) denotes the average value for the standard deviation from 1∘ to 360∘. Hence, as an approximation, 𝜃 threshold is set to 4° in Eq. 12. According to the time consumption of information acquisition for robots, t threshold is set to 2 s. Every movement cycle contains 20 motion points. Hence, in Eq. 12, n = 20-1.

    Fig. 11
    figure 11

    Standard deviation between predicted and actual visual angles

    Full size image

    Appendix C

    Figure 12 provides the belief values of different positions for each robot in nine independent tests. Each robot can pass the mirror test in all the nine tests.

    Fig. 12
    figure 12

    multi-robots mirror self-recognition test in nine times

    Full size image

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zeng, Y., Zhao, Y., Bai, J. et al. Toward Robot Self-Consciousness (II): Brain-Inspired Robot Bodily Self Model for Self-Recognition. Cogn Comput 10, 307–320 (2018). https://doi.org/10.1007/s12559-017-9505-1

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s12559-017-9505-1

    Keywords

    Search

    Navigation