Skip to main content
SpringerLink
Log in

Horizon Cyber-Vision: A Cybernetic Approach for a Cortical Visual Prosthesis

  • Conference paper
  • First Online:
Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications (IWINAC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13258))

Abstract

The way towards the next generation of visual cortical prosthesis is visualised through an engineering cycle based on a cybernetic paradigm. Our proposal is to develop a configurable and wearable system that will generate simulated prosthetic vision, while on the other hand, perform intracortical stimulation when applied to blind patients, so that it is expected that improvements with sighted volunteers, in combination with transformed reality strategies, will correlate with similar improvements in blind patients. The resulting cybernetic model involves modelling from stimuli to visual percepts, and in parallel, developing the best suited transformed reality strategy leading to a better perception of the environment. Deep learning approaches for object detection, monocular depth estimation, or structural edge detection, in combination with the use of an eye-tracking system, will lead to an integrated system that has proved to be wearable, optimised, modular, and computationally lightweight. To assess the cybernetic approach, behavioural experiments are proposed using two different scenarios. Firstly, a corridor with a series of obstacles and a controlled but more complex environment that resembles a city square, called StreetLab.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ayton, L.N., et al.: Harmonization of outcomes and vision endpoints in vision restoration trials: recommendations from the international hover taskforce. Transl. Vis. Sci. Technol. 9(8), 25 (2020)

    Google Scholar 

  2. Benson, N.C., Kupers, E.R., Barbot, A., Carrasco, M., Winawer, J.: Cortical magnification in human visual cortex parallels task performance around the visual field. Elife 10, e67685 (2021)

    Google Scholar 

  3. Brindley, G., Lewin, W.: Short-and long-term stability of cortical electrical phosphenes. J. Physiol. 196, 479–493 (1968)

    Article  CAS  Google Scholar 

  4. Brindley, G.S., Lewin, W.S.: The sensations produced by electrical stimulation of the visual cortex. J. Physiol. 196(2), 479–493 (1968)

    Article  CAS  Google Scholar 

  5. Brindley, G.: Effects of electrical stimulation of the visual cortex. Hum. Neurobiol. 1, 281–283 (1982)

    CAS  PubMed  Google Scholar 

  6. Caspi, A., et al.: Eye movements and the perceived location of phosphenes generated by intracranial primary visual cortex stimulation in the blind. Brain Stimul. 14(4), 851–860 (2021)

    Article  Google Scholar 

  7. Chen, S.C., Suaning, G.J., Morley, J.W., Lovell, N.H.: Simulating prosthetic vision: I. visual models of phosphenes. Vis. Res. 49(12), 1493–1506 (2009)

    Google Scholar 

  8. Chen, X., Wang, F., Fernandez, E., Roelfsema, P.R.: Shape perception via a high-channel-count neuroprosthesis in monkey visual cortex. Science 370(6521), 1191–1196 (2020)

    Article  CAS  Google Scholar 

  9. Chou, T.S., et al.: Carlsim 4: an open source library for large scale, biologically detailed spiking neural network simulation using heterogeneous clusters. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)

    Google Scholar 

  10. Craik, K.J.W.: The Nature of Explanation, vol. 445. CUP Archive, Cambridge (1952)

    Google Scholar 

  11. Dobelle, W., Mladejovsky, M.: Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind. J. Physiol. 243(2), 553–576 (1974)

    Article  CAS  Google Scholar 

  12. Dobelle, W.H., Mladejovsky, M.G., Girvin, J.: Artificial vision for the blind: electrical stimulation of visual cortex offers hope for a functional prosthesis. Science 183(4123), 440–444 (1974)

    Article  CAS  Google Scholar 

  13. Dobelle, W.H.: Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO J. 46(1), 3–9 (2000)

    Article  CAS  Google Scholar 

  14. Dobelle, W.H., Mladejovsky, M.G., Evans, J.R., Roberts, T., Girvin, J.: Braille reading by a blind volunteer by visual cortex stimulation. Nature 259(5539), 111–112 (1976)

    Article  CAS  Google Scholar 

  15. Eshraghian, J.K., et al.: Formulation and implementation of nonlinear integral equations to model neural dynamics within the vertebrate retina. Int. J. Neural Syst. 28(07), 1850004 (2018)

    Article  Google Scholar 

  16. Fernández, E., et al.: Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex. J. Clin. Investig. 131(23), e151331 (2021). https://doi.org/10.1172/JCI151331

  17. Hall, E.C., Gordon, J., Abel, L.A., Hainline, L., Abramov, I.: Nystagmus waveforms in blindness. Vis. Impair. Res. 2(2), 65–73 (2000)

    Article  Google Scholar 

  18. Hartline, H.K.: The response of single optic nerve fibers of the vertebrate eye to illumination of the retina. Am. J. Physiol. Leg. Content 121(2), 400–415 (1938)

    Article  Google Scholar 

  19. Kumarasinghe, K., Kasabov, N., Taylor, D.: Deep learning and deep knowledge representation in spiking neural networks for brain-computer interfaces. Neural Netw. 121, 169–185 (2020)

    Article  Google Scholar 

  20. Li, W.: Wearable computer vision systems for a cortical visual prosthesis. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 428–435 (2013)

    Google Scholar 

  21. Liu, L., et al.: Deep learning for generic object detection: a survey. Int. J. Comput. Vis. 128(2), 261–318 (2020)

    Article  Google Scholar 

  22. Liu, Y., Cheng, M.M., Fan, D.P., Zhang, L., Bian, J.W., Tao, D.: Semantic edge detection with diverse deep supervision. Int. J. Comput. Vis. 130(1), 179–198 (2022)

    Article  Google Scholar 

  23. Lozano, A., Soto-Sanchez, C., Garrigos, J., Martínez, J.J., Ferrández, J.M., Fernandez, E.: A 3d convolutional neural network to model retinal ganglion cell’s responses to light patterns in mice. Int. J. Neural Syst. 28(10), 1850043 (2018)

    Article  Google Scholar 

  24. Lui, W.L.D., Browne, D., Kleeman, L., Drummond, T., Li, W.H.: Transformative reality: augmented reality for visual prostheses. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 253–254. IEEE (2011)

    Google Scholar 

  25. Lui, W.L.D., Browne, D., Kleeman, L., Drummond, T., Li, W.H.: Transformative reality: improving bionic vision with robotic sensing. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 304–307. IEEE (2012)

    Google Scholar 

  26. McCulloch, W.S.: Embodiments of Mind. MIT press, Cambridge (2016)

    Google Scholar 

  27. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)

    Article  Google Scholar 

  28. Melanitis, N., Nikita, K.S.: Biologically-inspired image processing in computational retina models. Comput. Biol. Med. 113, 103399 (2019)

    Article  Google Scholar 

  29. Minaee, S., Boykov, Y.Y., Porikli, F., Plaza, A.J., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2021)

    Google Scholar 

  30. Mira, J.: Entre lo biológico y lo artificial: selección de publicaciones: con 24 escritos originales de colaboradores del autor. Red Temática en Tecnologías de Computación Artificial/Natural (RTNAC) (2014)

    Google Scholar 

  31. Muratore, D.G., Chichilnisky, E.J.: Artificial retina: a future cellular-resolution brain-machine interface. In: Murmann, B., Hoefflinger, B. (eds.) NANO-CHIPS 2030. TFC, pp. 443–465. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-18338-7_24

    Chapter  Google Scholar 

  32. Oswalt, D., et al.: Multi-electrode stimulation evokes consistent spatial patterns of phosphenes and improves phosphene mapping in blind subjects. Brain Stimul. 14(5), 1356–1372 (2021)

    Article  Google Scholar 

  33. Pio-Lopez, L., Poulkouras, R., Depannemaecker, D.: Visual cortical prosthesis: an electrical perspective. J. Med. Eng. Technol. 45(5), 394–407 (2021)

    Article  Google Scholar 

  34. Polimeni, J.R., Balasubramanian, M., Schwartz, E.L.: Multi-area visuotopic map complexes in macaque striate and extra-striate cortex. Vis. Res. 46(20), 3336–3359 (2006)

    Article  CAS  Google Scholar 

  35. Poma, X.S., Riba, E., Sappa, A.: Dense extreme inception network: towards a robust CNN model for edge detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (March 2020)

    Google Scholar 

  36. Rosenblueth, A., Wiener, N., Bigelow, J.: Behavior, purpose and teleology. Philos. Sci. 10(1), 18–24 (1943)

    Article  Google Scholar 

  37. Schiller, P.H., Tehovnik, E.J.: Visual prosthesis. Perception 37(10), 1529–1559 (2008)

    Article  Google Scholar 

  38. Schultze, M.: The retina is the membrane-like terminal expansion of the. Man. Hum. Comparat. Histol. 3, 218 (1873)

    Google Scholar 

  39. Shah, N.P., Chichilnisky, E.: Computational challenges and opportunities for a bi-directional artificial retina. J. Neural Eng. 17(5), 055002 (2020)

    Google Scholar 

  40. Van Steveninck, J.D.R., et al.: Real-world indoor mobility with simulated prosthetic vision: the benefits and feasibility of contour-based scene simplification at different phosphene resolutions. J. Vis. 22(2), 1 (2022)

    Google Scholar 

  41. Subbulakshmi Radhakrishnan, S., Sebastian, A., Oberoi, A., Das, S., Das, S.: A biomimetic neural encoder for spiking neural network. Nat. Commun. 12(1), 1–10 (2021)

    Article  Google Scholar 

  42. Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., Masquelier, T., Maida, A.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019)

    Article  Google Scholar 

  43. Wiener, N.: Cybernetics or Control and Communication in the Animal and the Machine. Technology Press, Cambridge (1948)

    Google Scholar 

  44. Zaidi, S.S.A., Ansari, M.S., Aslam, A., Kanwal, N., Asghar, M., Lee, B.: A survey of modern deep learning based object detection models. Digital Signal Process. 126, 103514 (2022)

    Article  Google Scholar 

  45. Zhao, C.Q., Sun, Q.Y., Zhang, C.Z., Tang, Y., Qian, F.: Monocular depth estimation based on deep learning: an overview. Sci. China Technol. Sci. 63(9), 1612–1627 (2020). https://doi.org/10.1007/s11431-020-1582-8

    Article  Google Scholar 

Download references

Acknowledgements

This project has received funding by grant RTI2018-098969-B-100 from the Spanish Ministerio de Ciencia Innovación y Universidades, by grant PROMETEO/2019/119 from the Generalitat Valenciana (Spain), by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 861423 (enTRAIN Vision) and by grant agreement No. 899287 (project NeuraViPer).

Author information

Authors and Affiliations

  1. Instituto de Bioingeniería, Univ. Miguel Hernández, Alicante, Spain

    Mikel Val Calvo, Roberto Morollón Ruiz, Leili Soo, Dorota Wacławczyk, Fabrizio Grani & Eduardo Fernández Jover

  2. CIBER-BBN, Madrid, Spain

    Eduardo Fernández Jover

  3. Dpto. de Electrónica, Tecnología de Computadoras y Proyectos, Univ. Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández

Authors
  1. Mikel Val Calvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberto Morollón Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leili Soo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dorota Wacławczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fabrizio Grani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. José Manuel Ferrández
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Eduardo Fernández Jover
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mikel Val Calvo .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Universidad Nacional de Educación a Distancia, Madrid, Spain

    José Ramón Álvarez-Sánchez

  3. Universidad Nacional de Educación a Distancia, Madrid, Spain

    Félix de la Paz López

  4. Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Val Calvo, M. et al. (2022). Horizon Cyber-Vision: A Cybernetic Approach for a Cortical Visual Prosthesis. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications. IWINAC 2022. Lecture Notes in Computer Science, vol 13258. Springer, Cham. https://doi.org/10.1007/978-3-031-06242-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06242-1_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06241-4

  • Online ISBN: 978-3-031-06242-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation