Invited: Neuromorphic Vision Modalities in the NimbleAI 3D Chip
Authors: 
Xabier Iturbe, Bernabe Linares-Barranco, Sio-Hoi Ieng, Arne Erdmann, Luca Peres, Oliver Rhodes, Rafael Tornero, Manolis Sifalakis, Marcel van de Burgwal, Amirreza Yousefzadeh, Maha Kooli, Riccardo Alidori, Pavel ZaykovAuthors Info & Claims
Xabier Iturbe, Bernabe Linares-Barranco, Sio-Hoi Ieng, Arne Erdmann, Luca Peres, Oliver Rhodes, Rafael Tornero, Manolis Sifalakis, Marcel van de Burgwal, Amirreza Yousefzadeh, Maha Kooli, Riccardo Alidori, Pavel ZaykovAuthors Info & ClaimsArticle No.: 358, Pages 1 - 4
Abstract
This paper provides an overview of the ongoing work to enable novel modalities of passive monocular neuromorphic vision in the NimbleAI sensing-processing architecture; namely, foveated and light-field event-driven vision with selective visual attention. The latter vision modality encodes 3D visual surroundings as sparse visual events in a 4D spatiotemporal domain, adding depth to current representation of visual information delivered by Dynamic Vision Sensors (DVS). The NimbleAI architecture implements hardware support for efficient execution of mainstream computer vision algorithms and AI models using these visual inputs. The architecture is designed to harness the latest advancements in 3D silicon integration, making it possible to squeeze sensing and spiking circuitry, memory, and processing engines into a miniature silicon volume.
References
[1]
Xabier Iturbe et al. NimbleAI: Towards neuromorphic sensing-processing 3D-integrated chips. In DATE, 2023.
[2]
Yole. Neuromorphic Computing, Memory and Sensing Report. 2024.
[3]
Patrick Lichtsteiner et al. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits, 43(2):566--576, 2008.
[4]
Amirreza Yousefzadeh et al. SENeCA: Scalable energy-efficient neuromorphic computer architecture. In AICAS, 2022.
[5]
Maha Kooli et al. Towards a truly integrated vector processing unit for memory-bound applications based on a cost-competitive computational sram design solution. ACM J. Emerg. Technol. Comput. Syst., 18(2), 2022.
[6]
Teresa Serrano and Bernabé Linares. A 128×128 1.5% contrast sensitivity 0.9% FPN 3 μs latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE J. Solid-State Circuits, 48(3):827--838, 2013.
[7]
Massimiliano Iacono et al. Proto-object based saliency for event-driven cameras. In IROS, 2019.
[8]
Jean-Nicolas Jérémie et al. Retinotopic mapping enhances the robustness of convolutional neural networks, 2024.
[9]
Christian Perwaß and Lennart Wietzke. Single lens 3D-camera with extended depth-of-field. SPIE Human Vision and Electronic Imaging XVII, (8291), 2012.
[10]
Xavier Lagorce et al. HOTS: A hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39(7):1346--1359, 2017.
[11]
Marc Osswald et al. A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Scientific Reports, 7, 2017.
[12]
Sio-Hoi Ieng et al. Event-based 3D motion flow estimation using 4D spatio temporal subspaces properties. Frontiers in Neuroscience, 10, 2017.
Index Terms
- Invited: Neuromorphic Vision Modalities in the NimbleAI 3D Chip
Index terms have been assigned to the content through auto-classification.
Recommendations
Coupling an a VLSI Neuromorphic vision chip to a neurotrophic model of synaptic plasticity
We couple a previously studied, biologically inspired neurotrophic model of activity-dependent competitive synaptic plasticity and neuronal development to a neuromorphic retina chip. Using this system, we examine the development and refinement of a ...
A neuromorphic multi-chip model of a disparity selective complex cell
NIPS'03: Proceedings of the 17th International Conference on Neural Information Processing SystemsThe relative depth of objects causes small shifts in the left and right retinal positions of these objects, called binocular disparity. Here, we describe a neuromorphic implementation of a disparity selective complex cell using the binocular energy model,...
3D model based tracking for omnidirectional vision: A new spherical approach
The current work addresses the problem of 3D model tracking in the context of monocular and stereo omnidirectional vision in order to estimate the camera pose. To this end, we track 3D objects modeled by line segments because the straight line feature ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 November 2024
Check for updates
Qualifiers
- Research-article
Funding Sources
- UKRI
- EU Horizon Europe
Conference
DAC '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,770 of 5,499 submissions, 32%
Upcoming Conference
DAC '25
- Sponsor:
- sigda
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 87Total Downloads
- Downloads (Last 12 months)87
- Downloads (Last 6 weeks)11
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in