skip to main content
10.1145/3649329.3689622acmconferencesArticle/Chapter ViewAbstractPublication PagesdacConference Proceedingsconference-collections
research-article

Invited: Neuromorphic Vision Modalities in the NimbleAI 3D Chip

Published: 07 November 2024 Publication History

Abstract

This paper provides an overview of the ongoing work to enable novel modalities of passive monocular neuromorphic vision in the NimbleAI sensing-processing architecture; namely, foveated and light-field event-driven vision with selective visual attention. The latter vision modality encodes 3D visual surroundings as sparse visual events in a 4D spatiotemporal domain, adding depth to current representation of visual information delivered by Dynamic Vision Sensors (DVS). The NimbleAI architecture implements hardware support for efficient execution of mainstream computer vision algorithms and AI models using these visual inputs. The architecture is designed to harness the latest advancements in 3D silicon integration, making it possible to squeeze sensing and spiking circuitry, memory, and processing engines into a miniature silicon volume.

References

[1]
Xabier Iturbe et al. NimbleAI: Towards neuromorphic sensing-processing 3D-integrated chips. In DATE, 2023.
[2]
Yole. Neuromorphic Computing, Memory and Sensing Report. 2024.
[3]
Patrick Lichtsteiner et al. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits, 43(2):566--576, 2008.
[4]
Amirreza Yousefzadeh et al. SENeCA: Scalable energy-efficient neuromorphic computer architecture. In AICAS, 2022.
[5]
Maha Kooli et al. Towards a truly integrated vector processing unit for memory-bound applications based on a cost-competitive computational sram design solution. ACM J. Emerg. Technol. Comput. Syst., 18(2), 2022.
[6]
Teresa Serrano and Bernabé Linares. A 128×128 1.5% contrast sensitivity 0.9% FPN 3 μs latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE J. Solid-State Circuits, 48(3):827--838, 2013.
[7]
Massimiliano Iacono et al. Proto-object based saliency for event-driven cameras. In IROS, 2019.
[8]
Jean-Nicolas Jérémie et al. Retinotopic mapping enhances the robustness of convolutional neural networks, 2024.
[9]
Christian Perwaß and Lennart Wietzke. Single lens 3D-camera with extended depth-of-field. SPIE Human Vision and Electronic Imaging XVII, (8291), 2012.
[10]
Xavier Lagorce et al. HOTS: A hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39(7):1346--1359, 2017.
[11]
Marc Osswald et al. A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Scientific Reports, 7, 2017.
[12]
Sio-Hoi Ieng et al. Event-based 3D motion flow estimation using 4D spatio temporal subspaces properties. Frontiers in Neuroscience, 10, 2017.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
DAC '24: Proceedings of the 61st ACM/IEEE Design Automation Conference
June 2024
2159 pages
ISBN:9798400706011
DOI:10.1145/3649329
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2024

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • UKRI
  • EU Horizon Europe

Conference

DAC '24
Sponsor:
DAC '24: 61st ACM/IEEE Design Automation Conference
June 23 - 27, 2024
CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25
62nd ACM/IEEE Design Automation Conference
June 22 - 26, 2025
San Francisco , CA , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 87
    Total Downloads
  • Downloads (Last 12 months)87
  • Downloads (Last 6 weeks)11
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media