CEDAR: Computing-in-pixel Edge-aware Detection and Reconstruction Architecture for High-resolution 3D Imaging
Authors: 
Bu Chen, Zhangcheng Huang, Qi Zheng, Weiyi Tang, Jingyi Wang, Hankun Lv, Chixiao Chen, Jianlu Wang, Qi LiuAuthors Info & Claims
Bu Chen, Zhangcheng Huang, Qi Zheng, Weiyi Tang, Jingyi Wang, Hankun Lv, Chixiao Chen, Jianlu Wang, Qi LiuAuthors Info & ClaimsArticle No.: 217, Pages 1 - 6
Abstract
Large-format single-photon avalanche diode (SPAD)-based direct time of flight (dToF) sensors are expected to be widely applied in future L5 full driving automation. However, the high-power in-pixel TDCs and the huge amount of data generated by multiframe histogram sampling impose limitations on the pixel format of SPAD-based dToF sensors. To tackle this challenge, we proposed the Computing-in-pixel Edge-aware Detection and Reconstruction (CEDAR) architecture. In this architecture, edge pixels are recognized by charge-domain convolution (CDC) computing, and noise pixels are eliminated by in-memory denoising (IMD). Only few TDCs in these edge pixels are activated, resulting in significant power and data savings. Afterward, the full-format image is reconstructed by a U-Net using the obtained depth information from these edge pixels. For the first time, we proposed a high-resolution 512 × 512 SPAD-based dToF sensor with a low power of 83.3 mW, a distance accuracy of 0.9 cm, and a frame rate of 60 fps. The high-resolution 3D image can be reconstructed by only 3.5% sparse edge pixels, achieving a PSNR of 35.2 dB. The CEDAR architecture can achieve 16× pixel format and image resolution improvement under the same constraint of power dissipation.
References
[1]
Li et al. Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems. IEEE Signal Processing Magazine, 37(4):50--61, 2020.
[2]
Campbell et al. Recent advances in avalanche photodiodes. Journal of Lightwave Technology, 34(2):278--285, 2015.
[3]
Morimoto et al. 3.2 megapixel 3d-stacked charge focusing spad for low-light imaging and depth sensing. In 2021 IEEE International Electron Devices Meeting (IEDM), pages 20.2.1--20.2.4, 2021.
[4]
Kazuhiro Morimoto et al. Megapixel time-gated spad image sensor for 2d and 3d imaging applications. Optica, 7(4):346--354, Apr 2020.
[5]
Zhang et al. A 30-frames/s, 252 × 144 spad flash lidar with 1728 dual-clock 48.8-ps tdcs, and pixel-wise integrated histogramming. IEEE Journal of Solid-State Circuits, 54(4):1137--1151, 2019.
[6]
Padmanabhan et al. 7.4 a 256×128 3d-stacked (45nm) spad flash lidar with 7-level coincidence detection and progressive gating for 100m range and 10klux background light. In 2021 IEEE International Solid-State Circuits Conference (ISSCC), volume 64, pages 111--113, 2021.
[7]
Stoppa et al. A reconfigurable qvga/q3vga direct time-of-flight 3d imaging system with on-chip depth-map computation in 45/40 nm 3d-stacked bsi spad cmos. In Proc. Int. Image Sensor Workshop, pages 53--56, 2021.
[8]
Erdogan et al. A 16.5 giga events/s 1024× 8 spad line sensor with per-pixel zoomable 50ps-6.4 ns/bin histogramming tdc. In 2017 Symposium on VLSI Circuits, pages C292--C293. IEEE, 2017.
[9]
Mauro Buttafava et al. Spad-based asynchronous-readout array detectors for image-scanning microscopy. Optica, 7(7):755--765, Jul 2020.
[10]
Cui et al. Toward implementing multichannels, ring-oscillator-based, vernier time-to-digital converter in fpgas: Key design points and construction method. IEEE Transactions on Radiation and Plasma Medical Sciences, 1(5):391--399, 2017.
[11]
M. et al. Elad. Super-resolution reconstruction of image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9):817--834, 1999.
[12]
Eilertsen et al. Hdr image reconstruction from a single exposure using deep cnns. ACM Trans. Graph., 36(6), nov 2017.
[13]
Zitnick et al. High-quality video view interpolation using a layered representation. ACM Trans. Graph., 23(3):600--608, aug 2004.
[14]
Ronneberger et al. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234--241. Springer, 2015.
[15]
Zhirong et al. Wu. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.
[16]
Song et al. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.
[17]
Henderson et al. 5.7 a 256×256 40nm/90nm cmos 3d-stacked 120db dynamic-range reconfigurable time-resolved spad imager. In 2019 IEEE International Solid-State Circuits Conference - (ISSCC), pages 106--108, 2019.
[18]
Kumagai et al. 7.3 a 189×600 back-illuminated stacked spad direct time-of-flight depth sensor for automotive lidar systems. In 2021 IEEE International Solid-State Circuits Conference (ISSCC), volume 64, pages 110--112, 2021.
Index terms have been assigned to the content through auto-classification.
Recommendations
Low-Power Low-Noise Front-End Circuit Design with Shared Pinned Photodiode 4T Active Pixel Sensor for 3D Imaging
ICEICE '12: Proceedings of the 2012 Second International Conference on Electric Information and Control Engineering - Volume 04There needs one low power and low noise image sensor technology to capture the 3D images based on CMOS process. In this paper, a novel front end circuit of CMOS sensor has proposed including the pixel readout circuit, noise reduction circuit and analog ...
Regularized Super-Resolution Reconstruction Based on Edge Prior
Intelligent Computing MethodologiesAbstractConsidering that there is no edge constraint in general regularized algorithms, an improved super-resolution algorithm with additional regularization is presented. The Difference Curvature (DC) regularization which containing the image edge ...
The research and implementation of super-resolution reconstruction for multi-frame blurring images
ICCC '11: Proceedings of the 2011 International Conference on Innovative Computing and Cloud ComputingIn this paper, we design a super-resolution restoration system for multi-frame blurring images. There are two modules in this system, namely that motion estimation and reconstruction of super-resolution. Firstly, the motion parameters are estimated by a ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 November 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- Shanghai Committee of Science and Technology
Conference
DAC '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,770 of 5,499 submissions, 32%
Upcoming Conference
DAC '25
- Sponsor:
- sigda
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 152Total Downloads
- Downloads (Last 12 months)152
- Downloads (Last 6 weeks)53
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in