skip to main content
10.1145/3665314.3670827acmconferencesArticle/Chapter ViewAbstractPublication PagesislpedConference Proceedingsconference-collections
research-article
Open access

Energy Efficiency Through In-Sensor Computing: ADC-less Real-Time Sensing for Image Edge Detection

Published: 09 September 2024 Publication History

Abstract

In-sensor computing has revolutionized modern vision-based applications, particularly in scenarios like autonomous vehicles and robotics where real-time or near-real-time processing is crucial. By enabling data processing at the sensor level, in-sensor computing eliminates the need to transmit data to cloud servers, significantly reducing latency and enhancing decision-making speed. Central to the in-sensor computing paradigm, CMOS image sensors (CISs) with edge computing, play a pivotal role in machine vision applications. The need for high resolution, low power, and real-time operation aligns seamlessly with the demands of modern vision-based applications. In this paper, we propose a novel approach for real-time image edge detection with an in-sensor, ADC-less sensing solution that achieves high energy efficiency and speed. The design utilizes the column-parallel architecture of existing CIS and the row-wise pixel readout scheme. Column voltages of three consecutive rows with a delay arrangement extract 4-bit edge pixels without deriving the actual digital image pixels. A time-to-digital conversion (TDC) technique using a 4-bit counter eliminates the requirement of power-hungry ADC. A 256(H) x 256(V) 2D CMOS pixel array with 10 μm pixel pitch is simulated using Spectre in TSMC 65nm low-power technology. CMOS pixels with wide dynamic range (WDR) capture the light intensity variation up to 92dB [10]. Simulation results show energy consumption of 2pW per pixel per frame, operating at a frame rate of 3.9kfps, all well-contained within a modest 0.5 mW power budget. The resultant frame rate emerges as notably superior in terms of speed, accompanied by a more than tenfold reduction in power consumption per edge frame-pixel compared to the existing prior art.

References

[1]
Karim Ali Ahmed, Hayate Okuhara, and Massimo Alioto. 2023. 55pW/pixel Peak Power Imager with Near-Sensor Novelty/Edge Detection and DC-DC Converter-Less MPPT for Purely Harvested Sensor Nodes. In 2023 IEEE International Solid-State Circuits Conference (ISSCC). 102--104.
[2]
John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8, 6 (1986), 679--698.
[3]
Jaehyuk Choi, Seokjun Park, Jihyun Cho, and Euisik Yoon. 2014. A 3.4-μ W Object-Adaptive CMOS Image Sensor With Embedded Feature Extraction Algorithm for Motion-Triggered Object-of-Interest Imaging. IEEE Journal of Solid-State Circuits 49, 1 (2014), 289--300.
[4]
Jichao Cui and Kun Tian. 2020. Edge Detection Algorithm Optimization and Simulation Based on Machine Learning Method and Image Depth Information. IEEE Sensors Journal 20, 20 (2020), 11770--11777.
[5]
Tzu-Hsiang Hsu, Yen-Kai Chen, Min-Yang Chiu, Guan-Cheng Chen, Ren-Shuo Liu, Chung-Chuan Lo, Kea-Tiong Tang, Meng-Fan Chang, and Chih-Cheng Hsieh. 2021. A 0.8 V Multimode Vision Sensor for Motion and Saliency Detection With Ping-Pong PWM Pixel. IEEE Journal of Solid-State Circuits 56, 8 (2021), 2516--2524.
[6]
Tzu-Hsiang Hsu, Yi-Ren Chen, Ren-Shuo Liu, Chung-Chuan Lo, Kea-Tiong Tang, Meng-Fan Chang, and Chih-Cheng Hsieh. 2021. A 0.5-V Real-Time Computational CMOS Image Sensor With Programmable Kernel for Feature Extraction. IEEE Journal of Solid-State Circuits 56, 5 (2021), 1588--1596.
[7]
Minhyun Jin, Hyeonseob Noh, Minkyu Song, and Soo Youn Kim. 2020. Design of an edge-detection cmos image sensor with built-in mask circuits. Sensors 20, 13 (2020), 3649.
[8]
Tanay Karnik, Dileep Kurian, Paolo Aseron, Richard Dorrance, Erkan Alpman, Angela Nicoara, Roman Popov, Leonid Azarenkov, Mikhail Moiseev, Li Zhao, et al. 2018. A cm-scale self-powered intelligent and secure IoT edge mote featuring an ultra-low-power SoC in 14nm tri-gate CMOS. In 2018 IEEE International Solid-State Circuits Conference-(ISSCC). IEEE, 46--48.
[9]
Hyeon-June Kim, Sun-Il Hwang, Jae-Hyun Chung, Jong-Ho Park, and Seung-Tak Ryu. 2017. A Dual-Imaging Speed-Enhanced CMOS Image Sensor for Real-Time Edge Image Extraction. IEEE Journal of Solid-State Circuits 52, 9 (2017), 2488--2497.
[10]
Sangwoo Kim, Taehyoung Kim, Kiwon Seo, and Gunhee Han. 2022. A Fully Digital Time-Mode CMOS Image Sensor with 22.9pJ/frame.pixel and 92dB Dynamic Range. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), Vol. 65. 1--3.
[11]
Soyeon Lee, Bohyeok Jeong, Keunyeol Park, Minkyu Song, and Soo Youn Kim. 2021. On-cmos image sensor processing for lane detection. Sensors 21, 11 (2021), 3713.
[12]
Martin Lefebvre, Ludovic Moreau, Rémi Dekimpe, and David Bol. 2021. 7.7 A 0.2-to-3.6TOPS/W Programmable Convolutional Imager SoC with In-Sensor Current-Domain Ternary-Weighted MAC Operations for Feature Extraction and Region-of-Interest Detection. In 2021 IEEE International Solid- State Circuits Conference (ISSCC), Vol. 64. 118--120.
[13]
P. Lichtsteiner, C. Posch, and T. Delbruck. 2006. A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change. In 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers. 2060--2069.
[14]
Liqiao Liu, Xu Ren, Kai Zhao, Yandong He, and Gang Du. 2023. FD-SOI-Based Pixel With Real-Time Frame Difference for Motion Extraction and Image Preprocessing. IEEE Transactions on Electron Devices 70, 2 (2023), 594--599.
[15]
Min-Jun Park and Hyeon-June Kim. 2023. A Real-Time Edge-Detection CMOS Image Sensor for Machine Vision Applications. IEEE Sensors Journal 23, 9 (2023), 9254--9261.
[16]
Min-Jun Park and Hyeon-June Kim. 2023. A Real-Time Edge-Detection CMOS Image Sensor for Machine Vision Applications. IEEE Sensors Journal 23, 9 (2023), 9254--9261.
[17]
Vincent Torre and Tomaso A. Poggio. 1986. On Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8, 2 (1986), 147--163.
[18]
Rick Van Krevelen and Ronald Poelman. 2010. A Survey of Augmented Reality Technologies, Applications and Limitations. International Journal of Virtual Reality (ISSN 1081-1451) 9 (06 2010), 1.
[19]
Yuanlong Wang, Zhiqiang Jing, Zijie Ji, Liangguo Wang, Guan Zhou, Qiang Gao, Wanzhong Zhao, and Shijuan Dai. 2022. Lane Detection Based on Two-Stage Noise Features Filtering and Clustering. IEEE Sensors Journal 22, 15 (2022), 15526--15536.
[20]
Ruoyu Xu, Wai Chiu Ng, Jie Yuan, Shouyi Yin, and Shaojun Wei. 2014. A 1/2.5 inch VGA 400 fps CMOS Image Sensor With High Sensitivity for Machine Vision. IEEE Journal of Solid-State Circuits 49, 10 (2014), 2342--2351.
[21]
Chin Yin, Chin-Fong Chiu, and Chih-Cheng Hsieh. 2016. A 0.5 V, 14.28-kframes/s, 96.7-dB Smart Image Sensor With Array-Level Image Signal Processing for IoT Applications. IEEE Transactions on Electron Devices 63, 3 (2016), 1134--1140.
[22]
Xiaopeng Zhong, Man-Kay Law, Chi-Ying Tsui, and Amine Bermak. 2020. A Fully Dynamic Multi-Mode CMOS Vision Sensor With Mixed-Signal Cooperative Motion Sensing and Object Segmentation for Adaptive Edge Computing. IEEE Journal of Solid-State Circuits 55, 6 (2020), 1684--1697.
[23]
Yu Zou, Massimo Gottardi, Michela Lecca, and Matteo Perenzoni. 2020. A Low-Power VGA Vision Sensor With Embedded Event Detection for Outdoor Edge Applications. IEEE Journal of Solid-State Circuits 55, 11 (2020), 3112--3121.
[24]
Y. Zou, M. Gottardi, D. Perenzoni, M. Perenzoni, and D. Stoppa. 2017. A 1.6 mW 320 x 240-pixel vision sensor with programmable dynamic background rejection and motion detection. In 2017 IEEE SENSORS. 1--3.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISLPED '24: Proceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design
August 2024
384 pages
ISBN:9798400706882
DOI:10.1145/3665314
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2024

Check for updates

Author Tags

  1. CMOS image sensor (CIS)
  2. column-parallel (CP) imaging
  3. wide dynamic range (WDR)
  4. image signal processor (ISP)
  5. time-to-digital conversion (TDC)
  6. ADC-less CIS
  7. on-chip image edge detection

Qualifiers

  • Research-article

Conference

ISLPED '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 398 of 1,159 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 291
    Total Downloads
  • Downloads (Last 12 months)291
  • Downloads (Last 6 weeks)65
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media