Skip to main content
Springer Nature Link
Log in

Hierarchical Design of a Secure Image Sensor with Dynamic Reconfiguration

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

This paper presents a secure reconfigurable hierarchical hardware architecture at the pixel and region level for smart image sensors to accelerate machine vision applications. The design maintains hierarchical processing that begins at the pixel level. It aims to reduce the computational burden on the sequential processor and increases the confidentiality of the sensor. We achieve this goal by preprocessing the data in parallel with event-based processing within the sensor and extract the local features, which are then forwarded to an encryption module. After that, an external processor can obtain the encrypted features to complete the vision application. This approach significantly accelerates the vision application by executing the low-level and mid-level image processing applications and simultaneously by reducing the data volume at the sensor level. The secure hardware architecture enables the vision application to perform in real-time with reliability. This hierarchical processing breaks the traditional sequential image processing and introduces parallelism for machine vision applications. We evaluate the design in FPGA and achieve the GDSII file in the ASIC platform at 800MHz. Simulation results show that the area overhead and power penalty for adding reconfiguration features stay in an acceptable range. Besides, removing redundant information, 84.01%, and 94.31% dynamic power can be saved at each pixel-level and region-level, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10

Similar content being viewed by others

References

  1. Koelling, T., & Wang, J. (2008). Advances in cmos image sensors. Security Dealer and Integrator, 30(10), 70–70, 72, 74, 10.

    Google Scholar 

  2. Tyrrell, B., Anderson, K., Baker, J., Berger, R., Brown, M., Colonero, C., Costa, J., Holford, B., Kelly, M., Ringdahl, E., Schultz, K., & Wey, J. (2009). Time delay integration and in-pixel spatiotemporal filtering using a nanoscale digital cmos focal plane readout. IEEE Transactions on Electron Devices, 56(11), 2516–2523.

    Article  Google Scholar 

  3. Gouveia, L.C.P., & Choubey, B. (2016). Advances on cmos image sensors. Sensor Review, 36(3), 231–239.

    Article  Google Scholar 

  4. Zhu, H., & Shibata, T. (2014). A real-time motion-feature-extraction vlsi employing digital-pixel-sensor-based parallel architecture. IEEE Transactions on Circuits and Systems for Video Technology, 24(10), 1787–1799.

    Article  Google Scholar 

  5. Bhowmik, P., Pantho, M.J.H., Asadinia, M., & Bobda, C. (2018). Design of a reconfigurable 3d pixel-parallel neuromorphic architecture for smart image sensor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 673–681).

  6. Li, H., & Zhong, C. (2016). A machine vision based autonomous navigation system for lunar rover: the model and key technique. Sensor Review, 36(4), 377–385.

    Article  Google Scholar 

  7. Nassi, J.J., & Callaway, E.M. (2009). Parallel processing strategies of the primate visual system. Nature reviews neuroscience, 10(5), 360.

    Article  Google Scholar 

  8. DiCarlo, J.J., Zoccolan, D., & Rust, N.C. (2012). How does the brain solve visual object recognition? Neuron, 73(3), 415–434.

    Article  Google Scholar 

  9. Larkum, M. (2013). A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex. Trends in neurosciences, 36(3), 141–151.

    Article  Google Scholar 

  10. Rao, R.P., & Ballard, D.H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1), 79.

    Article  Google Scholar 

  11. Huang, Y., & Rao, R.P. (2011). Predictive coding. Wiley Interdisciplinary Reviews:, Cognitive Science, 2(5), 580– 593.

    Google Scholar 

  12. John, A. (1997). Robinson. Efficient general-purpose image compression with binary tree predictive coding. IEEE Transactions on Image Processing, 6(4), 601–608.

    Article  Google Scholar 

  13. Pantho, M.J.H., Bhowmik, P., & Bobda, C. (2018). Pixel-parallel architecture for neuromorphic smart image sensor with visual attention. In 2018 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 245–250. IEEE.

  14. Delbrück, T., Linares-Barranco, B., Culurciello, E., & Posch, C. (2010). Activity-driven, Event-based vision sensors. In In Proceedings of IEEE International Symposium on Circuits and systems, pages 2426–2429, IEEE, 2010.

  15. Vahid, F., & Givargis, T.D. (2002). Embedded System design: A Unified Hardware/Software Introduction chapter 1. Wiley, John & Sons, Incorporated.

  16. Pham, D.M., & Aziz, S.M. (2013). Object extraction scheme and protocol for energy efficient image communication over wireless sensor networks. Computer Networks, 57(15), 2949–2960.

    Article  Google Scholar 

  17. Walter, D.L.-S., Balkir, S., Sayood, K., Schemm, N., & Hoffman, M.W. (2007). A cmos imager with focal plane compression using predictive coding. IEEE Journal of Solid-State Circuits, 42(11), 2555–2572.

    Article  Google Scholar 

  18. Miao, W., Lin, Q., & Nanjian, W. (2007). A novel vision chip for high-speed target tracking. Japanese journal of applied physics, 46(4S), 2220.

    Article  Google Scholar 

  19. Moini, A., Bouzerdoum, A., Eshraghian, K., Yakovleff, A., Nguyen, X.T., Blanksby, A., Beare, R., Abbott, D., & Bogner, R.E. (1997). An insect vision-based motion detection chip. IEEE Journal of Solid-State Circuits, 32(2), 279–284.

    Article  Google Scholar 

  20. Oike, Y., Ikeda, M., & Asada, K. (2005). A 375/spl times/365 high-speed 3-d range-finding image sensor using row-parallel search architecture and multisampling technique. IEEE journal of solid-state circuits, 40(2), 444–453.

    Article  Google Scholar 

  21. Zhang, W., Qiuyu, F., & Nan-Jian, W. (2011). A programmable vision chip based on multiple levels of parallel processors. IEEE Journal of Solid-State Circuits, 46(9), 2132–2147.

    Article  Google Scholar 

  22. Culurciello, E., Etienne-Cummings, R., & Boahen, K.A. (2003). A biomorphic digital image sensor. IEEE Journal of Solid-State Circuits, 38(2), 281–294.

    Article  Google Scholar 

  23. Shoushun, C., & Bermak, A. (2007). Arbitrated time-to-first spike cmos image sensor with on-chip histogram equalization. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 15(3), 346–357.

    Article  Google Scholar 

  24. Serrano-Gotarredona, T., & Linares-Barranco, B. (2013). A 128x128 1.5% contrast sensitivity 0.9% fpn 3 us latency 4 mw asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE Journal of Solid-State Circuits, 48(3), 827–838.

    Article  Google Scholar 

  25. Tobi Delbruck. (2008). Frame-free dynamic digital vision. In Proceedings of Intl, Symp. on Secure-Life electronics Advanced Electronics for Quality Life and Society, pp 21–26.

  26. Bhowmik, P., Pantho, M.J.H., & Bobda, C. (2019). Visual cortex inspired pixel-level re-configurable processors for smart image sensors. In Proceedings of the 56th Annual Design Automation Conference 2019, page 235. ACM.

  27. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jiménez, A., Serrano-Gotarredona, C., Pérez-Carrasco, J.A., Linares-Barranco, B., Linares-Barranco, A., Jiménez-moreno, G., & Civit-Ballcels, A. (2008). On real-time aer 2-d convolutions hardware for neuromorphic spike-based cortical processing. IEEE Transactions on Neural Networks, 19(7), 1196–1219.

    Article  Google Scholar 

  28. Camunas-Mesa, L., Acosta-Jimenez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., & Linares-Barranco, B. (2010). A 32x32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 meps throughput. IEEE Transactions on Circuits and Systems I:, Regular Papers, 58(4), 777–790.

    Article  Google Scholar 

  29. Bahrami, S., & Naderi, M. (2012). Image encryption using a lightweight stream encryption algorithm. Advances in Multimedia, 2012, 4.

    Article  Google Scholar 

  30. Janakiraman, S., Thenmozhi, K., Rayappan, J.B.B., & Amirtharajan, R. (2018). Lightweight chaotic image encryption algorithm for real-time embedded system Implementation and analysis on 32-bit microcontroller. Microprocessors and Microsystems, 56, 1–12.

    Article  Google Scholar 

  31. Gonċalves, D., & Costa, D. (2015). A survey of image security in wireless sensor networks. Journal of Imaging, 1(1), 4–30.

    Article  Google Scholar 

  32. Gamal, A.El., Yang, D.X.D., & Fowler, B.A. (1999). Pixel-level processing: why, what, and how?. In Sensors, Cameras, and Applications for Digital Photography, volume 3650, pages 2–14. International Society for Optics and Photonics.

  33. Massari, N., & et al. (2005). A cmos image sensor with programmable pixel-level analog processing. IEEE Transactions on Neural Networks, 16(6), 1673–1684.

    Article  Google Scholar 

  34. Yin, C., & Chih-Cheng, H.A. (2013). 0.5 V 34.4 uw 14.28 kfps 105db smart image sensor with array-level analog signal processing. In Solid-State Circuits Conference (a-SSCC) IEEE asian, pages 97–100, IEEE (p. 2013).

  35. Sakakibara, M., Ogawa, K., Sakai, S., Tochigi, Y., Honda, K., Kikuchi, H., Wada, T., Kamikubo, Y., Miura, T., Nakamizo, M., & et al. (2018). A 6.9-μ m pixel-pitch back-illuminated global shutter cmos image sensor with pixel-parallel 14-bit subthreshold adc. IEEE Journal of Solid-State Circuits, 99, 1–9.

    Google Scholar 

  36. Kondo, T, & et al. (2015). A 3d stacked cmos image sensor with 16mpixel global-shutter mode and 2mpixel 10000fps mode using 4 million interconnections. In VLSI Circuits (VLSI Circuits), 2015 Symposium on, pages C90–C91. IEEE.

  37. Simon, J. (2007). Shepherd. The tiny encryption algorithm. Cryptologia, 31(3), 233–245.

    Article  Google Scholar 

  38. Wheeler, D.J., & Needham, R.M. (1994). Tea, a tiny encryption algorithm. In International Workshop on Fast Software Encryption, pages 363–366. Springer.

Download references

Acknowledgments

This work was supported by the National Science Foundation (NSF) under Grant-1618606.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA

    Pankaj Bhowmik, Md Jubaer Hossain Pantho & Christophe Bobda

Authors
  1. Pankaj Bhowmik
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Md Jubaer Hossain Pantho
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Christophe Bobda
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Pankaj Bhowmik.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhowmik, P., Pantho, M.J.H. & Bobda, C. Hierarchical Design of a Secure Image Sensor with Dynamic Reconfiguration. J Sign Process Syst 92, 999–1015 (2020). https://doi.org/10.1007/s11265-020-01564-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-020-01564-9

Keywords