Skip to main content
SpringerLink
Log in

NEROvideo: a general-purpose CNN-UM video processing system

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Emulations of cellular nonlinear networks (CNN) on digital reconfigurable hardware have proved to be adequate for highly-efficient computation of massive data, exceeding the accuracy and flexibility of full-custom designs. Based on a recently-proposed architecture for the emulation of a large-scale CNN universal machine, a new real-time video processing system has been developed. Due to its free programmability and massively-parallel architecture the system is very suitable for high-speed computation of complex algorithms that follow the idea of spatio-temporal computing. Implemented on a state-of-the-art Xilinx Zynq system-on-chip, the proposed setup is capable of processing a \(640\times 480\)p video stream with up to 1,700 fps, depending on the respective algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Abbo, A.A., Kleihorst, R.P., Choudhary, V., Sevat, L., Wielage, P., Mouy, S., Vermeulen, B., Heijligers, M.: Xetal-II: a 107 GOPS, 600 mW massively parallel processor for video scene analysis. Solid-State Circ. IEEE J. 43(1), 192–201 (2008)

    Article  Google Scholar 

  2. Analog Devices Inc. 225 MHz, High Performance HDMI Transmitter with ARC: ADV7511 (2010). http://www.analog.de

  3. Arce, G.R.: Nonlinear signal processing: a statistical approach. Wiley, New Jersey (2005)

    MATH  Google Scholar 

  4. ARM Ltd. AMBA AXI Protocol Specification. http://www.arm.com/ (2003)

  5. Avnet Inc. DVI I/O FMC Module Hardware Guide. http://www.avnet.com (2010)

  6. Blug, A., Strohm, P., Carl, D., Hofler, H., Blug, B., Kailer, A.: On the potential of current CNN cameras for industrial surface inspection. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, IEEE, pp 1–6 (2012)

  7. Braunschweig, R., Müller, J., Müller, J., Tetzlaff, R.: NERO mastering 300k CNN cells. In: Circuit Theory and Design (ECCTD), 2013 European Conference on, pp. 1–4 (2013). doi:10.1109/ECCTD.2013.6662202

  8. Butter, M., Leis, M., Sandtke, M., McLean, M., Lincoln, J., Wilson, A.: The Leverage Effect of Photonics Technologies: The European Perspective. Final Report, March 2011. Study prepared for the European Commission, DG Information Society and Media under reference SMART 2009/0066 (2011)

  9. Cheng, C.C., Lin, C.H., Li, C.T., Chen, L.G.: ivisual: An intelligent visual sensor soc with 2790 fps cmos image sensor and 205 gops/w vision processor. Solid-State Circ., IEEE J. 44(1), 127–135 (2009). doi:10.1109/JSSC.2008.2007158

    Article  Google Scholar 

  10. Chua, L.O., Yang, L.: Cellular neural networks: theory. IEEE Trans. Circ. Syst. 35(10), 1257–1272 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chua, L.O., Roska, T., Venetianer, P.L.: The CNN is universal as the Turing machine. IEEE Trans. Circ. Syst. I-Regular Papers 40, 289–291 (1993). doi:10.1109/81.224308

    Article  MathSciNet  MATH  Google Scholar 

  12. Chua, L.O., Hasler, M., Moschytz, G.S., Neirynck, J.: Autonomous cellular neural networks: A unified paradigm for pattern formation and active wave propagation. Circ. Syst. I: Fundam. Theory Appl., IEEE Trans. 42(10), 559–577 (1995)

    MathSciNet  Google Scholar 

  13. Dudek, P.: An asynchronous cellular logic network for trigger-wave image processing on fine-grain massively parallel arrays. IEEE Trans. Circ. Syst. II: Exp. Briefs 53(5), 354–358 (2006)

    Article  MathSciNet  Google Scholar 

  14. Dudek, P., Hicks, P.J.: A general-purpose processor-per-pixel analog SIMD vision chip. Circ. Syst. I: Regular Papers, IEEE Trans. 52(1), 13–20 (2005)

    Google Scholar 

  15. Földesy, P., Zarandy, A., Rekeczky, C.: Configurable 3D-integrated focal-plane cellular sensor-processor array architecture. Int. J. Circ. Theory Appl. 36(5–6), 573–588 (2008)

    Article  Google Scholar 

  16. Hoefflinger, B.: Chips 2020. Springer (2012)

  17. Kahle, J.A., Day, M.N., Hofstee, H.P., Johns, C.R., Maeurer, T.R., Shippy, D.J.: Introduction to the Cell multiprocessor. Ibm J. Res. Dev. 49, 589–604 (2005). doi:10.1147/rd.494.0589

    Article  Google Scholar 

  18. Karacs, K, et al.: Software Library for Cellular Wave Computing Engines V. 3.1. Budapest, Hungary: MTA-SZTAKI and Pazmany University (2010)

  19. Kayaer, K., Tavsanoglu, V.: A new approach to emulate CNN on FPGAs for real time video processing. In: Cellular Neural Networks and Their Applications, 2008. CNNA 2008. 11th International Workshop on, IEEE, pp 23–28 (2008)

  20. Kirk, D.: NVIDIA CUDA software and GPU parallel computing architecture. ISMM 7, 103–104 (2007)

    Article  Google Scholar 

  21. Linan, G., Espejo, S., Domínguez-Castro, R., Rodríguez-Vázquez, A.: ACE4k: An analog I/O 64\(\times\) 64 visual microprocessor chip with 7-bit analog accuracy. Int. J. Circ. Theory Appl. 30(2–3), 89–116 (2002)

    Article  MATH  Google Scholar 

  22. Lopich, A., Dudek, P. A general-purpose vision processor with 160\(\times\) 80 pixel-parallel SIMD processor array. In: Custom Integrated Circuits Conference (CICC), 2013 IEEE, IEEE, pp 1–4 (2013)

  23. Martínez, J.J., Garrigós, J., Toledo, J., Fernández, E., Manuel Ferrández, J.: Implementation of a CNN-based retinomorphic model on a high performance reconfigurable computer. Neurocomputing 74(8), 1290–1297 (2011)

    Article  Google Scholar 

  24. Müller, J., Müller, J., Tetzlaff, R.: A new cellular nonlinear network emulation on FPGA for EEG signal processing in epilepsy. In: Proceedings of SPIE, vol 8068, p 80680M (2011)

  25. Müller, J., Becker, R., Müller, J., Tetzlaff, R. CESAR: Emulating Cellular Networks on FPGA. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, pp 1–5, (2012). doi:10.1109/CNNA.2012.6331409

  26. Nagy, Z., Szolgay, P.: Configurable multilayer CNN-UM emulator on FPGA. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 50(6), 774–778 (2003). doi:10.1109/TCSI.2003.812611

    Article  Google Scholar 

  27. Nicolosi, L., Abt, F., Blug, A., Heider, A., Tetzlaff, R., Höfler, H.: A novel spatter detection algorithm based on typical cellular neural network operations for laser beam welding processes. Measur. Sci. Technol. 23(015), 401 (2012)

    Google Scholar 

  28. Nieto, A., Brea, V., Vilarino, D.: SIMD array on FPGA for B/W image processing. In: Cellular Neural Networks and Their Applications. CNNA 2008. 11th International Workshop on, IEEE, pp 202–207 (2008)

  29. Rodriguez-Vazquez, A., Linan-Cembrano, G., Carranza, L., Roca-Moreno, E., Carmona-Galan, R., Jimenez-Garrido, F., Dominguez-Castro, R., Meana, S.E.: ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs. IEEE Trans. Circ. Syst. I: Regular Papers 51(5), 851–863 (2004). doi:10.1109/TCSI.2004.827621

    Article  Google Scholar 

  30. Rodriguez-Vazquez, A., Dominguez-Castro, R., Jimenez-Garrido, F., Morillas, S., Listan, J., Alba, L., Utrera, C., Espejo, S., Romay, R.: The Eye-RIS CMOS vision system. In: Casier, H., Steyaert, M., van Roermund, A.H.M. (eds.) Analog Circuit Design. Sensors, Actuators and Power Drivers; Integrated Power Amplifiers from Wireline to RF; Very High Frequency Front Ends, Springer, Internetausgabe, pp 15–32 (2008)

  31. Roska, T., Chua, L.O.: The CNN universal machine: an analogic array computer. IEEE Trans. Circ. Syst. II: Analog Digital Signal Process. 40(3), 163–173 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  32. Steiner, G., Sobottka, S.B., Koch, E., Schackert, G., Kirsch, M.: Intraoperative imaging of cortical cerebral perfusion by time-resolved thermography and multivariate data analysis. J. Biomed. Optics 16(1):016, 001–016,001 (2011)

  33. Venetianer, P.L., Werblin, F., Roska, T., Chua, L.O.: Analogic cnn algorithms for some image compression and restoration tasks. IEEE Trans. Circ. Syst. 1, Fundam. Theory Appl. 42(5), 278–284 (1995)

    Article  Google Scholar 

  34. Vörösházi, Z., Kiss, A., Nagy, Z., Szolgay, P.: Implementation of embedded emulated-digital CNN-UM global analogic programming unit on FPGA and its application. Int. J. Circ. Theory Appl. 36, 589–603 (2008). doi:10.1002/cta.507

    Article  Google Scholar 

  35. Xilinx Inc. LogiCORE IP AXI Video Direct Memory Access v3.01a. http://www.xilinx.com (2011)

  36. Xilinx Inc. LogiCORE IP AXI DMA v6.03a. http://www.xilinx.com (2012)

  37. Xilinx Inc. Zynq-7000 All Programmable SoC Overview. http://www.xilinx.com (2013)

  38. Yildiz, N., Cesur, E., Tavsanoglu, V.: Demonstration of the Second Generation Real-time Cellular Neural Network Processor: RTCNNP-v2. In: Cellular Nanoscale Networks and Their Applications (CNNA), 2012 13th International Workshop on, pp 1–2 (2012). doi:10.1109/CNNA.2012.6331471

  39. Zarándy, Á.: Focal-plane sensor-processor chips. Springer, Berlin (2011)

  40. Zarándy, Á., Rekeczky, C.: 2D operators on topographic and non-topographic architectures—implementation, efficiency analysis, and architecture selection methodology. Int. J. Circ. Theory Appl. 39(10), 983–1005 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical and Computer Engineering, Technische Universität Dresden, 01062, Dresden, Germany

    Jens Müller, Jan Müller & Ronald Tetzlaff

Authors
  1. Jens Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ronald Tetzlaff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens Müller.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Müller, J., Müller, J. & Tetzlaff, R. NEROvideo: a general-purpose CNN-UM video processing system. J Real-Time Image Proc 12, 763–774 (2016). https://doi.org/10.1007/s11554-014-0451-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0451-9

Keywords

Search

Navigation