Skip to main content
SpringerLink
Log in

Hardware accelerator IP cores for real time Radar and camera-based ADAS

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The paper presents the design of HDL macrocells, working as hardware accelerators to achieve real time processing of some key functions of Advanced Driver Assistance Systems (ADAS). Particularly, the macrocells process in real time images from Radar and video-camera sensors implementing computing-intensive operations such as: pixel-level filtering, correction of distortions in large field-of-view cameras, multi-camera image fusion for panoramic view of car’s surround, traffic and road signs recognition, detection of obstacles for collision avoidance, and estimation of their distance and speed. The HDL macrocells are fully configurable and are synthesized in low-cost Artix-7 FPGA family, satisfying automotive-grade requirements. They can be integrated as accelerators in embedded hardware–software platforms to build a complete ADAS solution. To this aim, the macrocells have been synthesized also in a Zynq FPSoC. With respect to the state of the art, the proposed macrocells stand for their low-power and real time performance .

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Zhang, B., Appia, V., Pekkucuksen, I., Liu, Y., Umit Batur, A., Shastry, P., Liu, S., Sivasankaran, S., Chitnis, K.: A surround view camera solution for embedded systems. In: IEEE CVPR, pp. 676–681 (2014)

  2. Sankaran, J., Zoran, N.: TDA2x, a SoC optimized for advanced driver assistance systems. In: IEEE ICASSP, pp. 2204–2208 (2014)

  3. Mody, M., Swami, P., Chitnis, K., Jagannathan, S., Desappan, K., Jain, A., Poddar, D., Nikolic, Z., Viswanath, P., Mathew, M., Nagori, S., Garud, H.: High performance front camera ADAS applications on TI’s TDA3X platform. In: International Conference on High Performance Computing, pp. 456–463 (2015)

  4. Souki, M., Boussaid, L., Abid, M.: An embedded system for real-time traffic sign recognizing. In: IEEE International Design and Test Workshop, pp. 273–276 (2008)

  5. Rohling, H.: Some radar topics waveform design, range CFAR and target recognition. Adv. Sens. Secur. Appl. 2, 293–322 (2006)

    Google Scholar 

  6. Stein, G., Rushinek, E., Hayun, G., Shashua, A.: A computer vision system on a chip: a case study from the automotive domain. In: IEEE CVPR, pp. 130–131 (2005)

  7. Souani, C., Faiedh, H., Besbes, K.: Efficient algorithm for automatic road sign recognition and its hardware implementation. J. Real-Time Image Process. 9(1), 79–93 (2014)

    Article  Google Scholar 

  8. M. Muller, Braun, A., Gerlach, J., Rosenstiel, W., Nienhüser, D., Zöllner, J., Bringmann, O.: Design of an automotive traffic sign recognition system targeting a multi-core SoC implementation. In: IEEE DATE, pp. 532–537 (2010)

  9. Velez, G., Cortés, A., Nieto, M., Velez, I., Otaegui, O.: A reconfigurable embedded vision system for advanced driver assistance. J. Real-Time Image Process. 10(4), 725–739 (2015)

    Article  Google Scholar 

  10. Turturici, M., Saponara, S., Franchi, E., Fanucci, L.: Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications. J. Real-Time Image Process. 9(3), 463–478 (2014)

    Article  Google Scholar 

  11. Neri, B., Saponara, S.: Advances in technologies, architectures and applications of highly-integrated low-power radars. IEEE Aerosp. Electron. Syst. Mag. 27(1), 25–36 (2012)

    Article  Google Scholar 

  12. LRR3: 3rd generation Long-range radar sensor. Bosch document 292000P03W-C/SMC2-200906-En, pp. 1–2 (2009)

  13. Jaakkola, A.: Low cost mobile laser scanning and its feasibility for environmental mapping. Aalto Univ. Publ. Ser. 65, 1–65 (2015)

    Google Scholar 

  14. Eldada, L.: Solid state LiDAR for ubiquitous 3D sensing. In: GPU Technology Conference (2016)

  15. Ali Altuncu, M., Guven, T., Becerikli, Y., Sahin, S.: Real-time system implementation for image processing with hardware/software co-design on the Xilinx Zynq platform. Int. J. Inform. Electron. Eng. 5(6), 473–477 (2015)

    Article  Google Scholar 

  16. Marsi, S., Saponara, S.: Integrated video motion estimator with Retinex-like pre-processing for robust motion analysis in automotive scenarios: algorithmic and real-time architecture design. J. Real-Time Image Process. 5(4), 275–289 (2010)

    Article  Google Scholar 

  17. Fanucci, L., Saponara, S., Bertini, L.: A parametric VLSI architecture for video motion. Integr. VLSI J. 31(1), 79–100 (2001)

    Article  Google Scholar 

  18. Saponara, S., Martina, M., Casula, M., Masera, G., Fanucci, L.: Motion estimation and CABAC VLSI coprocessors for real-time high-quality H.264/AVC video coding. Microprocess. Microsyst. 34, 316–328 (2010)

    Article  Google Scholar 

  19. Fanucci, L., Saletti, R., Saponara, S.: Parametrized and reusable VLSI macro cells for the low-power realization of 2-D discrete-cosine-transform. Microelectron. J. 32(12), 1035–1045 (2001)

    Article  Google Scholar 

  20. Anders, J., Mefenza, M., Bobda, C., Yonfa, F., Aklah, Z., Gunn, K.: A hardware/software prototyping system for driving assistance investigations. J. Real-Time Image Process. 11(3), 559–569 (2016)

    Article  Google Scholar 

  21. Wang, S., Zhang, P., Dai, Z., Wang, Y., Tao, R., Sun, S.: Research and practice of traffic lights and traffic signs recognition system based on multicore of FPGA. Commun. Netw. 5, 61–64 (2013)

    Article  Google Scholar 

  22. Costantino, N., Serventi, R., Tinfena, F., D’Abramo, P., Chassard, P., Tisserand, P., Saponara, S., Fanucci, L.: Design and test of an HV-CMOS intelligent power switch with integrated protections and self-diagnostic for harsh automotive applications. IEEE Trans. Ind. Electron. 58(7), 2715–2727 (2011)

    Article  Google Scholar 

  23. Saponara, S., Petri, E., Fanucci, L., Terreni, P.: Sensor modeling, low-complexity fusion algorithms, and mixed-signal IC prototyping for gas measures in low-emission vehicles. IEEE Trans. Instrum. Meas. 60(2), 372–384 (2011)

    Article  Google Scholar 

  24. Han, Y., Virupakshappa, K., Pinto, E., Oruklu, E.: Hardware/software co-design of a traffic sign recognition system using Zynq FPGAs. Electronics 4, 1062–1089 (2015)

    Article  Google Scholar 

  25. Kim, J., Rajkumar, R., Kato, S.: Towards adaptive GPU resource management for embedded real-time systems. ACM SIGBED Rev. 10(1), 14–17 (2013)

    Article  Google Scholar 

  26. Mallika, M. R., Srinath, M.B., Venkatesh, N., Neha, K.B., Murthy, V.S.: FMCW Radar signal generation using FPGA. In: Proceedings of 15th IRF International Conference, pp. 84–87 (2015)

  27. Lischi, S., Massini, R., Staglianò, D., Mussetti, L., Berizzi, F., Neri, B., Martorella, M.: X-band compact low cost multi-channel radar prototype for short range high resolution 3D-InISAR. In: IEEE Eurad, pp. 157–160 (2014)

  28. Lam, L., Lee, S.W., Suen, C.Y.: Thinning methodologies: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 14(9), 869–888 (1992)

    Article  Google Scholar 

  29. Hsiao, P.Y., Hua, C.H., Lin, C.C.: A novel FPGA architectural implementation of pipeline thinning algorithm. IEEE ISCAS 2, 593–596 (2004)

    Google Scholar 

  30. Hermanato, L.: Hardware implementation of fingerprint image thinning algorithm in FPGA device. In: IEEE ICNIT, pp. 187–191 (2010)

  31. Estable, S., Schick, J., Stein, F., Janssen, R., Ott, R., Ritter, W., Zheng, Y.: A real-time traffic sign recognition systems. In: IEEE International Vehicles Symposium, pp. 24–26 (1994)

  32. Campani, M., Cappello, M., Piccioli, G., Reggi, E., Straforini, M., Torre, V.: Visual routines for outdoor navigation. In: IEEE International Vehicles Symposium, pp. 93–107 (1993)

  33. Gavrila, D.: Traffic sign recognition revisited. In: Proceedings of 21st DAGM Symposium, pp. 86–93 (1999)

    Google Scholar 

  34. Zheng, Y., Ritter, W., Janssen, R.: An adaptive system for traffic sign recognition. In: IEEE Intelligent Vehicles Symposium (1994)

  35. Piccioli, G., De Micheli, E., Parodia, P., Campani, M.: Robust method for road sign detection and recognition. Image Vision Comput. 14, 209–223 (1996)

    Article  Google Scholar 

  36. Gil-Jimenez, P., Lafuente-Arroyo, S., Gomez-Moreno, H., Lopez-Ferreras F., Maldonado-Bascon, S.: Traffic sign shape classification evaluation. In: IEEE Intelligent Vehicles Symposium, pp. 607–612 (2005)

  37. Muyan-Özçelik, P., Ota, J.M., Owens, J.D.: Feature-based speed limit sign detection using a graphics processing unit. In: IEEE Intelligent Vehicles Symposium, pp. 195–200 (2011)

  38. McGehee, D.V., Mazzae, E.N., Baldwin, S.: Driver reaction time in crash avoidance research: validation of a driving simulator study on a test track. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 44(3), 320–323 (2000)

    Google Scholar 

  39. LogiLMD IP core for rear lane departure warning. Xilon Inc. v. 2.0.1, pp. 1–5 (2014)

  40. Hsiao, P.Y., Yeh, C.W.: A portable real-time lane departure warning system based on embedded calculating technique. IEEE 63rd Veh. Tech. Conf. 6, 2982–2986 (2006)

    Google Scholar 

  41. Jeng, M.J.: Lane detection system based on software and hardware codesign. In: IEEE ICARA, pp. 319–323 (2009)

  42. Xilinx Zynq 7000 Series (XC7Z045) 20W Reference Design, Report PMP10613.2. 1–10 (2015). Accessed online 5 Aug 2016 http://www.ti.com/lit/ug/tidu970/tidu970.pdf

  43. LogiADAK Zynq-7000 all programmable SoC automotive driver assistance kit. Xilon Inc., v. 3.0.1, pp. 1–10 (2015)

  44. Hughes, C., Glavin, M., Jones, E., Denny, P.: Review of geometric distortion compensation in fish-eye cameras, In: IET ISSC, pp. 162–167 (2008)

  45. Mallon, J., Whelan, P.: Precise radial un-distortion of images. IEEE ICPR 1, 18–21 (2004)

    Google Scholar 

  46. Hughes, C., Jones, E., Glavin, M., Denny, P.: Validation of polynomial-based equidistance fish-eye models. In: IET ISSC, pp. 1–6 (2009)

  47. Basu, A., Licardie, S.: Alternative models for fish-eye lenses. Pattern Recognit. Lett. 16(4), 433–441 (1995)

    Article  Google Scholar 

  48. Devernay, F., Faugeras, O.: Straight lines have to be straight: automatic calibration and removal of distortion from scenes of structured environments. J. Mach. Vision Appl. 13(1), 14–24 (2001)

    Article  Google Scholar 

  49. Ishii, C., Sudo, Y., Hashimoto, H.: An image conversion algorithm from fish eye image to perspective image for human eyes. IEEE/ASME Int. Conf. Adv. Intell. Mechatron. 2, 1009–1014 (2003)

    Google Scholar 

  50. Bertozzi, M., Broggi, A., Fascioli, A.: Stereo inverse perspective mapping: theory and applications. Image Vision Comput. 16, 585–590 (1998)

    Article  Google Scholar 

  51. Broggi, A., Bertozzi, M., Fascioli, A., Conte, G.: Automatic vehicle guidance, the experience of the ARGO autonomous vehicle. World Scientific Publishing, Singapore (1999)

    Book  Google Scholar 

  52. Broggi, A., Conte, G., Gregoretti, F., Sansoe’, C., Passerone, R., Reyneri, L.M.: Design and implementation of the PAPRICA parallel architecture. J. VLSI Signal Process. Syst. Signal Image Video Tech. 19, 5–18 (1998)

    Article  Google Scholar 

  53. Muad, A., Hussain, A., Samad, S., Mustaffa, M., Majlis, B.: Implementation of inverse perspective mapping algorithm for the development of an automatic lane tracking system. In: IEEE Tencon (2004)

  54. Lin, C.C., Wang, M.S.: A vision based top-view transformation model for a vehicle parking assistant. Sensors 12, 4431–4446 (2012)

    Article  Google Scholar 

  55. Oliveira, M., Santos, V., Sappac, A.: Multimodal inverse perspective mapping. Inform. Fusion 24, 108–121 (2015)

    Article  Google Scholar 

  56. Jepson, A.: Homograhy. http://www.cs.toronto.edu/~jepson/csc2503/tutorials/homography.pdf. Accessed online 5 Aug 2016

  57. Lee, J., Lee, W., Cho, I., Jeong, D.: Fast panoramic image generation method using morphological corner detection. Lecture Notes in Computer Science, vol. 3767, pp. 547–558 (2005)

    Chapter  Google Scholar 

  58. Shimizu, S., Kawai, J., Yamada, H.: Wraparound view system for motor vehicles. Fujitsu Sci. Tech. J. 46(1), 95–102 (2010)

    Google Scholar 

Download references

Acknowledgements

Discussions with R. Massini, B. Neri, B. Carnevale, and E. Franchi, from University of Pisa, are acknowledged.

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Pisa, via G. Caruso 16, 56122, Pisa, Italy

    Sergio Saponara

Authors
  1. Sergio Saponara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergio Saponara.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saponara, S. Hardware accelerator IP cores for real time Radar and camera-based ADAS. J Real-Time Image Proc 16, 1493–1510 (2019). https://doi.org/10.1007/s11554-016-0657-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0657-0

Keywords

Search

Navigation