Skip to main content

Advertisement

SpringerLink
Log in

Accelerating embedded image processing for real time: a case study

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Many image processing applications need real-time performance, while having restrictions of size, weight and power consumption. Common solutions, including hardware/software co-designs, are based on Field Programmable Gate Arrays (FPGAs). Their main drawback is long development time. In this work, a co-design methodology for processor-centric embedded systems with hardware acceleration using FPGAs is proposed. The goal of this methodology is to achieve real-time embedded solutions, using hardware acceleration, but achieving development time similar to that of software projects. Well established methodologies, techniques and languages from the software domain—such as Object-Oriented Paradigm design, Unified Modelling Language, and multithreading programming—are applied; and semiautomatic C-to-HDL translation tools and methods are used and compared. The methodology is applied to achieve an embedded implementation of a global vision algorithm for the localization of multiple robots in an e-learning robotic laboratory. The algorithm is specifically developed to work reliably 24/7 and to detect the robot’s positions and headings even in the presence of partial occlusions and varying lighting conditions expectable in a normal classroom. The co-designed implementation of this algorithm processes 1,600 × 1,200 pixel images at a rate of 32 fps with an estimated energy consumption of 17 mJ per frame. It achieves a 16× acceleration and 92 % energy saving, which compares favorably with the most optimized embedded software solutions. This case study shows the usefulness of the proposed methodology for embedded real-time image processing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. According to the Embedded Market Survey, an average embedded system project takes 12 months and a group of 13.4 engineers (i.e., 13.4 man-years) [4]. An example of a project prototyped in FPGA is the memory controller of the ERC32 processor of the European Space Agency, which took 10 man-years and 25,000 code lines [10].

References

  1. Altera.: Altera Introduces SoC FPGAs: Integrating ARM processor system and FPGA into 28-nm single-chip solution. http://www.altera.com/corporate/news_room/releases/2011/products/nr-soc-fpga.html (2011)

  2. Arpinen, T., Salminen, E., Hmlinen, TD., Hnnikinen, M.: MARTE profile extension for modeling dynamic power management of embedded systems. J. Syst. Archit. 58(5), 209–219 (2012)

    Article  Google Scholar 

  3. Bailey, B., Martin G.: ESL Models and their Application. Electronic System Level Design and Verification in Practice. Springer, Berlin (2010)

  4. Blanza, D., Holland, C.: Embedded market survey. EETimes and Embedded http://seminar2.techonline.com/~additionalresources/esd_apr2012/ubme_embeddedmarket2012_full.pdf (2012)

  5. Bouguet, JY.: Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/ (2008)

  6. Brezak, M., Petrović, I., Ivanjko, E.: Robust and accurate global vision system for real time tracking of multiple mobile robots. Robot. Auton. Syst. 56(3), 213–230 (2008)

    Article  Google Scholar 

  7. Bruce, J., Veloso, M.: Fast and accurate vision-based pattern detection and identification. In: IEEE International Conference on Robotics and Automation, ICRA, IEEE, pp. 1277–1282 (2003)

  8. ESA European Space Agency VHDL. http://www.esa.int/TEC/Microelectronics/SEMS7EV681F_0.html (2011)

  9. Gaisler, J.: A structured VHDL design method. In: Fault-Tolerant Microprocessors for Space Applications, Gaisler Research, pp. 41–50, http://www.gaisler.com/doc/vhdl2proc.pdf (2004)

  10. Gaisler, J.: A structured VHDL design method. http://www.gaisler.com/doc/structdes.pdf (2011)

  11. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman, Boston (1995)

  12. Gunay, N., Dadios, E.: A robust and accurate color-based global vision recognition of highly dynamic objects in real time. In: 8th Asian Control Conference (ASCC), IEEE, pp. 90 –95 (2011)

  13. Gupta, S., Dutt, N., Gupta, R., Nicolau, A.: SPARK: A high-level synthesis framework for applying parallelizing compiler transformations. In: 16th Intl. Conf. on VLSI Design, IEEE, pp. 461–467 (2003)

  14. Happe, M., Lubbers, E., Platzner, M.: A self-adaptive heterogeneous multi-core architecture for embedded real-time video object tracking. J. Real-Time Image Process pp. 1–16 (2011)

  15. Jacquard ROCCC 2.0. http://www.jacquardcomputing.com/roccc/ (2011)

  16. Keskin, O., Uyar, E.: A framework for multi robot guidance control. In: Holonic and Multi-Agent Systems for Manufacturing, Lecture Notes in Computer Science, vol. 5696, Springer, Berlin, pp. 315–323 (2009)

  17. Klančar, G., Kristan, M., Kovačič, S., Orqueda, O.: Robust and efficient vision system for group of cooperating mobile robots with application to soccer robots. ISA Trans. 43(3), 329–342 (2004)

    Google Scholar 

  18. Klančar, G., Matko, D., Blažič, S.: Wheeled mobile robots control in a linear platoon. J. Intell. Robot. Syst. 54, 709–731 (2009)

    Google Scholar 

  19. Kulich, M., Chudoba, J., Kosnar, K., Krajnik, T., Faigl, J., Preucil, L.: SyRoTek—distance teaching of mobile robotics. IEEE Trans. Educ. 56(1), 18–23 (2013)

    Article  Google Scholar 

  20. Mentor-Graphics CatapultC. http://www.mentor.com/esl/catapult http://www.mentor.com/esl/catapult/overview (2011)

  21. Michael, N., Mellinger, D., Lindsey, Q., Kumar, V.: The GRASP multiple micro UAV testbed . IEEE Robotics and Automation Magazine (2010)

  22. Mischkalla, F., He, D., Mueller, W.: Closing the gap between UML-based modeling, simulation and synthesis of combined HW/SW systems. In: Design, Automation Test in Europe Conference Exhibition (DATE), 2010, pp. 1201 –1206 (2010)

  23. Mueller, W., Rosti, A., Bocchio, S., Riccobene, E., Scandurra, P., Dehaene, W., Vanderperren, Y., Ku, L.: UML for ESL design—Basic principles, tools, and applications. In: IEEE/ACM International Conference on Computer Aided Design, pp. 73–80 (2006)

  24. Nallatech: DIME-C. http://www.nallatech.com/Development-Tools/dime-c.html (2011)

  25. NVIDIA: CUDA: Parallel Programming. http://www.nvidia.com (2012)

  26. OMG UML Profile for System on a Chip (SoC) Version 1.0.1 - formal/06-08-01. http://www.omg.org/spec/SoCP/1.0.1/ (2006)

  27. OMG.: Systems modeling language (SysML) Version 1.2 - formal/2010-06-01. http://www.omg.org/spec/SysML/1.2/ (2010)

  28. OMG UML Profile for MARTE: Modeling and analysis of real-time embedded systems Version 1.1 - formal/2011-06-02. http://www.omg.org/spec/MARTE/1.1/ (2011)

  29. Paul, J., Laika, A,. Claus, C., Stechele, W., El Sayed Auf, A., Maehle, E.: Real-time motion detection based on SW/HW-codesign for walking rescue robots. J. Real-Time Image Process. 1–16 (2012)

  30. Pedre, S., Krajník, T., Todorovich, E., Borensztejn, P.: Hardware/software co-design for real time embedded image processing: A case study. In: Alvarez, L., Mejai,l M., Gomez, L., Jacobo J. (eds): Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Lecture Notes in Computer Science, vol 7441, Springer Berlin Heidelberg, pp. 599–606 (2012)

  31. Pedre, S., Krajník, T., Todorovich, E., Borensztejn, P.: A co-design methodology for processor-centric embedded systems with hardware acceleration using FPGA. In: IEEE 8th Southern Programmable Logic Conference, IEEE, Brazil, pp. 7–14 (2012)

  32. Quadri, IR., Gamati, A., Boulet, P., Meftali, S., Dekeyser, JL.: Expressing embedded systems configurations at high abstraction levels with UML MARTE profile: Advantages, limitations and alternatives. J. Syst. Archit. 58(5), 178–194 (2012)

    Article  Google Scholar 

  33. Rao, R., Taylor, C., Kumar, V.: Experiments in robot control from uncalibrated overhead imagery. In: Ang, J., Marcelo, H., Khatib, O. (eds): Experimental Robotics IX, Springer Tracts in Advanced Robotics, vol 21, Springer Berlin Heidelberg, pp. 491–500 (2006)

  34. Riccobene, E., Scandurra, P.: Weaving executability into UML class models at PIM level. In: First European Workshop on Behaviour Modelling in Model Driven Architecture (BM-MDA), CTIT Workshop Proceedings Series, Enschede, The Netherlands, pp. 10–28 (2009)

  35. Riccobene, E., Scandurra, P., Rosti, A., Bocchio, S.: A SoC design methodology involving a UML 2.0 profile for SystemC. In: Proceedings of the Conference on Design, Automation and Test in Europe, vol. 2, IEEE Computer Society, Washington, DC, USA, DATE 2005, pp. 704–709 (2005)

  36. Riccobene, E., Scandurra, P., Rosti, A., Bocchio, S.: A model-driven design environment for embedded systems. In: Proceedings of the 43rd Annual Design Automation Conference, ACM, New York, NY, USA, DAC ’06, pp. 915–918 (2006)

  37. Rodriguez-Gomez, R., Fernandez-Sanchez, E., Diaz, J., Ros, E.: Codebook hardware implementation on FPGA for background subtraction. J. Real-Time Image Process. pp. 1–15 (2012)

  38. Santarini, M.: Zynq-7000 EPP sets stage for new era of innovations. Xcell J 75, 8–13 (2011)

    Google Scholar 

  39. Silva-Filho, A.G., et al.: An ESL approach for energy consumption analysis of cache memories in SoC platforms. Int. J. Reconfigurable Comput. 2011, 1–12 (2011)

  40. Sparx Systems (2011) Visual modelling platform. http://www.sparxsystems.com/products/ea/

  41. Steggles, P., Gschwind, S.: The UBISENSE smart space platform. In: Third International Conference on Pervasive Computing (2005)

  42. Technology, BD.: The AutoESL Auto Pilot High-Level Synthesis. Tool. Tech. Rep. (2010)

  43. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, Cambridge (2005)

  44. Vidal, J., de Lamotte, F., Gogniat, G., Soulard, P., Diguet, JP.: A co-design approach for embedded system modeling and code generation with UML and MARTE. In: Proceedings of the Conference on Design, Automation and Test in Europe, European Design and Automation Association, 3001 Leuven, Belgium, DATE ’09, pp. 226–231 (2009)

  45. Virginia, A.J., et al.: An empirical comparison of ANSI-C to VHDL compilers : SPARK, RORCC and DWARV. In: Annual Workshop on Circuits Systems and Signal Processing ProRISC, pp. 388–394 (2007)

  46. Xilinx Platform studio and the embedded development kit (EDK). http://www.xilinx.com/tools/platform.htm (2011a)

  47. Xilinx Xilinx introduces Zynq-7000 family, industry’s first extensible processing platform. http://press.xilinx.com/index.php?s=34135&item=18 (2011b)

  48. Xilinx Vivado design suite. http://www.xilinx.com/products/design-tools/vivado/index.htm (2012)

  49. Yankova, Y., Kuzmanov, G., Bertels, K., Gaydadjiev, G., Lu, Y., Vassiliadis, S.: DWARV: Delftworkbench automated reconfigurable VHDL generator. In: Intl Conf on Field Programmable Logic and Applications, IEEE, pp. 697–701 (2007)

  50. Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673 (1999)

Download references

Acknowledgments

Xilinx Design Suite was donated by Xilinx University Program. This work has been partially supported by Czech project No. 7AMB12AR022, by EU within ICT - 216240 and Argentinian projects MINCyT RC/11/20, UBACyT 200158 and PICT-2009-0041.

Author information

Authors and Affiliations

  1. Departamento de Computación, FCEN-UBA, Buenos Aires, Argentina

    Sol Pedre & Patricia Borensztejn

  2. Czech Technical University in Prague, Prague, Czech Republic

    Tomáš Krajník

  3. Departamento de Computación y Sistemas, FCE-UNICEN, Tandil, Argentina

    Elías Todorovich

Authors
  1. Sol Pedre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomáš Krajník
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elías Todorovich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patricia Borensztejn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sol Pedre.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pedre, S., Krajník, T., Todorovich, E. et al. Accelerating embedded image processing for real time: a case study. J Real-Time Image Proc 11, 349–374 (2016). https://doi.org/10.1007/s11554-013-0353-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0353-2

Keywords

Search

Navigation