Skip to main content
Springer Nature Link
Log in

A hybrid system for embedded machine vision using FPGAs and neural networks

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper presents a hybrid model for embedded machine vision combining programmable hardware for the image processing tasks and a digital hardware implementation of an artificial neural network for the pattern recognition and classification tasks. A number of possible architectural implementations are compared. A prototype development system of the hybrid model has been created, and hardware details and software tools are discussed. The applicability of the hybrid design is demonstrated with the development of a vision application: real-time detection and recognition of road signs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arnold, J., Buell, D.A., Davis, E.G.: Splash-2. In: ACM Symposium on Parallel Algorithms and Architectures (ACM’92), pp. 316–324 (1992)

  2. AXEON Ltd: URL http://www.axeon.com

  3. Batista, L.B., Gomes, H.M., Herbster, R.F.: Application of growing hierarchical self-organizing map in handwritten digit recognition. In: Proceedings of the Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI’03) (2003)

  4. Batlle J., Martí J., Ridao P. and Amat J. (2002). A new FPGA/DSP-based parallel architecture for real-time image processing. RealTime Imaging 8: 345–356

    Article  MATH  Google Scholar 

  5. Benkrid, K., Crookes, D., Smith, J., Benkrid, A.: High level programming for FPGA based image and video processing using hardware skeletons. In: Proceedings of the IEEE Symposium Field-Programmable Custom Computing Machines (FCCM’01) (2001)

  6. Bode M., Freyd O., Fischer J., Niedernostheide E.J. and Schulze H.J. (2001). Hybrid hardware for a highly parallel search in the context of learning classifiers. Artif. Intell. 130: 75–84

    Article  MATH  Google Scholar 

  7. Bouridane A., Crookes D., Donachy P., Alotaibi K. and Benkrid K. (1999). A high level FPGA-based abstract machine for image processing. J. Syst. Archit. 45: 809–824

    Article  Google Scholar 

  8. Brown, D., Craw, I., Lewthwaite, J.: A SOM based approach to skin detection with application in real time systems. In: Proceedings of the British Machine Vision Conference (BMVC’01) (2001)

  9. Campbell N.W., Thomas B.T. and Troscianko T. (1997). Automatic segmentation and classification of outdoor images using neural networks. Neural Syst. 8(1): 137–144

    Article  Google Scholar 

  10. Celoxica: Handel-C Language reference manual, Celoxica (2004)

  11. Celoxica Ltd: URL http://www.celoxica.com

  12. Crookes D. (1999). Architectures for high performance image processing: the future. J. Syst. Archit. 45: 739–748

    Article  Google Scholar 

  13. Dias F.M., Antunes A. and Mota A.M. (2004). Artificial neural networks: a review of commercial hardware. Eng. App. Artif. Intell. 17: 945–952

    Article  Google Scholar 

  14. Diniz P., Hall M., Park J., So B. and Ziegler H. (2005). Automatic mapping of C to FPGAs with the DEFACTO compilation and synthesis system. Microprocess. Microsyst. 29: 51–62

    Article  Google Scholar 

  15. Douville P. (2000). Real-time classification of traffic signs. Realtime Imaging 6: 185–193

    Article  Google Scholar 

  16. Draper, B., Najjar, W., Bohm, W., Hammers, J., Rinker, B., Ross, C., Chawathe, M., Bins, J.: Compiling and optimizing image processing algorithms for FPGA’s. In: Proceedings of the IEEE Computer Architectures for Machine Perception (CAMP’00) (2000)

  17. Drayer, T.H., IV, W.E.K., Tront, J.G., Conners, R.W.: A modular and reprogrammable real-time processing hardware. In: Proceedings of the IEEE FPGA-Based Custom Computing Machines (FCCM’95) (1995)

  18. Dunn, P.A., Kearney, P.D., Jensen, M.J., Davey, P.I.: A modular configurable logic processor system. CSIRO Manufacturing Science and Technology (2002)

  19. Egmont-Petersen M., de Ridder D. and Handels H. (2002). Image processing with neural networks - a review. Pattern Recognit. 35(10): 2279–2301

    Article  MATH  Google Scholar 

  20. Eide, A., Lindblad, T., Lindsey, C.S., Minerskjold, M., Sekhniaidze, G., Székely, G.: An implementation of the Zero Instruction Set Computer (ZISC036) on a PC/ISA-bus card. In: Proceedings of the Workshop on Neural Networks (WNN/FNN’94), pp. 319–330 (1994)

  21. Eppler W., Fischer T., Gemmeke H., Chilingarian A. and Vardanyan A. (2000). Neural chip SAND in online data processing of extensive air showers. Comput. Phys. Commun. 126: 63–66

    Article  MATH  Google Scholar 

  22. Estable, S., Schick, J., Stein, F., Janssen, R., Ott, R., Ritter, W.: Real-time traffic sign recognition system. In: Proceedings of Intelligent Vehicles’94 Symposium (1994)

  23. Fiesler, E., Duong, T., Trunov, A.: Design of neural network-based microchip for color segmentation. In: Proceedings of SPIE, vol. 4055 (2000)

  24. Fu L. (1994). Neural Networks in Computer Intelligence. McGraw-Hill, Inc, New york

    Google Scholar 

  25. General Vision: URL http://www.general-vision.com

  26. Greenbaum, J., Baxter, M.: Increased FPGA capacity enables scalable, flexible CCMs: An example from image processing. In: Proceedings of the IEEE FPGA-Based Custom Computing Machines (FCCM’97) (1997)

  27. Hamid, G.: An FPGA-based coprocessor for image processing. IEE Colloquium Integrated Imaging Sensors and Processing, pp. 6/1–6/4 (1994)

  28. Heemskerk J.N.H. (1995) Overview of neural hardware. Neurocomputers for brain-style processing: Design, implementation and application. Ph.D. thesis, Unit of Experimental and Theoretical Psychology, Leiden University, Leiden (1995)

  29. Hendry D.C., Duncan A.A. and Lightowler N. (2003). IP core implementation of a self-organizing neural network. IEEE Trans. Neural Netw. 14(5): 1085–1096

    Article  Google Scholar 

  30. Hsu S.H. and Huang C.L. (2001). Road sign detection and recognition using matching pursuit method. Image Vis. Comput. 19: 119–129

    Article  Google Scholar 

  31. Ienne P. and Kuhn G. (1995). Digital systems for neural networks. SPIE Opt. Eng. Crit. Rev. Ser. CR57: 314–345

    Google Scholar 

  32. Ienne, P., Viredaz, M.A.: Implementation of Kohonen’s self- organizing maps on MANTRA I. In: Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 273–279 (1994)

  33. Johansson, B.: Road sign recognition from a moving vehicle. citeseer.ist.psu.edu/570294.html

  34. Kessal L., Abel N. and Demigny D. (2003). Real-time image processing with dynamically reconfigurable architecture. RealTime Imaging 9: 297–313

    Article  Google Scholar 

  35. Kohonen T. (1982). Analysis of a simple self-organizing process. Biol. Cybern. 44(2): 135–140

    Article  MATH  MathSciNet  Google Scholar 

  36. Krumbiegel, D., Kraiss, K.F., Schrieber, S.: A connectionist traffic sign recognition system for onboard driver information. In: 5th IFAC/IFIP/IFORS/IEA Symposium on Anlaysis, Design and Evaluation of Man–Machine Systems, pp. 201–206 (1992)

  37. Lalonde, M., Li, Y.: Road sign recognition, survey of the state of the art. Tech. rep., Centre de recherche informatique de Montreal CRIM/IIT (1995)

  38. Le, D.X., Thoma, G.R., Wechsler, H.: Document image analysis using integrated image and neural processing. In: Proceedings of the International Conference on Document Analysis and Recognition (ICDAR’95) (1995)

  39. Liao, Y.: Neural networks in hardware: A survey. Tech. rep. bsit.csc.lsu.edu/~jianhua/shiv2.pdf (2001)

  40. Lightowler, N.: Modular maps: an implementation strategy for the self-organising map. Ph.D. thesis, University of Aberdeen, Aberdeen (1997)

  41. Lightowler, N., Allen, A.R., Grant, H., Hendry, D.C., Spracklen, C.T.: The modular map. IJCNN (1999)

  42. McBader, S., Lee, P.: An FPGA implementation of a flexible, parallel image processing architecture suitable for embedded vision systems. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium (IPDPS’03) (2003)

  43. Moerland, P.D., Fiesler, E.: Hardware-friendly learning algorithms for neural networks: an overview. In: Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems (MicroNeuro’96) (1996)

  44. Muthukumar, V., Rao, D.V.: Image processing algorithms on reconfigurable architecture using HandelC. In: Proceedings of the IEEE Euromicro Systems on Digital System Design (DSD’04), pp. 218–226 (2004)

  45. Paclik, P.: The automatical classification of road signs. Master’s thesis, Faculty of transportation science, Czech Technical University, Prague (1998)

  46. Paclik P., Novovicova J., Pudil P. and Somol P. (2000). Road sign classification using laplace kernel classifier. Pattern Recogn. Lett. 21: 1165–1173

    Article  MATH  Google Scholar 

  47. Piccioli G., Micheli E.D., Parodi P. and Campani M. (1996). Robust method for road sign detection and recognition. Image Vis. Compu. 14: 209–223

    Article  Google Scholar 

  48. Porrmann, M., Franzmeier, M., Kalte, H., Witkowski, U., Ruckert, U.: A reconfigurable SOM hardware accelerator. In: Proceedings of the European Symposium on Artificial Neural Networks ESANN’02, pp. 337–342 (2002)

  49. Porrmann, M., Witkowski, U., Kalte, H., Ruckert, U.: Implementation of artificial neural networks. In: Proceedings of the Euromicro Workshop on Parallel, Distributed and Network-based Processing (EUROMICRO-PSP’02), pp. 337–342 (2002)

  50. Prieto, M.S., Allen, A.R.: Using self-organizing maps in the detection and recognition of road signs. Image Vis. Comput. (2005, submitted)

  51. Rueping, S., Goser, K., Rueckert, U.: A chip for self-organizing feature maps. In: Proceedings of the International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 26–33 (1994)

  52. Ruping S., Porrmann M. and Ruckert U. (1998). SOM accelerator system. Neurocomputing 21: 31–50

    Article  Google Scholar 

  53. Salcic Z. and Sivaswamy J. (1999). Imeco: A reconfigurable FPGA-based image enhancement co-processor framework. RealTime Imaging 5: 385–395

    Article  Google Scholar 

  54. Sarle, W.S.: Neural Network FAQ. URL ftp://ftp.sas.com/pub/neural/FAQ.html

  55. Schoenauer, T., Jahnke, A., Roth, U., Klar, H.: Digital neurohardware: principles and perspectives. In: Proceedings of the Neuronal Networks in Applications (NN’98), pp. 101–106 (1998)

  56. Seiffert, U.: Artificial neural networks on massively parallel computer hardware. In: Proceedings of the European Symposium on Artificial Neural Networks (ESANN’02), pp. 319–330 (2002)

  57. Sheen, T.M.: Tools for portable parallel image processing. Ph.D. thesis, University of Aberdeen (1999)

  58. Sheen, T.M., Allen, A.R., Lawrence, A.E., Page, I.: Hardware compilation technology for embedded image processing. In: High Performance Architectures for Real-Time Image Processing: IEE Colloquium Digest 1998/197, pp. 9/1–9/6 (1998)

  59. Siegel H.J., Armstrong J.B. and Watson D.W. (1992). Mapping computer vision-related tasks onto reconfigurable parallel-processing systems. IEEE Comput. 25(2): 54–63

    Google Scholar 

  60. Tanougast C., Berviller Y., Brunet P., Weber S. and Rabah H. (2003). Temporal partitioning methodology optimizing FPGA resources for dynamically reconfigurable embedded real-time system. Microprocess. Microsyst. 27: 115–130

    Article  Google Scholar 

  61. Viredaz, M.A.: MANTRA I: an SIMD processor array for neural computation. In: Proceedings of the Euro-ARCH’93 Conference, pp. 99–110 (1993)

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Physical Sciences, University of Aberdeen, Aberdeen, AB24 3UE, Scotland, UK

    Miguel S. Prieto & Alastair R. Allen

Authors
  1. Miguel S. Prieto
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Alastair R. Allen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Alastair R. Allen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Prieto, M.S., Allen, A.R. A hybrid system for embedded machine vision using FPGAs and neural networks. Machine Vision and Applications 20, 379–394 (2009). https://doi.org/10.1007/s00138-008-0133-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0133-3

Keywords