Skip to main content
SpringerLink
Log in

Analog VLSI Implementation of Wide-field Integration Methods

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper a novel integrated, single-chip solution for autonomous navigation inspired by the computations in the insect visuomotor system is proposed. A generalization of the theory of wide field integration (WFI) is presented which supports the use of sensors with a limited field of view, and the system concept is validated based on experiments using a prototype single-chip WFI sensor. The VLSI design implements (1) an array of Elementary Motion Detectors (EMDs) to derive local estimates of optic flow, (2) a novel mismatch compensation approach to handle dissimilarities in local motion detector units, and (3) on-chip programmable optic flow pattern weighting (Wide-Field Integration) to extract relative speed and proximity with respect to the surrounding environment. Computations are performed in the analog domain and in parallel, providing outputs at 1 kHz while consuming only 42.6 μW of power. The resulting sensor is integrated with a ground vehicle and navigation of corridor-like environments is demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Abbott, D., Bouzerdoum, A., Eshraghian, K.: Future directions for motion detection based on the parallel computational intelligence of insects. In: Proceedings of the 23rd Euromicro Conference, pp. 244–249 (1997)

  2. Andreou, A., Strohbehn, K., Jenkins, R.: Silicon retina for motion computation. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 1373–1376 (1991)

  3. Arias-Estrada, M., Poussart, D., Tremblay, M.: Motion vision sensor architecture with asynchronous self-signaling pixels. In: Proceedings Fourth IEEE International Workshop on Computer Architecture for Machine Perception, pp. 75–83 (1997)

  4. Barrows, G., Miller, K., Krantz, B.: Fusing neuromorphic motion detector outputs for robust optic flow measurement. In: Proc. IEEE International Joint Conference on Neural Networks, vol. 4, pp. 2296–2301. Washington DC, USA (1999)

  5. Benson, R., Delbruck, T.: Direction selective silicon retina that uses null inhibition. In: Advances in Neural Information Processing Systems, vol. 4, pp. 756–763 (1992)

  6. Borst, A.: Modeling fly motion vision. In: Feng, J. (ed.) Computational Neuroscience: A Comprehensive Approach, pp. 397–429. CRC Press (2003)

  7. Borst, A., Haag, J.: Neural networks in the cockpit of the fly. J. Comp. Physiol. A 188, 419–437 (2002)

    Article  Google Scholar 

  8. Bouzerdoum, A., Moini, A., Yakovleff, A., Nguyen, X.T., Bogner, R.E., Eshraghian, K.: A smart visual micro-sensor. In: Proc. IEEE Int. Conf. on Systems, Man and Cybernetics, pp. 276–279 (1994)

  9. Brinkworth, R., Shoemaker, P., O’Carroll, D.: Characterization of a neuromorphic motion detection chip based on insect visual system. In: Proceedings of the 5th International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 289–294 (2009)

  10. Coombs, D., Herman, M., Hong, T.H., Nashman, M.: Real-time obstacle avoidance using central flow divergence, and peripheral flow. IEEE Trans. Robot. Autom. 14(1), 49–59 (1998)

    Article  Google Scholar 

  11. Delbruck, T.: Silicon retina with correlation-based, velocity tuned pixels. IEEE Trans. Neural Netw. 4, 529–541 (1993)

    Article  Google Scholar 

  12. Deutschmann, R., Koch, C.: Compact analog VLSI 2-D velocity sensor. In: Proc. IEEE International Conference on Intelligent Vehicles, vol. 1, pp. 359–364 (1998)

  13. Egelhaaf, M., Borst, A.: Transient and steady-state response properties of movement detectors. J. Opt. Soc. Am. A 6(1), 116–127 (1989)

    Article  Google Scholar 

  14. Egelhaaf, M., Borst, A.: Motion computation and visual orientation in flies. Comp. Biochem. Physiol. 104A, 659–673 (1993)

    Article  Google Scholar 

  15. Egelhaaf, M., Reichardt, W.: Dynamic response properties of movement detectors: theoretical analysis and electrophyiological investigation in the visual system of the fly. Biol. Cybern. 56, 69–87 (1987)

    Article  MATH  Google Scholar 

  16. Egelhaaf, M., Borst, A., Reichardt, W.: Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly’s nervous system. J. Opt. Soc. Am. A 6(7), 1070–1087 (1989)

    Article  Google Scholar 

  17. Egelhaaf, M., Kern, R., Krapp, H., Kretzberg, J., Kurtz, R., Warzecha, A.: Neural encoding of behaviourally relevant visual-motion information in the fly. Trends Neurosci. 25, 96–102 (2002)

    Article  Google Scholar 

  18. Etienne-Cummings, R., Fernando, S., der Spiegel, J.V., Mueller, P.: Real-time 2-D analog motion detector VLSI circuit. In: Proceedings of the International Joint Conference on Neural Networks, vol. 4, pp. 426–431 (1992)

  19. Etienne-Cummings, R., Fernando, S., Takahashi, N., Shtonov, V., der Spiegel, J.V., Mueller, P.: A new temporal domain optical flow measurement technique for focal plane VLSI implementation. In: IEEE Proceedings on Computer Architectures for Machine Perception, pp. 241–250 (1993)

  20. Etienne-Cummings, R., der Spiegel, J.V., Mueller, P.: A focal plane visual motion measurement sensor. IEEE Trans. Circuits Syst. I 44(1), 55–66 (1997)

    Article  Google Scholar 

  21. Floreano, D., Zufferey, J., Srinivasan, M., Elington, C.: Flying Insects and Robots. Springer (2009)

  22. Franceschini, N., Pichon, J., Blanes, C.: From insect vision to robot vision. Philos. Trans. R. Soc. Lond. B 337, 283–294 (1992)

    Article  Google Scholar 

  23. Gasparini, L., Nicola, M.D., Massari, N., Gottardi, M.: A micro-power asynchronous contrast-based vision sensor wakes-up on motion. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 1040–1043 (2008)

  24. Gilbert, B.: Comment on single-ended-input single-ended-output four-quadrant analog multiplier. IEEE J. Solid-State Circuits 7(5), 434 (1972)

    Article  Google Scholar 

  25. Gopalan, A., Harrison, R.: A CMOS imager with on-chip temporal filtering for motion pre-processing. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 2, pp. 336–339 (2002)

  26. Gottardi, M., Yang, W.: A CCD/CMOS image motion sensor. In: IEEE Int. Solid-State Circuits Conf., pp. 194–195 (1993)

  27. Green, W., Oh, P., Sevcik, K., Barrows, G.: Autonomous landing for indoor flying robots using optic flow. In: Proc. ASME Int. Mech. Eng. Congr. Expo., vol. 2, pp. 1347–1352 (2003)

  28. Green W., Oh P., Barrows G.: Flying insect inspired vision for autonomous aerial robot maneuvers in near-earth environments. In: Proc. IEEE Int. Conf. on Robotics and Automation, pp. 2347–2352 (2004)

  29. Gruev, V., Etienne-Cummings, R.: Active pixel sensor with on-chip normal flow computation on the read out. In: Proc. IEEE International Conference on Electronics, Circuits, and Systems, pp. 215–218 (2004)

  30. Harrison, R.R., Koch, C.: A silicon implementation of the fly’s optomotor control system. Neural Comput. 12(10), 2291–2304 (2000)

    Article  Google Scholar 

  31. Harrison, R.R.: A biologically inspired analog IC for visual collision detection. IEEE Trans. Circuits Syst. I 52(11), 2308–2318 (2005)

    Article  Google Scholar 

  32. Harrison, R.R., Koch, C.: A robust analog VLSI motion sensor based on the visual system of the fly. Auton. Robots 7, 211–224 (1999)

    Article  Google Scholar 

  33. Harrison, R.R., Koch, C.: A robust analog VLSI Reichardt motion sensor. Int. J. Analog Integr. Circuits Signal Process. 24, 213–229 (2000)

    Article  Google Scholar 

  34. Hasler, P., Minch, B., Diorio, C.: An autozeroing floating-gate amplifier. IEEE Trans. Circuits Syst. II 48(1), 74–82 (2001)

    Article  Google Scholar 

  35. Hausen, K.: Motion sensitive interneurons in the optomotor system of the fly, part I. the horizontal cells: structure and signals. Biol. Cybern. 45, 143–156 (1982)

    Article  Google Scholar 

  36. Hengstenberg, R., Hausen, K., Hengstenberg, B.: The number and structure of giant vertical cells (vs) in the lobula plate of the blowfly Calliphora erythrocephala. J. Comp. Physiol. 149, 163–177 (1982)

    Article  Google Scholar 

  37. Higgins, C., Korrapti, S.: An analog VLSI motion energy sensor based on the Adelson-Bergen algorithm. In: International ICSC Symposium on Biologically-Inspired Systems (2000)

  38. Higgins, C.M., Pant, V.: A biomimetic VLSI sensor for visual tracking of small moving targets. IEEE Trans. Circuits Syst. I 51(11), 489–502 (2005)

    Article  Google Scholar 

  39. Higgins, C.M., Pant, V., Deutschmann, R.: Analog VLSI implementation of spatio-temporal frequency tuned visual motion algorithms. IEEE Trans. Circuits Syst. I 52(3), 489–502 (2005)

    Article  Google Scholar 

  40. Horiuchi, T., Lazzaro, J., Moore, A., Koch, C.: A delay line based motion detection chip. In: Advances in Neural Information Processing Systems, vol. 3, pp. 406–412 (1991)

  41. Humbert, J.S., Hyslop, A.M.: Bio-inspired visuomotor convergence. IEEE Trans. Robot. 26(1), 121–130 (2010)

    Article  Google Scholar 

  42. Humbert, J.S., Murray, R.M., Dickinson, M.H.: A control-oriented analysis of bio-inspired visuomotor convergence. In: Proc. IEEE Conf. on Decision and Control, pp. 245–250. Seville, Spain (2005)

  43. Humbert, J.S., Murray, R.M., Dickinson, M.H.: Pitch-altitude control and terrain following based on bio-inspired visuomotor convergence. In: Proceedings of the AIAA Guidance, Navigation and Control Conference. San Francisco, CA (2005)

  44. Humbert, J.S., Murray, R.M., Dickinson, M.H.: Sensorimotor convergence in visual navigation and flight control systems. In: Proceedings of the 16th IFAC World Congress. Praha, Czech Republic (2005)

  45. Humbert, J.S., Hyslop, A.M., Chinn, M.: Experimental validation of wide-field integration methods for autonomous navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2144–2149 (2007)

  46. Humbert, J.S., Hyslop, A.M., Chinn, M.W.: Experimental validation of wide-field integration methods for autonomous navigation. In: Proceedings of the IEEE Conference on Intelligent Robots and Systems (IROS). San Diego, CA (2007)

  47. Indiveri, G., Kramer, J., Koch, C.: Analog VLSI architecture for computing heading direction. In: Proceedings of the Intelligent Vehicles ’95 Symposium, pp. 24–29 (1995)

  48. Indiveri, G., Kramer, J., Koch, C.: Parallel analog VLSI architectures for computation of heading direction and time-to-contact. In: Advances in Neural Information Processing Systems, vol. 8, pp. 720–726 (1996)

  49. Indiveri, G., Kramer, J., Koch, C.: System implementations of analog VLSI velocity sensors. IEEE Micro 16(5), 40–49 (1996)

    Article  Google Scholar 

  50. Jiang, H.C., Wu, C.Y.: A 2-D velocity- and direction-selective sensor with BJT-based silicon retina and temporal zero-crossing detector. IEEE J. Solid-State Circuits 34(2), 241–247 (1999)

    Article  Google Scholar 

  51. Koenderink, J., van Doorn, A.: Facts on optic flow. Biol. Cybern. 56, 247–254 (1997)

    Article  Google Scholar 

  52. Kramer, J.: Compact integrated motion sensor with three pixel interaction. IEEE Trans. Pattern Anal. Mach. Intell. 18, 455–460 (1996)

    Article  Google Scholar 

  53. Kramer, J., Sarpeshkar, R., Koch, C.: An analog VLSI velocity sensor. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 413–416 (1995)

  54. Kramer, J., Sarpeshkar, R., Koch, C.: Analog VLSI motion discontinuity detectors for image segmentation. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 620–623 (1996)

  55. Kramer, J., Sarpeshkar, R., Koch, C.: Pulse-based analog VLSI velocity sensors. IEEE Trans. Circuits Syst. II 44(2), 86–101 (1997)

    Article  Google Scholar 

  56. Krapp, H., Hengstenberg, R.: Estimation of self-motion by optic flow processing in single visual interneurons. Nature 384, 463–466 (1996)

    Article  Google Scholar 

  57. Krapp, H., Hengstenberg, B., Hengstenberg, R.: Dendritic structure and receptive-field organization of optic flow processing interneurons in the fly. J. Neurophysiol. 79, 1902–1917 (1998)

    Google Scholar 

  58. Liu, S.C.: A neuromorphic aVLSI model of global motion processing in the fly. IEEE Trans. Circuits Syst. II 47(12) 1458–1467 (2000)

    Article  Google Scholar 

  59. Liu, S.C., Usseglio-Viretta, A.: Fly-like visumotor responses of a robot using aVLSI motion-sensitive chips. Biol. Cybern. 85(6), 449–457 (2001)

    Article  Google Scholar 

  60. Lyon, R.: The optical mouse, and an architectural methodology for smart digital sensors. In: Proc. CMU Conference on VLSI Structures and Computations, pp. 1–19 (1981)

  61. Mehta, S., Etienne-Cummings, R.: Normal flow measurement visual motion sensor. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 959–962 (2006)

  62. Meitzler, R., Strohbehn, K., Andreou, A.: A silicon retina for 2-D position and motion computation. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 3, pp. 2096–2099 (1995)

  63. Miller, K., Barrows, G.: Feature tracking linear optic flow sensor chip. In: Proc. IEEE Int. Symp. Circuits Systems, pp. V116–V119 (1999)

  64. Moeckel, R., Liu, S.C.: Motion detection circuits for a time-to-travel algorithm. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 3079–3082 (2007)

  65. Moeckel, R., Liu, S.C.: Motion detection chips for robotic platforms. In: Floreano, D., Zufferey, J., Srinivasan, M., Elington, C. (eds.) Flying Insects and Robots, chap. 8, pp. 101–114 . Springer-Verlag, Berlin (2009)

    Chapter  Google Scholar 

  66. Moini, A., Bouzerdoum, A., Yakovleff, A., Abbott, D., Kim, O., Eshraghian, K., Bogner, R.E.: An analog implementation of early visual processing in insects. In: Proc. 1993 Int. Symp. VLSI Technology, Systems, and Applications, pp. 283–287 (1993)

  67. Moini, A., Bouzerdoum, A., Eshraghian, K., Yakovleff, A., Nguyen, X.T., Blanksby, A., Beare, R., Abbott, D., Bogner, R.E.: An insect vision-based motion detection chip. IEEE J. Solid-State Circuits 32(2), 279–284 (1997)

    Article  Google Scholar 

  68. Ozalevli, E., Hasler, P., Higgins, C.: Winner-Take-All-Based visual motion sensors. IEEE Trans. Circuits Syst. II 53(8), 717–721 (2006)

    Article  Google Scholar 

  69. Pant, V., Higgins, C.: A biomimetic focal plane speed computation architecture. In: Proceedings of the Computational Optical Sensing and Imaging Conference (2007)

  70. Reichardt, W.: Autocorrelation, a principle for the evaluation of sensory information by the central nervous system. In: Rosenblith, A. (ed.) Sensory Communication, pp. 303–317. MIT Press (1961)

  71. Sánchez-Sinencio, E., Silva-Martínez, J.: CMOS transconductance amplifiers, architectures and active filters: a tutorial. IEE Proc. Circuits Devices Syst. 147(1), 3–12 (2000)

    Article  Google Scholar 

  72. Santos-Victor, J., Sandini, G., Curotto, F., Garibaldi, S.: Divergent stereo for robot navigation: learning from bees. In: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp. 434–439 (1993)

  73. Sarpeshkar, R., Bair, W., Koch, C.: Visual motion computation in analog VLSI using pulses. In: Advances in Neural Information Processing Systems, vol. 5, pp. 781–788 (1993)

  74. Srinivasan, M., Zhang, S., Lehrer, M., Collet, T.: Honeybee navigation en route to the goal: visual flight control and odometry. J. Exp. Biol. 199, 237–244 (1996)

    Google Scholar 

  75. Srinivasan, M., Chahl, J., Weber, K., Nagle, S.V.M., Zhang, S.: Robot navigation inspired by principles of insect vision. Robot. Auton. Syst. 26, 203–216 (1999)

    Article  Google Scholar 

  76. Stocker, A.: Analog VLSI focal-plane array with dynamic connections for the estimation of piecewise-smooth optical flow. IEEE Trans. Circuits Syst. I 51(5), 963–973 (2004)

    Article  Google Scholar 

  77. Tammero, L.F., Dickinson, M.H.: The influence of visual landscape on the free flight behavior of the fruit fly Drosophila melanogaster. J. Exp. Biol. 205, 327–343 (2002)

    Google Scholar 

  78. Tanner, J., Mead, C.: An integrated analog optical motion sensor. VLSI Signal Proc. 2, 59–76 (1986)

    Google Scholar 

  79. Weber, K., Venkatesh, S., Srinivasan, M.: Insect inspired behaviours for the autonomous control of mobile robots. In: Proc. IEEE Int. Conf. on Pattern Recognition, pp. 156–160 (1996)

  80. Weber, K., Venkatesh, S., Srinivasan, M.: Robot navigation inspired by principles of insect vision. Robot. Auton. Syst. 26, 203–216 (1999)

    Article  Google Scholar 

  81. Wong, E.L., Cohen, M.H., Abshire, P.A.: A 128x128 floating gate imager with self-adapting fixed pattern noise reduction. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 5, pp. 5314–5317 (2005)

  82. Wu, C.Y., Huang, C.T.: A CMOS expansion/contraction motion sensor with a retinal processing circuit for Z-motion detection applications. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 3087–3090 (2007)

  83. Wu, C.Y., Huang, K.H.: A CMOS focal-plane motion sensor with BJT-based retinal smoothing network and modified correlation-based algorithm. IEEE Sens. J. 2(6), 549–558 (2002)

    Article  Google Scholar 

  84. Xu, P., Abshire, P., Humbert, J.S.: Motion image sensor with on-chip adaptation and programmable filtering. In: Proceedings of IEEE International Symposium on Circuits and Systems, Rio de Janeiro, Brazil, 15–18 May 2011

  85. Yakovleff, A., Moini, A., Bouzerdoum, A., Nguyen, X.T., Bogner, R.E., Eshraghian, K.: A micro-sensor based on insect vision. In: Proc. Workshop on Computer Architectures for Machine Perception, pp. 137–146 (1993)

  86. Yakovleff, A., Nguyen, X.T., Moini, A., Bouzerdoum, A., Bogner, R.E., Eshraghian, K.: Dual purpose interpretation of sensory information. In: Proc. Int. Conf. Robotics and Automation, pp. 1635–1640 (1994)

  87. Yakovleff, A., Abbott, D., Nguyen, X., Eshraghian, K.: Obstacle avoidance and motion induced navigation. In: Proc. Workshop on Computer Architectures for Machine Perception, pp. 384–393 (1995)

  88. Yamada, K., Soga, M.: A compact integrated visual motion sensor for ITS applications. IEEE Trans. Intell. Transp. Syst. 4(1), 35–42 (2003)

    Article  Google Scholar 

  89. Zhang, G., Yang, T., Gregori, S., Liu, J., Maloberti, F.: Ultra-low power motion-triggered image sensor for distributed wireless sensor network. In: Proceedings of IEEE Sensors, pp. 1141–1146 (2003)

  90. Zhang, G., Lee, H., Liu, J.: A 35nW/pixel 2D visual motion sensor. In: Proceedings of the 31st European Solid-State Circuits Conference, pp. 419–422 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA

    Peng Xu

  2. Texas Instruments, Inc., Columbia, MD, USA

    Peng Xu

  3. Department of Aerospace Engineering, University of Maryland, College Park, MD, USA

    James Sean Humbert

  4. Department of Electrical and Computer Engineering and Institute for Systems Research, University of Maryland, College Park, MD, USA

    Pamela Abshire

Authors
  1. Peng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Sean Humbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pamela Abshire
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James Sean Humbert.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, P., Humbert, J.S. & Abshire, P. Analog VLSI Implementation of Wide-field Integration Methods. J Intell Robot Syst 64, 465–487 (2011). https://doi.org/10.1007/s10846-011-9549-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-011-9549-5

Keywords

Search

Navigation