Abstract
In this paper a novel integrated, single-chip solution for autonomous navigation inspired by the computations in the insect visuomotor system is proposed. A generalization of the theory of wide field integration (WFI) is presented which supports the use of sensors with a limited field of view, and the system concept is validated based on experiments using a prototype single-chip WFI sensor. The VLSI design implements (1) an array of Elementary Motion Detectors (EMDs) to derive local estimates of optic flow, (2) a novel mismatch compensation approach to handle dissimilarities in local motion detector units, and (3) on-chip programmable optic flow pattern weighting (Wide-Field Integration) to extract relative speed and proximity with respect to the surrounding environment. Computations are performed in the analog domain and in parallel, providing outputs at 1 kHz while consuming only 42.6 μW of power. The resulting sensor is integrated with a ground vehicle and navigation of corridor-like environments is demonstrated.
Similar content being viewed by others
References
Abbott, D., Bouzerdoum, A., Eshraghian, K.: Future directions for motion detection based on the parallel computational intelligence of insects. In: Proceedings of the 23rd Euromicro Conference, pp. 244–249 (1997)
Andreou, A., Strohbehn, K., Jenkins, R.: Silicon retina for motion computation. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 1373–1376 (1991)
Arias-Estrada, M., Poussart, D., Tremblay, M.: Motion vision sensor architecture with asynchronous self-signaling pixels. In: Proceedings Fourth IEEE International Workshop on Computer Architecture for Machine Perception, pp. 75–83 (1997)
Barrows, G., Miller, K., Krantz, B.: Fusing neuromorphic motion detector outputs for robust optic flow measurement. In: Proc. IEEE International Joint Conference on Neural Networks, vol. 4, pp. 2296–2301. Washington DC, USA (1999)
Benson, R., Delbruck, T.: Direction selective silicon retina that uses null inhibition. In: Advances in Neural Information Processing Systems, vol. 4, pp. 756–763 (1992)
Borst, A.: Modeling fly motion vision. In: Feng, J. (ed.) Computational Neuroscience: A Comprehensive Approach, pp. 397–429. CRC Press (2003)
Borst, A., Haag, J.: Neural networks in the cockpit of the fly. J. Comp. Physiol. A 188, 419–437 (2002)
Bouzerdoum, A., Moini, A., Yakovleff, A., Nguyen, X.T., Bogner, R.E., Eshraghian, K.: A smart visual micro-sensor. In: Proc. IEEE Int. Conf. on Systems, Man and Cybernetics, pp. 276–279 (1994)
Brinkworth, R., Shoemaker, P., O’Carroll, D.: Characterization of a neuromorphic motion detection chip based on insect visual system. In: Proceedings of the 5th International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 289–294 (2009)
Coombs, D., Herman, M., Hong, T.H., Nashman, M.: Real-time obstacle avoidance using central flow divergence, and peripheral flow. IEEE Trans. Robot. Autom. 14(1), 49–59 (1998)
Delbruck, T.: Silicon retina with correlation-based, velocity tuned pixels. IEEE Trans. Neural Netw. 4, 529–541 (1993)
Deutschmann, R., Koch, C.: Compact analog VLSI 2-D velocity sensor. In: Proc. IEEE International Conference on Intelligent Vehicles, vol. 1, pp. 359–364 (1998)
Egelhaaf, M., Borst, A.: Transient and steady-state response properties of movement detectors. J. Opt. Soc. Am. A 6(1), 116–127 (1989)
Egelhaaf, M., Borst, A.: Motion computation and visual orientation in flies. Comp. Biochem. Physiol. 104A, 659–673 (1993)
Egelhaaf, M., Reichardt, W.: Dynamic response properties of movement detectors: theoretical analysis and electrophyiological investigation in the visual system of the fly. Biol. Cybern. 56, 69–87 (1987)
Egelhaaf, M., Borst, A., Reichardt, W.: Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly’s nervous system. J. Opt. Soc. Am. A 6(7), 1070–1087 (1989)
Egelhaaf, M., Kern, R., Krapp, H., Kretzberg, J., Kurtz, R., Warzecha, A.: Neural encoding of behaviourally relevant visual-motion information in the fly. Trends Neurosci. 25, 96–102 (2002)
Etienne-Cummings, R., Fernando, S., der Spiegel, J.V., Mueller, P.: Real-time 2-D analog motion detector VLSI circuit. In: Proceedings of the International Joint Conference on Neural Networks, vol. 4, pp. 426–431 (1992)
Etienne-Cummings, R., Fernando, S., Takahashi, N., Shtonov, V., der Spiegel, J.V., Mueller, P.: A new temporal domain optical flow measurement technique for focal plane VLSI implementation. In: IEEE Proceedings on Computer Architectures for Machine Perception, pp. 241–250 (1993)
Etienne-Cummings, R., der Spiegel, J.V., Mueller, P.: A focal plane visual motion measurement sensor. IEEE Trans. Circuits Syst. I 44(1), 55–66 (1997)
Floreano, D., Zufferey, J., Srinivasan, M., Elington, C.: Flying Insects and Robots. Springer (2009)
Franceschini, N., Pichon, J., Blanes, C.: From insect vision to robot vision. Philos. Trans. R. Soc. Lond. B 337, 283–294 (1992)
Gasparini, L., Nicola, M.D., Massari, N., Gottardi, M.: A micro-power asynchronous contrast-based vision sensor wakes-up on motion. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 1040–1043 (2008)
Gilbert, B.: Comment on single-ended-input single-ended-output four-quadrant analog multiplier. IEEE J. Solid-State Circuits 7(5), 434 (1972)
Gopalan, A., Harrison, R.: A CMOS imager with on-chip temporal filtering for motion pre-processing. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 2, pp. 336–339 (2002)
Gottardi, M., Yang, W.: A CCD/CMOS image motion sensor. In: IEEE Int. Solid-State Circuits Conf., pp. 194–195 (1993)
Green, W., Oh, P., Sevcik, K., Barrows, G.: Autonomous landing for indoor flying robots using optic flow. In: Proc. ASME Int. Mech. Eng. Congr. Expo., vol. 2, pp. 1347–1352 (2003)
Green W., Oh P., Barrows G.: Flying insect inspired vision for autonomous aerial robot maneuvers in near-earth environments. In: Proc. IEEE Int. Conf. on Robotics and Automation, pp. 2347–2352 (2004)
Gruev, V., Etienne-Cummings, R.: Active pixel sensor with on-chip normal flow computation on the read out. In: Proc. IEEE International Conference on Electronics, Circuits, and Systems, pp. 215–218 (2004)
Harrison, R.R., Koch, C.: A silicon implementation of the fly’s optomotor control system. Neural Comput. 12(10), 2291–2304 (2000)
Harrison, R.R.: A biologically inspired analog IC for visual collision detection. IEEE Trans. Circuits Syst. I 52(11), 2308–2318 (2005)
Harrison, R.R., Koch, C.: A robust analog VLSI motion sensor based on the visual system of the fly. Auton. Robots 7, 211–224 (1999)
Harrison, R.R., Koch, C.: A robust analog VLSI Reichardt motion sensor. Int. J. Analog Integr. Circuits Signal Process. 24, 213–229 (2000)
Hasler, P., Minch, B., Diorio, C.: An autozeroing floating-gate amplifier. IEEE Trans. Circuits Syst. II 48(1), 74–82 (2001)
Hausen, K.: Motion sensitive interneurons in the optomotor system of the fly, part I. the horizontal cells: structure and signals. Biol. Cybern. 45, 143–156 (1982)
Hengstenberg, R., Hausen, K., Hengstenberg, B.: The number and structure of giant vertical cells (vs) in the lobula plate of the blowfly Calliphora erythrocephala. J. Comp. Physiol. 149, 163–177 (1982)
Higgins, C., Korrapti, S.: An analog VLSI motion energy sensor based on the Adelson-Bergen algorithm. In: International ICSC Symposium on Biologically-Inspired Systems (2000)
Higgins, C.M., Pant, V.: A biomimetic VLSI sensor for visual tracking of small moving targets. IEEE Trans. Circuits Syst. I 51(11), 489–502 (2005)
Higgins, C.M., Pant, V., Deutschmann, R.: Analog VLSI implementation of spatio-temporal frequency tuned visual motion algorithms. IEEE Trans. Circuits Syst. I 52(3), 489–502 (2005)
Horiuchi, T., Lazzaro, J., Moore, A., Koch, C.: A delay line based motion detection chip. In: Advances in Neural Information Processing Systems, vol. 3, pp. 406–412 (1991)
Humbert, J.S., Hyslop, A.M.: Bio-inspired visuomotor convergence. IEEE Trans. Robot. 26(1), 121–130 (2010)
Humbert, J.S., Murray, R.M., Dickinson, M.H.: A control-oriented analysis of bio-inspired visuomotor convergence. In: Proc. IEEE Conf. on Decision and Control, pp. 245–250. Seville, Spain (2005)
Humbert, J.S., Murray, R.M., Dickinson, M.H.: Pitch-altitude control and terrain following based on bio-inspired visuomotor convergence. In: Proceedings of the AIAA Guidance, Navigation and Control Conference. San Francisco, CA (2005)
Humbert, J.S., Murray, R.M., Dickinson, M.H.: Sensorimotor convergence in visual navigation and flight control systems. In: Proceedings of the 16th IFAC World Congress. Praha, Czech Republic (2005)
Humbert, J.S., Hyslop, A.M., Chinn, M.: Experimental validation of wide-field integration methods for autonomous navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2144–2149 (2007)
Humbert, J.S., Hyslop, A.M., Chinn, M.W.: Experimental validation of wide-field integration methods for autonomous navigation. In: Proceedings of the IEEE Conference on Intelligent Robots and Systems (IROS). San Diego, CA (2007)
Indiveri, G., Kramer, J., Koch, C.: Analog VLSI architecture for computing heading direction. In: Proceedings of the Intelligent Vehicles ’95 Symposium, pp. 24–29 (1995)
Indiveri, G., Kramer, J., Koch, C.: Parallel analog VLSI architectures for computation of heading direction and time-to-contact. In: Advances in Neural Information Processing Systems, vol. 8, pp. 720–726 (1996)
Indiveri, G., Kramer, J., Koch, C.: System implementations of analog VLSI velocity sensors. IEEE Micro 16(5), 40–49 (1996)
Jiang, H.C., Wu, C.Y.: A 2-D velocity- and direction-selective sensor with BJT-based silicon retina and temporal zero-crossing detector. IEEE J. Solid-State Circuits 34(2), 241–247 (1999)
Koenderink, J., van Doorn, A.: Facts on optic flow. Biol. Cybern. 56, 247–254 (1997)
Kramer, J.: Compact integrated motion sensor with three pixel interaction. IEEE Trans. Pattern Anal. Mach. Intell. 18, 455–460 (1996)
Kramer, J., Sarpeshkar, R., Koch, C.: An analog VLSI velocity sensor. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 413–416 (1995)
Kramer, J., Sarpeshkar, R., Koch, C.: Analog VLSI motion discontinuity detectors for image segmentation. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 620–623 (1996)
Kramer, J., Sarpeshkar, R., Koch, C.: Pulse-based analog VLSI velocity sensors. IEEE Trans. Circuits Syst. II 44(2), 86–101 (1997)
Krapp, H., Hengstenberg, R.: Estimation of self-motion by optic flow processing in single visual interneurons. Nature 384, 463–466 (1996)
Krapp, H., Hengstenberg, B., Hengstenberg, R.: Dendritic structure and receptive-field organization of optic flow processing interneurons in the fly. J. Neurophysiol. 79, 1902–1917 (1998)
Liu, S.C.: A neuromorphic aVLSI model of global motion processing in the fly. IEEE Trans. Circuits Syst. II 47(12) 1458–1467 (2000)
Liu, S.C., Usseglio-Viretta, A.: Fly-like visumotor responses of a robot using aVLSI motion-sensitive chips. Biol. Cybern. 85(6), 449–457 (2001)
Lyon, R.: The optical mouse, and an architectural methodology for smart digital sensors. In: Proc. CMU Conference on VLSI Structures and Computations, pp. 1–19 (1981)
Mehta, S., Etienne-Cummings, R.: Normal flow measurement visual motion sensor. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 959–962 (2006)
Meitzler, R., Strohbehn, K., Andreou, A.: A silicon retina for 2-D position and motion computation. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 3, pp. 2096–2099 (1995)
Miller, K., Barrows, G.: Feature tracking linear optic flow sensor chip. In: Proc. IEEE Int. Symp. Circuits Systems, pp. V116–V119 (1999)
Moeckel, R., Liu, S.C.: Motion detection circuits for a time-to-travel algorithm. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 3079–3082 (2007)
Moeckel, R., Liu, S.C.: Motion detection chips for robotic platforms. In: Floreano, D., Zufferey, J., Srinivasan, M., Elington, C. (eds.) Flying Insects and Robots, chap. 8, pp. 101–114 . Springer-Verlag, Berlin (2009)
Moini, A., Bouzerdoum, A., Yakovleff, A., Abbott, D., Kim, O., Eshraghian, K., Bogner, R.E.: An analog implementation of early visual processing in insects. In: Proc. 1993 Int. Symp. VLSI Technology, Systems, and Applications, pp. 283–287 (1993)
Moini, A., Bouzerdoum, A., Eshraghian, K., Yakovleff, A., Nguyen, X.T., Blanksby, A., Beare, R., Abbott, D., Bogner, R.E.: An insect vision-based motion detection chip. IEEE J. Solid-State Circuits 32(2), 279–284 (1997)
Ozalevli, E., Hasler, P., Higgins, C.: Winner-Take-All-Based visual motion sensors. IEEE Trans. Circuits Syst. II 53(8), 717–721 (2006)
Pant, V., Higgins, C.: A biomimetic focal plane speed computation architecture. In: Proceedings of the Computational Optical Sensing and Imaging Conference (2007)
Reichardt, W.: Autocorrelation, a principle for the evaluation of sensory information by the central nervous system. In: Rosenblith, A. (ed.) Sensory Communication, pp. 303–317. MIT Press (1961)
Sánchez-Sinencio, E., Silva-Martínez, J.: CMOS transconductance amplifiers, architectures and active filters: a tutorial. IEE Proc. Circuits Devices Syst. 147(1), 3–12 (2000)
Santos-Victor, J., Sandini, G., Curotto, F., Garibaldi, S.: Divergent stereo for robot navigation: learning from bees. In: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp. 434–439 (1993)
Sarpeshkar, R., Bair, W., Koch, C.: Visual motion computation in analog VLSI using pulses. In: Advances in Neural Information Processing Systems, vol. 5, pp. 781–788 (1993)
Srinivasan, M., Zhang, S., Lehrer, M., Collet, T.: Honeybee navigation en route to the goal: visual flight control and odometry. J. Exp. Biol. 199, 237–244 (1996)
Srinivasan, M., Chahl, J., Weber, K., Nagle, S.V.M., Zhang, S.: Robot navigation inspired by principles of insect vision. Robot. Auton. Syst. 26, 203–216 (1999)
Stocker, A.: Analog VLSI focal-plane array with dynamic connections for the estimation of piecewise-smooth optical flow. IEEE Trans. Circuits Syst. I 51(5), 963–973 (2004)
Tammero, L.F., Dickinson, M.H.: The influence of visual landscape on the free flight behavior of the fruit fly Drosophila melanogaster. J. Exp. Biol. 205, 327–343 (2002)
Tanner, J., Mead, C.: An integrated analog optical motion sensor. VLSI Signal Proc. 2, 59–76 (1986)
Weber, K., Venkatesh, S., Srinivasan, M.: Insect inspired behaviours for the autonomous control of mobile robots. In: Proc. IEEE Int. Conf. on Pattern Recognition, pp. 156–160 (1996)
Weber, K., Venkatesh, S., Srinivasan, M.: Robot navigation inspired by principles of insect vision. Robot. Auton. Syst. 26, 203–216 (1999)
Wong, E.L., Cohen, M.H., Abshire, P.A.: A 128x128 floating gate imager with self-adapting fixed pattern noise reduction. In: Proc. IEEE Int. Symp. Circuits Systems, vol. 5, pp. 5314–5317 (2005)
Wu, C.Y., Huang, C.T.: A CMOS expansion/contraction motion sensor with a retinal processing circuit for Z-motion detection applications. In: Proc. IEEE Int. Symp. Circuits Systems, pp. 3087–3090 (2007)
Wu, C.Y., Huang, K.H.: A CMOS focal-plane motion sensor with BJT-based retinal smoothing network and modified correlation-based algorithm. IEEE Sens. J. 2(6), 549–558 (2002)
Xu, P., Abshire, P., Humbert, J.S.: Motion image sensor with on-chip adaptation and programmable filtering. In: Proceedings of IEEE International Symposium on Circuits and Systems, Rio de Janeiro, Brazil, 15–18 May 2011
Yakovleff, A., Moini, A., Bouzerdoum, A., Nguyen, X.T., Bogner, R.E., Eshraghian, K.: A micro-sensor based on insect vision. In: Proc. Workshop on Computer Architectures for Machine Perception, pp. 137–146 (1993)
Yakovleff, A., Nguyen, X.T., Moini, A., Bouzerdoum, A., Bogner, R.E., Eshraghian, K.: Dual purpose interpretation of sensory information. In: Proc. Int. Conf. Robotics and Automation, pp. 1635–1640 (1994)
Yakovleff, A., Abbott, D., Nguyen, X., Eshraghian, K.: Obstacle avoidance and motion induced navigation. In: Proc. Workshop on Computer Architectures for Machine Perception, pp. 384–393 (1995)
Yamada, K., Soga, M.: A compact integrated visual motion sensor for ITS applications. IEEE Trans. Intell. Transp. Syst. 4(1), 35–42 (2003)
Zhang, G., Yang, T., Gregori, S., Liu, J., Maloberti, F.: Ultra-low power motion-triggered image sensor for distributed wireless sensor network. In: Proceedings of IEEE Sensors, pp. 1141–1146 (2003)
Zhang, G., Lee, H., Liu, J.: A 35nW/pixel 2D visual motion sensor. In: Proceedings of the 31st European Solid-State Circuits Conference, pp. 419–422 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xu, P., Humbert, J.S. & Abshire, P. Analog VLSI Implementation of Wide-field Integration Methods. J Intell Robot Syst 64, 465–487 (2011). https://doi.org/10.1007/s10846-011-9549-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-011-9549-5