Skip to main content
SpringerLink
Log in

Real-time Motion Tracking from a Mobile Robot

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

A mobile robot needs to perceive the motions of external objects to perform tasks successfully in a dynamic environment. We propose a set of algorithms for multiple motion tracking from a mobile robot equipped with a monocular camera and a laser rangefinder. The key challenges are 1. to compensate the ego-motion of the robot for external motion detection, and 2. to cope with transient and structural noise for robust motion tracking. In our algorithms, the robot ego-motion is directly estimated using corresponding feature sets in two consecutive images, and the position and velocity of a moving object is estimated in image space using multiple particle filters. The estimates are fused with the depth information from the laser rangefinder to estimate the partial 3D position. The proposed algorithms have been tested with various configurations in outdoor environments. The algorithms were deployed on three different platforms; it was shown that various type of ego-motion were successfully eliminated and the particle filters were able to track motions robustly. The real-time capability of the tracking algorithm was demonstrated by integrating it into a robot control loop.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bajracharya M, Moghaddam B, Howard A, Brennan S, Matthies LH (2009) Results from a real-time stereo-based pedestrian detection system on a moving vehicle. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  2. Barfoot TD (2005) Online visual motion estimation using FastSLAM with SIFT features. In: Proceedings of the 2005 IEEE/RSJ international conference on intelligent robots and systems, Alberta, Canada, August 2005, pp 579–585

  3. Behrad A, Shahrokni A, Motamedi SA (2001) A robust vision-based moving target detection and tracking system. In: The proceeding of image and vision computing conference, University of Otago, Dunedin, November 2001

    Google Scholar 

  4. Black MJ, Anandan P (1996) The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput Vis Image Underst 63(1):75–104

    Article  Google Scholar 

  5. Bouguet J-Y (1999) Pyramidal implementation of the Lucas Kanade feature tracker: Description of the algorithm. Technical report, Intel Research Laboratory

  6. Censi A, Fusiello A, Roberto V (1999) Image stabilization by features tracking. In: Proceedings of the 10th international conference on image analysis and processing, Venice, Italy, September 1999, pp 665–667

  7. Cohen I, Medioni G (1999) Detecting and tracking objects in video surveillance. In: Proceedings of the IEEE computer vision and pattern recognition 99, Fort Collins, June 1999, pp 319–325

  8. Collins R, Lipton A, Fujiyoshi H, Kanade T (2001) Algorithms for cooperative multisensor surveillance. In: Proceedings of the IEEE, vol 89, pp 1456–1477, October 2001

  9. Cox IJ, Hingorani SL (1996) An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE Trans Pattern Anal Mach Intell 18(2):138–150

    Article  Google Scholar 

  10. Ess A, Schindler K, Leibe B, van Gool L (2009) Improved multi-person tracking with active occlusion handling. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  11. Foresti GL, Micheloni C (2003) A robust feature tracker for active surveillance of outdoor scenes. Electron Lett Comput Vis Image Anal 1(1):21–34

    Google Scholar 

  12. Forsyth DA, Ponce J (2003) Computer vision: a modern approach. Prentice Hall, Englewood Cliffs

    Google Scholar 

  13. Fox D (2001) KLD-sampling: Adaptive particle filter. In: Advances in neural information processing systems 14. MIT Press, Cambridge

    Google Scholar 

  14. Gutmann J-S, Schlegel C (1996) Amos: Comparison of scan matching approaches for self-localization in indoor environments. In: Proceedings of the 1st Euromicro workshop on advanced mobile robots, pp 61–67

  15. Harris C, Stephens M (1988) A combined corner and edge detector. In: Proceedings of the fourth Alvey vision conference, Manchester, pp 147–151

  16. Hue C, Le Cadre J-P, Pérez P (2001) A particle filter to track multiple objects. In: IEEE workshop on multi-object tracking, Vancouver, Canada, July 2001, pp 61–68

  17. Irani M, Anandan P (1998) Video indexing based on mosaic representation. IEEE Trans Pattern Anal Mach Intell 86(5):905–921

    Google Scholar 

  18. Irani M, Rousso R, Peleg S (1994) Recovery of ego-motion using image stabilization. In: Proceedings of the IEEE computer vision and pattern recognition, March 1994, pp 454–460

  19. Isard M, Blake A (1998) Condensation—conditional density propagation for visual tracking. Int J Comput Vis 29(1):5–28

    Article  Google Scholar 

  20. Jung B (2005) Cooperative target tracking using mobile robots. PhD thesis, University of Southern California, Los Angeles, CA

  21. Jung B, Sukhatme GS (2004) Detecting moving objects using a single camera on a mobile robot in an outdoor environment. In: International conference on intelligent autonomous systems, The Netherlands, March 2004, pp 980–987

  22. Kang J, Cohen I, Medioni G (2002) Continuous multi-views tracking using tensor voting. In: Proceedings of the IEEE workshop on motion and video computing, Orlando, Florida, December 2002, pp 181–186

  23. Kang J, Cohen I, Medioni G (2003) Continuous tracking within and across camera streams. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, Madison, Wisconsin, June 2003, pp 267–272

  24. Ke Y, Sukthankar R (2004) PCA-SIFT: A more distinctive representation for local image descriptors. In: Proceedings of the IEEE computer vision and pattern recognition, Washington, DC, June 2004, pp 506–513

  25. Kelly A (1994) A 3D state space formulation of a navigation Kalman filter for autonomous vehicles. Technical Report CMU-RI-TR-94-19, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, May 1994

  26. Lau B, Arras KO, Burgard W (2009) Multi-modal hypothesis group tracking and, group size estimation. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  27. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  28. Lu F, Milios E (1997) Robot pose estimation in unknown environments by matching 2D range scans. J Intell Robot Syst 18:249–275

    Article  Google Scholar 

  29. Luber M, Arras KO, Plagemann C, Burgard W (2009) Classifying dynamic objects: An unsupervised learning approach. Auton Robot 26(2–3):141–151

    Article  Google Scholar 

  30. Luber M, Tipaldi GD, Arras KO (2009) Spatially grounded multi-hypothesis tracking of people. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  31. Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on artificial intelligence, pp 674–697

  32. Maccormick J, Blake A (2000) A probabilistic exclusion principle for tracking multiple objects. Int J Comput Vis 39(1):57–71

    Article  MATH  Google Scholar 

  33. Murray D, Basu A (1994) Motion tracking with an active camera. IEEE Trans Pattern Anal Mach Intell 16(5):449–459

    Article  Google Scholar 

  34. Nordlund P, Uhlin T (1996) Closing the loop: Detection and pursuit of a moving object by a moving observer. Image Vis Comput 14:265–275

    Article  Google Scholar 

  35. Rasmussen C, Hager GD (2001) Probabilistic data association methods for tracking complex visual objects. IEEE Trans Pattern Anal Mach Intell 23(6):560–576

    Article  Google Scholar 

  36. Sclaroff S, Rosales R (1999) 3d trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: IEEE conference on computer vision and pattern recognition, pp 117–123

  37. Rowe S, Blake A (1996) Statistical mosaics for tracking. J Image Vis Comput 14:549–564

    Article  Google Scholar 

  38. Saripalli S, Montgomery JF, Sukhatme GS (2003) Visually-guided landing of an unmanned aerial vehicle. IEEE Trans Robot Autom 19(3):371–381

    Article  Google Scholar 

  39. Saripalli S, Roberts JM, Corke PI, Buskey G, Sukhatme GS (2003) A tale of two helicopters. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, October 2003, pp 805–810

  40. Schiele B, Andriluka M, Majer N, Roth S, Wojek C (2009) Visual people detection—different models, comparison and discussion. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  41. Se S, Barfoot T, Jasiobedzki P (2001) Visual motion estimation and terrain modeling for planetary rovers. In: IEEE international conference on robotics and automation, South Korea, May 2001, pp 2051–2058

  42. Shi J, Tomasi C (1994) Good features to track. In: Proceedings of IEEE conference on computer vision and pattern recognition, Seattle, Washington, June 1994, pp 593–600

  43. Spinello L, Triebel R, Siegwart R (2009) A trained system for multimodal perception in urban environments. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  44. Srinivasan S, Chellappa R (1997) Image stabilization and mosaicking using the overlapped basis optical flow field. In: Proceedings of IEEE international conference on image processing, October 1997, pp 356–359

  45. Stauffer C, Grimson EL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757

    Article  Google Scholar 

  46. Sullivan J, Nillius P, Carlsson S (2009) Multi-target tracking on a large scale: Experiences from football player tracking. In: Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking

  47. Thrun S, Fox D, Burgard W, Dellaert F (2001) Robust Monte Carlo localization for mobile robots. Artif Intell 128:99–141

    Article  MATH  Google Scholar 

  48. Tomasi C, Kanade T (1991) Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University, Pittsburgh, PA, April 1991

  49. Ulrich I, Borenstein J (1998) VFH+: Reliable obstacle avoidance for fast mobile robots. In: Proceeding of the IEEE international conference on robotics and automation, Leuven, Belgium, 16–21 May 1998, pp 1572–1577

  50. van Leeuwen MB, Groen FCA (2002) Motion interpretation for in-car vision systems. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. EPFL, Lausanne, pp 135–140

    Chapter  Google Scholar 

  51. Vermaak J, Doucet A, Perez P (2003) Maintaining multi-modality through mixture tracking. In: Proceedings of the 9th IEEE international conference on computer vision, pp 1110–1116

  52. Vidal R (2005) Multi-subspace methods for motion segmentation from affine, perspective and central panoramic cameras. In: Proceedings of the 2005 IEEE international conference on robotics and automation, pp 1216–1221

  53. Wang JYA, Adelson EH (2004) Representing moving images with layers. IEEE Trans Image Process 3(5):625–638

    Article  Google Scholar 

  54. Welch G, Bishop G (2006) An introduction to the Kalman filter. Technical Report 95-041, Department of Computer Science, University of North Carolina at Chapel Hill

  55. Wren CR, Azarbayejani A, Darrell T, Pentland A (1997) Pfinder: Real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785

    Article  Google Scholar 

  56. Xiao J, Shah M (2005) Accurate motion layer segmentation and matting. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition, Washington, DC, 2005, pp 698–703

  57. Yilmaz A, Shafique K, Lobo N, Li X, Olson T, Shah M (2001) Target-tracking in FLIR imagery using mean-shift and global motion compensation. In: Workshop on computer vision beyond the visible spectrum, Kauai, Hawaii, December 2001, pp 54–58

  58. Zoghlami I, Faugeras O, Deriche R (1997) Using geometric corners to build a 2D mosaic from a set of images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 420–425

Download references

Author information

Authors and Affiliations

  1. NavCom Technology, Inc., A John Deere Company, Torrance, USA

    Boyoon Jung

  2. Robotic Embedded Systems Laboratory, Center for Robotics and Embedded Systems, University of Southern California, Los Angeles, USA

    Gaurav S. Sukhatme

Authors
  1. Boyoon Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaurav S. Sukhatme
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Boyoon Jung.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jung, B., Sukhatme, G.S. Real-time Motion Tracking from a Mobile Robot. Int J of Soc Robotics 2, 63–78 (2010). https://doi.org/10.1007/s12369-009-0038-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-009-0038-y

Search

Navigation