Skip to main content
SpringerLink
Log in

Geolocation of Multiple Targets from Airborne Video Without Terrain Data

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The task of geolocating targets from airborne video is required for many applications in surveillance, law enforcement, reconnaissance, etc. The usual approaches to target geolocation involve terrain data, single target tracking, gimbal control of camera heads, altimeters, etc. The main goal of this research is to eliminate those requirements and still develop an accurate, efficient, and robust vision-based method for geolocation that can be carried out for multiple targets simultaneously. In that sense, our main contributions to the state-of-the-art in geolocation are fourfold: 1) to eliminate the requirement for gimbal control of the cameras or any particular path planning control for the UAV; 2) to perform instaneous geolocation of multiple targets; 3) to eliminate the requirements for geo-referenced terrain database (elevation maps) or for an altimeter that provides the UAV’s and target’s altitudes; and 4) to use one single camera while still maintaining good overall accuracy. In order to achieve that, the only requirements for our proposed method are: that the intrinsic parameters of the camera be known; that the on board camera be equipped with global positioning system (GPS) and inertial measurement unit (IMU); and that the height of the vehicle can be calculated using feature points extracted from the ground surrounding the image of the targets. To satisfy the first two requirements, we developed and tested a robust calibration procedure that can estimate not only the intrinsic parameters of the camera, but also the IMU-camera parameters (also know in the robotic circles as the hand-eye calibration). The last requirement was addressed using a pseudo-stereo vision technique that maximizes the distance between stereo pairs (baseline) while keeping large the number of common feature points extracted by the algorithm. The result is a method that can reach approximately 25 m of accuracy for an UAV flying at 155 m away from the target. Such performance is demonstrated by computer simulation, in-scale data using a model city, and real airborne video with ground truth.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Murphy, D., Cycon, J.: Applications for mini vtol uav for law enforcement. In: SPIE Proc. Sensors, C3I, Information, and Training Technologies for Law Enforcement. Boston, MA (1998)

  2. Z. S. S. A.-S. AE, T. U. of Crete: Survey of uav applications in civil markets. In: 9th IEEE Mediterranean Conference on Control and Automation. Dubrovnik, Croatia (2001)

  3. Pastor, J.L.C.B.E.S.E., Royo, P., Prats, X.: Project sky-eye, applying uavs to forest fire fighter support and monitoring. In: UAV 2007 Conference, Paris, France (2007)

  4. Comaniciu D., Meer P.: Mean shift:a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)

    Article  Google Scholar 

  5. Avidan, S.: Support vector tracking. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 184–191. Kauai, Hawaii (2001)

  6. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. In: IEEE Trans. Pattern Anal. Mach. Intell. Hilton Head, SC (2000)

  7. Isard, M., Blake, A.: Contour tracking by stochastic propagation of conditional density. In: Proceedings of the 4th Eurpean Conference on Computer Vision (ECCV), vol. 1, pp. 343–356. Berlin, Germany (1996)

  8. Witkin, A., Kass, M., Terzopulos, D.: Snakes:active contour models. Int. J. Comput. Vis. 321–331 (1988)

  9. Isard, M., Blake, A.: Condensation—conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)

    Article  Google Scholar 

  10. Papanikolopoulos, N.P., Khosla, P.K., Kanade, T.: Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision. IEEE Trans. Robot. Autom. 9, 14–35 (1993)

    Article  Google Scholar 

  11. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  12. Kadir, T., Brady, M.:, Sailency, scale and image description. Int. J. Comput. Vis. 45, 83–105 (2001)

    Article  MATH  Google Scholar 

  13. Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60, 63–86 (2004)

    Article  Google Scholar 

  14. Donoser, M., Bischof, H.: Efficient maximally stable extremal region(mser). In: CVPR 06, New York, NY (2006)

  15. Viola, P., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: IEEE CVPR, 2001, pp. 511–518 (2001)

  16. Ce Lui, A.T.J.S., Yuen, J., Freeman, W.T.: Sift flow:dense correspondence across difference scenes. In: European Conference on Computer Vision, Marseile, France (2008)

  17. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: ECCV 2004, vol. 3024, pp. 25–36. Prague, Czech Republic (2004)

  18. Bouguet, J.-Y.: Pyramidal Implementation of the Lucas Kande Feature Tracker. Intel Research Technical Report (2002)

  19. Lucas, B., Kanade, T.: An iterative image registration technique with an apllication to stereo vision. In Proc. of 7th International Joint Conference on Artificial Intelligence(IJCAI), pp. 674–679 (1981)

  20. Horn, B.K., Schunk, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  21. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 711–720 (1997)

    Article  Google Scholar 

  22. Shivani Agarwal, A.A., Roth, D.: Learning to detect objects in images via a sparse, part-based representation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1475–1490 (2004)

    Article  Google Scholar 

  23. Moelund, T., Granum, E.: A survey of computer vision based human motion capture. Comput. Vis. Image Underst. 81(3), 231–268 (2001)

    Article  Google Scholar 

  24. Moelund, T., Granum, E.: A survey of advances in vision based human motion capture and analysis. Comput. Vis. Image Underst. 104, 90–126(2006)

    Article  Google Scholar 

  25. Lee, M.W., Cohen, I.: Proposal maps driven mcmc for estimating human body pose in static images. In: Computer Vision Image Understanding (2004)

  26. Sidenbladh, H., Black, M.: Learning image statistics of people in images and video. Int. J. Comput. Vis. (2003)

  27. Chen, K.-W., Lai, C.-C., Hung, Y.-P., Chen, C.-S.: An adaptive learning methods for target tracking across multiple cameras. In: IEEE CVPR, 2008 (2008)

  28. Minwoo Park, Y.L., Collins, R.T.: Efficient mean shift belief propagation for vision tracking. In: IEEE CVPR, 2008 (2008)

  29. Bibby, C., Reid, I.: Robust real-time tracking using pixel-wise posteriors. In: ECCV 2008 (2008)

  30. Bohyung Han, S.-w.J., Davis, L.S.: Probabilistic fusion tracking using mixture kernel-based bayesian filtering. In: IEEE International Conference on Computer Vision 2007 (2007)

  31. Heikkila, J., Silven, O.: A real-time system for monitoring of cyclists and pedestrians. In Second IEEE Workshop on Visual Surveillance, pp. 74–81. Fort Collins, Colorado (1999)

  32. Tian, M.L.Y.-L. Hampapur, A.: Robust and efficient foreground analysis for real-time video surveillance. In: IEEE Conference on Computer Vision and Pattern Recognition (2005)

  33. Ya-Ming Wang, L.C. Huang, W.-Q.: 3-d human motion estimation using regularization with 2-d feature point tracking. In: IEEE International Conference on Machine Learning and Cybernetics 2003 (2003)

  34. Torr, P.H., Zisserman, A.: Feature based methods for structure and motion estimation. In IEEE International Conference on Computer Vision 1999 (1999)

  35. Xiaolu, L., Ogawara, K., Ikeuchi, K.: Marker-less human motion estimation using articulated deformable model. In: IEEE International Conference on Robotics and Automation, pp. 46–51 (2007)

  36. DeSouza, G.N., Kak, A.C.: A subsumptive, hierarchical, and distributed vision-based architecture for smart robotics. IEEE Trans. Syst. Man Cybern., Part B 34, 1988–2002 (2004)

    Article  Google Scholar 

  37. Yasushi Iwatani, K.W., Hashimoto, K.: Visual tracking with occlusion handling for visual servo control. In: IEEE International Conference on Robotics and Automation. Pasadena, CA (2008)

  38. DeSouza, G.N., Kak, A.C.: Vision for mobile robot navigation: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 237–267 (2002)

    Article  Google Scholar 

  39. Collins, R.: Mean-shift blob tracking through scale space. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 234–240. Madison, Wisconsin (2003)

  40. Changjjang Yang, R.D., Davis, L.: Efficient Mean-Shift Tracking via a New Similarity Measure. Sandiego, USA (2005)

  41. Liu, T., Chen, H.: Real-time tracking using trust region methods. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 397–402 (2004)

    Article  Google Scholar 

  42. Yoon, Y., DeSouza, G.N., Kak, A.C.: Real-time tracking and pose estimation for industrial objects using geometric features. In: Proceedings of 2003 IEEE International Conference on Robotics and Automation, Taiwan (2003)

  43. Dornaika, F., and Horaud, R.: Simultaneous robot-world and hand-eye calibration. IEEE Trans. Robot. Autom. 14, 617–622 (1998)

    Article  Google Scholar 

  44. Horaud, R., Dornaika, F.: Hand-eye calibration. Int. J. Rob. Res. 14, 195–210 (1995)

    Article  Google Scholar 

  45. DeSouza, G.N., Jones, A.H., Kak, A.C.: An world-independent approach for the calibration of mobile robotics active stereo heads. In: Proceedings of 2002 IEEE International Conference on Robotics and Automation, Washington DC, USA (2002)

  46. Shiu, Y.C., Ahmad, S.: Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB. IEEE Trans. Robot. Autom. 5, 16–29 (1989)

    Article  Google Scholar 

  47. Tsai, R.Y., Lenz, R.K.: A new technique for fully autonomous and efficient 3-d robotics hand/eye calibration. IEEE Trans. Robot. Autom. 5, 345–358 (1989)

    Article  Google Scholar 

  48. Hirsh, R., DeSouza, G.N., Kak, A.C.: An iterative approach to the hand-eye and base-world calibration problem. In: Proceedings of 2001 IEEE International Conference on Robotics and Automation, vol. 1, pp. 2171–2176. Seoul, Korea (2001)

  49. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)

    Article  Google Scholar 

  50. Weng, J., Cohen, P., Herniou, M.: Camera calibration with distortion models and accuracy evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 14, 965–980 (1992)

    Article  Google Scholar 

  51. Lobo, J., Dias, J.: Relative pose calibration between visual and inertial sensors. Int. J. Rob. Res. 26(6), 561–575 (2007)

    Article  Google Scholar 

  52. Kelly, J., Sukhatme, G.: Fast relative pose calibration for visual and inertial sensors. In: 11th International Symposium on Experimental Robotics 2008. Greece, Athens (2008)

  53. Lobo, J., and Dias, J.: Relative pose calibration between visual and inertial sensors. In: InterVis, Barcelona, Spain (2005)

  54. Mirzaei, F.M., Roumeliotis, S.I.: A kalman filter-based algorithm for imu-camera calibration: observability analysis and performance evaluation. IEEE Transactions on Robotics 24(5), 1143–1155 (2008)

    Article  Google Scholar 

  55. Redding, J., McLain, T.W., Beard, R.W., Taylor, C.: Vision-based target localization from a fixed-wing miniature air vehicle. In: Proceedings of 2006 American Control Conference, pp. 2862–2867. Minneapolis, MN, USA (2006)

  56. Barber, D.B., Redding, J., McLain, T.W., Beard, R.W., Taylor, C.: Vision-based target geo-location using a fixed-wing miniature air vehicle. J. Intell. Robot. Syst. 47, 361–382 (2006)

    Article  Google Scholar 

  57. Monda, M.J., Woolsey, C.A., Reddy, C.K.: Ground target localization and tracking in a riverine environment from a uav with a gimbaled camera. In: Proceedings of AIAA Guidance, Navigation and Control Conference, pp. 6747–6750. Hilton Head, SC (2007)

  58. DeLima, P., York, G., Pack, D.: Localization of ground targets using a flying sensor network. In: Proceedings of the IEEE international Conference on Sensor Networks, Ubiquitous, and Trusworthy Computing, pp. 194–199, Taichung, Taiwan (2006)

  59. Whitacre, W., Campbell, M., Wheeler, M., Stevenson, D.: Flight results from tracking ground targets using seascan uavs with gimballing cameras. In Proceedings of 2007 American Control Conference, New York, NY, USA (2007)

  60. Whang, I.H., Dobrokhodov, V.N., Kaminer, I.I., Jones, K.D.: On vision-based target tracking and range estimation for small uavs. In: Proceedings of AIAA Guidance, Navigation and Control Conference, San Franscisco, CA, USA (2005)

  61. Dobrokhodov, V.N., Kaminer, I.I., Jones, K.D., Ghabcheloo, R.: Vision-based tracking and motion estimation for moving targets using small uavs. In: Proceedings of 2006 American Control Conference, Minneapolis, MN, USA (2006)

  62. Wolfgang, B.: Generating one-meter terrain data for tactical simulations. Military Intelligence Professional Bulletin, 28, 36 (2002)

    Google Scholar 

  63. Rysdyk, R.: Uav path following for constant line-of-sight. In: Proceedings of the 2nd AIAA Unmanned Unlimited Systems, Technologies and Operations Aerospace, Land and Sea conference, San Diego, California, USA (2003)

  64. Shi, J., and Tomasi, C.: Good features to track. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600. Seattle, USA (1994)

  65. Tomasi, C., and Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132 (1991)

  66. Bouguet, J.Y.: Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calibdoc (2006)

  67. Faugeras, O.D.: Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press (1993)

  68. Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filter for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. Signal Process. 50, 174–188 (2002)

    Article  Google Scholar 

  69. Kalman, R.: A new approach to linear filtering and prediction problems. J. Basic Eng. 35–45 (1960)

  70. Doucet, G.N.J.A., De Freitas N., eds.: Sequential Monte Carlo Methods in Practice. Springer (2001)

  71. Gustafsson, N.B.U.F.J.J.R.K.F., Gunnarsson, F., Nordlund, P.J.: .Particle filter for positioning, navigation and tracking. IEEE Trans. Signal Process. 50, 425–437 (2002)

    Article  Google Scholar 

  72. Kollsman: Kollsman Servoed Altimeter with Automatic Pressure Stanby. Type A41322 10 015 (AAU34/A).

Download references

Author information

Authors and Affiliations

  1. University of Missouri, Missouri, Columbia

    Kyung Min Han & Guilherme N. DeSouza

Authors
  1. Kyung Min Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guilherme N. DeSouza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung Min Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, K.M., DeSouza, G.N. Geolocation of Multiple Targets from Airborne Video Without Terrain Data. J Intell Robot Syst 62, 159–183 (2011). https://doi.org/10.1007/s10846-010-9442-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-010-9442-7

Keywords

Search

Navigation