Abstract
We present an algorithm for underwater robots to visually detect and track human motion. Our objective is to enable human–robot interaction by allowing a robot to follow behind a human moving in (up to) six degrees of freedom. In particular, we have developed a system to allow a robot to detect, track and follow a scuba diver by using frequency-domain detection of biological motion patterns. The motion of biological entities is characterized by combinations of periodic motions which are inherently distinctive. This is especially true of human swimmers. By using the frequency-space response of spatial signals over a number of video frames, we attempt to identify signatures pertaining to biological motion. This technique is applied to track scuba divers in underwater domains, typically with the robot swimming behind the diver. The algorithm is able to detect a range of motions, which includes motion directly away from or towards the camera. Once detected, the motion of the diver relative to the vehicle is then tracked using an Unscented Kalman Filter, an approach for non-linear estimation. The efficiency of our approach makes it attractive for real-time applications on-board our underwater vehicle, and in future applications we intend to track scuba divers in real-time with the robot. The paper presents an algorithmic overview of our approach, together with experimental evaluation based on underwater video footage.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersen, B. G. (1969). Measurement of scuba diver performance in open ocean environment. In American Society of Mechanical Engineers, Papers (pp. 8–16).
Bainbridge, R. (1958). The speed of swimming of fish as related to size and to the frequency and amplitude of the tail beat. Journal of Experimental Biology, 35(1), 109–133.
Begg, R., & Kamruzzaman, J. (2005). A machine learning approach for automated recognition of movement patterns using basic, kinetic and kinematic gait data. Journal of Biomechanics, 38(3), 401–408.
Bhattcharyya, A. (1943). On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of Calcutta Mathematical Society, 35, 99–110.
Bruce, B. D., Stevens, J. D., & Malcolm, H. (2006). Movements and swimming behaviour of white sharks (Carcharodon carcharias) in Australian waters. Marine Biology, 150(2), 161–172.
Coral Reef Conservation Act. (2003). Coral reef conservation act of 2000. P.L. 106–562, 16 U.S.C. 6401 et seq.
Duda, R. O., Hart, P. E., & Stork, D. G. (2000). Pattern classification (2nd ed.). Wiley. ISBN 0471056693.
Dudek, G., Sattar, J., & Xu, A. (2007). A visual language for robot control and programming: A human-interface study. In Proceedings of the International Conference on Robotics and Automation ICRA, Rome, Italy (pp. 2507–2513).
Fleet, D. J., & Jepson, A. D. (1990). Computation of component velocity from local phase information. International Journal of Computer Vision, 5(1), 77–104.
Freeman, W. T., & Adelson, E. H. (1991). The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9), 891–906.
Giguere, P., Girdhar, Y., & Dudek, G. (2013). Wide-speed autopilot system for a swimming hexapod robot. In 2013 International Conference on Computer and Robot Vision (pp. 9–15).
Isard, M., & Blake, A. (1998). Condensation—conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5–28.
Julier, S. J., & Uhlmann, J. K. (2004). Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3), 401–422.
Land, M. F. (1990). Optics of the eyes of marine animals. In P. J. Herring, A. K. Campbell, M. Whitfield, & L. Maddock (Eds.), Light and life in the sea (pp. 149–166). Cambridge: Cambridge University Press.
Leigh, J. R. (2004). Control theory. Institution of Electrical Engineers. ISBN 0863413390.
Nixon, M. S., Tan, T. N., & Chellappa, R. (2005). Human identification based on gait., The Kluwer international series on biometrics New York: Springer.
Niyogi, S. A., & Adelson, E. H. (1994). Analyzing and recognizing walking figures in XYT. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 469–474).
Oppenheim, A. V., Willsky, A. S., & Hamid Nawab, S. (1996). Signals & systems (2nd ed.). Upper Saddle River: Prentice-Hall Inc. ISBN 0-13-814757-4.
Pendergast, D. R., Tedesco, M., Nawrocki, D. M., & Fisher, N. M. (1996). Energetics of underwater swimming with SCUBA. Medicine & Science in Sports & Exercise, 28(5), 573. ISSN 0195-9131.
Rashid, R. F. (1980). Toward a system for the interpretation of moving light display. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(6), 574–581.
Sattar, J., & Dudek, G. (2006). On the performance of color tracking algorithms for underwater robots under varying lighting and visibility. In Proceedings of the IEEE International Conference on Robotics and Automation ICRA, Orlando, Florida (pp. 3550–3555).
Sattar, J., & Dudek, G. (2007). Where is your dive buddy: Tracking humans underwater using spatio-temporal features. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Diego, California, USA (pp. 3654–3659).
Sattar, J., & Dudek, G. (2009a). A vision-based control and interaction framework for a legged underwater robot. In Proceedings of the Sixth Canadian Conference on Robot Vision (CRV), Kelowna, British Columbia (pp. 329–336).
Sattar, J., & Dudek, G. (2009b). Robust servo-control for underwater robots using banks of visual filters. In Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, Kobe, Japan (pp. 3583–3588).
Sattar, J., & Dudek, G. (2011). Towards quantitative modeling of task confirmations in human–robot dialog. In Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, Shanghai, China (pp. 1957–1963).
Sattar, J., Giguère, P., Dudek, G., & Prahacs, C. (2005). A visual servoing system for an aquatic swimming robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Edmonton, Alberta, Canada (pp. 1483–1488).
Sidenbladh, H., & Black, M. J. (2003). Learning the statistics of people in images and video. International Journal of Computer Vision, 54(1–3), 181–207. ISSN 0920-5691.
Sidenbladh, H., Black, M. J., & Fleet, D. J. (2000). Stochastic tracking of 3D human figures using 2D image motion. In Proceedings of the European Conference on Computer Vision (vol. 2, pp. 702–718).
Wannier, T., Bastiaanse, C., Colombo, G., & Dietz, V. (2001). Arm to leg coordination in humans during walking, creeping and swimming activities. Experimental Brain Research, 141(3), 375–379. doi:10.1007/s002210100875. ISSN 0014-4819, PMID: 11715082.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Sattar, J., Dudek, G. Visual identification of biological motion for underwater human–robot interaction. Auton Robot 42, 111–124 (2018). https://doi.org/10.1007/s10514-017-9644-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-017-9644-y