Skip to main content

Advertisement

SpringerLink
Log in

A Time-Delay Control Approach for a Stereo Vision Based Human-Machine Interaction System

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper, an approach to control a 6-DoF stereo camera for the purpose of actively tracking the face of a human observer in the context of Human-Robot Interaction (HRI) is proposed. The main objective in the presented work is to cope with the critical time-delay introduced by the computer vision algorithms used to acquire the feedback variable within the control system. In the studied HRI architecture, the feedback variable is represented by the 3D position of a human subject. We proposed a predictive control method which is able to handle the high time-delay inserted by the vision elements into the control system of the stereo camera. Also, along with the predictive control approach, a novel 3D nose detection algorithm is suggested for the computation of the feedback variable. The performance of the implemented platform is given through experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aloimonos, J., Weiss, I., Bandyopadhyay, A.: Active vision. Int. J. Comput. Vis. 1(4), 333–356 (1988)

    Article  Google Scholar 

  2. Ayasun, S., Gelen, A.: Stability analysis of a generator excitation control system with time delays. Electr. Eng. 91(1), 347–355 (2010)

    Article  Google Scholar 

  3. Bohren, J., Rusu, R.B., Jones, G., Marder-Eppstein, E., Pantofaru, C., Wise, M., Moesenlechner, L., Meeussen, W., Holzer, S.: Towards autonomous robotic butlers: lessons learned with the PR2. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Shanghai, China (2011)

  4. Bolme, D.S., Draper, B.A., Beveridge, J.R.: Average of synthetic exact filters. In: International Conference on Computer Vision and Pattern Recognition CVPR, pp. 2105–2112 (2009)

  5. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media, Sebastopol, Canada (2008)

  6. Brooks, A., Arkin, R.: Behavioral overlays for non-verbal communication expression on a humanoid robot. Auton. Robot. 22(1), 55–74 (2007)

    Article  Google Scholar 

  7. Bruce, V., Young, A.: In the Eye of the Beholder: The Science of Face Perception. Oxford University Press, Oxford (2000)

    Google Scholar 

  8. Comnac, V., Coman, S., Boldişor, C.: Sisteme Liniare Continue. Universităţii Transilvania, Brasov, Romania (2009)

    Google Scholar 

  9. Corke, P.: Visual Control of Robots : High-performance Visual Servoing. Research Studies Press Ltd, Taunton, England (1996)

    Google Scholar 

  10. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition - CVPR 2005, vol. 1, pp. 886–893 (2005). doi:10.1109/CVPR.2005.177

  11. De Paor, A.M., O’Malley, M.: Controllers of Ziegler-Nichols type for unstable process with time delay. Int. J. Control. 49(4), 1273–1284 (1989)

    Article  MATH  Google Scholar 

  12. Dorf, R., Bishop, R.: Modern Control Systems. Prentice-Hall, Inc., New Jersey (2010)

    Google Scholar 

  13. Ferreira, J.F., Castelo-Branco, M., Dias, J.: A hierarchical Bayesian framework for multimodal active perception. Adaptive Behaviour 20(3), 172–190 (2012)

    Article  Google Scholar 

  14. Franklin, G.F., Powell, D.J., Workman, M.L.: Digital Control of Dynamic Systems, 3rd edn. Prentice-Hall, Inc., Boston (1997)

    Google Scholar 

  15. Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gaudette, D., Miller, D.: When is the achievable discrete-time delay margin nonzero? IEEE Trans. Autom. Control 56(4), 886–890 (2011)

    Article  MathSciNet  Google Scholar 

  17. Goodwin, G., Graebe, S., Salgado, M.: Control System Design. Prentice-Hall, Inc, New Jersey (2001)

    Google Scholar 

  18. Grigorescu, S., Macesanu, G., Cocias, T., Puiu, D., Moldoveanu, F.: Robust camera pose and scene structure analysis for service robotics. Robot. Auton. Syst. 59(11), 899–909 (2011)

    Article  Google Scholar 

  19. Grigorescu, S.M., Lth, T., Fragkopoulos, C., Cyriacks, M., Grser, A.: A bci-controlled robotic assistant for quadriplegic people in domestic and professional life. Robotica 30, 419–431 (2012). doi:10.1017/S0263574711000737

    Article  Google Scholar 

  20. Huang, J., Lewis, F.L., Liu, K.: A neural net predictive control for telerobots with time delay. J. Intell. Robot. Syst. 29, 1–25 (2000)

    Article  MATH  Google Scholar 

  21. Kumar, S., Micheloni, C., Piciarelli, C.: Stereo localization using dual ptz cameras. In: International Conference on Computer Analysis of Images and Patterns, vol. 5702/2009, pp. 1061–1069. Munster, GE (2009)

  22. Macesanu, G., Ferreira, J.F., Dias, J.: A Bayesian hierarchy for gaze following. In: The 5th Inter. Conf. on Cognitive Systems. TU Vienna, Austria (2012)

  23. Macesanu, G., Grigorescu, S., Comnac, V.: Time-delay analysis of a robotic stereo active vision system. In: Proc. of the 15th Inter. Conf. on System Theory, Control, and Computing, pp. 1–6. Sinaia, Romania (2011)

  24. Macesanu, G., Grigorescu, S., Ferreira, J.F., Dias, J., Moldoveanu, F.: Real Time facial features tracking using an active vision system. In: Proc. of the 13th Inter. Conf. on Optimization of Electrical and Electronic Equipment, pp. 1493–1498. Brasov, Romania (2012)

  25. Michiels, W., Vyhlidal, T., Zitek, P.: Control design for time-delay systems based on quasi-direct pole placement. J. Process Control 20(3), 337–343 (2010)

    Article  Google Scholar 

  26. Mozos, O.M., Marton, Z.C., Beetz, M.: Furniture models learned from the WWW—using web catalogs to locate and categorize unknown furniture pieces in 3D laser scans. Robot. Autom. Mag. 18(2), 22–32 (2011)

    Article  Google Scholar 

  27. Nickel, K., Stiefelhagen, R.: Visual recognition of pointing gestures for human-robot interaction. Image Vis. Comput. 25(12), 1875–1884 (2007)

    Article  Google Scholar 

  28. Ojala, T., Pietikainen, M., Harwood, D.: Performance evaluation of texture measures with classification based on kullback discrimination of distributions. In: International Conference on Pattern Recognition, vol. 1, pp. 582–585. Jerusalem, Israel (1994)

  29. Park, C.B., Lee, S.W.: Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter. Image Vis. Comput. 29(1), 51–63 (2011)

    Article  Google Scholar 

  30. Pateraki, M., Baltzakis, H., Kondaxakis, P., Trahanias, P.: Tracking of facial features to support human-robot interaction. In: IEEE International Conference on Robotics and Automation, 2009 - ICRA ’09, pp. 3755–3760 (2009). doi:10.1109/ROBOT.2009.5152600

  31. Siciliano, B., Khatib, O.: Springer Handbook of Robotics. Springer, Berlin, Germany (2008)

    Book  MATH  Google Scholar 

  32. Sidner, C.L., Lee, C., Kidd, C.D., Lesh, N., Rich, C.: Explorations in engagement for humans and robots. Artif. Intell. 166(2), 140–164 (2005)

    Article  Google Scholar 

  33. Silva, G., Aniruddha, D., Bhattacharyya, S.P.: PID Controllers for Time-Delay Systems, 1st edn. Birkhauser Boston, New York (2004)

    Google Scholar 

  34. Silva, G., Datta, A., Bhattacharyya, S.: Determination of stabilizing feedback gains for second-order systems with time delay. In: Proc. of the 2001 American Control Conf., pp. 4658–4663. Arlington, Virginia (2001)

  35. Skodras, E., Fakotakis, N.: An unconstrained method for lip detection in color images. In: Proc. of the 2011 IEEE Inter. Conf. on Acoustics, Speech and Signal Processing, pp.  1013–1016 (2011)

  36. Sommerlade, E., Benfold, B., Reid, I.: Gaze directed camera control for face image acquisition. In: IEEE International Conference on Robotics and Automation, pp. 4227–4233 (2011)

  37. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. of the 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518. Kauai, USA (2001)

  38. Wan, D., Zhou, J.: Stereo vision using two ptz cameras. Comp. Vision Image Underst. 112(2), 184–194 (2008). doi:10.1016/j.cviu.2008.02.005. http://www.sciencedirect.com/science/article/pii/S1077314208000313

    Article  MathSciNet  Google Scholar 

  39. Wang, Q.G., Zhang, Z., Astrom, K.J., Chek, L.S.: Guaranteed dominant pole placement with PID controllers. J. Process Control 19(2), 349–352 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automation, Transilvania University of Brasov, Mihai Viteazu 5, 500174, Brasov, Romania

    Gigel Macesanu, Vasile Comnac, Florin Moldoveanu & Sorin M. Grigorescu

Authors
  1. Gigel Macesanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasile Comnac
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florin Moldoveanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sorin M. Grigorescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gigel Macesanu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Macesanu, G., Comnac, V., Moldoveanu, F. et al. A Time-Delay Control Approach for a Stereo Vision Based Human-Machine Interaction System. J Intell Robot Syst 76, 297–313 (2014). https://doi.org/10.1007/s10846-013-9994-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-013-9994-4

Keywords

Search

Navigation