Skip to main content
SpringerLink
Log in

Towards a Task-Aware Proactive Sociable Robot Based on Multi-state Perspective-Taking

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Robots are expected to cooperate with humans in day-to-day interaction. One aspect of such cooperation is behaving proactively. In this paper we will enable our robots, equipped with visuo-spatial perspective-taking capabilities, to behave proactively based on reasoning ‘where’ its human partner might perform a particular task with different effort levels. For this, the robot analyzes the agents’ abilities not only from the current state but also from a set of different states the agent might attain.

Depending on the task and the situation, the robot exhibits different types of proactive behaviors, such as, reaching out, suggesting a solution and providing clues by head movement, for two different tasks performed by the human partner: give and make accessible. These proactive behaviors are intended to be informative to reduce confusion of the human partner, to communicate the robot’s ability and intention and to guide the partner for better cooperation.

We have validated the behaviors by user studies, which suggest that such proactive behaviors reduce the ‘confusion’ and ‘effort’ of the users. Further, the participants reported the robot to be more ‘supportive and aware’ compared to the situations where the robot was non-proactive.

Such proactive behaviors could enrich multi-modal interaction and cooperation capabilities of the robot as well as help in developing more complex socially expected and accepted behaviors in the human centered environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Alili S, Alami R, Montreuil V (2009) A task planner for an autonomous social robot. In: Asama H, Kurokawa H, Ota J, Sekiyama K (eds) Distributed autonomous robotic systems, vol 8. Springer, Berlin, pp 335–344

    Google Scholar 

  2. Berlin M, Gray J, Thomaz AI, Breazeal C (2006) Perspective taking: an organizing principle for learning in human-robot interaction. In: Proceedings of the 21st national conference on artificial intelligence (AAAI’06), vol 2. AAAI Press, Menlo Park, pp 1444–1450

    Google Scholar 

  3. Breazeal C (2009) Role of expressive behaviour for robots that learn from people. Phil Trans R Soc B, Biol Sci 364:3527–3538

    Article  Google Scholar 

  4. Broquère X, Sidobre D, Nguyen K (2010) From motion planning to trajectory control with bounded jerk for service manipulator robots. In: IEEE international conference on robotics and automation, pp 4505–4510

    Google Scholar 

  5. Buss M, Carton D, Gonsior B, Kuehnlenz K, Landsiedel C, Mitsou N, de Nijs R, Zlotowski J, Sosnowski S, Strasser E, Tscheligi M, Weiss A, Wollherr D (2011) Towards proactive human-robot interaction in human environments. In: 2nd international conference on cognitive infocommunications (CogInfoCom), 2011, pp 1–6

    Google Scholar 

  6. Cakmak M, Srinivasa S, Lee MK, Forlizzi J, Kiesler S (2011) Human preferences for robot-human hand-over configurations. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), 2011, pp 1986–1993

    Google Scholar 

  7. Carlson T, Demiris Y (2008) Human-wheelchair collaboration through prediction of intention and adaptive assistance. In: IEEE international conference on robotics and automation (ICRA), pp 3926–3931

    Google Scholar 

  8. Cesta A, Cortellessa G, Pecora F, Rasconi R (2007) Supporting interaction in the robocare intelligent assistive environment. In: Proceedings of AAAI Spring symposium on interaction challenges for intelligent assistants. AAAI Press, Menlo Park, pp 18–25

    Google Scholar 

  9. Chella A, Dindo H, Infantino I (2006) A cognitive framework for imitation learning. Robot Auton Syst 54(5):403–408

    Article  Google Scholar 

  10. Choi HJ, Mark LS (2004) Scaling affordances for human reach actions. Hum Mov Sci 23(6):785–806

    Article  Google Scholar 

  11. Clark HH (2003) Pointing and placing. In: Kita S (ed) Pointing: where language, culture, and cognition meet. Erlbaum, Hillsdale, pp 243–268

    Google Scholar 

  12. Clodic A, Cao H, Alili S, Montreuil V, Alami R, Chatila R (2009) Shary: a supervision system adapted to human-robot interaction. In: Khatib O, Kumar V, Pappas G (eds) Experimental robotics. Springer tracts in advanced robotics, vol 54. Springer, Berlin, pp 229–238

    Chapter  Google Scholar 

  13. Cramer HS, Kemper NA, Amin A, Evers V (2009) The effects of robot touch and proactive behaviour on perceptions of human-robot interactions. In: Proceedings of the 4th ACM/IEEE international conference on human robot interaction (HRI), pp 275–276

    Chapter  Google Scholar 

  14. Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Philos Trans R Soc Lond B, Biol Sci 362(1480):679–704

    Article  Google Scholar 

  15. Dehais F, Sisbot EA, Alami R, Causse M (2011) Physiological and subjective evaluation of a human-robot object hand-over task. Appl Ergon 42(6):785–792

    Article  Google Scholar 

  16. Duong T, Bui H, Phung D, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-Markov model. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 838–845

    Google Scholar 

  17. Eimler SC, Krämer NC, Patten AMVD (2010) Prerequisites for human-agent- and human-robot interaction: towards an integrated theory. In: Trappl R (ed) European meetings on cybernetics and systems research (EMCSR)

    Google Scholar 

  18. Flanagan JR, Johansson RS (2003) Action plans used in action observation. Nature 6950:769–771. doi:10.1038/nature01861

    Article  Google Scholar 

  19. Gardner DL, Mark LS, Ward JA, Edkins H (2001) How do task characteristics affect the transitions between seated and standing reaches? Ecol Psychol 13(4):245–274

    Article  Google Scholar 

  20. Gharbi M, Cortes J, Simeon T (2008) A sampling-based path planner for dual-arm manipulation. In: IEEE/ASME international conference on advanced intelligent mechatronics, pp 383–388

    Google Scholar 

  21. Gredeback G, Kochukhova O (2010) Goal anticipation during action observation is influenced by synonymous action capabilities, a puzzling developmental study. Exp Brain Res 202:493–497

    Article  Google Scholar 

  22. Kozima HY (2001) In search of otogenetic prerequisites for embodied social intelligence. In: Proceedings of the workshop on emergence and development on embodied cognition (EDEC-2001); international conference on cognitive science (ICCS-2001), pp 30–34

    Google Scholar 

  23. Hoffman G (2010) Anticipation in human-robot interaction. In: Proceedings of AAAI Spring symposium

    Google Scholar 

  24. Holthaus P, Lutkebohle I, Hanheide M, Wachsmuth S (2010) Can I help you? In: Ge S, Li H, Cabibihan JJ, Tan Y (eds) Social robotics. Lecture notes in computer science, vol 6414. Springer, Berlin, pp 325–334

    Chapter  Google Scholar 

  25. Holthaus P, Pitsch K, Wachsmuth S (2011) How can I help? Int J Soc Robot 3:383–393. doi:10.1007/s12369-011-0108-9

    Article  Google Scholar 

  26. Huber M, Knoll A, Brandt T, Glasauer S (2009) Handing over a cube. In: Basic and clinical aspects of vertigo and dizziness. Annals of the New York Academy of Sciences, vol 1164, pp 380–382

    Google Scholar 

  27. Iio T, Shiomi M, Shinozawa K, Akimoto T, Shimohara K, Hagita N (2011) Investigating entrainment of people’s pointing gestures by robot’s gestures using a WOz method. Int J Soc Robot 3:405–414. doi:10.1007/s12369-011-0112-0

    Article  Google Scholar 

  28. Imai M, Ono T, Ishiguro H (2003) Physical relation and expression: joint attention for human-robot interaction. IEEE Trans Ind Electron 50(4):636–643

    Article  Google Scholar 

  29. Jordan JS, Hunsinger M (2008) Learned patterns of action effect anticipation contribute to the spatial displacement of continuously moving stimuli. Journal of Experimental Psychology: Human Perception and Performance, 113–124

  30. Kemp C, Edsinger A, Torres-Jara E (2007) Challenges for robot manipulation in human environments. IEEE Robot Autom Mag 14(1):20–29

    Article  Google Scholar 

  31. Kennedy WG, Bugajska MD, Harrison AM, Trafton JG (2009) “like-me” simulation as an effective and cognitively plausible basis for social robotics. Int J Soc Robot 1(2):181–194

    Article  Google Scholar 

  32. Khatib O, Demircan E, Sapio VD, Sentis L, Besier T, Delp S (2009) Robotics-based synthesis of human motion. J Physiol 103:211–219

    Google Scholar 

  33. Kockler H, Scheef L, Tepest R, David N, Bewernick B, Newen A, Schild H, May M, Vogeley K (2010) Visuospatial perspective taking in a dynamic environment: perceiving moving objects from a first-person-perspective induces a disposition to act. Conscious Cogn 19(3):690–701

    Article  Google Scholar 

  34. Kwon WY, Suh IH (2010) Probabilistic temporal prediction for proactive action selection. In: IROS workshop on probabilistic graphical models in robotics, GraphBot

    Google Scholar 

  35. Kwon WY, Suh IH (2011) Towards proactive assistant robots for human assembly tasks. In: Proceedings of the 6th international conference on human-robot interaction, HRI’11. ACM, New York, pp 175–176

    Google Scholar 

  36. L’Abbate M (2007) Modelling proactive behaviour of conversational interfaces. Ph.D. thesis, TU Darmstadt, Darmstadt

  37. Lemaignan S, Ros R, Sisbot E, Alami R, Beetz M (2012) Grounding the interaction: anchoring situated discourse in everyday human-robot interaction. Int J Soc Robot 4:181–199

    Article  Google Scholar 

  38. Li J, Chignell M (2011) Communication of emotion in social robots through simple head and arm movements. Int J Soc Robot 3:125–142. doi:10.1007/s12369-010-0071-x

    Article  Google Scholar 

  39. Lohse M (2010) Investigating the influence of situations and expectations on user behavior-empirical analyses in human-robot interaction. Ph.D. thesis, Universität Bielefeld

  40. Louwerse M, Bangerter A (2005) Focusing attention with deictic gestures and linguistic expressions. In: Proceedings of the 27th annual meeting of the cognitive science society

    Google Scholar 

  41. Marin-Urias L, Sisbot E, Pandey A, Tadakuma R, Alami R (2009) Towards shared attention through geometric reasoning for human robot interaction. In: 9th IEEE-RAS international conference on humanoid robots (Humanoids), pp 331–336

    Google Scholar 

  42. Nicolescu M, Mataric M (2001) Learning and interacting in human-robot domains. IEEE Trans Syst Man Cybern, Part A, Syst Hum 31(5):419–430

    Article  Google Scholar 

  43. Pandey A, Alami R (2010) Mightability maps: a perceptual level decisional framework for co-operative and competitive human-robot interaction. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 5842–5848

    Google Scholar 

  44. Pandey AK, Alami R (2011) Towards task understanding through multi-state visuo-spatial perspective taking for human-robot interaction. In: IJCAI workshop on agents learning interactively from human teachers (ALIHT-IJCAI)

    Google Scholar 

  45. Pardowitz M, Dillmann R (2007) Towards life-long learning in household robots: the piagetian approach. In: IEEE 6th international conference on development and learning (ICDL), pp 88–93

    Google Scholar 

  46. Ros R, Lemaignan S, Sisbot E, Alami R, Steinwender J, Hamann K, Warneken F (2010) Which one? Grounding the referent based on efficient human-robot interaction. In: IEEE RO-MAN, 2010, pp 570–575

    Google Scholar 

  47. Salichs M, Barber R, Khamis A, Malfaz M, Gorostiza J, Pacheco R, Rivas R, Corrales A, Delgado E, Garcia D (2006) Maggie: a robotic platform for human-robot social interaction. In: IEEE conference on robotics, automation and mechatronics, pp 1–7

    Google Scholar 

  48. Sapio V, Warren J, Khatib O (2006) Predicting reaching postures using a kinematically constrained shoulder model. In: Lennarcic J, Roth B (eds) Advances in robot kinematics. Springer, Dordrecht, pp 209–218

    Chapter  Google Scholar 

  49. Saut JP, Sahbani A, Perdereau V (2011) A generic motion planner for robot multi-fingered manipulation. Adv Robot 25(1–2):23–46

    Article  Google Scholar 

  50. Saut JP, Sidobre D (2012) Efficient models for grasp planning with a multi-fingered hand. Robot Auton Syst 60(3):347–357

    Article  Google Scholar 

  51. Scassellati B (2002) Theory of mind for a humanoid robot. Auton Robots 12:13–24

    Article  MATH  Google Scholar 

  52. Schmid A, Weede O, Worn H (2007) Proactive robot task selection given a human intention estimate. In: 16th IEEE international symposium on robot and human interactive communication (RO-MAN), pp 726–731

    Google Scholar 

  53. Schrempf O, Hanebeck U, Schmid A, Worn H (2005) A novel approach to proactive human-robot cooperation. In: IEEE international symposium on robot and human interactive communication (Ro-MAN), pp 555–560

    Google Scholar 

  54. Sciutti A, Bisio A, Nori F, Metta G, Fadiga L, Pozzo T, Sandini G (2012) Measuring human-robot interaction through motor resonance. Int J Soc Robot 4:223–234. doi:10.1007/s12369-012-0143-1

    Article  Google Scholar 

  55. Sebanz N, Knoblich G (2009) Prediction in joint action: what, when, and where. Top Cogn Sci 1(2):353–367

    Article  Google Scholar 

  56. Simeon T, Laumond PJ, Lamiraux F (2001) Move3d: a generic platform for path planning. In: 4th international symposium on assembly and task planning, pp 25–30

    Google Scholar 

  57. Sisbot EA, Alami R (2012) A human-aware manipulation planner. IEEE Trans Robot 28(5):1045–1057

    Article  Google Scholar 

  58. Taha T, Miro JV, Dissanayake G (2008) Pomdp-based long-term user intention prediction for wheelchair navigation. In: ICRA, pp 3920–3925

    Google Scholar 

  59. Tomasello M, Carpenter M, Call J, Behne T, Moll H (2005) Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci 28(5):675–691

    Google Scholar 

  60. Zwickel J (2009) Agency attribution and visuospatial perspective taking. Psychon Bull Rev 16:1089–1093

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LAAS, CNRS, 7 avenue du colonel Roche, BP 54200, 31031, Toulouse, France

    Amit Kumar Pandey, Muhammad Ali & Rachid Alami

  2. LAAS, Univ de Toulouse, 31400, Toulouse, France

    Amit Kumar Pandey, Muhammad Ali & Rachid Alami

Authors
  1. Amit Kumar Pandey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachid Alami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amit Kumar Pandey.

Additional information

This work has been conducted within the EU SAPHARI project (http://www.saphari.eu/) funded by the E.C. Division FP7-IST under Contract ICT-287513.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pandey, A.K., Ali, M. & Alami, R. Towards a Task-Aware Proactive Sociable Robot Based on Multi-state Perspective-Taking. Int J of Soc Robotics 5, 215–236 (2013). https://doi.org/10.1007/s12369-013-0181-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-013-0181-3

Keywords

Search

Navigation