Skip to main content

Advertisement

Log in

Human-directed coordinated control of an assistive mobile manipulator

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

This paper considers the control of a mobile manipulator with the end-effector commanded by a human user. The motivation is to empower mobility impaired individuals to tackle certain activities of daily living. The prototype robotic system consists of a redundant articulated arm mounted on a mobile base. The human user specifies the desired motion of certain degrees of freedom, such as the translation or rotation of the arm end-effector. An optimization-based control algorithm determines the full arm and base motion to minimize deviation from the human intent while satisfying certain equality and inequality constraints. These constraints may involve physical requirements of the system, e.g., collision avoidance and joint limits, and human user consideration, such as intuitiveness of the combined arm and base motion. This constraint optimization problem is strictly convex and may be efficiently solved as a quadratic problem. We demonstrate the effectiveness of the algorithm in simulation and experimentation, using a dual-arm Baxter robot mounted on a wheelchair as a prototype testbed. Human user commands the end-effector of one of the arms using a 3-degree-of-freedom interface, either a sip-puff or a joystick type of device. We demonstrate effective object retrieval and placement in an environment with obstacles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Baxter: Redefining robotics and manufacturing. http://www.rethinkrobotics.com/products/baxter (2014). Accessed 14 March 2014

  • Cai, H., Mostofi, Y,: To ask or not to ask: a foundation for the optimization of human-robot collaborations. In: Proceedings of 2015 American Control Conference, Chicago, IL, Jul 1–3 (2015)

  • Chen, T.L., Ciocarlie, M., Cousins, S., Grice, P., Hawkins, K., Hsiao, K., Kemp, C.C., King, C.H., Lazewatsky, D.A., Leeper, A., Nguyen, H., Paepcke, A., Pantofaru, C., Smart, W.D., Takayama, L.: Robots for humanity: a case study in assistive mobile manipulation. In: IEEE Robotics and Automation Magazine, Special issue on Assistive Robotics, vol 20(1) (2013)

  • Cipriani, C., Zaccone, F., Micera, S., Chiara Carrozza, M.: On the shared control of an EMG-controlled prosthetic hand: analysis of user prosthesis. IEEE Trans. Robot. 24(1), 170–184 (2008)

    Article  Google Scholar 

  • Cunningham, A., Keddy-Hector, W., Sinha, U., Whalen, D., Kruse, D., Braasch, J., Wen, J.T., Jamster.: A mobile dual-arm assistive robot with Jamboxx control. In: Conference on Automation Science and Engineering (CASE), pp. 509–514, Aug (2014)

  • Dietrich, A., Wimboeck, T., Albu-Schäffer, A., Hirzinger, G.: Integration of reactive, torque-based self-collision avoidance into a task hierarchy. IEEE Trans. Robot. 28(6), 1278–1293 (2012)

    Article  Google Scholar 

  • Dietrich, A., Wimboeck, T., Albu-Schäffer, A., Hirzinger, G.: Reactive whole-body control: dynamic mobile manipulation using a large number of actuated degrees of freedom. IEEE Robot. Autom. Mag. Special Issue Mobile Manipul. 19(2), 20–33 (2012)

    Article  Google Scholar 

  • Dragan, A.D., Srinivasa, S.S.: A policy-blending formalism for shared control. Int. J. Robot. Res. 32(7), 790–805 (2013)

    Article  Google Scholar 

  • Flacco, F., Luca, A.D.: Discrete-time redundancy resolution at the velocity level with acceleration/torque optimization properties. Robot. Autonom. Syst. (2015) (accepted and in press)

  • Franchi, A., Secchi, C., Ryll, M., Bulthoff, H.H., Giordano, P.R.: Shared control: balancing autonomy and human assistance with a group of quadrotor UAVs. IEEE Robot. Autom. Mag. 19(3), 57–68 (2012)

    Article  Google Scholar 

  • Jiang, J., Astolfi, A.: Shared-control for the kinematic model of a rear-wheel drive car. In: Proceedings of 2015 American Control Conference, Chicago, IL, Jul 1–3 (2015)

  • Kim, H., Biggs, S.J., Schloerb, D.W., Carmena, J.M., Lebedev, M.A., Nicolelis, M.A.L., Srinivasan, M.A.: Continuous shared control for stabilizing reaching and grasping with brain-machine interfaces. IEEE Trans. Biomed. Eng. 53(6), 1164–1173 (2006)

    Article  Google Scholar 

  • Kruse, D., Radke, R., Wen, J.: Collaborative human-robot manipulation of highly deformable materials. In: Proceedings of the IEEE Conference on Robotics and Automation (ICRA), Seattle, WA, May 26–30 (2015)

  • Lacey, G., MacNamara, S.: Context-aware shared control of a robot mobility aid for the elderly blind. Int. J. Robot. Res. 19(11), 1054–1065 (2000)

    Article  Google Scholar 

  • Leeper, A., Hsiao, K., Ciocarlie, M., Sucan, I., Salisbury, K.: Methods for collision-free arm teleoperation in clutter using constraints from 3d sensor data. In: IEEE-RAS International Conference on Humanoid Robots, Oct 15–17 (2013)

  • Luca, A.D., Oriolo, G., Giordano, P.R.: Image-based visual servoing schemes for nonholonomic mobile manipulators. Robotica 25, 131–145 (2007)

    Google Scholar 

  • Madigan, E.A., Newman, W.S.: What do users want from “smart wheelchairs”. In: NI 2012: Proceedings of the 11th International Congress on Nursing Informatics, vol. 2012. American Medical Informatics Association (2012)

  • Nudehi, S.S., Mukherjee, R., Ghodoussi, M.: A shared-control approach to haptic interface design for minimally invasive telesurgical training. IEEE Trans. Control Syst. Technol. 13(4), 588–592 (2005)

    Article  Google Scholar 

  • Ogren, P., Smith, C., Karayiannidis, Y., Kragic, D.: A multi objective control approach to online dual arm manipulation. In IFAC Symposium on Robot Control, SyRoCo, Dubrovnik, Croatia (2012)

    Google Scholar 

  • Pineau, J., Atrash, A.: SmartWheeler: a robotic wheelchair test-bed for investigating new models of human-robot interaction. In: AAAI Spring Symposium: Multidisciplinary Collaboration for Socially Assistive Robotics, pp. 59–64 (2007)

  • Storms, J.G., Vozar, S., Tilbury, D.M.: Predicting human performance during teleoperation. In: ACM/IEEE International Conference on Human-Robot Interaction, Bielefeld University, Germany, Mar 3–6 (2014)

  • Storms, J.G., Tilbury, D.M.: Blending of human and obstacle avoidance control for a high speed mobile robot. In: American Control Conference (ACC), Portland, Oregon, USA, Jun 4–6 (2014)

  • Stückler, J., Behnke, S.: Following human guidance to cooperatively carry a large object. In: 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia, October (2011)

  • Tan, J., Xi, N., Wang, Y.: Integrated task planning and control for mobile manipulators. Int. J. Robot. Res. 22(5), 337–354 (2003)

    Article  Google Scholar 

  • Tanner, H.G., Loizou, S.G., Kyriakopoulos, K.J.: Nonholonomic navigation and control of cooperating mobile manipulators. IEEE Trans. Robot. Autom. 19(1), 53–64 (2003)

    Article  Google Scholar 

  • Venator, E., Lee, G.S., Newman, W.: Hardware and software architecture of ABBY: An industrial mobile manipulator. In: 2013 IEEE International Conference on Automation Science and Engineering (CASE), pp. 324–329. IEEE (2013)

  • Wang, H., Liu, X.P.: Adaptive shared control for a novel mobile assistive robot. IEEE/ASME Trans. Mechatron. 19(6), 1725–1736 (2014)

    Article  Google Scholar 

  • Wang, Z., Mülling, K., Deisenroth, M.P., Amor, H.B., Vogt, D., Schölkopf, B., Peters, J.: Probabilistic movement modeling for intention inference in human–robot interaction. Int. J. Robot. Res. 32(7), 841–858 (2013)

    Article  Google Scholar 

  • Wason Technology. Robot Raconteur. https://robotraconteur.com (2014). Accessed 28 July 2015

  • White, G.D., Bhatt, R.M., Krovi, V.N.: Dynamic redundancy resolution in a nonholonomic wheeled mobile manipulator. Robotica 25(3), 147–156 (2007)

    Google Scholar 

  • White, G.D., Bhatt, R.M., Tang, C.P., Krovi, V.N.: Experimental evaluation of dynamic redundancy resolution in a nonholonomic wheeled mobile manipulator. IEEE/ASME Trans. Mechatron. 14(3), 349–357 (2009)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Andrew Cunningham and Jingyu Su for the wheelchair control system implementation, Dan Kruse for the Robot Raconteur Baxter Bridge, and John Wason for the development and sharing of Robot Raconteur. The enthusiasm, encouragement and feedback from David Whalen is a continuing source of inspiration. This work is supported primarily by the Center for Automation Technologies and Systems (CATS) under a block grant from the New York State Empire State Development Division of Science, Technology and Innovation (NYSTAR), and in part by the National Science Foundation Smart Lighting ERC under NSF Cooperative Agreement No. EEC-0812056.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lu Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, L., Wen, J.T. Human-directed coordinated control of an assistive mobile manipulator. Int J Intell Robot Appl 1, 104–120 (2017). https://doi.org/10.1007/s41315-016-0005-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-016-0005-3

Keywords

Navigation