Skip to main content

Advertisement

SpringerLink
Log in

Sparse Contextual Task Learning and Classification to Assist Mobile Robot Teleoperation with Introspective Estimation

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This report proposes a novel approach to learn from demonstrations and classify contextual tasks the human operator executes by remotely controlling a mobile robot with joystick, aiming to assist mobile robot teleoperation within a shared autonomy system in a task-appropriate manner. The proposed classifier is implemented with the Gaussian Process (GP). GP is superior in uncertainty estimation when predicting class labels (i.e. the introspective capability) over other state-of-art classification methods, such as Support Vector Machine (SVM), which is probably the most widely used approach on this topic to date. Moreover, to keep the learned model sparse to limit the amount of storage and computation required, full GP is approximated with a state-of-art Sparse Online Gaussian Process (SOGP) algorithm, to maintain scalability to large datasets without compromising classification performance. The proposed approach is extensively evaluated on real data and verified to outperform the baseline classifiers both in classification accuracy and uncertainty estimation in predicting class labels, while maintaining sparsity and real-time property to scale with large datasets. This demonstrates the feasibility of the proposed approach for online use in real applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alkan, A., Günay, M.: Identification of emg signals using discriminant analysis and svm classifier. Expert Syst. Appl. 39(1), 44–47 (2012)

    Article  Google Scholar 

  2. Allen, J.E., Guinn, C.I.: Mixed-initiative interaction. IEEE Intell. Syst. Appl. 14(5), 14–23 (1999)

    Article  Google Scholar 

  3. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)

    Article  Google Scholar 

  4. Berczi, L.P., Posner, I., Barfoot, T.D.: Learning to assess terrain from human demonstration using an introspective gaussian-process classifier. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 3178–3185. IEEE (2015)

  5. Bidder, O.R., Campbell, H.A., Gómez-Laich, A., Urgé, P., Walker, J., Cai, Y., Gao, L., Quintana, F., Wilson, R.P.: Love thy neighbour: automatic animal behavioural classification of acceleration data using the k-nearest neighbour algorithm. PloS one 9(2), e88,609 (2014)

    Article  Google Scholar 

  6. Billings, C.E.: Aviation automation: The search for a human-centered approach (1997)

  7. Britton, N., Yoshida, K., Walker, J., Nagatani, K., Taylor, G., Dauphin, L.: Lunar micro rover design for exploration through virtual reality tele-operation. In: Field and Service Robotics, pp. 259–272. Springer (2015)

  8. Burridge, R.R., Hambuchen, K.A.: Using prediction to enhance remote robot supervision across time delay. In: 2009. IROS 2009. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5628–5634. IEEE (2009)

  9. Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  10. Csató, L., Opper, M.: Sparse on-line gaussian processes. Neural Comput. 14(3), 641–668 (2002)

    Article  MATH  Google Scholar 

  11. Dragan, A., Srinivasa, S.S.: Formalizing assistive teleoperation. R: SS (2012)

  12. Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., Marshall, S.: A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiol. Measur. 35(11), 2191 (2014)

    Article  Google Scholar 

  13. Ferber, J.: Multi-agent systems: an introduction to distributed artificial intelligence, vol. 1. Addison-Wesley, Reading (1999)

    Google Scholar 

  14. Fong, T., Thorpe, C.: Vehicle teleoperation interfaces. Autonomous Robots 11(1), 9–18 (2001)

    Article  MATH  Google Scholar 

  15. Gao, M., Oberlȧnder, J., Schamm, T., Zȯllner, J.M.: Contextual Task-Aware Shared Autonomy for Assistive Mobile Robot Teleoperation. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2014)

  16. Gao, M., Oberlȧnder, J., Schamm, T., Zȯllner, J.M.: Shared autonomy for assisted mobile robot teleoperation by recognizing operator intention as contextual task. In: 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE (2014)

  17. Gao, M., Schamm, T., J.Marius, Z.: Contextual task recognition to assist mobile robot teleoperation with introspective estimation using gaussian process. In: 2016 IEEE International Conference on Autonomous Robot Systems and Competitions. IEEE (2016)

  18. Gao, M., Schamm, T., Zȯllner, J.M.: Multiple contextual task recognition for sharing autonomy to assist mobile robot teleoperation. In: 2015 International Conference on Intelligent Robotics and Application (IRA). Springer (2015)

  19. Gao, M., Schamm, T., Zȯllner, J.M.: Online and incremental contextual task learning and recognition for sharing autonomy to assist mobile robot teleoperation. In: 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE (2015)

  20. Geng, Y., Chen, J., Fu, R., Bao, G., Pahlavan, K.: Enlighten wearable physiological monitoring systems: On-body rf characteristics based human motion classification using a support vector machine. IEEE Trans. Mob. Comput. 15(3), 656–671 (2016)

    Article  Google Scholar 

  21. Griffiths, P.G., Gillespie, R.B.: Sharing control between humans and automation using haptic interface: primary and secondary task performance benefits. Human Factors: J. Human Factors Ergon. Soc. 47(3), 574–590 (2005)

    Article  Google Scholar 

  22. Grimmett, H., Paul, R., Triebel, R., Posner, I.: Knowing when we don’t know: Introspective classification for mission-critical decision making. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 4531–4538. IEEE (2013)

  23. Grimmett, H., Triebel, R., Paul, R., Posner, I.: Introspective classification for robot perception. Int. J. Robot. Res. 35(7), 743–762 (2016)

    Article  Google Scholar 

  24. Grollman, D.H., Jenkins, O.C.: Sparse incremental learning for interactive robot control policy estimation. In: 2008. ICRA 2008. IEEE International Conference on Robotics and Automation, pp. 3315–3320. IEEE (2008)

  25. Grollman, D.H., Jenkins, O.C.: Incremental learning of subtasks from unsegmented demonstration. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 261–266. IEEE (2010)

  26. Guo, S., Pang, M., Gao, B., Hirata, H., Ishihara, H.: Comparison of semg-based feature extraction and motion classification methods for upper-limb movement. Sensors 15(4), 9022–9038 (2015)

    Article  Google Scholar 

  27. Hauser, K.: Recognition, prediction, and planning for assisted teleoperation of freeform tasks. Auton. Robot. 35(4), 241–254 (2013)

    Article  MathSciNet  Google Scholar 

  28. Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Gaussian processes for object categorization. Int. J. Comput. Vis. 88(2), 169–188 (2010)

    Article  Google Scholar 

  29. Keskinpala, H.K., Adams, J.A., Kawamura, K.: PDA-based human-robotic interface. In: 2003. IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3931–3936. IEEE (2003)

  30. Khan, A.M., Tufail, A., Khattak, A.M., Laine, T.H.: Activity recognition on smartphones via sensor-fusion and kda-based svms. International Journal of Distributed Sensor Networks (2014)

  31. Kohlbrecher, S., Meyer, J., von Stryk, O., Klingauf, U.: A flexible and scalable slam system with full 3d motion estimation. In: Security and Rescue Robotics (SSRR) Proceedings of IEEE International Symposium on Safety. IEEE (2011)

  32. Lee, D.H., Noh, K.W., Park, S.H., Lee, J.M.: Accurate velocity control based on the distance of a mobile robot and obstacle using a haptic joystick. In: 2012 12th International Conference on Control, Automation and Systems (ICCAS), pp. 1649–1654. IEEE (2012)

  33. Liarokapis, M.V., Artemiadis, P.K., Katsiaris, P.T., Kyriakopoulos, K.J., Manolakos, E.S.: Learning human reach-to-grasp strategies: Towards emg-based control of robotic arm-hand systems. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 2287–2292. IEEE (2012)

  34. Lin, M.H., Song, K.T.: Design and experimental study of a shared-controlled omnidirectional mobile platform. In: 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3579–3584. IEEE (2014)

  35. Lozano-Perez, T., Cox, I.J., Wilfong, G.T.: Autonomous robot vehicles. Springer Science & Business Media, Berlin (2012)

    Google Scholar 

  36. Nielsen, C.W., Goodrich, M.A., Ricks, R.W.: Ecological interfaces for improving mobile robot teleoperation. IEEE Trans. Robot. 23(5), 927–941 (2007)

    Article  Google Scholar 

  37. Okada, Y., Nagatani, K., Yoshida, K., Tadokoro, S., Yoshida, T., Koyanagi, E.: Shared autonomy system for tracked vehicles on rough terrain based on continuous three-dimensional terrain scanning. J. Field Robot. 28(6), 875–893 (2011)

    Article  Google Scholar 

  38. Paul, R., Triebel, R., Rus, D., Newman, P.: Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pp. 2404–2410. IEEE (2012)

  39. Pei, L., Liu, J., Guinness, R., Chen, Y., Kuusniemi, H., Chen, R.: Using ls-svm based motion recognition for smartphone indoor wireless positioning. Sensors 12(5), 6155–6175 (2012)

    Article  Google Scholar 

  40. Pitzer, B., Styer, M., Bersch, C., DuHadway, C., Becker, J.: Towards perceptual shared autonomy for robotic mobile manipulation. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 6245–6251. IEEE (2011). https://doi.org/10.1109/ICRA.2011.5980259

  41. Rasmussen, C.E.: Gaussian processes for machine learning (2006)

  42. Sa, I., Hrabar, S., Corke, P.: Inspection of pole-like structures using a visual-inertial aided vtol platform with shared autonomy. Sensors 15(9), 22,003–22,048 (2015)

    Article  Google Scholar 

  43. Secchi, C., Franchi, A., Bulthoff, H.H., Robuffo Giordano, P.: Bilateral control of the degree of connectivity in multiple mobile-robot teleoperation. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 3645–3652. IEEE (2013)

  44. Sheridan, T.B.: Telerobotics, automation and human supervisory control. The MIT press, Cambridge (1992)

    Google Scholar 

  45. Tambe, M., Scerri, P., Pynadath, D.V.: Adjustable autonomy for the real world. J. Artif. Intell. Res. 17(1), 171–228 (2002)

    MathSciNet  MATH  Google Scholar 

  46. Varkey, J.P., Pompili, D., Walls, T.A.: Human motion recognition using a wireless sensor-based wearable system. Personal Ubiquit. Comput. 16(7), 897–910 (2012)

    Article  Google Scholar 

  47. Vozar, S.E.: A framework for improving the speed and performance of teleoperated mobile manipulators. Ph.D. thesis, US Army (2013)

  48. Wu, W., Dasgupta, S., Ramirez, E.E., Peterson, C., Norman, G.J.: Classification accuracies of physical activities using smartphone motion sensors. J. Med. Internet Res. 14(5), e130 (2012)

    Article  Google Scholar 

  49. Yu, N., Wang, K., Li, Y., Xu, C., Liu, J.: A haptic shared control algorithm for flexible human assistance to semi-autonomous robots. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5241–5246. IEEE (2015)

Download references

Author information

Authors and Affiliations

  1. Intelligent Systems and Production Engineering (ISPE), FZI Research Center for Information Technology, Haid-und-Neu-Str. 10-14, 76131, Karlsruhe, Germany

    Ming Gao & J. Marius Zöllner

Authors
  1. Ming Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Marius Zöllner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Gao.

Additional information

The authors would like to acknowledge the prior publication at 2016 ICARSC conference, and thank all test participants. The authors sincerely appreciate the constructive comments made by the editors and the reviewers, which significantly helps improve this report.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, M., Zöllner, J.M. Sparse Contextual Task Learning and Classification to Assist Mobile Robot Teleoperation with Introspective Estimation. J Intell Robot Syst 93, 571–585 (2019). https://doi.org/10.1007/s10846-017-0681-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0681-8

Keywords

Search

Navigation