Skip to main content
SpringerLink
Log in

A Taxonomy of Benchmark Tasks for Robot Manipulation

  • Chapter
  • First Online:
Book cover Robotics Research

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 2))

Abstract

This paper presents a taxonomy of benchmark manipulation tasks for service robots. Our contributions are threefold: (1) A review of relevant literature regarding manipulation tests in the robotics domain and related fields, such as physical therapy, assistive technologies and prosthetics. (2) Guidelines to design useful testing protocols to evaluate manipulation performance. (3) A proposed general taxonomy of benchmark manipulation tasks and sample tests per each class.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.cc.gatech.edu/~ahuaman3/docs/papers/supplementaryMaterial.pdf.

References

  1. Aaron, D., Jansen, C.: Development of the functional dexterity test (FDT): construction, validity, reliability, and normative data. J. Hand Ther. 16(1), 12–21 (2003)

    Article  Google Scholar 

  2. Balasubramanian, R., Xu, L., Brook, P., Smith, J., Matsuoka, Y.: Physical human interactive guidance: identifying grasping principles from human-planned grasps. The Human Hand as an Inspiration for Robot Hand Development. Springer, Switzerland (2014)

    Chapter  Google Scholar 

  3. Barreca, S., Gowland, C., Stratford, P., Huijbregts, M., Griffiths, J., Torresin, W., Dunkley, M., Miller, P., Masters, L.: Development of the chedoke arm and hand activity inventory: theoretical constructs, item generation, and selection. Topics Stroke Rehabil. 11(4), 31–42 (2004)

    Article  Google Scholar 

  4. Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis—a survey. IEEE Trans. Robot. 30, (2013)

    Google Scholar 

  5. Bryden, P., Roy, E.: A new method of administering the grooved pegboard test: performance as a function of handedness and sex. Brain Cogn. 58(3), xxx (2005)

    Article  Google Scholar 

  6. Choi, Y., Deyle, T., Chen, T., Glass, J., Kemp, C.: A list of household objects for robotic retrieval prioritized by people with ALS. In: IEEE International Conference on Rehabilitation Robotics (2009)

    Google Scholar 

  7. Collins, K., Palmer, A., Rathmill, K.: The development of a European benchmark for the comparison of assembly robot programming systems. Robot Technology and Applications. Springer, Heidelberg (1985)

    Google Scholar 

  8. Dantam, N., Ben Amor, H., Christensen, H., Stilman, M.: Online multi-camera registration for bimanual workspace trajectories. In: HUMANOIDS (2014)

    Google Scholar 

  9. Deimel, R., Brock, O.: A novel type of compliant, underactuated robotic hand for dexterous grasping. In: Proceedings of Robotics: Science and Systems, pp. 1687–1692 (2014)

    Google Scholar 

  10. Desrosiers, J., Hébert, R., Dutil, E., Bravo, G.: Development and reliability of an upper extremity function test for the elderly: the TEMPA. Can. J. Occup. Ther. 60(1), 9–16 (1993)

    Article  Google Scholar 

  11. Feix, T., Bullock, I., Dollar, A.: Analysis of human grasping behavior: correlating tasks, objects and grasps. IEEE Trans. Haptics 7, 430–441 (2014)

    Article  Google Scholar 

  12. Feix, T., Bullock, I., Dollar, A.: Analysis of human grasping behavior: object characteristics and grasp type. IEEE Trans. Haptics 7, 311–323 (2014)

    Article  Google Scholar 

  13. Feix, T., Romero, J., Ek, C., Schmiedmayer, H., Kragic, D.: A metric for comparing the anthropomorphic motion capability of artificial hands. IEEE Trans. Robot. 29(1), 82–93 (2013)

    Article  Google Scholar 

  14. Ferrari, C., Canny, J.: Planning optimal grasps. In: ICRA (1992)

    Google Scholar 

  15. Gloss, D., Wardle, M.: Use of the minnesota rate of manipulation test for disability evaluation. Percept. Mot. Skills 55(2), 527–532 (1982)

    Article  Google Scholar 

  16. Grebenstein, M.: The awiwi hand: an artificial hand for the DLR hand arm system. Approaching Human Performance, pp. 65–130. Springer, Switzerland (2014)

    Chapter  Google Scholar 

  17. Grunwald, G., Borst, C., Zöllner, J.E.A.: Benchmarking dexterous dual-arm/hand robotic manipulation. In: IROS Workshop on Performance Evaluation and Benchmarking (2008)

    Google Scholar 

  18. Hackett, D., Pippine, J., Watson, A., Sullivan, C., Pratt, G.: An overview of the DARPA autonomous robotic manipulation (ARM) program. J. Robot. Soc. Jpn. 31(4), 326–329 (2013)

    Article  Google Scholar 

  19. Iossifidis, I., Lawitzky, G., Knoop, S., Zöllner, R.: Towards benchmarking of domestic robotic assistants. Advances in Human-Robot Interaction. Springer, Heidelberg (2005)

    Google Scholar 

  20. Jebsen, R., Taylor, N., Trieschmann, R., Trotter, M., Howard, L.: An objective and standardized test of hand function. Arch. Phys. Med. Rehabil. 50(6), 311 (1969)

    Google Scholar 

  21. Kapandji, A.: Clinical test of apposition and counter-apposition of the thumb. Annales de chirurgie de la main: organe officiel des societes de chirurgie de la main 5(1) (1985)

    Google Scholar 

  22. Kim, J., Iwamoto, K., Kuffner, J., Ota, Y., Pollard, N.: Physically based grasp quality evaluation under pose uncertainty. IEEE Trans. Robot. 29, 1424 (2013)

    Article  Google Scholar 

  23. Kopp, B., Kunkel, A., Flor, H., Platz, T., Rose, U., Mauritz, K., Gresser, K., McCulloch, K., Taub, E.: The arm motor ability test: reliability, validity, and sensitivity to change of an instrument for assessing disabilities in activities of daily living. Arch. Phys. Med. Rehabil. 78(6), 615–620 (1997)

    Article  Google Scholar 

  24. Kyberd, P., Murgia, A., Gasson, M., Tjerks, T., Metcalf, C., Chappell, P., Warwick, K., Lawson, S., Barnhill, T.: Case studies to demonstrate the range of applications of the Southampton Hand Assessment Procedure. Br. J. Occup. Ther. 72(5), 212–218 (2009)

    Article  Google Scholar 

  25. Leidner, D., Borst, C., Hirzinger, G.: Things are made for what they are: solving manipulation tasks by using functional object classes. In: HUMANOIDS (2012)

    Google Scholar 

  26. Light, C.M., Chappell, P.H., Kyberd, P.: Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. Arch. Phys. Med. Rehabil. 83(6), 776–783 (2002)

    Article  Google Scholar 

  27. Lin, S., Chang, J., Chen, P., Mao, H.: Hand function measures for burn patients: a literature review. Burns (J. Int. Soc. Burn Inj.) 39(1), 16–23 (2013)

    Article  Google Scholar 

  28. Matheus, K., Dollar, A.: Benchmarking grasping and manipulation: properties of the objects of daily living. In: IROS (2010)

    Google Scholar 

  29. Mathiowetz, V., Weber, K., Volland, G., Kashman, N.: Reliability and validity of grip and pinch strength evaluations. J. Hand Surg. 9(2), 222–226 (1984)

    Article  Google Scholar 

  30. Mathiowetz, V., Volland, G., Kashman, N., Weber, K.: Adult norms for the box and block test of manual dexterity. Am. J. Occup. Ther. 39(6), 386–391 (1985)

    Article  Google Scholar 

  31. Mathiowetz, V., Rogers, S., Dowe-Keval, M., Donahoe, L., Rennells, C.: The purdue pegboard: norms for 14-to 19-year-olds. Am. J. Occup. Ther. 40(3), 174–179 (1986)

    Article  Google Scholar 

  32. Meeussen, W., Wise, M., Glaser, S., Chitta, S., McGann, C., Mihelich, P., Marder-Eppstein, E., Muja, M., Eruhimov, V., Foote, T., et al.: Autonomous door opening and plugging in with a personal robot. In: ICRA, pp. 729–736 (2010)

    Google Scholar 

  33. Morales, A., Chinellato, E., Sanz, P., Del Pobil, A., Fagg, A.H.: Learning to predict grasp reliability for a multifinger robot hand by using visual features. In: AISC (2004)

    Google Scholar 

  34. Mukai, T., Hirano, S., Nakashima, H., Kato, Y., Sakaida, Y., Guo, S., Hosoe, S.: Development of a nursing-care assistant robot RIBA that can lift a human in its arms. In: IROS (2010)

    Google Scholar 

  35. Ng, C., Ho, D., Chow, S.: The Moberg pickup test: results of testing with a standard protocol. J. Hand Ther. 12(4), 309–312 (1999)

    Article  Google Scholar 

  36. Poole, J., Burtner, P., Torres, T., McMullen, C., Markham, A., Marcum, M., Anderson, J., Qualls, C.: Measuring dexterity in children using the nine-hole peg test. J. Hand Ther. 18(3), 348–351 (2005)

    Article  Google Scholar 

  37. Roa, M., Hertkorn, K., Zacharias, F., Borst, C., Hirzinger, G.: Graspability map: a tool for evaluating grasp capabilities. In: IEEE/RSJ IROS, pp. 1768–1774 (2011)

    Google Scholar 

  38. Schoneveld, K., Wittink, H., Takken, T.: Clinimetric evaluation of measurement tools used in hand therapy to assess activity and participation. J. Hand Ther. 22(3), 221–236 (2009)

    Article  Google Scholar 

  39. Siciliano, B.: Advanced Bimanual Manipulation: Results from The DEXMART Project, vol. 80. Springer, Heidelberg (2012)

    Book  Google Scholar 

  40. Sollerman, C., Ejeskär, A.: Sollerman hand function test: a standardised method and its use in tetraplegic patients. Scand. J. Plast. Reconstr. Surg. Hand Surg. 29(2), 167–176 (1995)

    Article  Google Scholar 

  41. The Amazon Picking Challenge. http://amazonpickingchallenge.org/ (2014)

  42. Tsui, K., Feil-Seifer, D., Matarić, M.J., Yanco, H.: Performance evaluation methods for assistive robotic technology. Performance Evaluation and Benchmarking of Intelligent Systems, pp. 41–66. Springer, US (2009)

    Chapter  Google Scholar 

  43. van Lankveld, W., van’t Pad Bosch, P., Bakker, J., Terwindt, S., Franssen, M.: Sequential occupational dexterity assessment (SODA): a new test to measure hand disability. J. Hand Ther. 9(1), 27–32 (1996)

    Google Scholar 

  44. Williams, M., Hadler, N., Earp, J.: Manual ability as a marker of dependency in geriatric women. J. Chronic Dis. 35(2), 115–122 (1982)

    Article  Google Scholar 

  45. Wisspeintner, T., Van Der Zant, T., Iocchi, L., Schiffer, S.: Robocup@Home: scientific competition and benchmarking for domestic service robots. Interact. Stud. 10(3), 392–426 (2009)

    Article  Google Scholar 

  46. Wolf, S., Thompson, P., Morris, D., D.K., Winstein, C., Taub, E., Giuliani, C., Pearson, S.: The EXCITE trial: attributes of the wolf motor function test in patients with subacute stroke. Neurorehabil. Neural Repair 19(3), 194–205 (2005)

    Google Scholar 

  47. Workshop on Autonomous Grasping and Manipulation: An Open Challenge. http://grasping-challenge.org/ (2014)

  48. Yozbatiran, N., Der-Yeghiaian, L., Cramer, S.: A standardized approach to performing the action research arm test. Neurorehabil. Neural Repair 22(1), 78–90 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Ana Huamán Quispe, Heni Ben Amor & Henrik I. Christensen

Authors
  1. Ana Huamán Quispe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heni Ben Amor
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henrik I. Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Huamán Quispe .

Editor information

Editors and Affiliations

  1. Istituto Italiano di Tecnologia, Genova, Italy, University of Pisa, Pisa, Italy , Pisa, Italy

    Antonio Bicchi

  2. Inst. für Informatik, Albert-Ludwigs-Universität Freiburg Inst. für Informatik, Freiburg, Germany

    Wolfram Burgard

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Huamán Quispe, A., Ben Amor, H., Christensen, H.I. (2018). A Taxonomy of Benchmark Tasks for Robot Manipulation. In: Bicchi, A., Burgard, W. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-319-51532-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-51532-8_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-51531-1

  • Online ISBN: 978-3-319-51532-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation