Skip to main content
SpringerLink
Log in

In-Hand Object Recognition for Sensorized Soft Hand

  • Conference paper
  • First Online:
Intelligent Autonomous Systems 16 (IAS 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 412))

Included in the following conference series:

Abstract

This paper presents an anthropomorphic soft robotic hand integrated with multiple flexible force and flex sensors in the fingers and palm. This enables the soft hand to estimate the curvature and contact forces, as it grasps different types of objects. Here, we blend the soft sensing technology with neural network to develop an in-hand object recognition ability for the soft hand. We implemented three transfer learning based neural networks to recognize objects using the force and flex sensors data. The networks are trained with 10 items and are validated with 10 unknown items of similar shape and dimensions. This approach produced a classification accuracy of up to 88.20% on unknown test items, which was comparable to previous reports in the literature. While it proves to be an efficient learning method, we also note that the accuracy may be further enhanced by incorporating piecewise compliant grasping mechanism, which would allow the soft hand to better conform to objects of angular profiles.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Deimel, R., Brock, O.: A novel type of compliant, underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. 35(1–3), 161–185 (2015)

    Google Scholar 

  2. Tavakoli, M., Almeida, A.T.D.: Adaptive under-actuated anthropomorphic hand: ISR-SoftHand. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2014)

    Google Scholar 

  3. Godfrey, S.B., Adjoudani, A., Catalano, M., Bicchi, A.: A synergy-driven approach to a myoelectric hand. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–6 (2013)

    Google Scholar 

  4. Yap, H.K., Ng, H.Y., Yeow, C.-H.: High-force soft printable pneumatics for soft robotic applications. Soft Robot. 3(3), 144–158 (2016)

    Article  Google Scholar 

  5. Galloway, K.C., et al.: Soft robotic grippers for biological sampling on deep reefs. Soft Robot. 3(1), 23–33 (2016)

    Article  MathSciNet  Google Scholar 

  6. She, Y., Li, C., Cleary, J., Su, H.-J.: Design and fabrication of a soft robotic hand with embedded actuators and sensors. J. Mech. Robot. 7(2), 021007 (2015)

    Article  Google Scholar 

  7. Renda, F., Giorelli, M., Calisti, M., Cianchetti, M., Laschi, C.: Dynamic model of a multibending soft robot arm driven by cables. IEEE Trans. Robot. 30(5), 1109–1122 (2014)

    Article  Google Scholar 

  8. Chen, Y., Guo, S., Li, C., Yang, H., Hao, L.: Size recognition and adaptive grasping using an integration of actuating and sensing soft pneumatic gripper. Robot. Auton. Syst. 104, 14–24 (2018)

    Article  Google Scholar 

  9. Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)

    Article  Google Scholar 

  10. Wall, V., Zoller, G., Brock, O.: A method for sensorizing soft actuators and its application to the RBO hand 2. In: 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017)

    Google Scholar 

  11. Shih, B., Drotman, D., Christianson, C., Huo, Z., White, R., Christensen, H.I., Tolley, M.T.: Custom soft robotic gripper sensor skins for haptic object visualization. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)

    Google Scholar 

  12. Homberg, B.S., Katzschmann, R.K., Dogar, M.R., Rus, D.: Haptic identification of objects using a modular soft robotic gripper. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015)

    Google Scholar 

  13. Zhao, H., O’Brien, K., Li, S., Shepherd, R.F.: Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1(1), eaai7529 (2016)

    Google Scholar 

  14. Kazemi, M., Valois, J.-S., Bagnell, J.A., Pollard, N.: Human-inspired force compliant grasping primitives. Auton. Robot. 37(2), 209–225 (2014). https://doi.org/10.1007/s10514-014-9389-9

    Article  Google Scholar 

  15. Liarokapis, M.V., Calli, B., Spiers, A.J., Dollar, A.M.: Unplanned, model-free, single grasp object classification with underactuated hands and force sensors. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015)

    Google Scholar 

  16. Thuruthel, T.G., Shih, B., Laschi, C., Tolley, M.T.: Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4(26), eaav1488 (2019)

    Google Scholar 

  17. Shirafuji, S., Hosoda, K.: Detection and prevention of slip using sensors with different properties embedded in elastic artificial skin on the basis of previous experience. Robot. Auton. Syst. 61(1), 46–52 (2014)

    Article  Google Scholar 

  18. Rocha, R.P., Lopes, P.A., Almeida, A.T.D., Tavakoli, M., Majidi, C.: Fabrication and characterization of bending and pressure sensors for a soft prosthetic hand. J. Micromech. Microeng. 28(3), 034001 (2018)

    Article  Google Scholar 

  19. Truby, R.L., Wehner, M., Grosskopf, A.K., Vogt, D.M., Uzel, S.G.M., Wood, R.J., Lewis, J.A.: Soft robotics: soft somatosensitive actuators via embedded 3D printing. Adv. Mater. 30(15), 1870106 (2018)

    Article  Google Scholar 

  20. Choi, C., Schwarting, W., Delpreto, J., Rus, D.: Learning object grasping for soft robot hands. IEEE Robot. Autom. Lett. 3(3), 2370–2377 (2018)

    Article  Google Scholar 

  21. Jamone, L., Natale, L., Metta, G., Sandini, G.: Highly sensitive soft tactile sensors for an anthropomorphic robotic hand. IEEE Sens. J. 15(8), 4226–4233 (2015)

    Article  Google Scholar 

  22. Schmitz, A., Bansho, Y., Noda, K., Iwata, H., Ogata, T., Sugano, S.: Tactile object recognition using deep learning and dropout. In: IEEE-RAS International Conference on Humanoid Robots (2014)

    Google Scholar 

  23. Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  24. Albini, A., Denei, S., Cannata, G.: Human hand recognition from robotic skin measurements in human-robot physical interactions. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)

    Google Scholar 

  25. Luo, S., Mou, W., Althoefer, K., Lili, H.: Novel tactile-SIFT descriptor for object shape recognition. IEEE Sens. J. 15(9), 5001–5009 (2015)

    Article  Google Scholar 

  26. Gandarias, J.M., Garcia-Cerezo, A.J., Gomez-De-Gabriel, J.M.: CNN-based methods for object recognition with high-resolution tactile sensors. IEEE Sens. J. 19(16), 6872–6882 (2019)

    Article  Google Scholar 

  27. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)

    Google Scholar 

  29. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  30. Navarro, S.E., Gorges, N., Worn, H., Schill, J., Asfour, T., Dillmann, R.: Haptic object recognition for multi-fingered robot hands. In: 2012 IEEE Haptics Symposium (HAPTICS) (2012)

    Google Scholar 

  31. Luo, S., Mou, W., Althoefer, K., Liu, H.: Iterative closest labeled point for tactile object shape recognition. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016)

    Google Scholar 

  32. Gandarias, J.M., Gomez-De-Gabriel, J.M., Garcia-Cerezo, A.: Human and object recognition with a high-resolution tactile sensor. In: 2017 IEEE Sensors (2017)

    Google Scholar 

Download references

Acknowledgment

The authors would like to thank Andrei Nakagawa and Aaron Goh for offering valuable advice during the experiments and National Robotics Program for supporting this project. We also gratefully acknowledge the technical support of Nvidia Corporation through the Memorandum of Understanding with the Advanced Robotics Centre of the National University of Singapore on autonomous systems technologies.

Author information

Authors and Affiliations

  1. Advanced Robotics Center, National University of Singapore, Queenstown, Singapore

    Phone May Khin, Jin Huat Low, Marcelo H. Ang Jr. & Chen-Hua Yeow

  2. Department of Mechanical Engineering, National University of Singapore, Queenstown, Singapore

    Marcelo H. Ang Jr.

  3. Department of Biomedical Engineering, National University of Singapore, Queenstown, Singapore

    Chen-Hua Yeow

Authors
  1. Phone May Khin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Huat Low
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcelo H. Ang Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen-Hua Yeow
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phone May Khin .

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Marcelo H. Ang Jr

  2. Department of Precision Engineering, University of Tokyo, Tokyo, Japan

    Hajime Asama

  3. Singapore Institute of Manufacturing Technology, Singapore, Singapore

    Wei Lin

  4. Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore

    Shaohui Foong

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khin, P.M., Low, J.H., Ang, M.H., Yeow, CH. (2022). In-Hand Object Recognition for Sensorized Soft Hand. In: Ang Jr, M.H., Asama, H., Lin, W., Foong, S. (eds) Intelligent Autonomous Systems 16. IAS 2021. Lecture Notes in Networks and Systems, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-95892-3_27

Download citation

Publish with us

Policies and ethics

Search

Navigation