Skip to main content
SpringerLink
Log in

A Brief Review Focused on Tactile Sensing for Stable Robot Grasping Manipulation

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13457))

Included in the following conference series:

Abstract

In this paper, we briefly investigate recently published literature on robot grasping with tactile information to understand the effect introduced by tactile modality and summarize the current issues of tactile sensing. Moreover, this paper consists of a review of slip detection during grasping, a review of grasp stability assessment to estimate the current contact state and a review of regrasp to select appropriate grasp adjustment action. Finally, we discuss the current limitations and deficiencies that prevent researchers from using tactile sensors, making it challenging to incorporate tactile modalities into robot perception and properly utilize tactile information to achieve effective and stable grasp performances. We consider that the pipeline consisting of grasp outcome prediction and grasp action adjustment based on machine learning is an appropriate scheme to make full use of tactile information and its potential in robot grasping tasks. More studies in this field are expected in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wettels, N., Santos, V.J., Johansson, R.S., Loeb, G.E.: Biomimetic tactile sensor array. Adv. Robot. 22(8), 829–849 (2008)

    Article  Google Scholar 

  2. Tegin, J., Ekvall, S., Kragic, D., Wikander, J., Iliev, B.: Demonstration-based learning and control for automatic grasping. Intell. Serv. Robot. 2(1), 23–30 (2009)

    Article  Google Scholar 

  3. Goldfeder, C., Allen, P.K., Lackner, C., Pelossof, R.: Grasp planning via decomposition trees. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, Rome, Italy, April 2007, pp. 4679–4684 (2007)

    Google Scholar 

  4. Faccio, M., Bottin, M., Rosati, G.: Collaborative and traditional robotic assembly: a comparison model. Int. J. Adv. Manuf. Technol. 102(5–8), 1355–1372 (2019). https://doi.org/10.1007/s00170-018-03247-z

    Article  Google Scholar 

  5. Bekiroglu, Y., Huebner, K., Kragic, D.: Integrating grasp planning with online stability assessment using tactile sensing. In: 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011, pp. 4750–4755 (2011)

    Google Scholar 

  6. Li, M., Bekiroglu, Y., Kragic, D., Billard, A.: Learning of grasp adaptation through experience and tactile sensing. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, September 2014, pp. 3339–3346 (2014)

    Google Scholar 

  7. Dang, H., Allen, P.K.: Learning grasp stability. In: 2012 IEEE International Conference on Robotics and Automation, St Paul, MN, USA, May 2012, pp. 2392–2397 (2012)

    Google Scholar 

  8. Roa, M.A., Suárez, R.: Grasp quality measures: review and performance. Auton. Robot. 38(1), 65–88 (2014). https://doi.org/10.1007/s10514-014-9402-3

    Article  Google Scholar 

  9. Hyttinen, E., Kragic, D., Detry, R.: Learning the tactile signatures of prototypical object parts for robust part-based grasping of novel objects. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, May 2015, pp. 4927–4932 (2015)

    Google Scholar 

  10. Dang, H., Weisz, J., Allen, P.K.: Blind grasping: stable robotic grasping using tactile feedback and hand kinematics. In: 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011, pp. 5917–5922 (2011)

    Google Scholar 

  11. Laaksonen, J., Kyrki, V., Kragic, D.: Evaluation of feature representation and machine learning methods in grasp stability learning. In: 2010 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA, December 2010, pp. 112–117 (2010)

    Google Scholar 

  12. Bierbaum, A., Rambow, M., Asfour, T., Dillmann, R.: Grasp affordances from multi-fingered tactile exploration using dynamic potential fields. In: 2009 9th IEEE-RAS International Conference on Humanoid Robots, Paris, December 2009, pp. 168–174 (2009)

    Google Scholar 

  13. Chebotar, Y., Kroemer, O., Peters, J.: Learning robot tactile sensing for object manipulation. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, September 2014, pp. 3368–3375 (2014)

    Google Scholar 

  14. van Hoof, H., Hermans, T., Neumann, G., Peters, J.: Learning robot in-hand manipulation with tactile features. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, South Korea, November 2015, pp. 121–127 (2015)

    Google Scholar 

  15. Hellman, R.B., Tekin, C., van der Schaar, M., Santos, V.J.: Functional contour-following via haptic perception and reinforcement learning. IEEE Trans. Haptics 11(1), 61–72 (2018)

    Article  Google Scholar 

  16. Hasegawa, T., Honda, K.: Detection and measurement of fingertip slip in multi-fingered precision manipulation with rolling contact. In: Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590), Baden-Baden, Germany, pp. 43–48 ( 2001)

    Google Scholar 

  17. Holweg, E.G.M., Hoeve, H., Jongkind, W., Marconi, L., Melchiorri, C., Bonivento, C.: Slip detection by tactile sensors: algorithms and experimental results. In: Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, vol. 4, pp. 3234–3239 (1996)

    Google Scholar 

  18. Tsujiuchi, N., et al.: Slip detection with distributed-type tactile sensor. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No. 04CH37566), Sendai, Japan, 2004, vol. 1, pp. 331–336 (2004)

    Google Scholar 

  19. Howe, R.D., Cutkosky, M.R.: Sensing skin acceleration for slip and texture perception. In: Proceedings, 1989 International Conference on Robotics and Automation, Scottsdale, AZ, USA, 1989, pp. 145–150 (1989)

    Google Scholar 

  20. Romano, J.M., Hsiao, K., Niemeyer, G., Chitta, S., Kuchenbecker, K.J.: Human-inspired robotic grasp control with tactile sensing. IEEE Trans. Robot. 27(6), 1067–1079 (2011)

    Article  Google Scholar 

  21. Yussof, H.: Sensorization of robotic hand using optical three-axis tactile sensor: evaluation with grasping and twisting motions. J. Comput. Sci. 6(8), 955–962 (2010)

    Article  Google Scholar 

  22. Tremblay, M.R., Cutkosky, M.R.: Estimating friction using incipient slip sensing during a manipulation task. In: Proceedings IEEE International Conference on Robotics and Automation, Atlanta, GA, USA, 1993, pp. 429–434 (1993)

    Google Scholar 

  23. Kaboli, M., Yao, K., Cheng, G.: Tactile-based manipulation of deformable objects with dynamic center of mass. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico, November 2016, pp. 752–757 (2016)

    Google Scholar 

  24. Melchiorri, C.: Slip detection and control using tactile and force sensors. IEEE/ASME Trans. Mechatron. 5(3), 235–243 (2000)

    Article  Google Scholar 

  25. Gunji, D., et al.: Grasping force control of multi-fingered robot hand based on slip detection using tactile sensor. In: 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, May 2008, pp. 2605–2610 (2008)

    Google Scholar 

  26. Jamali, N., Sammut, C.: Slip prediction using Hidden Markov models: multidimensional sensor data to symbolic temporal pattern learning. In: 2012 IEEE International Conference on Robotics and Automation, St Paul, MN, USA, May 2012, pp. 215–222 (2012)

    Google Scholar 

  27. Su, Z., et al.: Force estimation and slip detection/classification for grip control using a biomimetic tactile sensor. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, South Korea, November 2015, pp. 297–303 (2015)

    Google Scholar 

  28. Veiga, F., Peters, J., Hermans, T.: Grip stabilization of novel objects using slip prediction. IEEE Trans. Haptics 11(4), 531–542 (2018)

    Article  Google Scholar 

  29. Veiga, F., van Hoof, H., Peters, J., Hermans, T.: Stabilizing novel objects by learning to predict tactile slip. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, September 2015, pp. 5065–5072 (2015)

    Google Scholar 

  30. Meier, M., Patzelt, F., Haschke, R., Ritter, H.J.: Tactile convolutional networks for online slip and rotation detection. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) Artificial Neural Networks and Machine Learning – ICANN 2016. LNCS, vol. 9887, pp. 12–19. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44781-0_2

    Chapter  Google Scholar 

  31. ZapataImpata, B., Gil, P., Torres, F.: Learning spatio temporal tactile features with a ConvLSTM for the direction of slip detection. Sensors 19(3), 523 (2019)

    Article  Google Scholar 

  32. Jiar, Y., Lee, K., Shi, G.: A high resolution and high compliance tactile sensing system for robotic manipulations. In: Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 1993), Yokohama, Japan, 1993, vol. 2, pp. 1005–1009 (1993)

    Google Scholar 

  33. Ohka, M., Mitsuya, Y., Hattori, K., Higashioka, I.: Data conversion capability of optical tactile sensor featuring an array of pyramidal projections. In: 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No. 96TH8242), Washington, DC, USA, 1996, pp. 573–580 (1996)

    Google Scholar 

  34. Maekawa, H., Tanie, K., Komoriya, K.: A finger-shaped tactile sensor using an optical waveguide. In: Proceedings of IEEE Systems Man and Cybernetics Conference - SMC, Le Touquet, France, 1993, pp. 403–408 (1993)

    Google Scholar 

  35. Ferrier, N.J., Brockett, R.W.: Reconstructing the shape of a deformable membrane from image data. Int. J. Robot. Res. 19(9), 795–816 (2000)

    Article  Google Scholar 

  36. Sato, K., Kamiyama, K., Nii, H., Kawakami, N., Tachi, S.: Measurement of force vector field of robotic finger using vision-based haptic sensor. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, September 2008, pp. 488–493 (2008)

    Google Scholar 

  37. Nagata, K., Ooki, M., Kakikur, M.: Feature detection with an image based compliant tactile sensor. In: Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No. 99CH36289), Kyongju, South Korea, 1999, vol. 2, pp. 838–843 (1999)

    Google Scholar 

  38. Kamiyama, K., Vlack, K., Mizota, T., Kajimoto, H., Kawakami, K., Tachi, S.: Vision-based sensor for real-time measuring of surface traction fields. IEEE Comput. Grap. Appl. 25(1), 68–75 (2005)

    Article  Google Scholar 

  39. Yuan, W., Li, R., Srinivasan, M.A., Adelson, E.H.: Measurement of shear and slip with a GelSight tactile sensor. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, May 2015, pp. 304–311 (2015)

    Google Scholar 

  40. Dong, S., Yuan, W., Adelson, E.H.: Improved GelSight tactile sensor for measuring geometry and slip. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, September 2017, pp. 137–144 (2017)

    Google Scholar 

  41. Li, J., Dong, S., Adelson, E.: Slip detection with combined tactile and visual information. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, May 2018, pp. 7772–7777 (2018)

    Google Scholar 

  42. Bekiroglu, Y., Kragic, D., Kyrki, V.: Learning grasp stability based on tactile data and HMMs. In: 19th International Symposium in Robot and Human Interactive Communication, Viareggio, Italy, September 2010, pp. 132–137 (2010)

    Google Scholar 

  43. Zapata-Impata, B.S., Gil, P., Torres, F.: Non-Matrix Tactile Sensors: How Can Be Exploited Their Local Connectivity For Predicting Grasp Stability? (2018)

    Google Scholar 

  44. Garcia-Garcia, A., Zapata-Impata, B.S., Orts-Escolano, S., Gil, P., Garcia-Rodriguez, J.: TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors (2019)

    Google Scholar 

  45. Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)

    Article  Google Scholar 

  46. Gao, Y., Hendricks, L.A., Kuchenbecker, K.J., Darrell, T.: Deep Learning for Tactile Understanding From Visual and Haptic Data (2015)

    Google Scholar 

  47. Koenig, A, Liu, Z., Janson, L., Howe, R.: Tactile Grasp Refinement using Deep Reinforcement Learning and Analytic Grasp Stability Metrics. arXiv:2109.11234 [cs, eess], September 2021, Accessed 27 October 2021. http://arxiv.org/abs/2109.11234

  48. Miller, A.T., Allen, P.K.: Examples of 3D grasp quality computations. In: Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), Detroit, MI, USA, 1999, vol. 2, pp. 1240–1246 (1999)

    Google Scholar 

  49. Zheng, Y., Qian, W.-H.: Improving grasp quality evaluation. Robot. Auton. Syst. 57(6–7), 665–673 (2009)

    Article  Google Scholar 

  50. Kwiatkowski, J., Cockburn, D., Duchaine, V.: Grasp stability assessment through the fusion of proprioception and tactile signals using convolutional neural networks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, September 2017, pp. 286–292 (2017)

    Google Scholar 

  51. Hyttinen, E., Kragic, D., Detry, R.: Estimating tactile data for adaptive grasping of novel objects. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, November 2017, pp. 643–648 (2017)

    Google Scholar 

  52. Dang, H., Allen, P.K.: Stable grasping under pose uncertainty using tactile feedback. Auton. Robot. 36(4), 309–330 (2013). https://doi.org/10.1007/s10514-013-9355-y

    Article  Google Scholar 

  53. Bekiroglu, Y., Song, D., Wang, L., Kragic, D.: A probabilistic framework for task-oriented grasp stability assessment. In: 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May 2013, pp. 3040–3047 (2013)

    Google Scholar 

  54. Cui, S., Wang, R., Wei, J, Li, F., Wang, S.: Grasp state assessment of deformable objects using visual-tactile fusion perception. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, May 2020, pp. 538–544 (2020)

    Google Scholar 

  55. Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Robot. Res. 37(4–5), 421–436 (2018)

    Article  Google Scholar 

  56. Yan, G., Schmitz, A., Funabashi, S., Somlor, S, Tomo, T.P., Sugano, S.: A robotic grasping state perception framework with multi-phase tactile information and ensemble learning. IEEE Robot. Autom. Lett., p. 1 (2022)

    Google Scholar 

  57. Bekiroglu, Y., Laaksonen, J., Jorgensen, J.A., Kyrki, V., Kragic, D.: Assessing grasp stability based on learning and haptic data. IEEE Trans. Robot. 27(3), 616–629 (2011)

    Article  Google Scholar 

  58. Cockbum, D., Roberge, J.-P., Le, T.-H.-L., Maslyczyk, A., Duchaine, V.: Grasp stability assessment through unsupervised feature learning of tactile images. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, Singapore, May 2017, pp. 2238–2244 (2017)

    Google Scholar 

  59. R. Calandra et al.: The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?,” arXiv:1710.05512 [cs, stat], October 2017, Accessed 08 Mar 2022. http://arxiv.org/abs/1710.05512

  60. Cui, S., Wang, R., Wei, J., Hu, J., Wang, S.: Self-attention based visual-tactile fusion learning for predicting grasp outcomes. IEEE Robot. Autom. Lett. 5(4), 5827–5834 (2020)

    Article  Google Scholar 

  61. Steffen, J., Haschke, R., Ritter, H.: Experience-based and tactile-driven dynamic grasp control. In: 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA, October 2007, pp. 2938–2943 (2007)

    Google Scholar 

  62. Kim, J., Iwamoto, K., Kuffner, J.J., Ota, Y., Pollard, N.S.: Physically-based grasp quality evaluation under uncertainty. In: 2012 IEEE International Conference on Robotics and Automation, St. Paul, MN, USA, May 2012, pp. 3258–3263 (2012)

    Google Scholar 

  63. Felip, J., Morales, A.: Robust sensor-based grasp primitive for a three-finger robot hand. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, pp. 1811–1816, October 2009

    Google Scholar 

  64. Chebotar, Y., Hausman, K., Kroemer, O., Sukhatme, G.S., Schaal, S.: Generalizing regrasping with supervised policy learning. In: Kulić, D., Nakamura, Y., Khatib, O., Venture, G. (eds.) 2016 International Symposium on Experimental Robotics. SPAR, vol. 1, pp. 622–632. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-50115-4_54

    Chapter  Google Scholar 

  65. Dang, H., Allen, P.K.: Grasp adjustment on novel objects using tactile experience from similar local geometry. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, pp. 4007–4012, November 2013

    Google Scholar 

  66. Chebotar, Y., Hausman, K., Su, Z., Sukhatme, G.S., Schaal, S.: Self-supervised regrasping using spatio-temporal tactile features and reinforcement learning. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea, pp. 1960–1966, October 2016

    Google Scholar 

  67. Hogan, F.R., Bauza, M., Canal, O., Donlon, E., Rodriguez, A.: Tactile regrasp: grasp adjustments via simulated tactile transformations. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, pp. 2963–2970, October 2018

    Google Scholar 

  68. Calandra, R., et al.: More than a feeling: learning to grasp and regrasp using vision and touch. IEEE Robot. Autom. Lett. 3(4), 3300–3307 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, 200240, China

    Zhenning Zhou, Zhuangzhuang Zhang, Kaiyi Xie, Xiaoxiao Zhu & Qixin Cao

Authors
  1. Zhenning Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhuangzhuang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaiyi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoxiao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qixin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qixin Cao .

Editor information

Editors and Affiliations

  1. Harbin Institute of Technology, Harbin, China

    Honghai Liu

  2. Huazhong University of Science and Technology, Wuhan, China

    Zhouping Yin

  3. Shenyang Institute of Automation, Shenyang, Liaoning, China

    Lianqing Liu

  4. Harbin Institute of Technology, Harbin, China

    Li Jiang

  5. Shanghai Jiao Tong University, Shanghai, China

    Guoying Gu

  6. Shenzhen Institutes of Advanced Technology, Shenzhen, China

    Xinyu Wu

  7. Harbin Institute of Technology, Harbin, China

    Weihong Ren

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, Z., Zhang, Z., Xie, K., Zhu, X., Cao, Q. (2022). A Brief Review Focused on Tactile Sensing for Stable Robot Grasping Manipulation. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13457. Springer, Cham. https://doi.org/10.1007/978-3-031-13835-5_57

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13835-5_57

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13834-8

  • Online ISBN: 978-3-031-13835-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation