Abstract
Robots with tactile sensors can distinguish the tactile property of the object, such as the spatial shape, in many robotic applications. The neuromorphic approach offers a new solution for information processing to encode tactile signals. Vision-based tactile sensing has gradually attracted attention in recent years. Although some work has been done on proving the capacity of tactile sensors, the soft neuromorphic method inspired by neuroscience for spatial shape sensing is remarkably rare. This paper presented a soft neuromorphic method for contact spatial shape sensing using a vision-based tactile sensor. The outputs from the sensor were fed into the Izhikevich neuron model to emit the spike trains for emulating the firing behavior of mechanoreceptors. 9 spatial shapes were evaluated with an active touch protocol. The neuromorphic spike trains were decoded for discriminating spatial shapes based on k-nearest neighbors (KNN). Three spike features were used: average firing rate (FR), the coefficient of variation of the interspike interval (ISI CV), and the first spike firing time (FST). The results demonstrated the ability to classify different shapes with an accuracy as high as 93.519%. Furthermore, we found that FST significantly improved spatial shape classification decoding performance. This work was a preliminary study to apply the neuromorphic way to convey the tactile information obtained from the vision-based tactile sensor. It paved the way for using the neuromorphic vision-based tactile sensor in neurorobotic applications.
Supported by the Shenzhen Science and Technology Program (Grant No. JCYJ20210324120214040), the Guangdong Science and Technology Research Council (Grant No. 2020B1515120064).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, S.C., Delbruck, T.: Neuromorphic sensory systems. Curr. Opin. Neurobiol. 20(3), 288–295 (2010). Sensory systems
Zhengkun, Y., Yilei, Z.: Recognizing tactile surface roughness with a biomimetic fingertip: a soft neuromorphic approach. Neurocomputing 244, 102–111 (2017)
Spigler, G., Oddo, C.M., Carrozza, M.C.: Soft-neuromorphic artificial touch for applications in neuro-robotics. In: 2012 4th IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 1913–1918 (2012)
Izhikevich, E.: Simple model of spiking neurons. IEEE Trans. Neural Netw. 14(6), 1569–1572 (2003)
Oballe-Peinado, S., Hidalgo-López, J.A., Sánchez-Durán, J.A., Castellanos-Ramos, J., Vidal-Verdú, F.: Architecture of a tactile sensor suite for artificial hands based on FPGAs. In: 2012 4th IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 112–117 (2012)
Lee, W.W., Cabibihan, J., Thakor, N.V.: Bio-mimetic strategies for tactile sensing. In: SENSORS 2013, pp. 1–4. IEEE (2013)
Sankar, S., et al.: Texture discrimination with a soft biomimetic finger using a flexible neuromorphic tactile sensor array that provides sensory feedback. Soft Robot. 8(5), 577–587 (2021)
Liu, H., Guo, D., Sun, F.: Object recognition using tactile measurements: kernel sparse coding methods. IEEE Trans. Instrum. Meas. 65(3), 656–665 (2016)
Xu, Z., Chen, M., Liu, C.: Object tactile character recognition model based on attention mechanism LSTM. In: 2020 Chinese Automation Congress (CAC), pp. 7095–7100 (2020)
Liu, H., Greco, J., Song, X., Bimbo, J., Seneviratne, L., Althoefer, K.: Tactile image based contact shape recognition using neural network. In: 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 138–143 (2012)
Du, Y., Zhang, G., Zhang, Y., Wang, M.Y.: High-resolution 3-dimensional contact deformation tracking for FingerVision sensor with dense random color pattern. IEEE Robot. Autom. Lett. 6(2), 2147–2154 (2021)
Zhang, Y., Yang, Y., He, K., Zhang, D., Liu, H.: Specific surface recognition using custom finger vision. In: 2020 International Symposium on Community-centric Systems (CcS), pp. 1–6 (2020)
Yang, Y., Wang, X., Zhou, Z., Zeng, J., Liu, H.: An enhanced FingerVision for contact spatial surface sensing. IEEE Sens. J. 21(15), 16492–16502 (2021)
Dahiya, R.S., Metta, G., Valle, M., Sandini, G.: Tactile sensing-from humans to humanoids. IEEE Trans. Robot. 26(1), 1–20 (2010)
Huang, X., et al.: Neuromorphic vision based contact-level classification in robotic grasping applications. Sensors 20(17), 4724 (2020)
Johnson, M.K., Adelson, E.H.: Retrographic sensing for the measurement of surface texture and shape. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1070–1077 (2009)
Jia, X., Li, R., Srinivasan, M.A., Adelson, E.H.: Lump detection with a gelsight sensor. In: 2013 World Haptics Conference (WHC), pp. 175–179 (2013)
Li, R., et al.: Localization and manipulation of small parts using gelsight tactile sensing. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3988–3993 (2014)
Dong, S., Yuan, W., Adelson, E.H.: Improved gelsight tactile sensor for measuring geometry and slip. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 137–144 (2017)
Yamaguchi, A., Atkeson, C.G.: Implementing tactile behaviors using FingerVision. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 241–248 (2017)
Yamaguchi, A., Atkeson, C.G.: Tactile behaviors with the vision-based tactile sensor FingerVision. Int. J. Humanoid Robot. 16(03), 1940002 (2019)
da Rocha, J.G.V., da Rocha, P.F.A., Lanceros-Mendez, S.: Capacitive sensor for three-axis force measurements and its readout electronics. IEEE Trans. Instrum. Meas. 58(8), 2830–2836 (2009)
Gupta, A.K., Nakagawa-Silva, A., Lepora, N.F., Thakor, N.V.: Spatio-temporal encoding improves neuromorphic tactile texture classification. IEEE Sens. J. 21(17), 19038–19046 (2021)
Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge (2002)
Johansson, R.S., Birznieks, I.: First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nat. Neurosci. 7(2), 170–177 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X., Yang, Y., Zhou, Z., Xiang, G., Liu, H. (2022). A Soft Neuromorphic Approach for Contact Spatial Shape Sensing Based on Vision-Based Tactile Sensor. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13457. Springer, Cham. https://doi.org/10.1007/978-3-031-13835-5_58
Download citation
DOI: https://doi.org/10.1007/978-3-031-13835-5_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13834-8
Online ISBN: 978-3-031-13835-5
eBook Packages: Computer ScienceComputer Science (R0)