Abstract
This paper investigates on the deep learning approaches for the social touch gesture recognition. Several types of neural network architectures are studied with a comprehensive experiment design. First, recurrent neural network using long short-term memory (LSTM) is adopted for modeling the gesture sequence. However, for both handcrafted features using geometric moment and feature extraction using convolutional neural network (CNN), LSTM cannot achieve satisfactory performances. Therefore, we propose to use the 3D CNN to model a fixed length of touch gesture sequence. Experimental results show that the 3D CNN approach can achieve a recognition accuracy of 76.1 % on the human-animal affective robot touch (HAART) database in the recognition of social touch gestures challenge 2015, which significantly outperforms the best submitted system of the challenge with a recognition accuracy of 70.9 %.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balli Altuglu, T., Altun, K.: Recognizing touch gestures for social human-robot interaction. In: Proceedings of the 2015 ACM International Conference on Multimodal Interaction (ICMI), pp. 407–413 (2015)
Cooney, M.D., Nishio, S., Ishiguro, H.: Recognizing affection for a touch-based interaction with a humanoid robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1420–1427 (2012)
Billard, A., Bonfiglio, A., Cannata, G., Cosseddu, P., Dahl, T., Dautenhahn, K., Mastrogiovanni, F., Metta, G., Natale, L., Robins, B., et al.: The ROBOSKIN project: challenges and results. In: Padois, V., Bidaud, P., Khatib, O. (eds.) Romansy 19–Robot Design, Dynamics and Control. CISM International Centre for Mechanical Sciences, pp. 351–358. Springer, Vienna (2013)
Knight, H., Toscano, R., Stiehl, W.D., Chang, A., Wang, Y., Breazeal, C.: Real-time social touch gesture recognition for sensate robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3715–3720 (2009)
Jung, M.M., Cang, X.L., Poel, M., MacLean, K.E.: Touch challenge ‘15: recognizing social touch gestures. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI), pp. 387–390 (2015)
Ta, V.-C., Johal, W., Portaz, M., Castelli, E., Vaufreydaz, D.: The Grenoble system for the social touch challenge at ICMI 2015. In: Proceedings of the 2015 ACM International Conference on Multimodal Interaction (ICMI), pp. 391–398 (2015)
Hughes, D., Farrow, N., Profita, H., Correll, N.: Detecting and identifying tactile gestures using deep autoencoders, geometric moments and gesture level features. In: Proceedings of the 2015 ACM International Conference on Multimodal Interaction (ICMI), pp. 415–422 (2015)
Falinie, Y. Gaus, A., Olugbade, T., Jan, A., Qin, R., Liu, J., Zhang, F., Meng, H., Bianchi-Berthouze, N.: Social touch gesture recognition using random forest and boosting on distinct feature sets. In: Proceedings of the 2015 ACM International Conference on Multimodal Interaction (ICMI), pp. 399–406 (2015)
Altuglu, T.B., Altun, K.: Recognizing touch gestures for social human-robot interaction. In: Proceedings of the 2015 ACM International Conference on Multimodal Interaction (ICMI), pp. 407–413 (2015)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Yu, K., Xu, W., Gong, Y.: Deep learning with kernel regularization for visual recognition. In: NIPS, pp. 1889–1896 (2008)
Ahmed, A., Yu, K., Xu, W., Gong, Y., Xing, E.: Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 69–82. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88690-7_6
Mobahi, H., Collobert, R., Weston, J.: Deep learning from temporal coherence in video. In: ICML, pp. 737–744 (2009)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Teh, C.-H., Chin, R.T.: On image analysis by the methods of moments. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 496–513 (1988)
Donahue, J., Hendricks, L.A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description (2014). CoRR, abs/1411.4389
Ji, S., Xu, W., Yang, M., Yu, K.: 3d convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors (2012). CoRR, abs/1207.0580
He, K., et al.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhou, N., Du, J. (2016). Recognition of Social Touch Gestures Using 3D Convolutional Neural Networks. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds) Pattern Recognition. CCPR 2016. Communications in Computer and Information Science, vol 662. Springer, Singapore. https://doi.org/10.1007/978-981-10-3002-4_14
Download citation
DOI: https://doi.org/10.1007/978-981-10-3002-4_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3001-7
Online ISBN: 978-981-10-3002-4
eBook Packages: Computer ScienceComputer Science (R0)