Abstract
In this paper, we propose a method that estimates contact force to hard surface from a single visual image of a finger by using a neural network. In general, it is hard to estimate applied force to hard object only from visual images as the object surface hardly moves. In this paper, we focus on the human side. When persons push an object, posture of hand reflects how hard he/she pushes the surface. Observation of human body condition will tell the haptic information. We used the Convolutional Neural Network to make the system learn the relationship between the applied force and the finger posture. We created a neural network model individually. The evaluation result shows that a root mean square error from the actual force is approximately 0.5 N for the best case, which is 2.5% to the dynamic range (0–20 N) of applied force.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kazuma, Y., Keisuke, H., Hiroyuki, S.: Measuring visio-tactile threshold for visio-tactile projector. In: Proceedings of SICE Annual Conference 2012, pp. 1996–2000 (2012)
Stephen, M., Harry, A.: Measurement of finger posture and three-axis fingertip touch force using fingernail sensors. IEEE Trans. Robot. Autom. 20, 26–35 (2004)
Thomas, G., John, H., Stephen, M.: 3-D fingertip touch force prediction using fingernail imaging with automated calibration. IEEE Trans. Rob. 31, 1116–1129 (2015)
Thomas, G., Lucas, L., Yu, S., John, H., Stephen, M.: 3D force prediction using fingernail imaging with automated calibration. In: 2010 IEEE Haptics Symposium, pp. 113–120 (2010)
Cornelia, F., Fang, W., Yezhou, Y., Konstantinos, Z., Yi, Z., Francisco, B., Michael, P.: Prediction of Manipulation Actions. arXiv.org. https://arxiv.org/abs/1610.00759. Accessed 20 Jan 2018
Tu-Hoa, P., Nikolaos, K., Antonis, A., Abderrahmane, K.: Hand-object contact force estimation from markerless visual tracking. IEEE Trans. Patt. Anal. Mach. Intell. PP, 1 (2017)
Wonjun, H., Soo-Chul, L.: Inferring Interaction Force from Visual Information without Using Physical Force Sensors. Sensors, Basel (2017)
Florian, E., Andreas, D., Marcus, T., Gudrun, K.: Inverted FTIR: easy multitouch sensing for flatscreens. In: ITS 2009 Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, pp. 29–32 (2009)
Chris, H., Hrvoje, B., Andrew, W.: OmniTouch: wearable multitouch interaction everywhere. In: UIST 2011 Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 441–450 (2011)
Alex, K., Ilya, S., Geoffrey, H.: ImageNet classification with deep convolutional neural networks. In: NIPS 2012 Proceedings of the 25th International Conference on Neural Information Processing Systems, vol. 1, pp. 1097–1105 (2012)
Alex, G.: Generating Sequences With Recurrent Neural Network. arXiv.org. https://arxiv.org/abs/1308.0850. Accessed 20 Jan 2018
Shaoqing, R., Kaiming, H., Ross, G., Jian, S.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.org. https://arxiv.org/abs/1506.01497. Accessed 20 Jan 2018
Acknowledgments
This work is supported by JST PRESTO 17939983.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Inoue, Y., Makino, Y., Shinoda, H. (2018). Estimation of the Pressing Force from Finger Image by Using Neural Network. In: Prattichizzo, D., Shinoda, H., Tan, H., Ruffaldi, E., Frisoli, A. (eds) Haptics: Science, Technology, and Applications. EuroHaptics 2018. Lecture Notes in Computer Science(), vol 10894. Springer, Cham. https://doi.org/10.1007/978-3-319-93399-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-93399-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93398-6
Online ISBN: 978-3-319-93399-3
eBook Packages: Computer ScienceComputer Science (R0)