Skip to main content
SpringerLink
Log in

Egocentric-Vision based Hand Posture Control System for Reconnaissance Robots

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

To facilitate full-loaded commandos to control reconnaissance robots, in this paper, we propose a wearable hand posture control system based on egocentric-vision by imitating the sign language interaction way among commandos. Considering the characteristics of the egocentric-vision on the battlefield, such as complicated backgrounds, large ego-motions and extreme transitions in lighting, a new hand detector based on Binary Edge HOG Block (BEHB) features is proposed to extract articulated postures from the egocentric-vision. Different from many other methods that use skin color cues, our proposed hand detector adopts contour cues and part-based voting idea. This means that our algorithm can be used on the battlefield even in dark environment, because infrared cameras can be used to get contour images rather than skin color images. The experiment result shows that the proposed hand detector can get a better posture detection result on the NUS hand posture dataset II. To improve hand recognition accuracy, a deep ensemble hybrid classifier is proposed by combing hybrid CNN-SVM classifier and ensemble technique. Compared with other state-of-art algorithms, the proposed classifier yields a recognition accuracy of 97.72 % on the NUS hand posture dataset II. At last, to reduce misjudgments during consecutive posture switches, a vote filter is proposed and applied to the sequence of the recognition results. The scout experiment shows that our wearable hand posture control system is more suitable than traditional hand-held controllers for full-loaded commandos to control reconnaissance robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)

    Article  Google Scholar 

  2. Li, C., Kitani, K.M.: Pixel-level hand detection in ego-centric videos. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2013)

  3. Betancourt, A., Lopez, M.M., Regazzoni, C.S., Rauterberg, M.: A sequential classifier for hand detection in the framework of egocentric vision. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2014)

  4. Dardas, N.H., Georganas, N.D.: Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE T Instrum. Meas. 60(11), 3592–3607 (2011)

    Article  Google Scholar 

  5. Kontschieder, P., Riemenschneider, H., Donoser, M., Bischof, H.: Discriminative Learning of Contour Fragments for Object Detection. Paper presented at the Proceedings Of The British Machine Vision Conference (2011)

  6. Shotton, J., Blake, A., Cipolla, R.: Contour-based learning for object detection. Paper presented at the IEEE International Conference on Computer Vision (2005)

  7. Haydar, J., Dalal, B., Hussainy, S., Khansa, L.E.: ASL fingerspelling translator glove. Int. J. Comput. Sci. Issues (IJCSI) 9(6) (2012)

  8. Hsu, R.C., Jian, J.W., Lin, C.C., Lai, C.H., Liu, C.T.: Remotely controlling of mobile robots using gesture captured by the Kinect and recognized by machine learning method. Paper presented at the Proceedings of SPIE (2013)

  9. Wang, C., Liu, Z., Chan, S.C.: Superpixel-based hand gesture recognition with kinect depth camera. IEEE T. Multimed. 17(1), 29–39 (2015)

    Article  Google Scholar 

  10. Yang, H.D., Park, A.Y., Lee, S.W.: Gesture spotting and recognition for human-robot interaction. IEEE T. Robot. 23(2), 256–270 (2007)

    Article  Google Scholar 

  11. Li, S.Z., Yu, B., Wu, W., Su, S.Z., Ji, R.R.: Feature learning based on SAE-PCA network for human gesture recognition in RGBD images. Neurocomputing 151(2), 565–573 (2015)

    Article  Google Scholar 

  12. Paulraj, M.P., Yaacob, S., Desa, H., Hema, C.R., Ab Majid, W.: Extraction of head and hand gesture features for recognition of sign language. Paper presented at the International Conference On Electronic Design (2008)

  13. Pisharady, P.K., Vadakkepat, P., Loh, A.P.: Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds. INT J. Comput. Vis. 101(3SI), 403–419 (2013)

    Article  Google Scholar 

  14. Leibe, B., Leonardis, A., Schiele, B.: Robust object detection with interleaved categorization and segmentation. INT J. Comput. Vis. 77(1-3), 259–289 (2008)

    Article  Google Scholar 

  15. Opelt, A., Pinz, A., Zisserman, A.: A boundary-fragment-model for object detection. Paper presented at the 9th European Conference on Computer Vision (ECCV) (2006)

  16. Yarlagadda, P., Monroy, A., Ommer, B.: Voting by grouping dependent parts. Paper presented at the 11th European Conference on Computer Vision(ECCV) (2010)

  17. Schlecht, J., Ommer, B.: Contour-based object detection. Paper presented at the 22nd British Machine Vision Conference (2011)

  18. Gu, C.H., Lim, J.J., Arbelaez, P., Malik, J.: Recognition using Regions Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2009)

  19. Mittal, A., Zisserman, A., Torr, P.: Hand detection using multiple proposals. Paper presented at the 22nd British Machine Vision Conference (2011)

  20. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. Paper presented at the IEEE COMPUTER SOCIETY Conference On Computer Vision And Pattern Recognition (2005)

  21. Tang, A., Lu, K., Wang, Y.F., Huang, J., Li, H.Q.: A real-time hand posture recognition system using deep neural networks. ACM T Intel. Syst. Tec. 6(212) (2015)

  22. Niu, X.X., Suen, C.Y.: A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn. 45(4), 1318–1325 (2012)

    Article  Google Scholar 

  23. Huang, F.J., Lecun, Y.: Large-scale learning with SVM and convolutional for generic object categorization. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)

  24. Wozniak, M., Grana, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inform. Fusion 16(SI), 3–17 (2014)

    Article  Google Scholar 

  25. Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (2012)

  26. Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. Paper presented at the 11th International Conference on Document Analysis and Recognition (ICDAR) (2011)

  27. Maqsood, I., Khan, M.R., Abraham, A.: An ensemble of neural networks for weather forecasting. Neural Comput. Appl. 13(2), 112–122 (2004)

    Article  Google Scholar 

  28. Qian, K., Song, A.G., Bao, J.T., Zhang, H.T.: Small Teleoperated Robot for Nuclear Radiation and Chemical Leak Detection. Int. J Adv. Robot. Syst. 9(70) (2012)

  29. Maire, M., Arbelaez, P., Fowlkes, C., Malik, J.: Using contours to detect and localize junctions in natural images. Paper presented at the IEEE COMPUTER Society Conference On Computer Vision And Pattern Recognition (2008)

  30. Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE T Pattern Anal. 26(5), 530–549 (2004)

    Article  Google Scholar 

  31. Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM T Intel. Syst. Tec. 2(273SI) (2011)

  32. Sancho-Asensio, A., Orriols-Puig, A., Golobardes, E.: Robust on-line neural learning classifier system for data stream classification tasks. Soft Comput. 18(8), 1441–1461 (2014)

    Article  Google Scholar 

  33. Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., Poggio, T.: Robust object recognition with cortex-like mechanisms. IEEE T Pattern Anal. 29(3), 411–426 (2007)

    Article  Google Scholar 

  34. Triesch, J., von der Malsburg, C.: A system for person-independent hand posture recognition against complex backgrounds. IEEE T Pattern Anal. 23(12), 1449–1453 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Instrument Science and Engineering, Southeast University, Nanjing, Jiangsu Province, China

    Peng Ji, Aiguo Song, Ping Yi, Xiaonong Xu & Huijun Li

  2. Information Engineering School, Nanchang University, Nanchang, China

    Pengwen Xiong

Authors
  1. Peng Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aiguo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengwen Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ping Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaonong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huijun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aiguo Song.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, P., Song, A., Xiong, P. et al. Egocentric-Vision based Hand Posture Control System for Reconnaissance Robots. J Intell Robot Syst 87, 583–599 (2017). https://doi.org/10.1007/s10846-016-0440-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-016-0440-2

Keywords

Search

Navigation