Skip to main content
SpringerLink
Log in

Arabic sign language continuous sentences recognition using PCNN and graph matching

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Many previous researchers have tried developing sign languages recognition systems in general and Arabic sign language specifically. They succeeded to achieve acceptable results for isolated gestures level, but none of them investigated the recognition of connected sequence of gestures. This paper focuses on how to recognize real-time connected sequence of gestures using graph-matching technique, also how the continuous input gestures are segmented and classified. Graphs are a general and powerful data structure useful for the representation of various objects and concepts. This work is a component of a real-time Arabic Sign Language Recognition system that applied pulse-coupled neural network for static posture recognition in its first phase. This work can be adapted and applied to different sign languages and other recognition problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Doner B (1993) Hand shape identification and tracking for sign language interpretation. Looking at People Workshop, Chambery

    Google Scholar 

  2. Bauer B, Hienz H (2000) Relevant features for video-based continuous sign language recognition. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition, pp 64–75

  3. Chao M-W, Lin C-H, Chang C–C, Lee T-Y (2011) A graph-based shape matching scheme for 3D articulated objects. Comput Animat Virtual Worlds 22(2–3):295–305

    Google Scholar 

  4. Gupta M (2011) Design pattern mining using greedy algorithm for multi-labelled graphs. Int J Inf Commun Technol 3(4):314–323

    Google Scholar 

  5. Bauer B, Hienz H (2000) Relevant features for video-based continuous sign language recognition. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition, pp 64–75

  6. Tanibata N, Shimada N, Shirai Y (2001) Extraction of hand features for recognition of sign language words. In: Proceedings of the 15th international conference on vision interface. Calgary, Canada

    Google Scholar 

  7. Chen F-S, Fu C-M, Huang C-L (2003) Hand gesture recognition using a real-time tracking method and hidden Markov models. Image Vis Comput 21:745–758

    Google Scholar 

  8. Zieren J, Kraiss K-F (2004) Non-intrusive sign language recognition for human-computer interaction. In: 9th IFAC/IFIP/IFORS/IEA symposium analysis, design, and evaluation of human-machine systems, Atlanta, GA, pp 221–228

  9. Zieren J. Kraiss, K-F (2005) Robust person- independent visual sign language recognition, proceedings of pattern recognition and image analysis, second Iberian conference, Estoril, Portugal

  10. Dreuw P, Forster J, Gweth Y, Stein D, Ney H, Martínez G, Vergés-Llahí J, Crasborn O, Ormel E, Du W, Hoyoux T, Piater J, Moya JM, Wheatley M (2010) SignSpeak understanding, recognition, and translation of sign languages. In: 4th Workshop on the representation and processing of sign languages: corpora and sign language technologies, language resources and evaluation conference (LREC)

  11. Al-Rousan M, Aljarrah O, Hussain M (2001) Automatic recognition of arabic sign language finger spelling. Int J Comput Their Appl 8(2):80–88 (Issue 1076-5204)

    Google Scholar 

  12. Jarrah O, Halawani A (2001) Recognition of gestures in Arabic Sign Language using neuro-fuzzy systems. Artif Intell, pp 117–138

  13. Assaleh K, Al-Rousan M (2005) Recognition of Arabic sign language alphabet using polynomial classifiers. EURASIP J Appl Signal Process Soc 13:2136–2145

    Article  Google Scholar 

  14. Mohandes M, Deriche M (2005) Image based Arabic sign language recognition. Signal processing and its applications, 2005. In: Proceedings of the eighth international symposium, vol 1, pp 86–89

  15. Tolba MF, Abdellwahab MS, Abulle-Ela M, Samir A (2010) Image signature improving by PCNN for Arabic sign language recognition. Can J Artif Intell Mach Learn Pattern Recogn 1(1):1–6

    Google Scholar 

  16. Official Microsoft Xbox website, introduction of Kinect, http://www.xbox.com/en-US/kinect

  17. Countdown to Kinect: 17 Controller-Free Games Launch in November. Microsoft News Center, https://www.microsoft.com/presspass/press/2010/oct10/10-18mskinectuspr.mspx

  18. Kinect Fact Sheet, Microsoft News Center, June 2010, http://www.microsoft.com/presspass/presskits/xbox/docs/KinectFS.docx

  19. Integrating Speech and Hearing Challenge Individuals. YouTube channel of Dr. Natheer Khasawneh, http://www.youtube.com/user/knatheer#p/a/u/1/vVL398dUU5Q

  20. Topouzelis K, Karathanassi V, Pavlakis P, Rokos (2003) A neural network approach to oil spill detection using SAR data. In: 54th International astronautical congress, Bremen, Germany

  21. Ma YD, Li L, Wang YF (2006) The principles of pulse coupled neural networks and its application. Science Press, Beijing

    Google Scholar 

  22. Eckhorn R, Reitboeck HJ, Arndt M, Dicke P (1990) Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex. Neural Comput 2(3):293–307

    Google Scholar 

  23. Forgá R (2008) Feature generation improving by optimized PCNN, applied machine intelligence and informatics. SAMI 2008. In: 6th International symposium, pp 203–207, Issue, 21–22, Jan. 2008

  24. Kinser JM, Nguyen C (2000) Pulse image processing using centripetal autowaves. Proc SPIE 4052:278–284

    Google Scholar 

  25. Qiu H, Hancock ER (2003) Graph partition for matching. In: Proceedings of 4th international workshop on graph-based representations in pattern recognition. Springer LNCS2726, Berlin, pp 178—189

  26. Delalandre M, Trupin E, Ogier JM (2004) Local structural analysis: a primer. In: Proceedings of workshop on graphics recognition. Springer, LNCS3088, Berlin, pp 220–231

  27. Qiu H, Hancock ER (2003) Graph partition for matching. In: Proceedings of 4th international workshop on graph-based representations in pattern recognition. Springer LNCS2726, Berlin, pp 178–189

  28. Messmer BT, Bunke H (1999) A decision tree approach to graph and subgraph isomorphism detection. Pattern Recogn 32(12):1979–1998

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Scientific Computing Department, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt

    M. F. Tolba & Ahmed Samir

  2. Sadat Academy for Management Sciences, Cairo, Egypt

    Magdy Aboul-Ela

Authors
  1. M. F. Tolba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed Samir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Magdy Aboul-Ela
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Samir.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tolba, M.F., Samir, A. & Aboul-Ela, M. Arabic sign language continuous sentences recognition using PCNN and graph matching. Neural Comput & Applic 23, 999–1010 (2013). https://doi.org/10.1007/s00521-012-1024-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1024-0

Keywords

Search

Navigation