Abstract
Deep learning models are considered a revolutionary learning paradigm in artificial intelligence and machine learning, piquing the interest of image recognition and computer vision experts. Because deep learning models have gained popularity and improved outcomes in the literature, this work provides a deep learning method based on a holistic approach to recognize offline handwritten Gurumukhi words. The holistic approach to word recognition treats a word as a separate entity rather than its component letters. Three characteristics are extracted from word pictures to train a Convolutional Neural Network (CNN), namely, zoning, diagonal, and centroid. Five performance measures are used to assess trained CNN performance, namely, Accuracy, True Positive Rate (TPR), False Positive Rate (FPR), Root Mean Square Error (RMSE), and Area Under Curve (AUC). The proposed model is trained and assessed using a 40,000 words benchmark dataset based on 70:30 partitioning technique, in which 70% of the data is used to train the model and 30% of the data is used to test the trained model. To assess the efficacy of the suggested technique, a fivefold cross validation process is performed. Using the partitioning method and cross-validation approach, the best accuracy rates of 95.11% and 94.96% are obtained after 30 epochs, respectively which surpassed the existing state-of-the-art offline handwritten Gurumukhi word recognition systems.
Similar content being viewed by others
Data availability
The datasets generated during the current study are available from the corresponding author on reasonable request.
References
Abdurahman F, Sisay E, Fante KA (2021) AHWR-Net: Offline Handwritten Amharic Word Recognition using Convolutional Recurrent Neural Network. SN Appl. Sci. 3(8):1–11
Acharyya A, Rakshit S, Sarkar R, Basu S, Nasipuri M (2013) Handwritten Word Recognition Using MLP Based Classifier: A Holistic Approach. Int. J. Comput. Sci Issues 10(2):422–427
Akbarpour S (2011) Improved feature extraction and lexicon reduction methods classified by support vector machine for Farsi handwritten word recognition system. Dissertation, University Putra Malaysia
AlbawiS, Mohammed TA and Al-Zawi S (2017) Understanding of a Convolutional Neural Network. Proceedings of IEEE Int. Conf. Eng. Technol (ICET), 1–6
Alkhawaldeh RS (2021) Arabic (Indian) digit handwritten recognition using recurrent transfer deep architecture. Soft Comput 25(4):3131–3141
Almaza´n J, Gordo A, Forne´s A and Valveny E (2014) Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mac, 36(12):2552–2566
Almodfer R, Xiong S, Mudhsh M and Duan P (2017) Multi-column deep neural network for offline Arabic handwriting recognition. Proc Int Conf Artificial Neural Netw 260–267
Amrouch M and Rabi M (2017) Deep neural networks features for Arabic handwriting recognition. Proc Int Conf Adv. Inf. Syst. 138–149
Arani SA, Kabir E, Ebrahimpour R (2019) Handwritten Farsi word recognition using NN-based fusion of HMM classifiers with different types of features. Int J Image Graph 19(01):1950001
Awni M, Khalil MI and Abbas HM (2019) Deep-learning ensemble for offline Arabic handwritten words recognition. Proceedings of IEEE 14th International Conf Comput Eng Syst (ICCES), 40–45
Bhowmik S, Polley S, Roushan MG, Malakar S, Sarkar R, Nasipuri M (2015) A holistic word recognition technique for handwritten Bangla words. Int. J. Pattern Recognit (IJAPR) 2(2):142–159
Bhowmik S, Malakar S, Sarkar R, Basu S, Kundu M, Nasipuri M (2019) Off-Line Bangla Handwritten Word Recognition: A Holistic Approach. Neural Comput Appl 31:5783–5798
Bianne-Bernard AL, Menasri F, Mohamad RAH, Mokbel C, Kermorvant C, Likforman-Sulem L (2011) Dynamic and contextual information in HMM modeling for handwritten word recognition. IEEE Trans Pattern Anal Mach Intell 33(10):2066–2080
BlucheT and Messina R (2017) Gated Convolutional Recurrent Neural Networks for Multilingual Handwriting Recognition. Proceedings of IEEE 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) 1:646–651
BlucheT, Ney H and Kermorvant C (2013) Tandem HMM with Convolutional Neural Network for handwritten word recognition. Proc IEEE Int Conf Acoust, Speech and Signal Processing, 2390–2394
Bluche T, Ney H and Kermorvant C (2014) A Comparison of Sequence-Trained Deep Neural Networks and Recurrent Neural Networks Optical Modeling for Handwriting Recognition. Proc Int Conf Statistical Language Speech Process 199–210
Blumenstein M, Verma B (1999) A New Segmentation Algorithm for Handwritten Word Recognition. Proc. Int. Jt. Conf. Neural Netw’99(IJCNN’99) 4:2893–2898
Bonyani M, Jahangard S, Daneshmand M (2021) Persian handwritten digit, character and word recognition using deep learning. Int J Doc Anal Recognit (IJDAR) 24(1):133–143
Boualam M, Elfakir Y, Khaissidi G, Mrabti M (2022) Arabic Handwriting Word Recognition Based on Convolutional Recurrent Neural Network. Proceedings of WITS 2020:877–885
BouazizS, Mezghani A and Kanoun S (2014) Arabic Handwritten Word Recognition with Large Vocabulary Based on Explicit Segmentation. Proc Int Conf Info Communication Technologies Innovation and Application (ICTIA), 1–4
Bozinovic MR, Srihari NS (1989) Off-Line Cursive Script Word Recognition. IEEE Trans Pattern Anal Mach Intell 11(1):68–83
Chergui L, Kef M (2015) SIFT descriptors for Arabic handwriting recognition. Int J Comput Vis Robot 5(4):441–461
Cilia ND, De Stefano C, Fontanella F, Marrocco C, Molinara M, Scotto Di Freca A (2020) An end-to- end deep learning system for medieval writer identification. Pattern Recogn Lett 129:137–143
Dasgupta J, Bhattacharya K, Chanda B (2016) A Holistic Approach for Off-Line Handwritten Cursive Word Recognition using Directional Feature based on Arnold Transform. Pattern Recogn Lett 79:73–79
De Oliveira JJ, de A Freitas CO, de Carvalho JM and Sabourin R (2009) Handwritten Word Recognition Using Multi-view Analysis. In: Bayro-Corrochano E., Eklundh JO. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2009. Lecture Notes in Computer Science, vol 5856. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10268-4_44
Dreuw P, Rybach D, Heigold G, Ney H (2012) RWTH OCR: A Large Vocabulary Optical Character Recognition System for Arabic Scripts. In: Märgner V, El Abed H (eds) Guide to OCR for Arabic Scripts. Springer, London, pp 215–254
Edelman S, Flash T, Ullman S (1990) Reading Cursive Handwriting by Alignment of Letter Prototypes. Int J Comput Vision 5(3):303–331
El-Sawy A, Hazem EB and Loey M (2016) CNN for handwritten Arabic digits recognition based on LeNet-5. Proc Inte Conf Adv. Intell. Syst Inform 566–575
Elleuch M, Tagougui N and Kherallah M (2015) Towards unsupervised learning for Arabic handwritten recognition using deep architectures. Proc 22nd Int Conf Neural Info Proc Syst 363–372
Elleuch M, Maalej R, Kherallah M (2016) A new design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition. Procedia Comput. Sci. 80:1712–1723
Espana-Boquera S, Castro-Bleda MJ, Gorbe-Moya J, Zamora-Martinez F (2011) Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN models. IEEE Trans Pattern Anal Mach Intell 33(4):767–779
Ghadikolaie MFY, Kabir E, Razzazi F (2016) Sub-word based offline handwritten Farsi word recognition using recurrent neural network. ETRI J 38(4):703–713
Ghosh M, Malakar S, Bhowmik S, Sarkar R and Nasipuri M (2019) Feature Selection for Handwritten Word Recognition Using Memetic Algorithm. In: Mandal J., Dutta P., Mukhopadhyay S. (eds) Advances in Intelligent Computing. Studies in Computational Intelligence, vol 687. Springer, Singapore. https://doi.org/10.1007/978-981-10-8974-9_6
Gimenez A, Khoury I, Andres-Ferrer J, Juan A (2012) Handwriting word recognition using windowed Bernoulli HMMs. Pattern Recogn Lett 35(1):149–156
Golzari S, Khalili A, Sabzi R (2022) Combining convolutional neural networks with SVM classifier for recognizing Persian and Arabic handwritten words. Multimedia Tools and Applications 81:33785–33799
Haghighi F, Omranpour H (2021) Stacking ensemble model of deep learning and its application to Persian/ Arabic handwritten digits recognition. Knowledge Based Systems 220:106940
Hossain MT, Hasan MW and Das AK (2021) Bangla Handwritten Word Recognition System Using Convolutional Neural Network. Proc 15th Inte Conf Ubiquitous Info Manag Communication (IMCOM), 1–8
Imani Z, Ahmadyfard A, Zohrevand A (2016) Holistic Farsi handwritten word recognition using gradient features. Journal of AI and Data Mining 4(1):19–25
JayadevanR, Kolhe SR, Patil PM and Pal U (2011) Database development and recognition of handwritten Devanagari legal amount words. Proc Inter Conf Doc. Anal. Recognit (ICDAR), 304–308.
Jayech K, Mahjoub M, Amara NB (2016) Arabic Handwritten Word Recognition Based on Dynamic Bayesian Network. Inter Arab J Info Technol 13(6B):1024–1031
Jino PJ, Balakrishnan K (2017) Offline Handwritten Recognition of Malayalam District Name - A Holistic Approach. Int J Eng Technol 9(2):1–8
Jino PJ, Balakrishnan K and Bhattacharya U (2019) Offline Handwritten Malayalam Word Recognition Using a Deep Architecture. In: Bansal J., Das K., Nagar A., Deep K., Ojha A. (eds) Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing, vol 816. Springer, Singapore. https://doi.org/10.1007/978-981-13-1592-3_73
Kaur H and Kumar M (2019) Benchmark Dataset: Offline Handwritten Gurmukhi City Names for Postal Automation. In: Sundaram S., Harit G. (eds) Document Analysis and Recognition. DAR 2018. Communications in Computer and Information Science, vol 1020. Springer, Singapore. https://doi.org/10.1007/978-981-13-9361-7_14
Kaur H, Kumar M (2021) Offline Handwritten Gurumukhi Word Recognition using eXtreme Gradient Boosting Methodology. Soft Comput 25:4451–4464
Kaur H, Kumar M (2021) On the Recognition of Offline Handwritten Word using Holistic Approach and AdaBoost Methodology. Multimedia Tools and Applications 80:11155–11175
Kaur H and Kumar M (2021c) Performance Evaluation ofVarious Feature Selection Techniques for Offline Handwritten Gurumukhi Place Name Recognition. In: Singh T.P., Tomar R., Choudhury T., Perumal T., Mahdi H.F. (eds) Data Driven Approach Towards Disruptive Technologies. Studies in Autonomic, Data- driven and Industrial Computing. Springer, Singapore. https://doi.org/10.1007/978-981-15-9873- 9_44
Khosravi S, Chalechale A (2022) Chimp Optimization Algorithm to Optimize a Convolutional Neural Network for Recognizing Persian/Arabic Handwritten Words. Math Probl Eng. https://doi.org/10.1155/2022/4894922
Kumar R and Sharma R (2013) An efficient post processing algorithm for online handwriting Gurmukhi character recognition using set theory. International Journal of Int. J. Pattern Recognit. Artif. 27(4): 1353002–1–1353002–17
Kumar N, Gupta S (2018) Offline Handwritten Gurmukhi Word Recognition Using Deep Neural Networks. Int J Pure Appl Math 119(12):14749–14767
Kumar M, Jindal MKand Sharma RK (2011) k-nearest neighbor based offline handwritten Gurmukhi character recognition. Proc Int Conf Image Info Proc 1–4.
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444
Lee H, Verma B (2011) Binary Segmentation Algorithm for English Cursive Handwriting Recognition. Pattern Recogn 45(4):1306–1317
Leila C, Maamar K and Salim C (2011) Combining neural networks for Arabic handwriting recognition. Proc IEEE 10th Int Sympos Program Syst 74–79
Loey M, El-Sawy A and EL-Bakry H (2017) Deep learning autoencoder approach for handwritten Arabic digits recognition. arXiv preprint arXiv:1706.067
Maalej R and Kherallah M (2019) Maxout into MDLSTM for offline Arabic handwriting recognition. Proc Int Conf Neural Info Process 534-545
Madhvanath S, Govindaraju V (2001) The Role of Holistic Paradigms in Handwritten Word Recognition. IEEE Trans Pattern Anal Mach Intell 23(2):149–164
Mhiri M, Desrosiers C, Cheriet M (2018) Convolutional pyramid of bidirectional character sequences for the recognition of handwritten words. Pattern Recogn Lett 111:87–93
Mondal R, Malakar S, Barney Smith EH, Sarkar R (2022) Handwritten English word recognition using a deep learning based object detection architecture. Multimedia Tools and Applications 81(1):975–1000
Mustafa ME, Elbashir MK (2020) A Deep Learning Approach for Handwritten Arabic Names Recognition. Int J Adv Comput Sci App (IJACSA) 11(1):678–682
Namane A, Guessoum A and Meyrueis P (2005) New Holistic Handwritten Word Recognition and Its Application to French Legal Amount. In: Singh S., Singh M., Apte C., Perner P. (eds) Pattern Recognition and Data Mining. ICAPR 2005. Lecture Notes in Computer Science, vol 3686, Springer, Berlin, Heidelberg. https://doi.org/10.1007/11551188_72
Nanehkaran YA, Zhang D, Salimi S, Chen J, Tian Y, Al-Nabhan N (2021) Analysis and comparison of machine learning classifiers and deep neural networks techniques for recognition of Farsi handwritten digits. J Supercomput 77(4):3193–3222
Nurseitov D, Bostanbekov K, Kanatov M, Alimova A, Abdallah A, Abdimanap G (2020) Classification of Handwritten Names of Cities and Handwritten Text Recognition using Various Deep Learning Models. Adv Sci Technol Eng Syst J 5(5):934–943
Pal U, Roy K, Kimura F (2009) A Lexicon-Driven Handwritten City Name Recognition Scheme for Indian Postal Automation. IEICE Trans Inf Syst 92(5):1146–1158
Parseh M, Rahmanimanesh M, Keshavarzi P (2020) Persian handwritten digit recognition using combination of convolutional neural network and support vector machine methods. Int Arab J Info Technol 17(4):572–578
Patel MS, Reddy SL and Naik AJ (2015) An Efficient Way of Handwritten English Word Recognition. In: Satapathy S, Biswal B, Udgata S, Mandal J (eds) Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA) 2014. Advances in Intelligent Systems and Computing, vol 328. Springer, Cham. https://doi.org/10.1007/978-3-319-12012-6_62
Pham V, Bluche T, Kermorvant C and Louradour J (2014)Dropout Improves Recurrent Neural Networks for Handwriting Recognition. Proc 14th Int Conf Frontiers in Handwriting Recognit 285–290
Poznanski A and Wolf L (2016) CNN-N-gram for handwriting word recognition. Proc IEEE Conf Comput Vision Pattern Recognit (CVPR), 2305–2314.
Pramanik R, Bag S (2020) Segmentation-Based Recognition System for Handwritten Bangla and Devanagari Words Using Conventional Classification and Transfer Learning. IET Image Proc 14(5):959–972
Pramanik R, Bag S (2021) Handwritten Bangla city name word recognition using CNN-based transfer learning and FCN. Neural Comput Appl 33:9329–9341
PuigcerverJ (2017) Are multidimensional recurrent layers really necessary for handwritten text recognition? Proc IEEE 14th IAPR Inter Conf Document Anal Recognit (ICDAR), 1:67–72
Rahal N, Tounsi M, Hamdani TM and Alimi AM (2019)Handwritten words and digits recognition using Deep Learning based Bag of Features Framework. Proc Int Conf Doc Anal Recognit (ICDAR), 701–706.
Rothacker L and Fink GA (2016) Robust Output Modeling in Bag-of-Features HMMs for Handwriting Recognition. Proc 15th Int Conf Frontiers in Handwriting Recognit (ICFHR), 199–204
Roy A, Bhowmik KT, Parui KS and Roy U (2005a) A Novel Approach to Skew Detection and Character Segmentation for Handwritten Bangla Words. Proc Digital Image Computing: Techniques and Applications (DICTA), 1–8.
RoyK, Vajda S, Pal U, Chaudhuri BB and Belaid A (2005b) A System for Indian Postal Automation. Proc 8th Int Conf Doc Anal Recognit (ICDAR'05), 1060–1064
Saeed U, Muhammad T, Alghamdi AS, Alkatheiri MS (2020) Automatic recognition of handwritten Arabic using maximally stable extremal region features. Opt Eng 59(5):051405
Safarzadeh VM and Jafarzadeh P (2020) Offline Persian handwriting recognition with CNN and RNN-CTC. Proc IEEE 25th Int Comput Conf Comput Soc Iran (CSICC), 1–10
ScheidlH (2018) Handwritten text recognition in historical documents, Master’s thesis, Technische Universitat¨ Wien, Vienna, 2018, diplom-Ingenieur in Visual Computing
SharmaA, Kumar R and Sharma RK (2009) Rearrangement of recognized strokes in online handwritten Gurmukhi words recognition. Proc 10th Int Conf Doc Anal Recognit (ICDAR), 1241–1245
Sharma S, Gupta S, Gupta D, Juneja S, Singal G, Dhiman G, Kautish S (2022) Recognition of Gurmukhi handwritten city names using deep learning and cloud computing. Sci Program. https://doi.org/10.1155/2022/5945117
Shrestha A, Mahmood A (2019) Review of deep learning algorithms and architectures. IEEE Access 7:53040–53065
Singh S and Sharma A (2019) Online handwritten Gurmukhi words recognition: An inclusive study. ACM Transactions on Asian and Low-Resource Language Information Processing, 18(3):21:1–21:55
Singh S, Sharma A, Chhabra I (2016) Online handwritten Gurmukhi strokes dataset based on minimal set of words. ACM Transactions on Asian and Low-Resource Language Information Processing 16:1–20
Singh S, Chauhan VK, Smith EHB (2020) A self- controlled RDP approach for feature extraction in online handwriting recognition using deep learning. Appl Intell 50(7):2093–2104
Singh S, Sharma A, Chauhan VK (2021) Online handwritten Gurmukhi word recognition using fine-tuned Deep Convolutional Neural Network on offline features. Machine Learning with Applications 5:100037. https://doi.org/10.1016/j.mlwa.2021.100037
Sudholt S and Fink GA (2016) PHOCNet: a deep convolutional neural network for word spotting in handwritten documents. Proc 15th Int Conf Frontiers in Handwriting Recognit 277–282
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res 15(56):1929–1958
Talaat M, Wahbi and Musa MEM (2016) Holistic approach for Arabic word recognition. Int J Comput Appl. Res. Technol, 5(3):141–146
WafaMohamed Musa MEM A (2009) Recognition of Arabic handwritten names using Probabilistic Neural Networks. Computer Studies Journal 1(1):1–12
Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9(4):611–629
Yan R, Peng L, Xiao S, Johnson MT, Wang S (2019) Dynamic temporal residual network for sequence modeling. International Journal on Document Analysis and Recognition (IJDAR) 22(3):235–246
Zamani Y, Souri Y, Rashidi H and Kasaei S (2015) Persian handwritten digit recognition by random Forest and convolutional neural networks. Proc 9th Iranian IEEE Conf Machine Vision and Image Process 37–40
Zhang TY, Suen CY (1984) A Fast Parallel Algorithm for Thinning Digital Patterns. Commun ACM 27(3):236–239
Zohrevand A, Imani Z (2021) Holistic Persian Handwritten Word Recognition Using Convolutional Neural Network. Int J Eng 34(8):2028–2037
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
The authors have created their own dataset for performing the experiments in the considered work.
Informed consent
All the authors are agreed for this submission.
Conflict of interest
The authors declare that they have no conflict of interest in this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kaur, H., Bansal, S., Kumar, M. et al. Worddeepnet: handwritten gurumukhi word recognition using convolutional neural network. Multimed Tools Appl 82, 46763–46788 (2023). https://doi.org/10.1007/s11042-023-15527-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-15527-2