Skip to main content
SpringerLink
Log in

Handwritten computer science words vocabulary recognition using concatenated convolutional neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Handwriting recognition is a multi-step process that includes data collection, preprocessing, feature extraction, and classification in order to create a final prediction. This process becomes more and more delicate when dealing with the scriptures of college or secondary school learners. The primary purpose of this research is to offer an improved model for classifying images of computer science words vocabulary written by learners. Indeed, the aim is to develop a reliable handwriting recognition system for the benefit of the educational field. The proposed recognition model based on the combination of four pre-trained CNNs models, namely ResNet50 V2, MobileNet V2, ResNet101 V2, and Xception. Our earlier established Computer Science Vocabulary Dataset (CSVD) is used to build and validate the proposed concatenated model. Then, we have applied preprocessing operations to reduce irregularities, like fuzzy letters and distorted undefined symbols. The proposed CNN model is trained on the concatenated features generated by the four pre-trained CNNs using a parallel deep feature extraction approach. To evaluate the performance of our recognition system, we have used different common evaluation measures. The average accuracy of the proposed system for handwritten words vocabulary is 99.97%, and the overall loss rate is 3.56%. In addition, these performances have been compared with alternative state-of-the-art models and better performance has been observed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The details about the data used in this manuscript are available at the corresponding author.

Notes

  1. http://www.ifnenit.com/

  2. https://www.kaggle.com/mloey1/ahcd1

References

  1. Abadi M et al. (2016) “TensorFlow: A system for large-scale machine learning,” arXiv:1605.08695 [cs]. Accessed: Apr. 24, 2021. [Online]. Available: http://arxiv.org/abs/1605.08695

  2. Abbaszadeh Arani SAA, Kabir E, Ebrahimpour R (2018) Combining RtL and LtR HMMs to recognise handwritten Farsi words of small- and medium-sized vocabularies. IET Comput Vis 12(6):925–932. https://doi.org/10.1049/iet-cvi.2017.0645

    Article  Google Scholar 

  3. Almodfer R, Xiong S, Mudhsh M, Duan P (2017) Multi-column deep neural network for offline Arabic handwriting recognition. In: Lintas A, Rovetta S, Verschure PFMJ, Villa AEP (eds) Artificial neural networks and machine learning – ICANN 2017, vol 10614. Springer International Publishing, Cham, pp 260–267. https://doi.org/10.1007/978-3-319-68612-7_30

    Chapter  Google Scholar 

  4. Alrobah N, Albahli S (2021) A hybrid deep model for recognizing Arabic handwritten characters. IEEE Access 9:87058–87069. https://doi.org/10.1109/ACCESS.2021.3087647

    Article  Google Scholar 

  5. Al-Sarem M, Alsaeedi A, Saeed F, Boulila W, AmeerBakhsh O (2021) “A Novel Hybrid Deep Learning Model for Detecting COVID-19-Related Rumors on Social Media Based on LSTM and Concatenated Parallel CNNs,” Appl Sci, vol. 11, no. 17, Art no 17, https://doi.org/10.3390/app11177940.

  6. Alyafeai Z, AlShaibani MS, Ahmad I (2020) “A Survey on Transfer Learning in Natural Language Processing,” arXiv:2007.04239 [cs, stat] Accessed: Nov. 15, 2021. [Online]. Available: http://arxiv.org/abs/2007.04239

  7. Alyahya H, Ismail MMB, Al-Salman A (2020) Deep ensemble neural networks for recognizing isolated Arabic handwritten characters. TIPCV 6(21):68–79. https://doi.org/10.19101/TIPCV.2020.618051

    Article  Google Scholar 

  8. Amir M., Mousavi H, Bossaghzadeh A (2020) “Improving Persian Digit Recognition by Combining Deep Neural Networks and SVM and Using PCA,” in 2020 International conference on machine vision and image processing (MVIP), Iran, pp. 1–5. https://doi.org/10.1109/MVIP49855.2020.9116893.

  9. Aqab S, Usman M (2020) Handwriting recognition using artificial intelligence neural network and image processing. IJACSA 11(7). https://doi.org/10.14569/IJACSA.2020.0110719

  10. Balaha HM, Ali HA, Saraya M, Badawy M (2021) A new Arabic handwritten character recognition deep learning system (AHCR-DLS). Neural Comput Applic 33(11):6325–6367. https://doi.org/10.1007/s00521-020-05397-2

    Article  Google Scholar 

  11. Bianco S, Cadene R, Celona L, Napoletano P (2018) Benchmark analysis of representative deep neural network architectures. IEEE Access 6:64270–64277. https://doi.org/10.1109/ACCESS.2018.2877890

    Article  Google Scholar 

  12. Biggerstaff BJ (2008) Confidence intervals for the difference of two proportions estimated from pooled samples. JABES 13(4):478–496. https://doi.org/10.1198/108571108X379055

    Article  MathSciNet  MATH  Google Scholar 

  13. Bisong E (2019) “Introduction to Scikit-learn,” in Building machine learning and deep learning models on Google cloud platform, Berkeley, CA: Apress,pp. 215–229. https://doi.org/10.1007/978-1-4842-4470-8_18.

  14. Blockeel H, Webb GI, Auer P, Webb GI (2011) Overfitting. In: Sammut C, Webb GI (eds) Encyclopedia of machine learning. Springer US, Boston, MA, pp 744–744. https://doi.org/10.1007/978-0-387-30164-8_623

    Chapter  Google Scholar 

  15. Bonyani M, Jahangard S, Daneshmand M (2021) Persian handwritten digit, character and word recognition using deep learning. IJDAR 24(1–2):133–143. https://doi.org/10.1007/s10032-021-00368-2

    Article  Google Scholar 

  16. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30(7):1145–1159. https://doi.org/10.1016/S0031-3203(96)00142-2

    Article  Google Scholar 

  17. Burstein J (2009) Opportunities for Natural Language Processing Research in Education. In: Gelbukh A (ed) Computational Linguistics and Intelligent Text Processing, vol 5449. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 6–27. https://doi.org/10.1007/978-3-642-00382-0_2

    Chapter  Google Scholar 

  18. Cireşan D, Meier U, Schmidhuber J (2012) “Multi-column Deep Neural Networks for Image Classification,” https://doi.org/10.48550/ARXIV.1202.2745.

  19. de S. Neto AF, Bezerra BLD, Toselli AH (2020) Towards the natural language processing as spelling correction for offline handwritten text recognition systems. Appl Sci 10(21):7711. https://doi.org/10.3390/app10217711

    Article  Google Scholar 

  20. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 248–255. https://doi.org/10.1109/CVPR.2009.5206848.

  21. El Gannour O, Hamida S, Cherradi B, Raihani A, Moujahid H (2020) “Performance Evaluation of Transfer Learning Technique for Automatic Detection of Patients with COVID-19 on X-Ray Images”. In: 2020 IEEE 2nd international conference on electronics, control, optimization and computer science (ICECOCS), Kenitra, Morocco , pp. 1–6. https://doi.org/10.1109/ICECOCS50124.2020.9314458.

  22. El Gannour O et al (2021) Concatenation of Pre-Trained Convolutional Neural Networks for Enhanced COVID-19 Screening Using Transfer Learning Technique. Electronics 11(1):103. https://doi.org/10.3390/electronics11010103

    Article  Google Scholar 

  23. El-Sawy A (2017) CNN for handwritten Arabic digits recognition based on LeNet-5. In: Hassanien AE, Shaalan K, Gaber T, Azar AT, Tolba MF (eds) Proceedings of the international conference on advanced intelligent systems and informatics 2016, vol 533. Springer International Publishing, Cham, pp 566–575. https://doi.org/10.1007/978-3-319-48308-5_54

    Chapter  Google Scholar 

  24. Fan G-F, Zhang L-Z, Yu M, Hong W-C, Dong S-Q (2022) Applications of random forest in multivariable response surface for short-term load forecasting. Int J Electr Power Energy Syst 139:108073. https://doi.org/10.1016/j.ijepes.2022.108073

    Article  Google Scholar 

  25. Farahbakhsh E, Kozegar E, Soryani M (2017) “Improving persian digit recognition by combining data augmentation and AlexNet,” in 2017 10th Iranian Conference on Machine Vision and Image Processing (MVIP), Isfahan, Iran, . 265–270. https://doi.org/10.1109/IranianMVIP.2017.8342362.

  26. Fürnkranz J, … de Raedt L (2011) Model evaluation. In: Sammut C, Webb GI (eds) Encyclopedia of machine learning. Springer US, Boston, pp 683–683. https://doi.org/10.1007/978-0-387-30164-8_550

    Chapter  Google Scholar 

  27. Hamed Y, Ibrahim Alzahrani A, Shafie A, Mustaffa Z, Che Ismail M,Kok Eng K, “Two steps hybrid calibration algorithm of support vector regression and K-nearest neighbors,” Alexandria Eng J, vol. 59, no. 3, Art no 3, 2020, https://doi.org/10.1016/j.aej.2020.01.033.

  28. Hamida S, Cherradi B, Raihani A, Ouajji H (2019)“Performance Evaluation of Machine Learning Algorithms in Handwritten Digits Recognition,” in 2019 1st international conference on smart systems and data science (ICSSD), Rabat, Morocco. 1–6. https://doi.org/10.1109/ICSSD47982.2019.9003052.

  29. Hamida S, Cherradi B, Terrada O, Raihani A, Ouajji H, Laghmati S (2020) “A Novel Feature Extraction System for Cursive Word Vocabulary Recognition using Local Features Descriptors and Gabor Filter”. In: 2020 3rd International Conference on Advanced Communication Technologies and Networking (CommNet), Marrakech, Morocco 1–7. https://doi.org/10.1109/CommNet49926.2020.9199642.

  30. Hamida S, Cherradi B, Ouajji H, Raihani A (2020) Convolutional neural network architecture for offline handwritten characters recognition. In: Serrhini M, Silva C, Aljahdali S (eds) Innovation in information systems and technologies to support learning research, vol 7. Springer International Publishing, Cham, pp 368–377. https://doi.org/10.1007/978-3-030-36778-7_41

    Chapter  Google Scholar 

  31. Hamida S, Cherradi B, Ouajji H (2020) “Handwritten Arabic Words Recognition System Based on HOG and Gabor Filter Descriptors,” in 2020 1st international conference on innovative research in applied science, engineering and technology (IRASET), Meknes, Morocco,, pp. 1–4. https://doi.org/10.1109/IRASET48871.2020.9092067.

  32. Hamida S, Cherradi B, El Gannour O, Terrada O, Raihani A, Ouajji H (2021) “New Database of French Computer Science Words Handwritten Vocabulary,” in 2021 International congress of advanced technology and engineering (ICOTEN), Taiz, Yemen, pp. 1–5. https://doi.org/10.1109/ICOTEN52080.2021.9493438.

  33. Hamida S, El Gannour O, Cherradi B, Raihani A, Moujahid H, Ouajji H (Nov. 2021) A novel COVID-19 diagnosis support system using the stacking approach and transfer learning technique on chest X-ray images. J Healthcare Eng 2021:1–17. https://doi.org/10.1155/2021/9437538

    Article  Google Scholar 

  34. Han X et al. (2021) “Pre-Trained Models: Past, Present and Future”. AI Open. p. S2666651021000231, https://doi.org/10.1016/j.aiopen.2021.08.002.

  35. He K, Zhang X, Ren S,Sun J, (2016) “Deep Residual Learning for Image Recognition,” in 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA. pp. 770–778. https://doi.org/10.1109/CVPR.2016.90.

  36. He K, Zhang X, Ren S,Sun J (2016) “Identity Mappings in Deep Residual Networks,” arXiv:1603.05027 [cs], Accessed: Mar. 14, 2021. [Online]. Available: http://arxiv.org/abs/1603.05027

  37. He K, Girshick R, Dollar P (2019) “Rethinking ImageNet Pre-Training,” in 2019 IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea (south), pp. 4917–4926. https://doi.org/10.1109/ICCV.2019.00502.

  38. Jyotsna S Chauhan ES, Doegar A (2016) “Binarization techniques for degraded document images — A review,” in 2016 5th International Conference on Reliability, Infocom technologies and optimization (trends and future directions) (ICRITO), Noida, India, pp. 163–166. https://doi.org/10.1109/ICRITO.2016.7784945.

  39. Khaled (2014) Natural Language Processing and its Use in Education. ijacsa 5(12). https://doi.org/10.14569/IJACSA.2014.051210

  40. Lincy RB, Gayathri R (Feb. 2021) Optimally configured convolutional neural network for Tamil handwritten character recognition by improved lion optimization model. Multimed Tools Appl 80(4):5917–5943. https://doi.org/10.1007/s11042-020-09771-z

    Article  Google Scholar 

  41. Liu C-L, Suen CY (Dec. 2009) A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters. Pattern Recogn 42(12):3287–3295. https://doi.org/10.1016/j.patcog.2008.10.007

    Article  MATH  Google Scholar 

  42. Marti U-V, Bunke H (Nov. 2002) The IAM-database: an English sentence database for offline handwriting recognition. Int J Doc Anal Recognit 5(1):39–46. https://doi.org/10.1007/s100320200071

    Article  MATH  Google Scholar 

  43. Memon J, Sami M, Khan RA (2021) “Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR),” arXiv:2001.00139 [cs], Dec. 2019, Accessed: Nov. 15,. [Online]. Available: http://arxiv.org/abs/2001.00139

  44. Michalak H, Okarma K (2019) Improvement of Image Binarization Methods Using Image Preprocessing with Local Entropy Filtering for Alphanumerical Character Recognition Purposes. Entropy 21(6):562. https://doi.org/10.3390/e21060562

    Article  MathSciNet  Google Scholar 

  45. Mishra P, Passos D (2021) Realizing transfer learning for updating deep learning models of spectral data to be used in new scenarios. Chemom Intell Lab Syst 212:104283. https://doi.org/10.1016/j.chemolab.2021.104283

    Article  Google Scholar 

  46. Mouhcine R, Mustapha A, Zouhir M (2018) Recognition of cursive Arabic handwritten text using embedded training based on HMMs. J Electrical Syst Info Technol 5(2):245–251. https://doi.org/10.1016/j.jesit.2017.02.001

    Article  Google Scholar 

  47. Moujahid H, Cherradi B, Bahatti L, Terrada O, Hamida S (2020) Convolutional neural network based classification of patients with pneumonia using X-ray lung images. Advances Sci, Technol Eng Syst J 5:167–175

    Article  Google Scholar 

  48. Mozaffari S, Soltanizadeh H (2009)“ICDAR 2009 Handwritten Farsi/Arabic Character Recognition Competition,” in 2009 10th International Conference on Document Analysis and Recognition, Barcelona, Spain, pp. 1413–1417. https://doi.org/10.1109/ICDAR.2009.283.

  49. Nadkarni PM, Ohno-Machado L, Chapman WW (Sep. 2011) Natural language processing: an introduction. J Am Med Inform Assoc 18(5):544–551. https://doi.org/10.1136/amiajnl-2011-000464

    Article  Google Scholar 

  50. Palatnik de Sousa I (2018) Convolutional ensembles for Arabic handwritten character and digit recognition. PeerJ Comput Sci 4:e167. https://doi.org/10.7717/peerj-cs.167

    Article  Google Scholar 

  51. Paper D (2021) TensorFlow 2.X in the Colaboratory cloud: an introduction to deep learning on Google’s cloud service. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-6649-6

    Book  Google Scholar 

  52. Ruder S, Peters ME, Swayamdipta S, Wolf T (2019) “Transfer Learning in Natural Language Processing,” in Proceedings of the 2019 Conference of the North, Minneapolis, Minnesota, pp. 15–18. https://doi.org/10.18653/v1/N19-5004.

  53. Saddami K, Munadi K, Away Y, Arnia F (2019) Effective and fast binarization method for combined degradation on ancient documents. Heliyon 5(10):e02613. https://doi.org/10.1016/j.heliyon.2019.e02613

    Article  Google Scholar 

  54. Saeed K, Tabędzki M, Rybnik M, Adamski M (Jun. 2010) K3M: a universal algorithm for image skeletonization and a review of thinning techniques. Int J Appl Math Comput Sci 20(2):317–335. https://doi.org/10.2478/v10006-010-0024-4

    Article  MATH  Google Scholar 

  55. Shaffi N, Hajamohideen F (2021) uTHCD: a new benchmarking for Tamil handwritten OCR. IEEE Access 9:101469–101493. https://doi.org/10.1109/ACCESS.2021.3096823

    Article  Google Scholar 

  56. Shams M, Amira A, Wael Z (2020) Arabic Handwritten Character Recognition based on Convolution Neural Networks and Support Vector Machine. IJACSA 11(8). https://doi.org/10.14569/IJACSA.2020.0110819

  57. Shi C, Pustejovsky J, Verhagen M (2014) “A Conceptual Framework of Online Natural Language Processing Pipeline Application”. In: Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT, Dublin, Ireland. 53–59. https://doi.org/10.3115/v1/W14-5206.

  58. Simonnet D, Girard N, Anquetil E, Renault M, Thomas S (2019) Evaluation of children cursive handwritten words for e-education. Pattern Recogn Lett 121:133–139. https://doi.org/10.1016/j.patrec.2018.07.021

    Article  Google Scholar 

  59. Simonyan K, Zisserman A (2015) “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556 [cs], Accessed: Mar. 14, 2021. [Online]. Available: http://arxiv.org/abs/1409.1556

  60. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” arXiv:1602.07261 [cs], Accessed: Nov. 16, 2021. [Online]. Available: http://arxiv.org/abs/1602.07261

  61. Wang L, Qian X, Zhang Y, Shen J, Cao X (Jul. 2020) Enhancing sketch-based image retrieval by CNN semantic re-ranking. IEEE Trans Cybern 50(7):3330–3342. https://doi.org/10.1109/TCYB.2019.2894498

    Article  Google Scholar 

  62. Wei W, Wu J, Zhu C (Mar. 2020) Special issue on deep learning for natural language processing. Computing 102(3):601–603. https://doi.org/10.1007/s00607-019-00788-3

    Article  MathSciNet  MATH  Google Scholar 

  63. Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9(4):611–629. https://doi.org/10.1007/s13244-018-0639-9

    Article  Google Scholar 

  64. Younessy Ghadikolaie MF, Kabir E, Razzazi F (2016) Sub-word-based offline handwritten Farsi word recognition using recurrent neural network. ETRI J, Apr. https://doi.org/10.4218/etrij.16.0115.0542

  65. Zhuang F et al. (2020) “A Comprehensive Survey on Transfer Learning,” arXiv:1911.02685 [cs, stat], Accessed: Nov. 15, 2021. [Online]. Available: http://arxiv.org/abs/1911.02685

  66. Zoizou A, Zarghili A, Chaker I (2020) A new hybrid method for Arabic multi-font text segmentation, and a reference corpus construction. J King Saud University - Comput Info Sci 32(5):576–582. https://doi.org/10.1016/j.jksuci.2018.07.003

    Article  Google Scholar 

Download references

Funding

This research received no external funding.

Author information

Authors and Affiliations

  1. Electrical Engineering and Intelligent Systems (EEIS) Laboratory, ENSET of Mohammedia, Hassan II University of Casablanca, 28830, Mohammedia, Morocco

    Soufiane Hamida, Oussama El Gannour, Bouchaib Cherradi, Hassan Ouajji & Abdelhadi Raihani

  2. STIE Team, CRMEF Casablanca-Settat, provincial section of El Jadida, 24000, El Jadida, Morocco

    Bouchaib Cherradi

Authors
  1. Soufiane Hamida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oussama El Gannour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bouchaib Cherradi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hassan Ouajji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdelhadi Raihani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bouchaib Cherradi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest regarding the publication of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamida, S., El Gannour, O., Cherradi, B. et al. Handwritten computer science words vocabulary recognition using concatenated convolutional neural networks. Multimed Tools Appl 82, 23091–23117 (2023). https://doi.org/10.1007/s11042-022-14105-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14105-2

Keywords

Search

Navigation