Skip to main content
SpringerLink
Log in

Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

The problem of handwritten digit recognition has long been an open problem in the field of pattern classification and of great importance in industry. The heart of the problem lies within the ability to design an efficient algorithm that can recognize digits written and submitted by users via a tablet, scanner, and other digital devices. From an engineering point of view, it is desirable to achieve a good performance within limited resources. To this end, we have developed a new approach for handwritten digit recognition that uses a small number of patterns for training phase. To improve the overall performance achieved in classification task, the literature suggests combining the decision of multiple classifiers rather than using the output of the best classifier in the ensemble; so, in this new approach, an ensemble of classifiers is used for the recognition of handwritten digit. The classifiers used in proposed system are based on singular value decomposition (SVD) algorithm. The experimental results and the literature show that the SVD algorithm is suitable for solving sparse matrices such as handwritten digit. The decisions obtained by SVD classifiers are combined by a novel proposed combination rule which we named reliable multi-phase particle swarm optimization. We call the method “Reliable” because we have introduced a novel reliability parameter which is applied to tackle the problem of PSO being trapped in local minima. In comparison with previous methods, one of the significant advantages of the proposed method is that it is not sensitive to the size of training set. Unlike other methods, the proposed method uses just 15 % of the dataset as a training set, while other methods usually use (60–75) % of the whole dataset as the training set. To evaluate the proposed method, we tested our algorithm on Farsi/Arabic handwritten digit dataset. What makes the recognition of the handwritten Farsi/Arabic digits more challenging is that some of the digits can be legally written in different shapes. Therefore, 6000 hard samples (600 samples per class) are chosen by K-nearest neighbor algorithm from the HODA dataset which is a standard Farsi/Arabic digit dataset. Experimental results have shown that the proposed method is fast, accurate, and robust against the local minima of PSO. Finally, the proposed method is compared with state of the art methods and some ensemble classifier based on MLP, RBF, and ANFIS with various combination rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abdi, M.J., Salimi, H.: Farsi handwriting recognition with mixture of RBF experts based on particle swarm optimization. Int. J. Inf. Sci. Comput. Math. 2, 129–136 (2010)

    Google Scholar 

  2. Alaei, A., Pal, U., Nagabhushan, P.: Using modified contour features and SVM based classifier for the recognition of Persian/Arabic handwritten numerals, 2009 Seventh International Conference on Advances in Pattern Recognition

  3. Anasuya Devi, H.K.: “Thresholding. A pixel-level image processing methodology preprocessing technique for an OCR system for the Brahmi script”. Ancient Asia (2009)

  4. Bellman, R.: Introduction to Matrix Analysis, 2nd edn. McGraw-Hill, NY (1970)

    MATH  Google Scholar 

  5. Broumandnia, A., Shanbehzadeh, J., RezakhahVarnoosfaderani, M.: Persian/Arabic handwritten word recognition using M-band packet wavelet transform. Image Vis. Comput. Elsevier 26, 829–842 (2008)

    Article  Google Scholar 

  6. Brown, G.: Diversity in neural network ensembles. PhD thesis, University of Birmingham; September (2003)

  7. Chandra, A, Yao, X.: DIVACE: diverse and accurate ensemble learning algorithm. In: Proceeding of the international conference on intelligent data engineering and automated learning (IDEAL: lecture notes in computer science, vol. 3117. Berlin Springer 2004, 619–25 (2004)

  8. Chen, C.H., Wang, P.S.P.: Handbook of Pattern Recognition and Computer Vision, 3rd edn. World Scientific, Singapore (2005)

    Book  MATH  Google Scholar 

  9. Cordella, L.P., Foggia, P., Sansone, C., Tortorella, F., Vento, M.: Reliability parmeters to improve combination strategies in multi-expert systems. Pattern Anal. Appl. 2, 205–214 (1999)

    Article  Google Scholar 

  10. Dehghan, M., Faez, K.: Farsi handwritten character recognition with moment invariants. In: Proceedings of 13th International Conference on Digital Signal Processing. Vol. 2, 507–510 (1997)

  11. Dimauro, G., Impedovo, S., Pirlo, G., Salzo, A.: Automatic bankcheck processing: a new engineered system. Mach. Percept. Artif. Intell. 28, 5–42 (1997)

    Article  Google Scholar 

  12. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2001)

    MATH  Google Scholar 

  13. Ebrahimpour, R., Esmkhani, A., Faridi, S.: Farsi handwritten digit recognition based on mixture of RBF experts. IEICE Electron. Express 7(14), 1014–1019 (2010)

    Article  Google Scholar 

  14. Eldén, L.: Matrix Methods in Data Mining and Pattern Recognition. SIAM, Philadelphia 3, 4 (2007)

  15. Fang, Y., Tan, T., Wang, Y.: Fusion of global and local features for face verification. In: 16th International conference on pattern recognition (ICPR 2002), vol. 2, pp. 382–385 (2002)

  16. Foley, D.H.: Considerations of sample and feature size. IEEE Trans. Inf. Theory 18(5), 618–626 (1972)

    Article  MATH  Google Scholar 

  17. Frade, F., De la Torre, F., Gross, R., Baker, S., Kumar, V.: Representational oriented component analysis (ROCA) for face recognition with one sample image per training class. In: Proceedings, IEEE Conference on Computer Vision and Pattern Recognition vol. 2, pp. 266–273 (2005)

  18. Hanmandlu, M., Grover, J., Madasu, V.K., Vasikarla, S.: “Input fuzzy modeling for the recognition of handwritten Hindi numeral”. International conference on informational technology, vol. 2, pp. 208–213 (2007)

  19. Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)

    Article  Google Scholar 

  20. Harifi, A., Aghagolzadeh, A.: A new pattern for handwritten Persian/Arabic digit recognition. J. Inf. Technol. 3, 249–252 (2004)

    Google Scholar 

  21. http://farsiocr.ir/farsi-digit-dataset

  22. Jain, A.K., Chandrasekaran, B.: Dimensionality and sample size considerations in pattern recognition practice. In: Krishnaiah, P., Kanal, L. (eds.) Handbook of Statistics volume 2, pp. 835–855. Amsterdam, North Holland (1982)

    Google Scholar 

  23. Jang, J.-S.R.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans. Syst. Man Cybern. 23(3), 665–685 (1993)

    Article  Google Scholar 

  24. Jolliffe, I.T.: Principal Component Analysis. Springer, New York (1986)

    Book  Google Scholar 

  25. Kennedy, J, Eberhart, RC.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks (ICNN 1995), vol. 4, pp. 1942–1948 (1995)

  26. Keysers, D., Deselaers, T., Gollan, C., Ney, H.: Deformation models for image recognition. IEEE Trans. Patt. Anal. Mach. Intell. 29(8), 1422–1435 (2007)

    Article  Google Scholar 

  27. Khosravi, H., Kabir, E.: Introducing a very large dataset of handwritten Farsi digit and a study on their varieties. Pattern Recognit. Lett. 28, 1133–1141 (2007)

    Article  Google Scholar 

  28. Kim, C., Oh, J., Choi, C.-H.: Combined subspace method using global and local features for face recognition. In: Proceedings of the International Joint Conference on, Neural Networks, pp. 2030–2035 (2005)

  29. Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

    Article  Google Scholar 

  30. Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion an experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)

    Article  MATH  Google Scholar 

  31. Lauer, F., Suen, C.Y., Bloch, G.: A trainable feature extractor for handwritten digit recognition. Pattern Recognit. 40(6), 1816–1824 (2007)

    Article  MATH  Google Scholar 

  32. LeCun, Y., Bottou, L., Bengio, Y.: Reading checks with graph transformer networks. In: Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing vol. 1, 151–154 (1997)

  33. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (November 1998)

    Google Scholar 

  34. Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: benchmarking of the state-of the- art techniques. Pattern Recognit. 36(10), 2271–2285 (2003)

    Article  MATH  Google Scholar 

  35. Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: investigation of normalization and feature extraction techniques. Pattern Recognit. 37, 256–279 (2004)

    Google Scholar 

  36. Liu, C.-L., Sako, H.: Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition. Pattern Recognit. 39(4), 669–681 (2006)

    Article  MATH  Google Scholar 

  37. Liu, C.L., Suen, C.Y.: A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters. Pattern Recognit. 42, 3287–3295 (2008)

    Article  Google Scholar 

  38. Marc’Aurelio, R., Poultney, C., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy based model. In MIT. Press, editor, Proceedings Advances in Neural Information Processing Systems, (2006)

  39. Hosseini, H.M.M., Bouzerdoum, A.: A Combined Method for Persian and Arabic Handwritten Digit Recognition, Australian New Zealand Conference on Intelligent Information System, pp. 80–83 (1996)

  40. Mori, S., Suen, C.Y, Yamamoto, K: “Historical review of OCR research and developing”. In: Proceedings IEEE vol. 80, July, pp. 1029–1058 (1992)

  41. Mori, S., Nishida, H., Yamada, H.: Optical Character Recognition. Wiley, New York (1999)

    Google Scholar 

  42. Mowlaei, A., Faez, K.: Recognition of isolated handwritten Persian/Arabic characters and numerals using support vector machines. In: Proceedings IEEE 13th Workshop on Neural Networks for Signal Processing, pp. 547–554 (2003)

  43. Mowlaei, A., Faez, K., Haghighat, A.: Feature extraction with wavelet transform for recognition of isolated handwritten Farsi/Arabic characters and numerals. Digit. Signal Process. 2, 923–926 (2002)

    Google Scholar 

  44. Mozaffari, S., Faez, K., Ziaratban, M.: “Structural Decomposition and Statistical description of Farsi/Arabic handwritten numeric characters. In: Proceedings of the 8th International Conference on Document Analysis and Recognition vol. 1, 237–241 (2005)

  45. Nabavi-Karizi, SH., Abadi, M., Kabir, E.: A PSO-based weighting method for linear combination of neural networks. Comput. Electr. Eng. doi:10.1016/j.comeleceng2008.04.006 (2008)

  46. OToole, A.J., Abdi, H.: Low-dimensional representation of faces in higher dimensions of the face space. Opt. Soc. Am. 10, 411 (1993)

    Google Scholar 

  47. Pan, W.M., Bui, T.D., Suen, C.Y.: Isolated handwritten Farsi numerals recognition using sparse and over-complete representations. In: 10th International Conference on Document Analysis and Recognition. 586–590 (2009)

  48. Sanguansat, P.: Two-dimensional principal component analysis and its extensions, principal component analysis. In: Sanguansat, P. (ed.), ISBN: 978-953-51-0195-6, InTech, Available from: http://www.intechopen.com/books/principal-component-analysis/2dpca-and-its-extensions (2012)

  49. Shi, Y.: Particle swarm optimization. IEEE Connect. 2(1), 8–13 (2004)

    Google Scholar 

  50. Rajashekararadhya, S.V., Ranjan, P.V., ManjunathAradhya, V.N.:“Isolated handwritten Kannada and Tamil numeral recognition: a novel approach”, First International Conference on Emerging Trends in Engineering and Technology ICETET, No. 8, pp. 1192–1195 (2008)

  51. Raudys, S.J., Pikelis, V.: On dimensionality, sample size, classification error, and complexity of classification algorithms in pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2, 243–251 (1980)

    Google Scholar 

  52. Raudys, S.J., Jain, A.K.: Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Trans. Pattern Anal. Mach. Intell. 13(3), 252–264 (1991)

    Article  Google Scholar 

  53. Sadri, J., Suen, C.Y., Bui, T.D.: “Application of support vector machines for recognition of handwritten Arabic/Persian digits”. In: Proceedings of the 2nd Conference on Machine Vision and Image Processing & Applications. vol. 1, pp. 300–307 (2003)

  54. Sadri, J., Suen, C.Y., Bui, T.D.: “Application of support vector machines for recognition of handwritten Arabic/Persian digits. In: Proceedings of the 2nd Conference on Machine Vision and Image Processing & Applications. vol. 1, 300–307 (2003)

  55. Savas, B., Elden, L.: Handwritten digit classification using higher order singular value decomposition. Pattern Recognit. 40, 993–1003 (2007)

    Article  MATH  Google Scholar 

  56. Shi, M., Fujisawa, Y., Wakabayashi, T., Kimura, F.: Handwritten numeral recognition using gradient and curvature of gray scale image. Pattern Recognit. 35(10), 2051–2059 (2002)

    Google Scholar 

  57. Shirali-Shahreza, M.H., Faez, K., Khotanzad, A.: Recognition of hand-written Persian/Arabic numerals by shadow coding and an edited probabilistic neural network. Proc. Int. Conf. Image Process. 3, 436–439 (1995)

    Google Scholar 

  58. Simard, P.Y., Steinkraus, D., Platt, J. : Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings International Conference on Document Analysis and Recognition (ICDAR), pp. 958–962, (2003)

  59. Simard, P.Y., LeCun, Y.A., Denker, J.S., Victorri, B.: Transformation invariance in pattern recognition tangent distance and tangent propagation. Intern. J. Imag. Syst. Technol. 11(3), 181–197 (2000)

    Google Scholar 

  60. Sirovich, L., Kirby, M.: Low-dimensional procedure for characterization of human faces. J. Opt. Soc. Am. 4, 519–524 (1987)

    Google Scholar 

  61. Soltanzadeh, H., Rahmati, M.: Recognition of persian handwritten digits using image profiles of multiple orientations. Pattern Recognit. Lett. 25(14), 1569–1576 (2004)

    Google Scholar 

  62. Srihari, S., Keubert, E.: Integration of handwritten address interpretation technology into the United States Postal Service remote computer reader system. In: Proceedings Fourth International Conference on Document Analysis and Recognition, vol. 2, 892–896 (1997)

  63. Suen, C., Liu, K., Strathy, N.: Sorting and recognizing cheques and financial documents. In: Proceedings of third IAPR workshop on document analysis systems, pp 1–18, (1998)

  64. Tan, X., Chen, S., Zhou, Z., Zhang, F.: Face recognition from a single image perperson: a survey. Pattern Recognit. 39, 1725–1745 (2006)

    Article  MATH  Google Scholar 

  65. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  66. Wshah, S., Shi, Z., Govindaraju, V.: “Segmentation of Arabic handwriting based on both contour and skeleton segmentation”, 10th International Conference on Document Analysis and Recognition, (2009)

  67. Yang, J., Zhang, D., Frangi, A.F., Yang, J.Y.: “Two-dimensional PCA: a new approach to appearance-based face representation and recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 131–137 (2004)

  68. Zhang, L., Tjondronegoro, D.: Selecting, optimizing and fusing ’Salient’ gabor features for facial expression recognition. In: Leung, C.S., Lee, M., Chan, J.H. (eds.) Neural Information Processing. LNCS, vol. 5863, pp. 724–732. Springer, Heidelberg (2009)

  69. Zhang, D., Zhou, Z.-H.: (2D)\(^{2}\)PCA: 2-directional 2-dimensional PCA for efficient face representation and recognition. Neurocomputing 69(1–3), 224–231 (2005)

    Article  Google Scholar 

  70. Zhang, P., Bui, T., Suen, C.: A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognit. 40(12), 3415–3429 (2007)

    Article  MATH  Google Scholar 

  71. Ziaratban, M., Faez, K., Faradji, F.: “Language-based feature extraction using template-matching in Farsi/Arabic handwritten numeral recognition. In: Proceedings of 9th International Conference on Document Analysis and Recognition. vol. 1, 297–301 (2007)

  72. Ziaratbanv, M., Faez, K., Faradji, F.: Language-based feature extraction using template-matching in Farsi/Arabichandwritten numeral recognition. In Proceedings Ninth International Conference on Document Analysis and Recognition, vol. 1, pp. 297–301, (2007)

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their constructive comments and suggestions. They also grateful to Dr. Noori who always made us clear with their valuable remarks and helpful discussions. In addition, they are indebted to Miss. Akram Ebrahimi, Miss. Fateme Behjati, and Mr.Vahid Ashrafian for their comments and help in proofreading this manuscript.

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Science, University of Tehran, Tehran, Iran

    Hamid Salimi & Davar Giveki

  2. Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain

    Davar Giveki

Authors
  1. Hamid Salimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Davar Giveki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Salimi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Salimi, H., Giveki, D. Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule. IJDAR 16, 371–386 (2013). https://doi.org/10.1007/s10032-012-0195-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-012-0195-7

Keywords

Search

Navigation