Skip to main content
Springer Nature Link
Log in

A voting-based technique for word spotting in handwritten document images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Word spotting in handwritten document images is a field of immense interest due to its widespread applications. Recognition-free and recognition-based approaches are the two comprehensively studied regimes for the said problem out of which the first one is more realistic for practical applications. In literature, several works have been found that have used contour and distance-based measures for matching of the profiles of two word images. Although this is a prudent choice for printed words, the same often faces bottlenecks for unconstrained handwriting. To this end, this work applies dynamic time warping algorithm on logarithmic profiles of handwritten word images to lessen the uncontrolled profile variation that occurs due to elongation while writing some characters. We have considered both global and local interpretations of a word image by dividing it vertically into a number of sub-parts. This multi-view analysis provides close-up views of different approximations for the same word image. Finally, a voting scheme is evoked to produce the final decision. Besides, we have adopted a pruning method to pre-filter the target word images prior to applying the voting-based word matching scheme. The method has been tested on word images, taken from Qatar University Writer Identification database. We have obtained satisfactory results as compared to many state-of-the-art methods that also include deep learning-based feature extraction models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Al Aghbari Z, Brook S (2009) HAH manuscripts: a holistic paradigm for classifying and retrieving historical Arabic handwritten documents. Expert Syst Appl 36:10942–10951. https://doi.org/10.1016/j.eswa.2009.02.024

    Article  Google Scholar 

  2. Al Maadeed S, Ayouby W, Hassaïne A, Aljaam JM (2012) Quwi: an Arabic and English handwriting dataset for offline writer identification. In: 2012 international conference on Frontiers in handwriting recognition. IEEE, pp 746–751

  3. Athitsos V, Alon J, Sclaroff S, Kollios G (2004) Boostmap: A method for efficient approximate similarity rankings. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. IEEE, pp II–II

  4. Basu S, Das N, Sarkar R, Kundu M, Nasipuri M, Basu DK (2009) A hierarchical approach to recognition of handwritten Bangla characters. Pattern Recogn 42:1467–1484. https://doi.org/10.1016/j.patcog.2009.01.008

    Article  MATH  Google Scholar 

  5. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509–522

    Article  Google Scholar 

  6. Bhowmik S, Malakar S, Sarkar R, Basu S, Kundu M, Nasipuri M (2018) Off-line Bangla handwritten word recognition: a holistic approach. Neural Comput & Applic 31:5783–5798. https://doi.org/10.1007/s00521-018-3389-1

    Article  Google Scholar 

  7. Cao H, Govindaraju V (2007) Template-free word spotting in low-quality manuscripts. In: International Conference on Advances in Pattern Recognition. World Scientific, pp. 1–5

  8. Chao P, Kao C-Y, Ruan Y-S, et al (2019) Hardnet: a low memory traffic network. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 3552–3561

  9. Deng J, Dong W, Socher R, et al (2010) ImageNet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition. pp. 248–255

  10. Djeddi C, Al-Maadeed S, Gattal A, et al (2015) ICDAR2015 competition on multi-script writer identification and gender classification using ‘QUWI’database. In: 2015 13th international conference on document analysis and recognition (ICDAR). IEEE, pp 1191–1195

  11. Fischer A, Keller A, Frinken V, Bunke H (2012) Lexicon free handwritten word spotting using character HMMs. Pattern Recogn Lett 33:934–942

    Article  Google Scholar 

  12. Frinken V, Fischer A, Manmatha R, Bunke H (2011) A novel word spotting method based on recurrent neural networks. IEEE Trans Pattern Anal Mach Intell 34:211–224

    Article  Google Scholar 

  13. Giotis AP, Sfikas G, Gatos B, Nikou C (2017) A survey of document image word spotting techniques. Pattern Recogn 68:310–332. https://doi.org/10.1016/j.patcog.2017.02.023

    Article  Google Scholar 

  14. Jegou H, Douze M, Schmid C (2010) Product quantization for nearest neighbor search. IEEE Trans Pattern Anal Mach Intell 33:117–128

    Article  Google Scholar 

  15. Khayyat M, Lam L, Suen CY (2014) Learning-based word spotting system for Arabic handwritten documents. Pattern Recogn 47:1021–1030. https://doi.org/10.1016/j.patcog.2013.08.014

    Article  Google Scholar 

  16. Khurshid K, Faure C, Vincent N (2009) A novel approach for word spotting using merge-split edit distance. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). Springer, pp 213–220

  17. Kovalchuk A, Wolf L, Dershowitz N (2014) A simple and fast word spotting method. In: 2014 14th international conference on Frontiers in handwriting recognition. IEEE, pp 3–8

  18. Leydier Y, Lebourgeois F, Emptoz H (2007) Text search for medieval manuscript images. Pattern Recogn 40:3552–3567

    Article  Google Scholar 

  19. Liang Y, Fairhurst MC, Guest RM (2012) A synthesised word approach to word retrieval in handwritten documents. Pattern Recogn 45:4225–4236. https://doi.org/10.1016/j.patcog.2012.05.024

    Article  Google Scholar 

  20. Malakar S, Mohanta D, Sarkar R et al (2011) A new global Thresholding approach for document image Binarization. Int J Inf Process 6:48–59

    Google Scholar 

  21. Malakar S, Ghosh M, Sarkar R, Nasipuri M (2019) Development of a two-stage segmentation-based word searching method for handwritten document images. J Intell Syst 29:719–735

    Article  Google Scholar 

  22. Manmatha R, Han C, Riseman EM (1996) Word spotting: a new approach to indexing handwriting. In: Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp. 631–637

  23. Mondal T, Ragot N, Ramel J yves, Pal U (2018) Comparative study of conventional time series matching techniques for word spotting. Pattern Recogn 73:47–64. https://doi.org/10.1016/j.patcog.2017.07.011

  24. Mondal T, Ragot N, Ramel JY, Pal U (2016) Flexible sequence matching technique: an effective learning-free approach for word spotting. Pattern Recogn 60:596–612. https://doi.org/10.1016/j.patcog.2016.05.011

    Article  Google Scholar 

  25. Mukherjee P (2019) A two-stage approach for word searching in handwritten document images. MTech Thesis under Jadavpur University, Kolkata

  26. Pantke W, Dennhardt M, Fecker D, et al (2014) An historical handwritten Arabic dataset for segmentation-free word spotting - HADARA80P. In: Proceedings of international conference on Frontiers in handwriting recognition, ICFHR. IEEE, pp. 15–20

  27. Rath TM, Manmatha R (2003) Features for word spotting in historical manuscripts. In: Seventh international conference on document analysis and recognition, 2003. Proceedings. IEEE, pp. 218–222

  28. Rath TM, Manmatha R (2007) Word spotting for historical documents. Int J Doc Anal Recognit 9:139–152

    Article  Google Scholar 

  29. Retsinas G, Louloudis G, Stamatopoulos N, Gatos B (2016) Keyword spotting in handwritten documents using projections of oriented gradients. In: 2016 12th IAPR workshop on document analysis systems (DAS). IEEE, pp 411–416

  30. Retsinas G, Louloudis G, Stamatopoulos N, Gatos B (2019) Efficient learning-free keyword spotting. IEEE Trans Pattern Anal Mach Intell 41:1587–1600

    Article  Google Scholar 

  31. Riba P, Llados J, Fornes A (2015) Handwritten word spotting by inexact matching of grapheme graphs. In: Proceedings of the international conference on document analysis and recognition, ICDAR. IEEE, pp. 781–785

  32. Riesen K, Bunke H (2009) Approximate graph edit distance computation by means of bipartite graph matching. Image Vis Comput 27:950–959

    Article  Google Scholar 

  33. Rodríguez-Serrano JA, Perronnin F (2009) Handwritten word-spotting using hidden Markov models and universal vocabularies. Pattern Recogn 42:2106–2116. https://doi.org/10.1016/j.patcog.2009.02.005

    Article  MATH  Google Scholar 

  34. Rothacker L, Fink GA (2015) Segmentation-free query-by-string word spotting with bag-of-features HMMs. In: Proceedings of the international conference on document analysis and recognition, ICDAR. IEEE, pp. 661–665

  35. Rothacker L, Rusinol M, Fink GA (2013) Bag-of-features HMMs for segmentation-free word spotting in handwritten documents. In: Proceedings of the international conference on document analysis and recognition, ICDAR. IEEE, pp. 1305–1309

  36. Roy PP, Ramel J, Ragot N (2011) Word retrieval in historical document using character-primitives. In: 2011 international conference on document analysis and recognition. Pp 678–682

  37. Rusiñol M, Aldavert D, Toledo R, Lladós J (2015) Efficient segmentation-free keyword spotting in historical document collections. Pattern Recogn 48:545–555. https://doi.org/10.1016/j.patcog.2014.08.021

    Article  Google Scholar 

  38. Saabni R (2013) The multi angular descriptor (mad): a binary and gray images descriptor for shape recognition. In: proceedings of the 2nd international workshop on historical document imaging and processing. ACM, pp 53–58

  39. Saabni R, Bronstein A (2012) Fast keyword searching using “boostmap” based embedding. In: Frontiers in handwriting recognition (ICFHR), 2012 international conference on. IEEE, pp 734–739

  40. Scot G, Loguet-Higgins H (1991) An algorithm for associating the features of two patterns. Proc R Soc Lond B 224:21–26

    Google Scholar 

  41. Sfikas G, Retsinas G, Gatos B (2016) Zoning aggregated hypercolumns for keyword spotting. In: 2016 15th international conference on Frontiers in handwriting recognition (ICFHR). IEEE, pp 283–288

  42. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd international conference on learning representations, ICLR 2015 - conference track proceedings

  43. Singh PK, Chowdhury SP, Sinha S, et al (2017) Page-to-word extraction from unconstrained handwritten document images. In: Proceedings of the First International Conference on Intelligent Computing and Communication. Springer, pp. 517–525

  44. Sudholt S, Fink GA (2016) PHOCNet : a deep convolutional neural network for word spotting in handwritten documents. In: international conference on Frontiers in handwriting recognition (ICFHR). IEEE, pp 277–282

  45. Terasawa K, Tanaka Y (2009) Slit style HOG feature for document image word spotting. In: 2009 10th international conference on document analysis and recognition. IEEE, pp 116–120

  46. Toselli AH, Vidal E (2013) Fast HMM-filler approach for key word spotting in handwritten documents. In: 2013 12th international conference on document analysis and recognition. IEEE, pp 501–505

  47. Wang P, Eglin V, Garcia C, et al (2014) A coarse-to-fine word spotting approach for historical handwritten documents based on graph embedding and graph edit distance. In: Proceedings - International Conference on Pattern Recognition. IEEE, pp. 3074–3079

  48. Wang P, Eglin V, Garcia C, et al (2014) A novel learning-free word spotting approach based on graph representation. In: proceedings - 11th IAPR international workshop on document analysis systems, DAS 2014. IEEE, pp 207–211

Download references

Acknowledgements

We would like to thank CMATER research laboratory of the Computer Science and Engineering Department, Jadavpur University, India for providing us the infrastructural support. This work is partially supported by the PURSE-II and UPE-II, Jadavpur University projects. Ram Sarkar is thankful to DST, Govt. of India, for the grant (EMR/2016/007213) to carry out this research.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Jadavpur University, Kolkata, India

    Shamik Majumder & Subhrangshu Ghosh

  2. Department of Computer Science, Asutosh College, Kolkata, India

    Samir Malakar

  3. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Ram Sarkar & Mita Nasipuri

Authors
  1. Shamik Majumder
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Subhrangshu Ghosh
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Samir Malakar
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ram Sarkar
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Mita Nasipuri
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Samir Malakar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Majumder, S., Ghosh, S., Malakar, S. et al. A voting-based technique for word spotting in handwritten document images. Multimed Tools Appl 80, 12411–12434 (2021). https://doi.org/10.1007/s11042-020-10363-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10363-0

Keywords