Skip to main content
SpringerLink
Log in

ClusTi: Clustering Method for Table Structure Recognition in Scanned Images

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

OCR (Optical Character Recognition) for scanned paper invoices is very challenging due to the variability of 19 invoice layouts, different information fields, large data tables, and low scanning quality. In this case, table structure recognition is a critical task in which all rows, columns, and cells must be accurately positioned and extracted. Existing methods such as DeepDeSRT only dealt with high-quality born-digital images (e.g., PDF) with low noise and apparent table structure. This paper proposes an efficient method called CluSTi (Clustering method for recognition of the Structure of Tables in invoice scanned Images). The contributions of CluSTi are three-fold. Firstly, it removes heavy noises in the table images using a clustering algorithm. Secondly, it extracts all text boxes using state-of-the-art text recognition. Thirdly, based on the horizontal and vertical clustering algorithm with optimized parameters, CluSTi groups the text boxes into their correct rows and columns, respectively. The method was evaluated on three datasets: i) 397 public scanned images; ii) 193 PDF document images from ICDAR 2013 competition dataset; and iii) 281 PDF document images from ICDAR 2019’s numeric tables. The evaluation results showed that CluSTi achieved an F1-score of 87.5%, 98.5%, and 94.5%, respectively. Our method also outperformed DeepDeSRT with an F1-score of 91.44% on only 34 images from the ICDAR 2013 competition dataset. To the best of our knowledge, CluSTi is the first method to tackle the table structure recognition problem on scanned images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. table-detection-dataset. https://github.com/sgrpanchal31/table-detection-datasethttps://github.com/sgrpanchal31/table-detection-dataset

  2. Agrawal R, Gehrke J, Gunopulos D, Raghavan P (1998) Automatic subspace clustering of high dimensional data for data mining applications. ACM

  3. Baek Y, Lee B, Han D, Yun S, Lee H (2019) Character region awareness for text detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9365–9374

  4. Ben-Hur A, Horn D, Siegelmann HT, Vapnik V (2001) Support vector clustering. J Mach Learn Res 2(Dec):125–137

    MATH  Google Scholar 

  5. Clinchant S, Déjean H., Meunier JL, Lang EM, Kleber F (2018) Comparing machine learning approaches for table recognition in historical register books. In: 2018 13th IAPR International workshop on document analysis systems (DAS). IEEE, pp 133–138

  6. Deng D, Liu H, Li X, Cai D (2018) Pixellink: Detecting scene text via instance segmentation Thirty-Second AAAI conference on artificial intelligence

  7. Ester M, Kriegel HP, Sander J, Xu X, et al. (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, pp 226–231

  8. Farahmand A, Sarrafzadeh H, Shanbehzadeh J (2013) Document image noises and removal methods. In: Proceedings of the international multiconference of engineers and computer scientists(Ed.) Newswood Ltd, pp 436–440

  9. Fields CR (2001) Probabilistic models for segmenting and labeling sequence data. In: ICML 2001

  10. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315 (5814):972–976

    Article  MathSciNet  Google Scholar 

  11. Hartigan JA, Wong MA (1979) Algorithm as 136: A k-means clustering algorithm. J R Stat Soc Ser C Appl Stat 28(1):100–108

    MATH  Google Scholar 

  12. He P, Huang W, He T, Zhu Q, Qiao Y, Li X (2017) Single shot text detector with regional attention. In: Proceedings of the IEEE international conference on computer vision, pp 3047–3055

  13. He T, Tian Z, Huang W, Shen C, Qiao Y, Sun C (2018) An end-to-end textspotter with explicit alignment and attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5020–5029

  14. He W, Zhang XY, Yin F, Liu CL (2017) Deep direct regression for multi-oriented scene text detection. In: Proceedings of the IEEE international conference on computer vision, pp 745–753

  15. Hu J, Kashi RS, Lopresti DP, Wilfong G (2000) Table structure recognition and its evaluation. In: Document recognition and retrieval VIII. International society for optics and photonics, vol 4307, pp 44–55

  16. Jain AK, Maheswari S (2012) Survey of recent clustering techniques in data mining. Int J Comput Sci Manag Res 3(2):68–75

    Google Scholar 

  17. Kboubi F, Chabi AH, Ahmed MB (2005) Table recognition evaluation and combination methods. In: Eighth international conference on document analysis and recognition (ICDAR’05). IEEE, pp 1237–1241

  18. Le Vine N, Zeigenfuse M, Rowan M (2019) Extracting tables from documents using conditional generative adversarial networks and genetic algorithms. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

  19. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  20. Liu X, Liang D, Yan S, Chen D, Qiao Y, Yan J (2018) Fots: Fast oriented text spotting with a unified network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5676–5685

  21. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  22. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: The IEEE conference on computer vision and pattern recognition (CVPR)

  23. Paliwal SS, Vishwanath D, Rahul R, Sharma M, Vig L (2019) Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images. In: 2019 International conference on document analysis and recognition (ICDAR). IEEE, pp 128–133

  24. Qasim SR, Mahmood H, Shafait F (2019) Rethinking table recognition using graph neural networks. In: 2019 International conference on document analysis and recognition (ICDAR). IEEE, pp 142–147

  25. Rajalingam N, Ranjini K (2011) Hierarchical clustering algorithm-a comparative study. Int J Comput Appl 19(3):42–46

    Google Scholar 

  26. Rani1Y, Rohil H (2013) A study of hierarchical clustering algorithm. ter S & on Te SIT-2, p 113

  27. Rashid SF, Akmal A, Adnan M, Aslam AA, Dengel A (2017) Table recognition in heterogeneous documents using machine learning. In: 2017 14th IAPR International conference on document analysis and recognition (ICDAR), vol 1. IEEE, pp 777–782

  28. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  29. Reynolds AP, Richards G, de la Iglesia B, Rayward-Smith VJ (2006) Clustering rules: a comparison of partitioning and hierarchical clustering algorithms. J Math Model Algorithms 5(4):475–504

    Article  MathSciNet  Google Scholar 

  30. Rokach L, Maimon O (2005) Clustering methods. In: Data mining and knowledge discovery handbook. Springer, pp 321–352

  31. Sasaki Y, et al. (2007) The truth of the f-measure. Teach Tutor mater 1(5):1–5

    Google Scholar 

  32. Satopaa V, Albrecht J, Irwin D, Raghavan B (2011) Finding a “kneedle” in a haystack: Detecting knee points in system behavior. In: 2011 31st international conference on distributed computing systems workshops. IEEE, pp 166–171

  33. Scholkmann F, Boss J, Wolf M (2012) An efficient algorithm for automatic peak detection in noisy periodic and quasi-periodic signals. Algorithms 5(4):588–603

    Article  Google Scholar 

  34. Schreiber S, Agne S, Wolf I, Dengel A, Ahmed S (2017) Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In: 2017 14th IAPR International conference on document analysis and recognition (ICDAR), vol 1. IEEE, pp 1162–1167

  35. Soille P (2013) Morphological image analysis: principles and applications. Springer Science & Business Media

  36. Sudana O, Putra D, Sudarma M, Hartati RS, Wirdiani A (2018) Image clustering of complex balinese character with dbscan algorithm. J Eng Technol 6(1):548–558

    Google Scholar 

  37. Xu Q, Wang Q, Xu C, Qu L (2017) Collective vertex classification using recursive neural network. arXiv:1701.06751

  38. Xu R, Wunsch D (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16 (3):645–678

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sorbonne University, Polytech Sorbonne, F-75005, Paris, France

    Arthur Zucker & Younes Belkada

  2. Viettel CyberSpace Center, 41st floor, Keangnam Landmark 72, Hanoi, Vietnam

    Hanh Vu

  3. Computer science and Engineering Department, Thuyloi University, 175 TaySon, DongDa, Hanoi, Vietnam

    Van Nam Nguyen

Authors
  1. Arthur Zucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Younes Belkada
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanh Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Van Nam Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van Nam Nguyen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zucker, A., Belkada, Y., Vu, H. et al. ClusTi: Clustering Method for Table Structure Recognition in Scanned Images. Mobile Netw Appl 26, 1765–1776 (2021). https://doi.org/10.1007/s11036-021-01759-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-021-01759-9

Keywords

Search

Navigation