short-paper

Challenges in chart image classification: a comparative study of different deep learning methods

Authors:

Sanasam Ranbir Singh,

Prabin K. BoraAuthors Info & Claims

DocEng '21: Proceedings of the 21st ACM Symposium on Document Engineering

Article No.: 29, Pages 1 - 4

https://doi.org/10.1145/3469096.3474931

Published: 16 August 2021 Publication History

Abstract

Charts are commonly used forms of visualizing scientific observations from research findings or commercial trends. They provide an abstraction of the underlying information in a more understandable way. Over time, different forms of charts are developed. With the increase in the number of scientific documents present on the internet with different types of charts, automatic chart classification is becoming an important task for various applications. There have been several studies on chart classification with methods ranging from traditional machine learning approaches like SVM, KNN, and HMM to recent deep learning models like VGG, ResNet, and Xception. However, inconsistencies in experimental results are evident. This paper evaluates nine of the recently proposed deep learning-based models on three datasets (one curated and annotated by authors, and two publicly available), and systematically studies their performances over various setups to understand the reason for observing inconsistent results.

References

[1]

J. Amara, Pawandeep Kaur, Michael Owonibi, and B. Bouaziz. 2017. Convolutional neural network based chart image classification. 25th WSCG 2017.

[2]

Tiago Araújo, Paulo Chagas, João Alves, Carlos Santos, Beatriz Sousa Santos, and Bianchi Serique Meiguins. 2020. A Real-World Approach on the Problem of Chart Recognition Using Classification, Detection and Perspective Correction. Sensors 20, 16 (2020).

[3]

F. Bajić, J. Job, and K. Nenadić. 2019. Chart Classification Using Simplified VGG Model. In IWSSI. 229--233.

[4]

Abhijit Balaji, Thuvaarakkesh Ramanathan, and Venkateshwarlu Sonathi. 2018. Chart-Text: A Fully Automated Chart Image Descriptor. CVPR (12 2018).

[5]

P. Chagas, R. Akiyama, A. Meiguins, C. Santos, F. Saraiva, B. Meiguins, and J. Morais. 2018. Evaluation of Convolutional Neural Network Architectures for Chart Image Classification. In IJCNN. 1--8.

[6]

P. Chagas, A. Freitas, R. Daisuke, B. Miranda, T. D. O. D. Araújo, C. Santos, B. Meiguins, and J. M. D. Morais. 2017. Architecture Proposal for Data Extraction of Chart Images Using Convolutional Neural Network. In IV. 318--323.

[7]

Wenjing Dai, Meng Wang, Zhibin Niu, and Jiawan Zhang. 2018. Chart Decoder: Generating Textual and Numeric Information from Chart Images Automatically. Journal of Visual Languages Computing 48 (08 2018).

[8]

K. Davila, B. U. Kota, S. Setlur, V. Govindaraju, C. Tensmeyer, S. Shekhar, and R. Chaudhry. 2019. ICDAR 2019 Competition on Harvesting Raw Tables from Infographics (CHART-Infographics). In ICDAR. 1594--1599.

[9]

K. Davila, S. Setlur, D. Doermann, U. K. Bhargava, and V. Govindaraju. 2020. Chart Mining: A Survey of Methods for Automated Chart Analysis. TPAMI (2020).

[10]

Krešimir Nenadić Filip Bajić, Josip Job. 2020. Data Visualization Classification Using Simple Convolutional Neural Network Model. IJECE 11 (2020), 43--51.

[11]

Robert P. Futrelle, Mingyan Shao, Chris Cieslik, and Andrea Elaina Grimes. 2003. Extraction, Layout Analysis and Classification of Diagrams in PDF Documents. In ICDAR. IEEE Computer Society, 1007.

[12]

J. Gao, Y. Zhou, and K. E. Barner. 2012. View: Visual Information Extraction Widget for improving chart images accessibility. In 19th IEEE ICIP. 2865--2868.

[13]

K. V. Jobin, A. Mondal, and C. V. Jawahar. 2019. DocFigure: A Dataset for Scientific Document Figure Classification. In ICDARW, Vol. 1. 74--79.

[14]

Daekyoung Jung, Wonjae Kim, Hyunjoo Song, Jeong-in Hwang, Bongshin Lee, Bohyoung Kim, and Jinwook Seo. 2017. ChartSense: Interactive Data Extraction from Chart Images. ACM, 6706--6717.

[15]

V. Karthikeyani and S. Nagarajan. 2012. Machine Learning Classification Algorithms to Recognize Chart Types in Portable Document Format (PDF) Files. IJCA 39 (02 2012), 1--5.

[16]

Jorge Poco and Jeffrey Heer. 2017. Reverse-Engineering Visualizations: Recovering Visual Encodings from Chart Images. CGF 36 (2017), 353--363.

Digital Library

[17]

Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. ReVision: Automated Classification, Analysis and Redesign of Chart Images (UIST '11). ACM, 10 pages.

[18]

Noah Siegel, Zachary Horvitz, Roie Levin, Santosh Divvala, and Ali Farhadi. 2016. FigureSeer: Parsing Result-Figures in Research Papers. ECCV 9911, 664--680.

[19]

P. NBless X. Liu, D. Klabjan. 2019. Data extraction from charts via single deep neural network. In arXiv preprint arXiv:1906.11906.

[20]

Yanping Zhou and Chew Lim Tan. 2001. Learning-based scientific chart recognition. In 4th IAPR GREC2001. 482--492.

Cited By

Dhote AJaved MDoermann D(2024)Swin-chartPattern Recognition Letters10.1016/j.patrec.2024.08.012185:C(203-209)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.patrec.2024.08.012
Lu SJiao ZWang XShao LWu JWang MLi LGao Q(2023)LineSegNet: A network architecture for extracting continuous lines from complex backgrounds2023 2nd International Conference on Image Processing and Media Computing (ICIPMC)10.1109/ICIPMC58929.2023.00035(169-176)Online publication date: 26-May-2023
https://doi.org/10.1109/ICIPMC58929.2023.00035
Thiyam JSingh SBora P(2023)Chart classification: a survey and benchmarking of different state-of-the-art methodsInternational Journal on Document Analysis and Recognition10.1007/s10032-023-00443-w27:1(19-44)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10032-023-00443-w
Show More Cited By

Recommendations

Chart classification: an empirical comparative study of different learning models
ICVGIP '21: Proceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image Processing

Charts are powerful tools for visualizing and comparing data. Representation of information through charts grows with time due to its easy and aesthetically attractive structure. With the increase in the number of documents with various chart types, ...
Chart classification: a survey and benchmarking of different state-of-the-art methods
Abstract
With the increase in the number of documents with various types of charts available on the internet, automatic chart classification has become an essential task for various downstream applications such as chart data recovery, chart replenishment. ...
Classifying Chart Based on Structural Dissimilarities using Improved Regularized Loss Function
Abstract
Classification of charts is a major challenge because each chart class has variations due to the styles, appearances, structure, and noises caused due to changing data values. These variations differ across all chart types and sub-types. Hence, it ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DocEng '21: Proceedings of the 21st ACM Symposium on Document Engineering

August 2021

178 pages

ISBN:9781450385961

DOI:10.1145/3469096

General Chairs:
Patrick Healy
University of Limerick, Ireland
,
Mihai Bilauca
University of Limerick, Ireland
,
Program Chair:
Alexandra Bonnici
University of Malta, Malta

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

DocEng '21

Sponsor:

SIGWEB

DocEng '21: ACM Symposium on Document Engineering 2021

August 24 - 27, 2021

Limerick, Ireland

Acceptance Rates

Overall Acceptance Rate 194 of 564 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
303
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dhote AJaved MDoermann D(2024)Swin-chartPattern Recognition Letters10.1016/j.patrec.2024.08.012185:C(203-209)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.patrec.2024.08.012
Lu SJiao ZWang XShao LWu JWang MLi LGao Q(2023)LineSegNet: A network architecture for extracting continuous lines from complex backgrounds2023 2nd International Conference on Image Processing and Media Computing (ICIPMC)10.1109/ICIPMC58929.2023.00035(169-176)Online publication date: 26-May-2023
https://doi.org/10.1109/ICIPMC58929.2023.00035
Thiyam JSingh SBora P(2023)Chart classification: a survey and benchmarking of different state-of-the-art methodsInternational Journal on Document Analysis and Recognition10.1007/s10032-023-00443-w27:1(19-44)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10032-023-00443-w
Bajić FJob J(2023)Review of chart image detection and classificationInternational Journal on Document Analysis and Recognition10.1007/s10032-022-00424-526:4(453-474)Online publication date: 12-Jan-2023
https://dl.acm.org/doi/10.1007/s10032-022-00424-5
Bajić FOrel OHabijan M(2022)A Multi-Purpose Shallow Convolutional Neural Network for Chart ImagesSensors10.3390/s2220769522:20(7695)Online publication date: 11-Oct-2022
https://doi.org/10.3390/s22207695
Thiyam JSingh SBora P(2022)Effect of attention and triplet loss on chart classification: a study on noisy charts and confusing chart pairsJournal of Intelligent Information Systems10.1007/s10844-022-00741-560:3(731-758)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/s10844-022-00741-5
Bajić FJob J(2021)Chart Classification Using Siamese CNNJournal of Imaging10.3390/jimaging71102207:11(220)Online publication date: 21-Oct-2021
https://doi.org/10.3390/jimaging7110220

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten