research-article

Dissimilarity-Based Regularized Learning of Charts

Authors:

Mithilesh Kumar ChaubeAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 17, Issue 4

Article No.: 131, Pages 1 - 23

https://doi.org/10.1145/3458884

Published: 12 November 2021 Publication History

Abstract

Chart images exhibit significant variabilities that make each image different from others even though they belong to the same class or categories. Classification of charts is a major challenge because each chart class has variations in features, structure, and noises. However, due to the lack of affiliation between the dissimilar features and the structure of the chart, it is a challenging task to model these variations for automatic chart recognition. In this article, we present a novel dissimilarity-based learning model for similar structured but diverse chart classification. Our approach jointly learns the features of both dissimilar and similar regions. The model is trained by an improved loss function, which is fused by a structural variation-aware dissimilarity index and incorporated with regularization parameters, making the model more prone toward dissimilar regions. The dissimilarity index enhances the discriminative power of the learned features not only from dissimilar regions but also from similar regions. Extensive comparative evaluations demonstrate that our approach significantly outperforms other benchmark methods, including both traditional and deep learning models, over publicly available datasets.

References

[1]

Mohamed Abouelenien and Xiaohui Yuan. 2013. Boosting for learning from multiclass data sets via a regularized loss function. In Proceedings of the 2013 IEEE International Conference on Granular Computing (GrC’13). IEEE, Los Alamitos, CA, 4–9.

[2]

Jihen Amara, Pawandeep Kaur, Michael Owonibi, and Bassem Bouaziz. 2017. Convolutional neural network based chart image classification. In Proceedings of the 25th International Conference in Central Europe on Computer Graphics, Visualization, and Computer Vision.

[3]

Jennifer Beddoe. 2014. Study.com—Bar Graph Definition, Types and Examples. Retrieved September 16, 2021 from https://study.com/academy/lesson/bar-graph-definition-types-examples.html.

[4]

Jie Cao, Yinping Qiu, Dongliang Chang, Xiaoxu Li, and Zhanyu Ma. 2019. Dynamic attention loss for small-sample image classification. In Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC’19). IEEE, Los Alamitos, CA, 75–79.

[5]

Paulo Chagas, Rafael Akiyama, Aruanda Meiguins, Carlos Santos, Filipe Saraiva, Bianchi Meiguins, and Jefferson Morais. 2018. Evaluation of convolutional neural network architectures for chart image classification. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN’18). IEEE, Los Alamitos, CA, 1–8.

[6]

Min Chen and Amos Golan. 2015. What may visualization processes optimize? IEEE Transactions on Visualization and Computer Graphics 22, 12 (2015), 2619–2632.

Digital Library

[7]

Beibei Cheng, R. Joe Stanley, Sameer Antani, and George R. Thoma. 2013. Graphical figure classification using data fusion for integrating text and image features. In Proceedings of the 2013 12th International Conference on Document Analysis and Recognition. IEEE, Los Alamitos, CA, 693–697.

Digital Library

[8]

Gong Cheng, Junwei Han, Peicheng Zhou, and Dong Xu. 2018. Learning rotation-invariant and Fisher discriminative convolutional neural networks for object detection. IEEE Transactions on Image Processing 28, 1 (2018), 265–278.

Digital Library

[9]

Jinho Choi, Sanghun Jung, Deok Gun Park, Jaegul Choo, and Niklas Elmqvist. 2019. Visualizing for the non-visual: Enabling the visually impaired to use visualization. Computer Graphics Forum (2019), 249–260.

[10]

Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, Los Alamitos, CA, 886–893.

Digital Library

[11]

Kenny Davila, Bhargava Urala Kota, Srirangaraj Setlur, Venu Govindaraju, Christopher Tensmeyer, Sumit Shekhar, and Ritwick Chaudhry. 2019. ICDAR 2019 competition on harvesting raw tables from infographics (CHART-infographics). In Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR’19). IEEE, Los Alamitos, CA, 1594–1599.

[12]

Ahmet Demirkaya, Jiasi Chen, and Samet Oymak. 2020. Exploring the role of loss functions in multiclass classification. In Proceedings of the 2020 54th Annual Conference on Information Sciences and Systems (CISS’20). IEEE, Los Alamitos, CA, 1–5.

[13]

Mohammad Haghighat, Mohamed Abdel-Mottaleb, and Wadee Alhalabi. 2016. Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition. IEEE Transactions on Information Forensics and Security 11, 9 (2016), 1984–1996.

Digital Library

[14]

Geoffrey Hinton. 2018. Neural Networks for Machine Learning Online Course. Retrieved September 16, 2021 from https://www.coursera.org/learn/neural-networks/home/welcome.

[15]

Chaoqun Hong, Jun Yu, Jane You, Xuhui Chen, and Dapeng Tao. 2015. Multi-view ensemble manifold regularization for 3D object recognition. Information Sciences 320 (2015), 395–405.

Digital Library

[16]

Weihua Huang and Chew Lim Tan. 2007. A system for understanding imaged infographics and its applications. In Proceedings of the 2007 ACM Symposium on Document Engineering. 9–18.

Digital Library

[17]

Weihua Huang, Chew Lim Tan, and Wee Kheng Leow. 2004. Elliptic arc vectorization for 3D pie chart recognition. In Proceedings of the 2004 International Conference on Image Processing (ICIP’04), Vol. 5. IEEE, Los Alamitos, CA, 2889–2892.

[18]

Weihua Huang, Siqi Zong, and Chew Lim Tan. 2007. Chart image classification using multiple-instance learning. In Proceedings of the 2007 IEEE Workshop on Applications of Computer Vision (WACV’07). IEEE, Los Alamitos, CA, 27–27.

Digital Library

[19]

Bo Jiang and Doudou Lin. 2018. Graph Laplacian regularized graph convolutional networks for semi-supervised learning. arXiv:1809.09839.

[20]

Daekyoung Jung, Wonjae Kim, Hyunjoo Song, Jeong-In Hwang, Bongshin Lee, Bohyoung Kim, and Jinwook Seo. 2017. ChartSense: Interactive data extraction from chart images. In Proceedings of the 2017 Chi Conference on Human Factors in Computing Systems. 6706–6717.

Digital Library

[21]

Aranzazu Jurio, Humberto Bustince, Miguel Pagola, Pedro Couto, and Witold Pedrycz. 2014. New measures of homogeneity for image processing: An application to fingerprint segmentation. Soft Computing 18, 6 (2014), 1055–1066.

Digital Library

[22]

Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2017. Figureqa: An annotated figure dataset for visual reasoning. arXiv:1710.07300.

[23]

V. Karthikeyani and S. Nagarajan. 2012. Machine learning classification algorithms to recognize chart types in portable document format (PDF) files. International Journal of Computer Applications 39, 2 (2012), 1–5.

[24]

Daehyun Kim, Balaji Polepalli Ramesh, and Hong Yu. 2011. Automatic figure classification in bioscience literature. Journal of Biomedical Informatics 44, 5 (2011), 848–858.

Digital Library

[25]

Vijay Kotu and Bala Deshpande. 2018. Data Science: Concepts and Practice. Morgan Kaufmann.

[26]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems—Volume 1(NIPS’12). 1097–1105.

Digital Library

[27]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278–2324.

[28]

Weifeng Liu, Xueqi Ma, Yicong Zhou, Dapeng Tao, and Jun Cheng. 2018.

-Laplacian regularization for scene recognition. IEEE Transactions on Cybernetics 49, 8 (2018), 2927–2940.

[29]

David G. Lowe. 1999. Object recognition from local scale-invariant features. In Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 2. IEEE, Los Alamitos, CA, 1150–1157.

Digital Library

[30]

Ales Mishchenko and Natalia Vassilieva. 2011. Model-based chart image classification. In Proceedings of the International Symposium on Visual Computing. 476–485.

Digital Library

[31]

John Negotia. 2016. Pie Chart and Donut Chart. https://code.tutsplus.comRetrieved November 14, 2016 from.

[32]

M.-E. Nilsback and Andrew Zisserman. 2006. A visual vocabulary for flower classification. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2. IEEE, Los Alamitos, CA, 1447–1454.

Digital Library

[33]

Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 7 (2002), 971–987.

Digital Library

[34]

Aude Oliva and Antonio Torralba. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 3 (2001), 145–175.

Digital Library

[35]

Ram Krishna Pandey, A. G. Ramakrishnan, and Souvik Karmakar. 2019. Effects of modifying the input features and the loss function on improving emotion classification. In Proceedings of the 2019 IEEE Region 10 Conference (TENCON’19). IEEE, Los Alamitos, CA, 1159–1162.

[36]

Yong Peng, Suhang Wang, Xianzhong Long, and Bao-Liang Lu. 2015. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 149 (2015), 340–353.

Digital Library

[37]

Jorge Poco and Jeffrey Heer. 2017. Reverse-engineering visualizations: Recovering visual encodings from chart images. Computer Graphics Forum 36 (2017), 353–363.

Digital Library

[38]

V. Shiv Naga Prasad, Behjat Siddiquie, Jennifer Golbeck, and Larry S. Davis. 2007. Classifying computer generated charts. In Proceedings of the 2007 International Workshop on Content-Based Multimedia Indexing. IEEE, Los Alamitos, CA, 85–92.

[39]

Alla Redko. 2014. Oracle Docs. Retrieved September 16, 2021 from docs.oracle.com.

[40]

Aliaksei Sandryhaila and Jose M. F. Moura. 2013. Classification via regularization on graphs. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing. IEEE, Los Alamitos, CA, 495–498.

[41]

Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. 393–402.

Digital Library

[42]

Shankar Setty, Moula Husain, Parisa Beham, Jyothi Gudavalli, Menaka Kandasamy, Radhesyam Vaddi, Vidyagouri Hemadri, et al. 2013. Indian movie face database: A benchmark for face recognition under wide variations. In Proceedings of the 2013 4th National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics (NCVPRIPG’13). IEEE, Los Alamitos, CA, 1–5.

[43]

Mingyan Shao and R. Futrelle. 2005. Graphics recognition in PDF documents. In Proceedings of the 6th International Conference on Graphics Recognition (GREC’05).

[44]

Sudhindra Shukla and Ashok Samal. 2008. Recognition and quality assessment of data charts in mixed-mode documents. International Journal of Document Analysis and Recognition 11, 3 (2008), 111.

Digital Library

[45]

Noah Siegel, Zachary Horvitz, Roie Levin, Santosh Divvala, and Ali Farhadi. 2016. FigureSeer: Parsing result-figures in research papers. In Proceedings of the European Conference on Computer Vision. 664–680.

[46]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.

[47]

Kaikai Song, Feng Li, Fei Long, Junping Wang, and Qiang Ling. 2018. Discriminative deep feature learning for semantic-based image retrieval. IEEE Access 6 (2018), 44268–44280.

[48]

Ben Starr. 2015. How to Design Area Charts. https://visage.co/data-visualization-101-area-charts/Retrieved January 13, 2015 from.

[49]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.

[50]

Vinh Thong Ta, Olivier Lezoray, and Abderrahim Elmoataz. 2007. Graph based semi and unsupervised classification and segmentation of microscopic images. In Proceedings of the 2007 IEEE International Symposium on Signal Processing and Information Technology. IEEE, Los Alamitos, CA, 1160–1165.

[51]

Binbin Tang, Xiao Liu, Jie Lei, Mingli Song, Dapeng Tao, Shuifa Sun, and Fangmin Dong. 2016. DeepChart: Combining deep convolutional networks and deep belief networks in chart classification. Signal Processing 124 (2016), 156–161.

Digital Library

[52]

Lin Wang, Chaoli Wang, Zhanquan Sun, Shuqun Cheng, and Lei Guo. 2020. Class balanced loss for image classification. IEEE Access 8 (2020), 81142–81153.

[53]

Shannon Williams. n.d. Lucid Charts. Retrieved September 16, 2021 from https://www.lucidchart.com/blog/how-to-make-a-bubble-chart-in-excel.

[54]

Yong Xu, Zuofeng Zhong, Jian Yang, Jane You, and David Zhang. 2016. A new discriminative sparse representation method for robust face recognition via

{2}

regularization. IEEE Transactions on Neural Networks and Learning Systems 28, 10 (2016), 2233–2242.

[55]

Minxiang Ye, Vladimir Stankovic, Lina Stankovic, and Gene Cheung. 2019. Deep graph regularized learning for binary classification. In Proceedings of the 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’19). IEEE, Los Alamitos, CA, 3537–3541.

[56]

Yuan Yuan, Lichao Mou, and Xiaoqiang Lu. 2015. Scene recognition by manifold regularized deep learning architecture. IEEE Transactions on Neural Networks and Learning Systems 26, 10 (2015), 2222–2233.

[57]

Shiliang Zhang, Ming Lei, Bin Ma, and Lei Xie. 2019. Robust audio-visual speech recognition using bimodal DFSMN with multi-condition training and dropout regularization. In Proceedings of the 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’19). IEEE, Los Alamitos, CA, 6570–6574.

[58]

Y. P. Zhou and Chew Lim Tan. 2001. Learning-based scientific chart recognition. In Proceedings of the 4th IAPR International Workshop on Graphics Recognition (GREC’01). 482–492.

Cited By

Chandra SSaxena SKumar SChaube MK G SAlsamhi SCurry ESaif A(2023)A Novel Framework for Detection of Digital Face Video Manipulation using Deep Learning2023 3rd International Conference on Computing and Information Technology (ICCIT)10.1109/ICCIT58132.2023.10273909(348-352)Online publication date: 13-Sep-2023
https://doi.org/10.1109/ICCIT58132.2023.10273909
Thiyam JSingh SBora P(2022)Effect of attention and triplet loss on chart classification: a study on noisy charts and confusing chart pairsJournal of Intelligent Information Systems10.1007/s10844-022-00741-560:3(731-758)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/s10844-022-00741-5

Index Terms

Dissimilarity-Based Regularized Learning of Charts
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Challenges in chart image classification: a comparative study of different deep learning methods
DocEng '21: Proceedings of the 21st ACM Symposium on Document Engineering

Charts are commonly used forms of visualizing scientific observations from research findings or commercial trends. They provide an abstraction of the underlying information in a more understandable way. Over time, different forms of charts are ...
Chart classification: an empirical comparative study of different learning models
ICVGIP '21: Proceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image Processing

Charts are powerful tools for visualizing and comparing data. Representation of information through charts grows with time due to its easy and aesthetically attractive structure. With the increase in the number of documents with various chart types, ...
Classifying Chart Based on Structural Dissimilarities using Improved Regularized Loss Function
Abstract
Classification of charts is a major challenge because each chart class has variations due to the styles, appearances, structure, and noises caused due to changing data values. These variations differ across all chart types and sub-types. Hence, it ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 17, Issue 4

November 2021

529 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3492437

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2021

Accepted: 01 March 2021

Revised: 01 January 2021

Received: 01 August 2020

Published in TOMM Volume 17, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
195
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chandra SSaxena SKumar SChaube MK G SAlsamhi SCurry ESaif A(2023)A Novel Framework for Detection of Digital Face Video Manipulation using Deep Learning2023 3rd International Conference on Computing and Information Technology (ICCIT)10.1109/ICCIT58132.2023.10273909(348-352)Online publication date: 13-Sep-2023
https://doi.org/10.1109/ICCIT58132.2023.10273909
Thiyam JSingh SBora P(2022)Effect of attention and triplet loss on chart classification: a study on noisy charts and confusing chart pairsJournal of Intelligent Information Systems10.1007/s10844-022-00741-560:3(731-758)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/s10844-022-00741-5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents