Abstract
Information retrieval research has shown significant improvement and provided techniques that retrieve documents in image or text form. However, retrieval of multi-modal documents has been given very less attention. We aim to build a system for retrieval of documents with embedded information graphics (Info-graphics). Info-graphics are images of bar charts and line graphs appearing with textual components in magazines, newspapers, and journals. In this paper, we present multi-modal document image retrieval framework by learning an optimal fusion of information from text and info-graphics regions. The evaluation of the proposed concept is demonstrated on documents collected from various sources such as magazines and journals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), Washington, D.C., vol. 1, pp. 886–893. IEEE Computer Society (2005)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), 5:1–5:60 (2008)
Demir, S., Carberry, S., McCoy, K.F.: Generating textual summaries of bar charts. In: Proceedings of the Fifth International Natural Language Generation Conference, pp. 7–15 (2008)
Elzer, S., Carberry, S., Zukerman, I.: The automated understanding of simple bar charts. Artif. Intell. 175(2), 526–555 (2011)
Garg, R., Hassan, E., Chaudhury, S.: Document indexing framework for retrieval of degraded document images. In: 13th International Conference on Document Analysis and Recognition, ICDAR 2015, Nancy, France, 23–26 August 2015, pp. 1261–1265 (2015)
Gaur, V., Hassan, E., Chaudhury, S.: Design of multi-kernel distance based hashing with multiple objectives for image indexing. In: ICPR, pp. 2637–2642 (2014)
Hassan, E., Chaudhury, S., Gopal, M.: Feature combination in kernel space for distance based image hashing. IEEE Trans. Multimedia 14(4), 1179–1195 (2012)
Hassan, E., Chaudhury, S., Gopal, M.: Multi-modal information integration for document retrieval. In: ICDAR, pp. 1200–1204 (2013)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002)
Lapata, M.: Image and natural language processing for multimedia information retrieval. In: Proceedings of the 32nd European Conference on Advances in Information Retrieval, p. 12 (2010)
Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl. 2(1), 1–19 (2006)
Li, Z., Carberry, S., Fang, H., McCoy, K.F., Peterson, K.: Infographics retrieval: a new methodology. In: Métais, E., Roche, M., Teisseire, M. (eds.) Natural Language Processing and Information Systems. LNCS, vol. 8455, pp. 101–113. Springer, Heidelberg (2014). doi:10.1007/978-3-319-07983-7_15
Li, Z., Carberry, S., Fang, H., McCoy, K.F., Peterson, K., Stagitis, M.: A novel methodology for retrieving infographics utilizing structure and message content. Data Knowl. Eng. 100(PB), 191–210 (2015)
Li, Z., Stagitis, M., Carberry, S., McCoy, K.F.: Towards retrieving relevant information graphics. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013, pp. 789–792 (2013)
Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)
Moreno, P.J., Ho, P.P., Vasconcelos, N.: A Kullback-Leibler divergence based kernel for SVM classification in multimedia applications. In: Advances in Neural Information Processing Systems, pp. 1385–1392 (2004)
Prasad, V.S.N., Siddiquie, B., Golbeck, J., Davis, L.S.: Classifying computer generated charts. In: 2007 International Workshop on Content-Based Multimedia Indexing, pp. 85–92 (2007)
Saleh, B., Dontcheva, M., Hertzmann, A., Liu, Z.: Learning style similarity for searching infographics. In: Proceedings of the 41st Graphics Interface Conference, pp. 59–64. Canadian Information Processing Society (2015)
Savva, M., Kong, N., Chhajta, A., Fei-Fei, L., Agrawala, M., Heer, J.: ReVision: automated classification, analysis and redesign of chart images. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 393–402 (2011)
Smith, R.: An overview of the tesseract OCR engine. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition, Washington, D.C., vol. 02, pp. 629–633. IEEE Computer Society (2007)
Wei, X., Croft, W.B.: LDA-based document models for ad-hoc retrieval. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 178–185 (2006)
Wu, P.: Recognizing the intended message of line graphs: methodology and applications. Ph.D. thesis, Newark, DE, USA (2012)
Wu, P., Carberry, R.: Toward extractive summarization of multimodal documents. In: Proceedings of the Canadian AI Workshop on Text Summarization, pp. 53–64 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Garg, R., Chaudhury, S. (2017). Info-Graphics Retrieval: A Multi-kernel Distance Based Hashing Scheme. In: Mukherjee, S., et al. Computer Vision, Graphics, and Image Processing. ICVGIP 2016. Lecture Notes in Computer Science(), vol 10481. Springer, Cham. https://doi.org/10.1007/978-3-319-68124-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-68124-5_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68123-8
Online ISBN: 978-3-319-68124-5
eBook Packages: Computer ScienceComputer Science (R0)