Abstract
The approach in which data is stored in Excel spreadsheets is not the most ideal approach to sort out and access it. Despite using a spreadsheet is in a growing exponentially in the last years, the increases in volume and complexity of these data have led to increased requirements to preserve these data and reuse it. Most of the applications focused on extracting structured data from semi-structured data on the web by HTML and XML formats. Nowadays, there is an explosion of semi-structured documents that generate outside the web. Metadata is important to extract the spreadsheet data. This paper produced an automated relation extractor tool that lets ordinary users extract structured relational tables from spreadsheets without previous experience. To define a structure for the spreadsheet table, this paper introduces a framework that automatically extracts relational data (tables) from spreadsheets and converts from Low-Quality data to High-Quality data. The paper considers some perspective approaches for the clustering-based table detection, heuristic-based table metadata extraction and rule-based table analysis for attribute names to automate insertion into a structured database to facilitate, integrate and reuse the data stored in spreadsheets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
* Y and N imply Yes and No respectively, and UI refers to User Interface.
References
Horry, Y.: Financial information description language and visualization/analysis tools. Comput. Lang. Syst. Struct. 50, 31–52 (2017)
Roy, S., Hermans, F., Aivaloglou, E., Winter, J., van Deursen, A.: Evaluating automatic spreadsheet metadata extraction on a large set of responses from MOOC participants. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1, pp. 135–145. IEEE, March 2016
Bhowan, U., Sacristan, P., O’Malley, L., Miranda, A.A., Corcoran, M.: US Patent Application No. 15/810,885 (2019)
Hu, J., Kashi, R.S., Lopresti, D., Wilfong, G.T.: Evaluating the performance of table processing algorithms. Int. J. Doc. Anal. Recogn. 4(3), 140–153 (2002)
Coüasnon, B., Lemaitre, A.: Recognition of Tables and Forms. Handbook of Document Image Processing and Recognition, pp. 647–677 (2014)
Pinto, D., McCallum, A., Wei, X., Croft, W.B.: Table extraction using conditional random fields. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 235–242. ACM, July 2003
Chen, Z., Cafarella, M.: Integrating spreadsheet data via accurate and low-effort extraction. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1126–1135. ACM, August 2014
Barowy, D.W., Gulwani, S., Hart, T., Zorn, B.: FlashRelate: extracting relational data from semi-structured spreadsheets using examples. ACM SIGPLAN Not. 50(6), 218–228 (2015)
Goto, K., Ohta, Y., Inakoshi, H., Yugami, N.: Extraction algorithms for hierarchical header structures from spreadsheets. In: EDBT/ICDT Workshops (2016)
Cunha, J., Saraiva, J., Visser, J.: From spreadsheets to relational databases and back. In: Proceedings of the 2009 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation, pp. 179–188. ACM, January 2009
Le, V., Gulwani, S.: FlashExtract: a framework for data extraction by examples. In: ACM SIGPLAN Notices, vol. 49, no. 6, pp. 542–553. ACM, June 2014
Chen, Z., Cafarella, M.: Automatic web spreadsheet data extraction. In: Proceedings of the 3rd International Workshop on Semantic Search over the Web, p. 1. ACM, August 2013
Gulwani, S., Harris, W.R., Singh, R.: Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012)
Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: DeepDeSRT: deep learning for detection and structure recognition of tables in document images. In: the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1162–1167. IEEE, November 2017
Bendre, M., Venkataraman, V., Zhou, X., Chang, K., Parameswaran, A.: Towards a holistic integration of spreadsheets with databases: a scalable storage engine for presentational data management. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 113–124. IEEE, April 2018
Chen, Z., Cafarella, M.: Integrating spreadsheet data via accurate and low-effort extraction. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1126–1135. ACM, August 2014
Spoth, W., Arab, B.S., Chan, E.S., Gawlick, D., Ghoneimy, A., Glavic, B., Hammerschmidt, B., Kennedy, O., Lee, S., Liu, Z.H., Niu, X.: Adaptive schema databases. In: 8th Biennial Conference on Innovative Data Systems Research, CIDR 2017, Online Proceedings, Chaminade, CA, USA, 8–11 January 2017, January 2017
Santos, M.Y., e Sá, J.O., Costa, C., Galvão, J., Andrade, C., Martinho, B., Lima, F.V., Costa, E.: A big data analytics architecture for industry 4.0. In: World Conference on Information Systems and Technologies, pp. 175–184. Springer, Cham, April 2017
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Awad, A., Elgohary, R., Moawad, I., Roushdy, M. (2020). Metadata Extraction for Low-Quality Semi-structured Spreadsheets. In: Hassanien, AE., Azar, A., Gaber, T., Oliva, D., Tolba, F. (eds) Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). AICV 2020. Advances in Intelligent Systems and Computing, vol 1153. Springer, Cham. https://doi.org/10.1007/978-3-030-44289-7_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-44289-7_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44288-0
Online ISBN: 978-3-030-44289-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)