Abstract
A large part of information available on the Web is hidden to conventional research engines because Web pages containing such information are dynamically generated as answers to query submitted by search form filled in by keywords. Such pages are referred as Deep Web pages and contain huge amount of relevant information for different application domain. For these reasons there is a constant high interest in efficiently extracting data from Deep Web data sources. In this paper we present a spatial instance learning method from Deep Web pages that exploits both the spatial arrangement and the visual features of data records and data items/fields produced by layout engines of web browsers. The proposed method is independent from the Deep Web pages encoding and from the presentation layout of data records. Furthermore, it allows for recognizing data records in Deep Web pages having multiple data regions. In the paper the effectiveness of the proposed method is proven by experiments carried out on a dataset of 100 Web pages randomly selected from most known Deep Web sites. Results obtained by using the proposed method show that the method has a very high precision and recall and that system works much better than MDR and ViNTS approaches applied to the same dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baumgartner, R., Flesca, S., Gottlob, G.: Visual web information extraction with lixto. In: VLDB, pp. 119–128. Morgan Kaufmann Publishers Inc., San Francisco (2001)
Baumgartner, R., Gottlob, G., Herzog, M.: Scalable web data extraction for online market intelligence. VLDB 2(2), 1512–1523 (2009)
Cai, D., Yu, S., Wen, J., Ma, W.-Y.: Extracting content structure for web pages based on visual representation. In: Zhou, X., Zhang, Y., Orlowska, M.E. (eds.) APWeb 2003. LNCS, vol. 2642, pp. 406–417. Springer, Heidelberg (2003)
Crescenzi, V., Mecca, G., Merialdo, P.: Roadrunner: Towards automatic data extraction from large web sites. In: VLDB, pp. 109–118. Morgan Kaufmann Publishers Inc., San Francisco (2001)
Gottlob, G., Koch, C., Baumgartner, R., Herzog, M., Flesca, S.: The lixto data extraction project: back and forth between theory and practice. In: PODS, pp. 1–12 (2004)
Hsu, C.N., Dung, M.T.: Generating finite-state transducers for semi-structured data extraction from the web. Inf. Syst. 23, 521–538 (1998)
Liu, B., Grossman, R., Zhai, Y.: Mining data records in web pages. In: KDD 2003: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 601–606. ACM, New York (2003)
Liu, W., Meng, X., Meng, W.: Vide: A vision-based approach for deep web data extraction. IEEE Trans. on Knowl. and Data Eng. 22(3), 447–460 (2010)
Mehta, R.R., Mitra, P., Karnick, H.: Extracting semantic structure of web documents using content and visual information. In: Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, WWW 2005, pp. 928–929. ACM, New York (2005)
Muslea, I., Minton, S., Knoblock, C.A.: Hierarchical wrapper induction for semistructured information sources. Autonomous Agents and Multi-Agent Systems 4(1-2), 93–114 (2001)
Navarrete, I., Sciavicco, G.: Spatial reasoning with rectangular cardinal direction relations. In: ECAI, pp. 1–9 (2006)
Oro, E., Ruffolo, M., Staab, S.: Sxpath - extending xpath towards spatial querying on web documents. PVLDB 4(2), 129–140 (2010)
Papadakis, N.K., Skoutas, D., Raftopoulos, K., Varvarigou, T.A.: Stavies: A system for information extraction from unknown web data sources through automatic web wrapper generation using clustering techniques. TKDE 17(12), 1638–1652 (2005)
Sahuguet, A., Azavant, F.: Building intelligent web applications using lightweight wrappers. DKE 36(3), 283–316 (2001)
Simon, K., Lausen, G.: Viper: augmenting automatic information extraction with visual perceptions. In: Proceedings of the 14th ACM International Conference on Information and Knowledge Management, CIKM 2005, pp. 381–388. ACM, New York (2005)
Zhai, Y., Liu, B.: Web data extraction based on partial tree alignment. In: WWW, pp. 76–85. ACM, New York (2005)
Zhai, Y., Liu, B.: Structured data extraction from the web based on partial tree alignment. TKDE 18(12), 1614–1628 (2006)
Zhao, H., Meng, W., Wu, Z., Raghavan, V., Yu, C.: Fully automatic wrapper generation for search engines. In: WWW, pp. 66–75. ACM, New York (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Oro, E., Ruffolo, M. (2011). Towards a Spatial Instance Learning Method for Deep Web Pages. In: Perner, P. (eds) Advances in Data Mining. Applications and Theoretical Aspects. ICDM 2011. Lecture Notes in Computer Science(), vol 6870. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23184-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-23184-1_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23183-4
Online ISBN: 978-3-642-23184-1
eBook Packages: Computer ScienceComputer Science (R0)