ABSTRACT
Forms are our gates to the web. They enable us to access the deep content of web sites. Automatic form understanding unlocks this content for applications ranging from crawlers to meta-search engines and is essential for improving usability and accessibility of the web. Form understanding has received surprisingly little attention other than as component in specific applications such as crawlers. No comprehensive approach to form understanding exists and previous works disagree even in the definition of the problem. In this paper, we present OPAL, the first comprehensive approach to form understanding. We identify form labeling and form interpretation as the two main tasks involved in form understanding. On both problems OPAL pushes the state of the art: For form labeling, it combines signals from the text, structure, and visual rendering of a web page, yielding robust characterisations of common design patterns. In extensive experiments on the ICQ and TEL-8 benchmarks and a set of 200 modern web forms OPAL outperforms previous approaches by a significant margin. For form interpretation, we introduce a template language to describe frequent form patterns. These two parts of OPAL combined yield form understanding with near perfect accuracy (> 98%).
- Z. Bar-Yossef and M. Gurevich. Random Sampling from a Search Engine's Index. J. ACM, 55(5), 2008. Google ScholarDigital Library
- L. Barbosa and J. Freire. Combining Classifiers to identify Online Databases. In WWW, 2007. Google ScholarDigital Library
- M. Benedikt and C. Koch. XPath leashed. CSUR, 2007. Google ScholarDigital Library
- A. Bilke and F. Naumann. Schema Matching using Duplicates. In ICDE, 2005. Google ScholarDigital Library
- M. J. Cafarella, E. Y. Chang, A. Fikes, A. Y. Halevy, W. C. Hsieh, A. Lerner, J. Madhavan, and S. Muthukrishnan. Data Management Projects at Google. SIGMOD Rec., 37(1), 2008. Google ScholarDigital Library
- E. C. Dragut, T. Kabisch, C. Yu, and U. Leser. A Hierarchical Approach to Model Web Query Interfaces for Web Source Integration. In VLDB, 2009. Google ScholarDigital Library
- O. Kalijuvee, O. Buyukkokten, H. Garcia-Molina, and A. Paepcke. Efficient Web Form Entry on PDAs. In WWW, 2001. Google ScholarDigital Library
- R. Khare and Y. An. An Empirical Study on using Hidden Markov Model for Search Interface Segmentation. In CIKM, 2009. Google ScholarDigital Library
- R. Khare, Y. An, and I.-Y. Song. Understanding Deep Web Search Interfaces: A Survey. SIGMOD Rec., 39(1), 2010. Google ScholarDigital Library
- J. Madhavan, D. Ko, L. Kot, V. Ganapathy, A. Rasmussen, and A. Halevy. Google's Deep Web Crawl. In VLDB, 2008. Google ScholarDigital Library
- A. Maiti, A. Dasgupta, N. Zhang, and G. Das. HDSampler: Revealing Data behind Web Form Interfaces. In SIGMOD, 2009. Google ScholarDigital Library
- I. Navarrete and G. Sciavicco. Spatial Reasoning with Rectangular Cardinal Direction Relations. In ECAI, 2006.Google Scholar
- H. Nguyen, T. Nguyen, and J. Freire. Learning to Extract From Labels. In VLDB, 2008. Google ScholarDigital Library
- S. Raghavan and H. Garcia-Molina. Crawling the Hidden Web. In VLDB, 2001. Google ScholarDigital Library
- . Wu, A. Doan, C. Yu, and W. Meng. Modeling and Extracting Deep-Web Query Interfaces. In Adv.@ in Inf.@ & Intelligent S., 2009.Google Scholar
- K. Chang, Z. Zhang, B. He. Understanding Web Query Interfaces: Best-Effort Parsing with Hidden Syntax. In SIGMOD, 2004. Google ScholarDigital Library
- T. Furche, G. Gottlob, G. Grasso, X. Guo, G. Orsi, C. Schallhart. Real understanding of real estate forms. In WIMS '11, 2011 Google ScholarDigital Library
Index Terms
- OPAL: automated form understanding for the deep web
Recommendations
OPAL: a passe-partout for web forms
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide WebWeb forms are the interfaces of the deep web. Though modern web browsers provide facilities to assist in form filling, this assistance is limited to prior form fillings or keyword matching. Automatic form understanding enables a broad range of ...
The ontological key: automatically understanding and integrating forms to access the deep Web
Forms are our gates to the Web. They enable us to access the deep content of Web sites. Automatic form understanding provides applications, ranging from crawlers over meta-search engines to service integrators, with a key to this content. Yet, it has ...
Current challenges in web crawling
ICWE'13: Proceedings of the 13th international conference on Web EngineeringWeb crawling, a process of collecting web pages in an automated manner, is the primary and ubiquitous operation used by a large number of web systems and agents starting from a simple program for website backup to a major web search engine. Due to an ...
Comments