Abstract
Due to globalization and the technological advances of the last decades, a large amount of data is created every day, especially on the Web. Web Services are one of the main artifacts created on the Web; they can provide access to sources of data of many sizes and types. In this work, we approach the challenge of designing and evaluating a Web Crawler to find two OGC (Open Geospatial Consortium) web services standards: WFS (Web Feature Service) and WMS (Web Map Service). Commercial search engines, e.g., Google and Bing, are indubitably doing a useful job as general-purpose search engines. However, some applications require domain-specific search engines and Web crawlers to find detailed information on the Web. Therefore, this work presents SpatiHarvest, a Web crawler that focuses on the task of discovering WFSes and WMSes. SpatiHarvest is a combination of some of the most advanced techniques found in the literature. Through experiments, we demonstrate that it is capable of finding more WFSes and WMSes, with less effort, when compared to the state of the art.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
The Australian government information management office (2019). https://www.finance.gov.au/agimo-archive/better-practice-checklists/docs/BPC17.pdf
Raghavan, S., Garcia-Molina, H.: Proceedings of the 27th International Conference on Very Large Data Bases, VLDB 2001, pp. 129–138. Morgan Kaufmann Publishers Inc., San Francisco (2001). http://dl.acm.org/citation.cfm?id=645927.672025
Vieira, K., Barbosa, L., da Silva, A.S., Freire, J., Moura, E.: World Wide Web 19(3), 449–474 (2015). https://doi.org/10.1007/s11280-015-0331-7
Bergman, M.K.: The deep web: surfacing hidden value. White paper (2001). https://doi.org/10.3998/3336451.0007.104
I. Cyclone Interactive Multimedia Group, I. Cyclone Interactive Multimedia Group. The digital universe of opportunities: rich data and the increasing value of the internet of things (2019). https://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm
Li, W., Wang, S., Bhatia, V.: Comput. Environ. Urban Syst. 59, 195 (2016). https://doi.org/10.1016/j.compenvurbsys.2016.07.004
Baeza-Yates, R.A., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)
Hearst, M.A., Dumais, S.T., Osuna, E., Platt, J., Scholkopf, B.: IEEE Intell. Syst. Their Appl. 13(4), 18 (1998). https://doi.org/10.1109/5254.708428
Chakrabarti, S., van den Berg, M., Dom, B.: Comput. Netw. 31, 1623 (2000). https://doi.org/10.1016/S1389-1286(99)00052-3
Batsakis, S., Petrakis, E.G., Milios, E.: Data Knowl. Eng. 68(10), 1001 (2009). https://doi.org/10.1016/j.datak.2009.04.002
Huang, C.Y., Chang, H.: ISPRS Int. J. Geo-Inf. 5, 8 (2016). https://doi.org/10.3390/ijgi5080136
Broder, A., Mitzenmacher, M.: Internet Math. 1, 485–509 (2003). https://doi.org/10.1080/15427951.2004.10129096
Rodrigues, K., Cristo, M., de Moura, E.S., da Silva, A.: IEEE Trans. Knowl. Data Eng. 27(08), 2261 (2015). https://doi.org/10.1109/TKDE.2015.2407354
Jamali, M., Sayyadi, H., Hariri, B.B., Abolhassani, H.: Web Intelligence, pp. 753–756 (2006). https://doi.org/10.1109/WI.2006.19
Debashis, H., Amritesh, K., Lizashree, M.: Int. J. Comput. Appl. 3, 23–29 (2010). https://doi.org/10.5120/767-1074
Pal, A., Tomar, D.S., Shrivastava, S.C.: ArXiv (2009)
Natural language toolkit (2019). https://www.nltk.org/
urllib - URL handling modules (2019). https://docs.python.org/3/library/urllib.html
learn: machine learning in Python - scikit-learn 0.16.1 documentation (2019). https://scikit-learn.org/
The most popular database for modern apps (2019). https://www.mongodb.com/
Barbosa, L., Freire, J.: Proceedings of the 16th International Conference on World Wide Web, WWW 2007, pp. 441–450. ACM, New York (2007). https://doi.org/10.1145/1242572.1242632
Gupta, A., Anand, P.: 2015 International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE), pp. 619–622 (2015). https://doi.org/10.1109/ABLAZE.2015.7154936
Cho, J., Garcia-Molina, H.: Proceedings of the 11th International Conference on World Wide Web, WWW 2002, pp. 124–135. ACM, New York (2002). https://doi.org/10.1145/511446.511464
Heydon, A., Najork, M.: World Wide Web 2(4), 219 (1999). https://doi.org/10.1023/A:1019213109274
Anandhi, R., Chitra, K.: Int. J. Comput. Appl. 52, 12 (2012). https://doi.org/10.5120/8172-1485
Lopez-Pellicer, F., Florczyk, A., Béjar, R., Muro-Medrano, P., Zarazaga, F.: Online Inf. Rev. 35, 909–927 (2011). https://doi.org/10.1108/14684521111193193
Acknowledgements
Project partially sponsored by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (Fapemig).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Macêdo Alexandrino, V., Comarela, G., Soares da Silva, A., Lisboa-Filho, J. (2020). A Focused Crawler for Web Feature Service and Web Map Service Discovering. In: Di Martino, S., Fang, Z., Li, KJ. (eds) Web and Wireless Geographical Information Systems. W2GIS 2020. Lecture Notes in Computer Science(), vol 12473. Springer, Cham. https://doi.org/10.1007/978-3-030-60952-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-60952-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60951-1
Online ISBN: 978-3-030-60952-8
eBook Packages: Computer ScienceComputer Science (R0)