Abstract
This article focuses on systems designed to extract the location of a job listing from a job advertisement on web pages. It presents the use of classifiers to improve the reliability of automata used to collect this information. Three different algorithms - SVM, Random Forest and XGBoost - were used for this purpose. Accuracy, precision, recall and F1 score were used to evaluate the performance of each algorithm. While XGBoost performed best with an accuracy and F1 score of nearly 94%, all three algorithms showed very similar results. This suggests that each of the three algorithms can be effectively used to improve the accuracy of indexing robots in identifying jobs in job listings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, J., Niu, Y., Nie, H.: Web document classification based on fuzzy k-NN algorithm. In: 2009 International Conference on Computational Intelligence and Security, 2009, pp. 193–196 (2009). https://doi.org/10.1109/CIS.2009.28
Dilip Patel, A., Pandya, V.N.: Web page classification based on context to the content extraction of articles. In: 2017 2nd International Conference for Convergence in Technology (I2CT), 2017, pp. 539–541 (2017). https://doi.org/10.1109/I2CT.2017.8226188
Weir, G. R. S., Dos Santos, E., Cartwright, B., Frank, R.: Positing the problem: enhancing classification of extremist web content through textual analysis. In: 2016 IEEE International Conference on Cybercrime and Computer Forensic (ICCCF), 2016, pp. 1–3 (2016). https://doi.org/10.1109/ICCCF.2016.7740431
Shen, D.: Text categorization. In: Liu, L., Ozsu, M.T. (eds.) Encyclopedia of Database Systems. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-39940-9_414
Joachims, T.: Text classification. In: Learning to Classify Text Using Support Vector Machines. The Springer International Series in Engineering and Computer Science, vol. 668. Springer, Boston (2002). https://doi.org/10.1007/978-1-4615-0907-3_2
Singh, A.K., Goyal, N.: Detection of malicious webpages using deep learning. In: 2021 IEEE International Conference on Big Data (Big Data), 2021, pp. 3370–3379 (2021). https://doi.org/10.1109/BigData52589.2021.9671622
Atari, M., Al-Mousa, A.: A machine-learning based approach for detecting phishing URLs. In: 2022 International Conference on Intelligent Data Science Technologies and Applications (IDSTA), 2022, pp. 82–88 (2022). https://doi.org/10.1109/IDSTA55301.2022.9923050
Liao, G.D., You, M.Z., Wang, G.C.: Experimental economic research for matching job positions. In: 2015 International Conference on Behavioral, Economic and Socio-cultural Computing (BESC), 2015, pp. 96–101 (2015). https://doi.org/10.1109/BESC.2015.7365965
Elizabeth, M.G., Rivera, M.F.M., Alfonso, C.G.E.: Grouping mixed documents: Mexico job offers case study. In: Mata-Rivera, M.F., Zagal-Flores, R., Arellano Verdejo, J., Lazcano Hernandez, H.E. (eds.) GIS LATAM 2020. CCIS, vol. 1276, pp. 151–159. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59872-3_11
Di Meglio, E., Grassia, M., Misuraca, M.: The ideal candidate. Analysis of professional competences through text mining of job offers. In: Fabbris, L. (eds.) Effectiveness of University Education in Italy. Physica-Verlag HD, Heidelberg (2007). https://doi.org/10.1007/978-3-7908-1751-5_19
Pektor, O., Walek, B., Martinik, I.: A data mining approach for finding similar job positions to upgrade a job position in a system for evaluating competencies. In: 2020 21th International Carpathian Control Conference (ICCC), 2020, pp. 1–5 (2020). https://doi.org/10.1109/ICCC49264.2020.9257298
Liu, B., Menczer, F.: Web crawling. In: Web Data Mining. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19460-3_8
Khder, M.: Web scraping or web crawling: state of art, techniques, approaches and application. Int. J. Adv. Soft Comput. Appl. 13(3), 145–168 (2021). https://doi.org/10.15849/IJASCA.211128.11
Talun, A., Drozda, P., Bukowski, L., Scherer, R.: FastText and XGBoost content-based classification for employment web scraping. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12416, pp. 435–444. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61534-5_39
Drozda, P., Talun, A., Bukowski, L.: Emplobot-design of the system. In: Proceedings of the 28th International Workshop on Concurrency, Specification and Programming (2019)
Chen, T., Guestrin, C.E: XGBoost: a scalable tree boosting system. In: KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794, August 2016. https://doi.org/10.1145/2939672.2939785
Małyszko, J., Abramowicz, W., Filipowska, A., Wagner, T.: Lemmatization of multi-word entity names for polish language using rules automatically generated based on the corpus analysis. In: Vetulani, Z., Mariani, J., Kubis, M. (eds.) LTC 2015. LNCS (LNAI), vol. 10930, pp. 74–84. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93782-3_6
Gallay, L., Šimko, M.: Utilizing vector models for automatic text lemmatization. In: Freivalds, R.M., Engels, G., Catania, B. (eds.) SOFSEM 2016. LNCS, vol. 9587, pp. 532–543. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49192-8_43
Ho, T.K.: Random decision forests. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, 1995, vol. 1, pp. 278–282 (1995). https://doi.org/10.1109/ICDAR.1995.598994
Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1, 81–106 (1986). https://doi.org/10.1007/BF00116251
Costa, V., Pedreira, C.: Recent advances in decision trees: an updated survey. Artif. Intell. Rev. (2022). https://doi.org/10.1007/s10462-022-10275-5
Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996). https://doi.org/10.1007/BF00058655
Breiman, L.: Arcing classifiers. Ann. Stat. 26(3), 801–824 (1998). pp. 801–823. https://www.jstor.org/stable/120055
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995). https://doi.org/10.1007/BF00994018
Acknowledgements
This work is part of the Emplobot project No POIR.01.01.01-00-1135/17 “Development of autonomous artificial intelligence using the learning of deep neural networks with strengthening, automating recruitment processes” funded by the National Centre for Research and Development.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Drozda, P., Nowak, B.A., Talun, A., Bukowski, L. (2023). Evaluating Web Crawlers with Machine Learning Algorithms for Accurate Location Extraction from Job Offers. In: Nguyen, N.T., et al. Advances in Computational Collective Intelligence. ICCCI 2023. Communications in Computer and Information Science, vol 1864. Springer, Cham. https://doi.org/10.1007/978-3-031-41774-0_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-41774-0_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41773-3
Online ISBN: 978-3-031-41774-0
eBook Packages: Computer ScienceComputer Science (R0)