Skip to main content

Web Crawler and Classifier for News Articles

  • Conference paper
  • First Online:
Advances in Computational Intelligence (MICAI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13613))

Included in the following conference series:

  • 635 Accesses

Abstract

In this work, we present a crawler that collects news articles and a classifier that identifies the section to which these articles belong. Due to a large number of available sources of information, a tool for gathering and filtering news articles about specific interests is necessary. For instance, a person might be interested in news about sports or science, and it could be necessary to check several websites to obtain this kind of news finally. Therefore, in this work, we propose a web application that uses a crawler to collect news articles from different websites automatically, then a classifier determines the section of each news article, and finally, the news articles that match the section of interest are displayed in the web application.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://reutersinstitute.politics.ox.ac.uk/.

  2. 2.

    https://www.eleconomista.com.mx/.

  3. 3.

    https://www.comscore.com/.

  4. 4.

    For instance, the Economy section is called Finanzas (finance) in some websites.

  5. 5.

    https://www.sopitas.com/noticias/policia-saludable-peso-ssc-cdmx/.

References

  1. Kumar, M., Bhatia, R., Rattan, D.: A survey of web crawlers for information retrieval. WIREs Data Min. Knowl. Disc. 7, e1218 (2017)

    Google Scholar 

  2. Agre, G.H., Mahajan, N.V.: Keyword focused web crawler. In: 2015 2nd International Conference on Electronics and Communication Systems (ICECS), pp. 1089–1092 (2015)

    Google Scholar 

  3. Aggarwal, C.C.: Machine Learning For Text. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-96623-2

  4. Gambino, O.J., Ortega-Pacheco, J.D., Mendoza, C.V.G., Felix-Mata, M.: Automatic detection and registration of events by analyzing email content. Res. Comput. Sci. 130, 35–43 (2016)

    Article  Google Scholar 

  5. Wu, X., Wu, G.Q., Xie, F., Zhu, Z., Hu, X.G.: News filtering and summarization on the web. IEEE Intell. Syst. 25, 68–76 (2010)

    Article  Google Scholar 

  6. Rao, V., Sachdev, J.: A machine learning approach to classify news articles based on location. In: 2017 International Conference on Intelligent Sustainable Systems, pp. 863–867 (2017)

    Google Scholar 

  7. García-Mendoza, C.-V., Gambino Juárez, O.: News article classification of Mexican newspapers. In: Mata-Rivera, M.F., Zagal-Flores, R. (eds.) WITCOM 2018. CCIS, vol. 944, pp. 101–109. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03763-5_9

    Chapter  Google Scholar 

  8. Farias, G., Vergara, S., Fabregas, E., Hermosilla, G., Dormido-Canto, S., Dormido, S.: Clasificador de noticias usando autoencoders. In: 2018 IEEE International Conference on Automation/XXIII Congress of the Chilean Association of Automatic Control (ICA-ACCA), pp. 1–6 (2018)

    Google Scholar 

  9. Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly Media, Inc., Sebastopol (2009)

    Google Scholar 

  10. Roy, A., Cruz, R.M., Sabourin, R., Cavalcanti, G.D.: A study on combining dynamic selection and data preprocessing for imbalance learning. Neurocomputing 286, 179–192 (2018)

    Article  Google Scholar 

  11. Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learning from Imbalanced Data Sets. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98074-4

    Book  Google Scholar 

  12. Yap, B.W., Rani, K.A., Rahman, H.A.A., Fong, S., Khairudin, Z., Abdullah, N.N.: An application of oversampling, undersampling, bagging and boosting in handling imbalanced datasets. In: Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013), pp. 13–22 (2014)

    Google Scholar 

  13. Lin, W.C., Tsai, C.F., Hu, Y.H., Jhang, J.S.: Clustering-based undersampling in class-imbalanced data. Inf. Sci. 409–410, 17–26 (2017)

    Article  Google Scholar 

  14. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

No acknowledgments given We thank the support of Insituto Politécnico Nacional (IPN), ESCOM-IPN, SIP-IPN projects numbers: SIP-20220620, SIP-2083, SIP-20220925 COFAA-IPN, EDI-IPN and CONACyT-SNI.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Omar Juárez Gambino .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

García-Mendoza, CV., Juárez Gambino, O. (2022). Web Crawler and Classifier for News Articles. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds) Advances in Computational Intelligence. MICAI 2022. Lecture Notes in Computer Science(), vol 13613. Springer, Cham. https://doi.org/10.1007/978-3-031-19496-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19496-2_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19495-5

  • Online ISBN: 978-3-031-19496-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics