Skip to main content
SpringerLink
Log in

A Novel Approach to News Archiving from Newswires

  • Conference paper
  • First Online:
Information and Communication Technology and Applications (ICTA 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1350))

  • 935 Accesses

Abstract

A news archive is the core operational tool a media relations team depends on in order to effectively feed a data-hungry organization. An ingrained approach to news archiving in existence is the use of a relational database. As a consequence, integrating search engines that support full-text search is practically impossible due to the strict data schema that is defined in relational database systems. Therefore, there is a need for news archives that support full-text search with relevance ranking of news. In this paper, an approach that supports full-text search is proposed. The process is started by crawling newswire websites for news that are relevant with respect to some predefined keywords and extracting them. Then, they are stored in a data structure known as an inverted-index which supports full-text search, aggregation, and relevance ranking of search results. Search results are ranked and returned to a user in the order of decreasing relevance to the search term. We were able to provide a software solution written in java, the jsoup library for HTML parsing, and an elasticsearch implementation of a search engine. We tested our solution on nine newswires using ten keywords and were able to retrieve a total of 42 relevant news matching seven keywords. The approach proposed in this paper when compared to the manual approach performed better in terms of retrieval speed and accuracy. We conclude that three main components are important in a good digital archive: relevance, extraction, and search. This work is an integration of a good relevance marking technique, an extraction method, and a search engine.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.elastic.co/.

  2. 2.

    https://jsoup.org/.

  3. 3.

    https://www.java.com/en/.

  4. 4.

    https://github.com/MudiLukman/news-archive.

  5. 5.

    https://www.jetbrains.com/idea/.

References

  1. García, R., Perdrix, F., Gil, R.M.: Ontological infrastructure for a semantic newspaper. In: Proceedings 15th World Wide Web Conference on Semantic Web Annotations for Multimedia (SWAMM), pp. 1–12 (2006)

    Google Scholar 

  2. Crescenzi, V., Mecca, G., Merialdo, P.: RoadRunner: towards automatic data extraction from large web sites. In: VLDB 2001 - Proceedings of 27th International Conference on Very Large Data Bases, pp. 109–118 (2001)

    Google Scholar 

  3. Laender, A.H.F. Ribeiro-Neto, B.A., da Silver, A.S., Teixerira, J.S.: A brief survey of web data extraction tools. ACM SIGMOD Rec. 31(2), p. 84 (2002). https://doi.org/10.1145/565117.565137

  4. Wong, K.-F. et al.: Utilizing microblogs for automatic news highlights extraction. In: Series on Language Processing, Pattern Recognition, and Intelligent Systems. Social Media Content Analysis, pp. 277–296 (2017). https://doi.org/10.1142/9789813223615_0019

  5. Prasad, J., Paepcke, A.: {CoreEx}: content extraction from online news articles. In: CIKM 2008 Proceeding 17th ACM Conference on Information and Knowledge. Management. pp. 1391–1392 (2008). https://dl.acm.org/doi/10.1145/1458082.1458295

  6. Liu, Q., Shao, M., Wu, L., Zhao, G.: Main content extraction from web pages based on node characteristics. J. Comput. Sci. Eng. 11(2), 39–48 (2017). https://doi.org/10.5626/JCSE.2017.11.2.39

    Article  Google Scholar 

  7. Connolly, T., Begg, C.: Database Systems: A practical Approach to Design, Implementation and Management (Sixth Edition). Pearson, Boston (2014)

    Google Scholar 

  8. Dehghani, M., Rothe, S, Alfonseca, P., Fleury, P.: Learning to attend, copy, and generate for session-based query suggestion. In: Proceedings of the 2017 ACM International Conference on Information and Knowledge Management, pp. 1747–1756 (2017). https://doi.org/10.1145/3132847.3133010

  9. Negm, N., ElKafrawy, P., Salem, A.B.: A survey of web information extraction tools. Int. J. Comput. Appl. 43(7), 19–27 (2012). https://doi.org/10.5120/6115-8296

    Article  Google Scholar 

  10. Yarlagadda, M., Gangadhara Rao, K., Srikrishna, A.: Frequent itemset-based feature selection and Rider Moth Search Algorithm for document clustering. J. King Saud Univ. Comput. Inf. Sci. (2019). https://doi.org/10.1016/j.jksuci.2019.09.002

  11. Bun, K.K., Ishizuka, M.: Topic extraction from news archive using TF*PDF algorithm In: Proceedings of the Third International Conference on Web Information Systems Engineering WISE 2002, IEEE, pp. 73–82 (2002). https://doi.org/10.1109/wise.2002.1181645

  12. Zhang, H., Boons, F., Batista-Navarro, R.: Whose story is it anyway? automatic extraction of accounts from news articles. Inf. Process. Manag. 56, 1837–1848 (2019). https://doi.org/10.1016/j.ipm.2019.02.012

    Article  Google Scholar 

  13. Weninger, T., Hsu, W., Han, J.: CETR - content extraction via tag ratios. In: Proceedings of the 19th International Conference on the World Wide Web, WWW 2010. pp. 971–980 (2010). https://doi.org/10.1145/1772690.1772789

  14. Yenicag, A., İnternet, H., İçin, S., İçerik, S-B., Yöntemi C.: A Template-Independent Content Extraction Approach for News (2012)

    Google Scholar 

  15. Karnalim, O.: Improving scalability of java archive search engine through recursion conversion and multithreading. CommIT (Commun. Inf. Technol.) J. 10, 15–26 (2016). https://doi.org/10.21512/commit.v10i1.832

  16. Lin, J., Nogueira, R., Yates, A.: Pretrained Transformers for Text Ranking: BERT and Beyond (2020)

    Google Scholar 

  17. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL HLT 2019 – 2019 Conference North American Chapter of the Association for Computational Linguistics Human Language Technologies - Proceeings Conference volume 1, pp. 4171–4186 (2019)

    Google Scholar 

  18. Croft, W.B., Metzler, D., Strohman, T.: Search Engines Information Retrieval in Practice. Pearson, Prentice Hall (2015)

    Google Scholar 

  19. Nakamura, T.A., Calias, P.H., de Castro Reis, D., Lemos, A.P.: An anatomy for neural search engines. Inf. Process. Manag. 480, 339–353 (2019). https://doi.org/10.1016/j.ins.2018.12.041

    Article  MathSciNet  Google Scholar 

  20. Guo, J., Fan, Y., Ai, Q., Croft, W.B.: A deep relevance matching model for Ad-hoc retrieval. In: Proceedings of the 25th ACM International Conference on Information and Knowledge Management, pp. 55–64 (2016). https://doi.org/10.1145/2983323.2983769

  21. Liu, B.: Web data mining: exploring hyperlinks, contents, and usage data. In: Carey, M.J. and Ceri, S. Data Centric Systems and Applications. (Second Edition). Springer, Berlin (2015). https://doi.org/10.1007/978-3-642-19460-3

  22. Dixit, B., Kuć, R., Rogoziński, M., Chhajed, S.: Elasticsearch A Complete Guide. Packt Publishing, Sebastopol (2017)

    Google Scholar 

  23. Gormley, C., Tong, Z.: Elasticsearch: The Definitive Guide: A Distributed Real-Time Search and Analytics Engine. O’Reilly Media, Sebastopol (2015)

    Google Scholar 

  24. Brutlag, J.D. Hutchinson, H., Stone, M.: User preference and search engine latency. In: JSM Proceedings, Quality and Productivity Research Conference, American Statistical Association, vol. 12, pp. 1–13 (2008)

    Google Scholar 

  25. Schildt, H.: Java The Complete Reference (Eleventh Edition). McGraw-Hill, New York (2018)

    Google Scholar 

  26. Liang, Y.D.: Introduction to Java Programming Comprehensive Version. Pearson, Prentice Hall (2016)

    Google Scholar 

  27. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein C..: Introduction to Algorithms. MIT Press (3rd Edition), Cambridge (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Electrical Engineering G.S.S.T, Kumamoto University, Kumamoto, Japan

    Bilkisu Larai Muhammad-Bello

  2. Federal University of Technology Minna, Minna, Niger, Nigeria

    Bilkisu Larai Muhammad-Bello, Mudi Lukman & Mudi Salim

Authors
  1. Bilkisu Larai Muhammad-Bello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mudi Lukman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mudi Salim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bilkisu Larai Muhammad-Bello .

Editor information

Editors and Affiliations

  1. Covenant University, Ota, Nigeria

    Sanjay Misra

  2. Federal University of Technology Minna, Minna, Nigeria

    Bilkisu Muhammad-Bello

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muhammad-Bello, B.L., Lukman, M., Salim, M. (2021). A Novel Approach to News Archiving from Newswires. In: Misra, S., Muhammad-Bello, B. (eds) Information and Communication Technology and Applications. ICTA 2020. Communications in Computer and Information Science, vol 1350. Springer, Cham. https://doi.org/10.1007/978-3-030-69143-1_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69143-1_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69142-4

  • Online ISBN: 978-3-030-69143-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation