Skip to main content
SpringerLink
Log in

Link-Rot in Web-Sourced Multimedia Datasets

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13833))

Included in the following conference series:

  • 1331 Accesses

Abstract

The Web is increasingly used as a source for content of datasets of various types, especially multimedia content. These datasets are then often distributed as a collection of URLs, pointing to the original sources of the elements. As these sources go offline over time, the datasets experience decay in the form of link-rot. In this paper, we analyze 24 Web-sourced datasets with a combined total of over 270 million URLs and find that over 20% of the content is no longer available. We discuss the adverse effects of this decay on the reproducibility of work based on such data and make some recommendations on how they could be mediated in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://laion.ai/blog/laion-5b/.

  2. 2.

    https://vimeo.com.

  3. 3.

    https://archive.org.

  4. 4.

    https://flickr.com.

  5. 5.

    This is to be seen as a general trend rather than a definite insight, as due to lack of release dates for individual URLs and the resulting crudity of the analysis, none of the correlations pass any reasonable threshold for statistical significance.

  6. 6.

    Observed status codes below 400 and above 600: 101, 300, 301, 302, 303, 304, 307, 600, 617, 651, 670, 724, 750, 903, 999. Italicized codes have no generally accepted definition.

  7. 7.

    Observed error codes between 400 and 500: 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 412, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 426, 429, 444, 445, 447, 449, 451, 456, 463, 465, 470, 471, 473, 477, 478, 479, 490, 493, 498. Italicized codes have no generally accepted definition.

  8. 8.

    Observed error codes between 500 and 600: 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 511, 512, 520, 521, 522, 523, 524, 525, 526, 529, 530, 533, 534, 535, 543, 555, 556, 567, 591. Italicized codes have no generally accepted definition.

References

  1. Abu-El-Haija, S., et al.: Youtube-8m: a large-scale video classification benchmark. CoRR abs/1609.08675 (2016), http://arxiv.org/abs/1609.08675

  2. Brickley, D., Burgess, M., Noy, N.: Google dataset search: building a search engine for datasets in an open web ecosystem. In: The World Wide Web Conference, pp. 1365–1375. WWW 2019, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3308558.3313685

  3. Burnhill, P., Mewissen, M., Wincewicz, R.: Reference rot in scholarly statement: threat and remedy. Insights 28(2) (2015). https://doi.org/10.1629/uksg.237

  4. Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.T.: NUS-WIDE: a real-world web image database from national university of Singapore. In: Proceedings of the ACM International Conference on Image and Video Retrieval. CIVR 2009, Association for Computing Machinery, New York, NY, USA (July 8–10, 2009). https://doi.org/10.1145/1646396.1646452

  5. Dai, A.M., Olah, C., Le, Q.V.: Document embedding with paragraph vectors. In: NIPS Deep Learning Workshop (2014)

    Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009). https://doi.org/10.1109/CVPR.2009.5206848

  7. Dividino, R., Kramer, A., Gottron, T.: An investigation of HTTP header information for detecting changes of linked open data sources. In: Presutti, V., Blomqvist, E., Troncy, R., Sack, H., Papadakis, I., Tordai, A. (eds.) ESWC 2014. LNCS, vol. 8798, pp. 199–203. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11955-7_18

    Chapter  Google Scholar 

  8. Caba Heilbron, F., Victor Escorcia, B.G., Niebles, J.C.: ActivityNet: a large-scale video benchmark for human activity understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–970 (June 2015). https://doi.org/10.1109/CVPR.2015.7298698

  9. Gemmeke, J.F., et al.: Audio set: an ontology and human-labeled dataset for audio events. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776–780. IEEE, New Orleans, LA (2017). https://doi.org/10.1109/ICASSP.2017.7952261

  10. Heller, S., Rossetto, L., Schuldt, H.: The PS-battles dataset - an image collection for image manipulation detection. CoRR abs/1804.04866, arXiv:1804.04866 (Apr 2018). https://doi.org/10.48550/ARXIV.1804.04866

  11. Jiang, Y.G., Ye, G., Chang, S.F., Ellis, D., Loui, A.C.: Consumer video understanding: a benchmark database and an evaluation of human and machine performance. In: Proceedings of ACM International Conference on Multimedia Retrieval (ICMR), oral session, pp. 1–8. ICMR 2011, Association for Computing Machinery, New York, NY, USA (2011). https://doi.org/10.1145/1991996.1992025

  12. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, June 23–28, 2014, pp. 1725–1732. IEEE Computer Society (2014). https://doi.org/10.1109/CVPR.2014.223

  13. Klein, M., et al.: Scholarly context not found: one in five articles suffers from reference rot. PLOS ONE 9(12), 1–39 (12 2014). https://doi.org/10.1371/journal.pone.0115253

  14. Krasin, I., et al.: OpenImages: a public dataset for large-scale multi-label and multi-class image classification. (2017)

    Google Scholar 

  15. Malmaud, J., Huang, J., Rathod, V., Johnston, N., Rabinovich, A., Murphy, K.: What’s cookin’? Interpreting cooking videos using text, speech and vision (2015). https://doi.org/10.48550/ARXIV.1503.01558

  16. Meneses, L., Furuta, R., Shipman, F.: Identifying “Soft 404’’ error pages: analyzing the lexical signatures of documents in distributed collections. In: Zaphiris, P., Buchanan, G., Rasmussen, E., Loizides, F. (eds.) TPDL 2012. LNCS, vol. 7489, pp. 197–208. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33290-6_22

    Chapter  Google Scholar 

  17. Over, P., Awad, G., Smeaton, A.F., Foley, C., Lanagan, J.: Creating a web-scale video collection for research. In: Proceedings of the 1st Workshop on Web-scale multimedia corpus, pp. 25–32. WSMC 2009, Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1631135.1631141

  18. Real, E., Shlens, J., Mazzocchi, S., Vanhoucke, V., Pan, X.: Youtube-boundingboxes: a large high-precision human-annotated dataset for object detection in video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464–7473 (July 2017). https://doi.org/10.48550/ARXIV.1702.00824

  19. Rossetto, L., Schuldt, H.: Web video in numbers - an analysis of web-video metadata (2017). https://doi.org/10.48550/ARXIV.1707.01340

  20. Rossetto, L., Schuldt, H., Awad, G., Butt, A.A.: V3C – a research video collection. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, W.-H., Vrochidis, S. (eds.) MMM 2019. LNCS, vol. 11295, pp. 349–360. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05710-7_29

    Chapter  Google Scholar 

  21. Sanderson, R., Phillips, M., de Sompel, H.V.: Analyzing the persistence of referenced web resources with memento. CoRR abs/1105.3459 (2011). http://arxiv.org/abs/1105.3459

  22. Sanderson, R., Van de Sompel, H., Burnhill, P., Grover, C.: Hiberlink: towards time travel for the scholarly web. In: Proceedings of the 1st International Workshop on Digital Preservation of Research Methods and Artefacts, p. 21. Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2499583.2500370

  23. Schmidt, L., Sharifi, M., Lopez-Moreno, I.: Large-scale speaker identification. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1650–1654. IEEE (2014). https://doi.org/10.1109/ICASSP.2014.6853878

  24. Schuhmann, C., et al.: LAION-400M: open dataset of clip-filtered 400 million image-text pairs. CoRR abs/2111.02114 (2021). https://arxiv.org/abs/2111.02114

  25. Sharma, P., Ding, N., Goodman, S., Soricut, R.: Conceptual captions: a cleaned, hypernymed, image alt-text dataset for automatic image captioning. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2556–2565. Association for Computational Linguistics, Melbourne, Australia (Jul 2018). https://doi.org/10.18653/v1/P18-1238, https://aclanthology.org/P18-1238

  26. Singh, S., Subramanya, A., Pereira, F., McCallum, A.: Wikilinks: a large-scale cross-document coreference corpus labeled via links to Wikipedia. Tech. Rep. UM-CS-2012-015, University of Massachusetts, Amherst (2012)

    Google Scholar 

  27. Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., Zisserman, A.: A short note on the kinetics-700-2020 human action dataset (2020). https://doi.org/10.48550/ARXIV.2010.10864

  28. Sun, J.J., Liu, T., Cowen, A.S., Schroff, F., Adam, H., Prasad, G.: EEV: a large-scale dataset for studying evoked expressions from video. arXiv preprint arXiv:2001.05488 abs/2001.05488 (2021). https://doi.org/10.48550/ARXIV.2001.05488

  29. Thomee, B., et al.: YFCC100M: the new data in multimedia research. Communications of the ACM 59(2), 64–73 (Jan 2016). https://doi.org/10.1145/2812802

  30. Wang, L.L., et al.: CORD-19: the covid-19 open research dataset. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. Association for Computational Linguistics (Jul 2020). https://www.aclweb.org/anthology/2020.nlpcovid19-acl.1

  31. Zhou, K., Grover, C., Klein, M., Tobin, R.: No more 404s: predicting referenced link rot in scholarly articles for pro-active archiving. In: Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 233–236. JCDL 2015, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2756406.2756940

  32. Zhou, K., Tobin, R., Grover, C.: Extraction and analysis of referenced web links in large-scale scholarly articles. In: IEEE/ACM Joint Conference on Digital Libraries, JCDL 2014, London, United Kingdom, September 8–12, 2014, pp. 451–452. IEEE Computer Society, New York, NY, USA (2014). https://doi.org/10.1109/JCDL.2014.6970220

  33. Zhou, T., Tucker, R., Flynn, J., Fyffe, G., Snavely, N.: Stereo magnification: learning view synthesis using multiplane images. ACM Trans. Graph. (Proc. SIGGRAPH) 37(4), 1–12 (07 2018). https://doi.org/10.1145/3197517.3201323

  34. Zittrain, J., Albert, K., Lessig, L.: Perma: scoping and addressing the problem of link and reference rot in legal citations. Legal Inf. Manag. 14(2), 88–99 (2014). https://doi.org/10.1017/S1472669614000255

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the Swiss National Science Foundation, Project “MediaGraph” (Grant Number 202125).

Author information

Authors and Affiliations

  1. Department of Informatics, University of Zurich, Zurich, Switzerland

    Viktor Lakic, Luca Rossetto & Abraham Bernstein

Authors
  1. Viktor Lakic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Rossetto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abraham Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Rossetto .

Editor information

Editors and Affiliations

  1. University of Bergen, Bergen, Norway

    Duc-Tien Dang-Nguyen

  2. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  3. Radboud University Nijmegen, Nijmegen, The Netherlands

    Martha Larson

  4. Dublin City University, Dublin, Ireland

    Alan F. Smeaton

  5. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  6. National Institute of Information and Communications Technology, Tokyo, Japan

    Minh-Son Dao

  7. Department of Information Science and Media Studies, University of Bergen, Bergen, Norway

    Christoph Trattner

  8. La Trobe University, Melbourne, VIC, Australia

    Phoebe Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lakic, V., Rossetto, L., Bernstein, A. (2023). Link-Rot in Web-Sourced Multimedia Datasets. In: Dang-Nguyen, DT., et al. MultiMedia Modeling. MMM 2023. Lecture Notes in Computer Science, vol 13833. Springer, Cham. https://doi.org/10.1007/978-3-031-27077-2_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27077-2_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27076-5

  • Online ISBN: 978-3-031-27077-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation