Skip to main content
SpringerLink
Log in

Vitrivr-Explore: Guided Multimedia Collection Exploration for Ad-hoc Video Search

  • Conference paper
  • First Online:
Similarity Search and Applications (SISAP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12440))

Included in the following conference series:

Abstract

vitrivr is an open-source system for indexing and retrieving multimedia data based on its content and it has been a fixture at the Video Browser Showdown (VBS) in the past years. While vitrivr has proven to be competitive in content-based retrieval due to the many different query modes it supports, its functionality is rather limited when it comes to exploring a collection or searching result sets based on content. In this paper, we present vitrivr-explore, an extension to the vitrivr stack that allows to explore multimedia collections using relevance feedback. For this, our implementation integrates into the existing features of vitrivr and exploits self-organizing maps. Users initialize the exploration by either starting with a query or just picking examples from a collection while browsing. Exploration can be based on a mixture of semantic and visual features. We describe our architecture and implementation and present first results of the effectiveness of vitrivr-explore in a VBS-like evaluation. These results show that vitrivr-explore is competitive for Ad-hoc Video Search (AVS) tasks, even without user initialization.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://tfhub.dev/google/imagenet/mobilenet_v1_050_192/quantops/feature_vector/3.

  2. 2.

    https://github.com/dashaub/kohonen4j.

  3. 3.

    https://github.com/lucaro/DRES.

  4. 4.

    https://github.com/klschoef/vbsserver.

References

  1. Gasser, R., Rossetto, L., Heller, S., Schuldt, H.: Cottontail DB: an open source database system for multimedia retrieval and analysis. In: Proceedings of the 28th ACM International Conference on Multimedia (2020)

    Google Scholar 

  2. Heller, S., Amiri Parian, M., Gasser, R., Sauter, L., Schuldt, H.: Interactive lifelog retrieval with vitrivr. In: Proceedings of the Third Annual Workshop on Lifelog Search Challenge, LSC 2020, p. 1–6 (2020)

    Google Scholar 

  3. Heller, S., Sauter, L., Schuldt, H., Rossetto, L.: Multi-stage queries and temporal scoring in vitrivr. In: 2020 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp. 1–5. IEEE (2020)

    Google Scholar 

  4. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  5. Jónsson, B.Þ., et al.: Exquisitor: interactive learning at large. arXiv preprint arXiv:1904.08689 (2019)

  6. Kratochvíl, M., Veselý, P., Mejzlík, F., Lokoč, J.: SOM-Hunter: video browsing with relevance-to-SOM feedback loop. In: Ro, Y.M., et al. (eds.) MMM 2020. LNCS, vol. 11962, pp. 790–795. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_71

    Chapter  Google Scholar 

  7. Lokoč, J., Bailer, W., Schoeffmann, K., Münzer, B., Awad, G.: On influential trends in interactive video retrieval: video browser showdown 2015–2017. IEEE Trans. Multimedia 20(12), 3361–3376 (2018)

    Article  Google Scholar 

  8. Lokoč, J., et al.: Interactive search or sequential browsing? A detailed analysis of the video browser showdown 2018. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 15(1), 29 (2019)

    Google Scholar 

  9. Rossetto, L., et al.: Interactive video retrieval in the age of deep learning-detailed evaluation of VBS 2019. IEEE Trans. Multimedia (2020). https://ieeexplore.ieee.org/abstract/document/9037125/

  10. Rossetto, L., Giangreco, I., Heller, S., Tănase, C., Schuldt, H.: Searching in video collections using sketches and sample images – the Cineast system. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9517, pp. 336–341. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-27674-8_30

    Chapter  Google Scholar 

  11. Rossetto, L., Amiri Parian, M., Gasser, R., Giangreco, I., Heller, S., Schuldt, H.: deep learning-based concept detection in vitrivr. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, W.-H., Vrochidis, S. (eds.) MMM 2019. LNCS, vol. 11296, pp. 616–621. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05716-9_55

    Chapter  Google Scholar 

  12. Rossetto, L., Schuldt, H., Awad, G., Butt, A.A.: V3C – a research video collection. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, W.-H., Vrochidis, S. (eds.) MMM 2019. LNCS, vol. 11295, pp. 349–360. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05710-7_29

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Basel, Basel, Switzerland

    Silvan Heller, Mahnaz Parian, Maurizio Pasquinelli & Heiko Schuldt

  2. University of Mons, Mons, Belgium

    Mahnaz Parian

Authors
  1. Silvan Heller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahnaz Parian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maurizio Pasquinelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heiko Schuldt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvan Heller .

Editor information

Editors and Affiliations

  1. National Institute of Informatics, Tokyo, Japan

    Shin'ichi Satoh

  2. ISTI-CNR, Pisa, Italy

    Lucia Vadicamo

  3. University of Southern Denmark, Odense M, Denmark

    Arthur Zimek

  4. ISTI-CNR, Pisa, Italy

    Fabio Carrara

  5. University of Bologna, Bologna, Italy

    Ilaria Bartolini

  6. IT University of Copenhagen, Copenhagen, Denmark

    Martin Aumüller

  7. IT University of Copenhagen, Copenhagen, Denmark

    Björn Þór Jónsson

  8. IT University of Copenhagen, Copenhagen, Denmark

    Rasmus Pagh

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Heller, S., Parian, M., Pasquinelli, M., Schuldt, H. (2020). Vitrivr-Explore: Guided Multimedia Collection Exploration for Ad-hoc Video Search. In: Satoh, S., et al. Similarity Search and Applications. SISAP 2020. Lecture Notes in Computer Science(), vol 12440. Springer, Cham. https://doi.org/10.1007/978-3-030-60936-8_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60936-8_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60935-1

  • Online ISBN: 978-3-030-60936-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation