skip to main content
10.1145/1743384.1743426acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
invited-talk

Multimodal retrieval and ranking: more than waveforms

Published: 29 March 2010 Publication History

Abstract

Those of us that are engineers like to think in terms of pure content-analysis problems: Analyze the waveform to find the most likely chord; Look at rating data to determine what songs our friends will like; Or recognize the genre from a bag-of-features model. But these signals are inherently noisy and hard to analyze. One can solve the pure problem, but the results are often not very good.
In this talk I argue for the importance of a multimodal approach to our problems. Very seldom do we know only one thing about an object, whether it is a piece of music, an image or a video. We often have text that describes it, or know where the object exists within the WWW. People are connected to objects, and these connections tell us much about the object. Objects come with context. The most important problem for music and media information retrieval is how to combine noisy information from many different domains and the context to deliver the best experience to our users.
My talk will be illustrated with a number of real-world audio and image examples from the analysis, recognition, search and recommendation fields. In the post precision/recall world, how can we take advantage of user-generated data, so completely dependent on the user's personal context, to improve our systems? These issues are even more important as we extend our systems to work across cultures.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MIR '10: Proceedings of the international conference on Multimedia information retrieval
March 2010
600 pages
ISBN:9781605588155
DOI:10.1145/1743384

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 March 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. analysis
  2. multimedia
  3. multimodal
  4. recommendation
  5. search

Qualifiers

  • Invited-talk

Conference

MIR '10
Sponsor:
MIR '10: International Conference on Multimedia Information Retrieval
March 29 - 31, 2010
Pennsylvania, Philadelphia, USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 189
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 21 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media