Multimedia Retrieval Using Time Series Representation and Relevance Feedback

Abstract

Multimedia data is ubiquitous and is involved in almost every aspect of our lives. Likewise, much of the world’s data is in the form of time series, and as will be shown, many other types of data, such as video, image, and handwriting, can be transformed into time series. This fact has fueled enormous interest in time series retrieval in the database and data mining community. However, much of this work’s narrow focus on efficiency and scalability has come at the cost of usability and effectiveness. In this work, we explore the utility of the multimedia data transformation into a much simpler one-dimensional time series representation. With this time series data, we can exploit the capability of Dynamic Time Warping, which results in a more accurate retrieval. We can also use a general framework that learns a distance measure with arbitrary constraints on the warping path of the Dynamic Time Warping calculation for both classification and query retrieval tasks. In addition, incorporating a relevance feedback system and query refinement into the retrieval task can further improve the precision/recall to a great extent.