Fast view-based 3D model retrieval via unsupervised multiple feature fusion and online projection learning

Highlights

• An Unsupervised Multiple Feature Fusion (UMFF) algorithm is proposed to fuse multiple features for 3D model representation.

• The ${\ell }_1$-graph method is used to learn roust and datum-adaptive graph to preserve local geometric structure information.

• An efficient Online Projection Learning (OPL) algorithm is designed to solve the out-of-sample problem.

Abstract

Since each visual feature only reflects a unique characteristic about a 3-dimensional (3D) model and different visual features have diverse discriminative power in model representation, it would be beneficial to fuse multiple visual features in 3D model retrieval. To this end, we propose a fast view-based 3D model retrieval framework in this article. This framework comprises two parts: the first one is an Unsupervised Multiple Feature Fusion algorithm (UMFF), which is used to learn a compact yet discriminative feature representation from the original multiple visual features; and the second one is an efficient Online Projection Learning algorithm (OPL), which is designed to fast transfer the input multiple visual features of a newcome model into its corresponding low-dimensional feature representation. In this framework, many existing ranking algorithms such as the simple distance-based ranking method can be directly adopted for sorting all 3D models in the database using the learned new feature representation and returning the top ranked models to the user. Extensive experiments on two public 3D model databases demonstrate the efficiency and the effectiveness of the proposed approach over its competitors. The proposed framework cannot only dramatically improve the retrieval performance but also reduce the computational cost in dealing with the newcome models.

Introduction

With the rapid development of computer hardware and computer graphics techniques, especially the modeling and rendering techniques, more and more 3-dimensional (3D) models have been created and used in a wide range of applications [1], [2]. Several public large-scale 3D model databases [3], [4] are also available on the internet. To manage and reuse the abundant 3D models, efficient and effective 3D model retrieval techniques and systems become crucially important.

In some early researches, the keyword-based methods were applied to retrieve 3D models whose tags are similar to the query from database. These methods require the user to label all 3D models in the database in advance, which is very tedious and time-consuming. Moreover, due to the high ambiguity and subjectivity of keyword, it is difficult to choose a correct and meaningful keyword as the tag for a given 3D model. It is also hard to measure the similarity between two visual or geometric similar 3D models with different keywords. So, these methods are inefficient and of low accuracy in many cases.

To overcome the shortcomings of keyword-based methods, a growing number of researchers have focused their attentions on the content-based 3D model retrieval techniques [5], [6], [7], [8]. In [9], the authors have shown that the content-based methods consistently outperform the keyword-based methods. Since efficient feature representation is the cornerstone of these content-based methods, a great deal of effort has been devoted to extracting or constructing various visual and geometric features including the low-level features (e.g., geometric moment [10] and surface features [11]) and the high-level structure features (e.g., MATE feature [12]) from 3D models as feature representation. However, many existing methods tended to use only one type of visual or geometric feature representation to characterize the 3D model, which results in poor performance. Recently, more and more researchers have found that correctly combining multiple features can significantly improve the algorithms’ performance in a large variety of areas [13], [14], [15], [16], [17], [18], [19]. In these work, how to combine/fuse multiple features becomes an important problem. An intuitive way is to concatenate multiple features into a new high-dimensional feature vector, which is very simple while lacks of physical meaning. It would be more smart to take both the complementary information between different feature representation and the intrinsic geometric structure information of each feature representation into account in fusing these multiple features. More importantly, how to deal with the newcome samples, which is named as the out-of-sample problem, is another open problem. However, not much work has been done to solve these problems well, especially for the task of 3D model retrieval.

To address the above mentioned issues, we propose a fast view-based 3D model retrieval framework. The framework comprises two parts: one is an Unsupervised Multiple Feature Fusion algorithm (UMFF) and the other one is an efficient Online Projection Learning algorithm (OPL). We first use UMFF to learn a more compact yet discriminative feature representation from the original multiple features. Then, OPL is designed to reduce the high computational cost in solving the objective function of UMFF for the newcome models by learning a projection matrix. Once the projection matrix is obtained, the input multiple high-dimensional features of the newcome 3D model can be fast transferred into the new compact yet discriminative feature representation. So the out-of-sample problem can be overcome via OPL in this work. In the proposed framework, many ranking algorithms such as the distance-based ranking method can be directly used to sort the 3D models and return the top ranked models to the user.

The rest of this article is organized as follows. Section 2 reviews some related work on the topic of 3D model retrieval. Then, the details of our proposed framework is introduced in Section 3. Section 4 presents the experimental results on two public 3D model databases. Finally, Section 5 concludes our work and sketches several direction of future work.