Novel Sketch-Based 3D Model Retrieval via Cross-domain Feature Clustering and Matching

Abstract

To date, with the rapid advancement of scanning hardware and CAD software, we are facing technically challenging on how to search and find a desired model from a huge shape repository in a fast and accurate way in this bigdata digital era. Sketch-based 3D model retrieval is a flexible and user-friendly approach to tackling the existing challenges. In this paper, we articulate a novel way for model retrieval by means of sketching and building a 3D model retrieval framework based on deep learning. The central idea is to dynamically adjust the distance between the learned features of sketch and model in the encoded latent space through the utility of several deep neural networks. In the pre-processing phase, we convert all models in the shape database from meshes to point clouds because of its lightweight and simplicity. We first utilize two deep neural networks for classification to generate embeddings of both input sketch and point cloud. Then, these embeddings are fed into our clustering deep neural network to dynamically adjust the distance between encodings of the sketch domain and the model domain. The application of the sketch embedding to the retrieval similarity measurement could continue to improve the performance of our framework by re-mapping the distance between encodings from both domains. In order to evaluate the performance of our novel approach, we test our framework on standard datasets and compare it with other state-of-the-art methods. Experimental results have validated the effectiveness, robustness, and accuracy of our novel method.