Elsevier

Computers & Graphics

Volume 81, June 2019, Pages 82-91
Computers & Graphics

Special Section on CAD & Graphics 2019
A dimensional reduction guiding deep learning architecture for 3D shape retrieval

https://doi.org/10.1016/j.cag.2019.04.002Get rights and content

Highlights

  • A method for extracting short descriptors from lengthy descriptors is developed.

  • The dimension reduction results are strengthened by an attraction/repulsion model.

  • A deep residual network is trained for generating the short descriptors.

  • The short descriptors improve the retrieval speed greatly.

Abstract

The state-of-the-art shape descriptors are usually lengthy for gaining high retrieval precision. With the rapidly growing number of 3-dimensional models, the retrieval speed becomes a prominent problem in shape retrieval. In this paper, by exploiting the capabilities of the dimensionality reduction methods and the deep convolutional residual network (ResNet), we developed a method for extracting short shape descriptors (with just 2 real numbers, named 2-descriptors) from lengthy descriptors, while keeping or even improving the retrieval precision of the original lengthy descriptors. Specifically, an attraction and repulsion model is devised to strengthen the direct dimensionality reduction results. In this way, the dimensionality reduction results turn into desirable labels for the ResNet. Moreover, to extract the 2-descriptors using ResNet, we transformed it as a classification problem. For this purpose, the range of each component of the dimensionality reduction results (including two components in total) is uniformly divided into n intervals corresponding to n classes. Experiments on 3D shape retrieval show that our method not only accelerates the retrieval speed greatly but also improves the retrieval precisions of the original shape descriptors.

Introduction

3D shape retrieval is a crucial topic in computer vision and pattern recognition, which aims to retrieve the most relevant shapes to the query shape, based on the shape descriptors. In the field of shape retrieval, there are two competing ingredients: retrieval accuracy and retrieval speed. While high retrieval accuracy requires long shape descriptors, quick retrieval speed needs short descriptors. The state-of-the-art studies on shape retrieval mainly focus on improving accuracy, thus producing lots of lengthy shape descriptors, sometimes with high dimensions [1], [2], [3]. Although the accuracies of the retrieval algorithms based on these lengthy descriptors are acceptable, the retrieval speeds are usually very slow in huge shape database. More seriously, when the traditional index structures are employed for the fast query, the space and time requirements grow exponentially in the dimension [4]. Although there are indexing methods that avoid the curse of dimensionality [5], they do not support the exact search, and the dimension is also an important factor especially when facing a huge shape database. Therefore, due to the rapidly growing number of 3D shapes, it has become an urgent task to develop high-speed shape retrieval algorithms for the huge shape database, while keeping high retrieval accuracy.

In this paper, we develop an effective method for extracting short shape descriptors from lengthy or high dimensional descriptors, while keeping or even improving the retrieval accuracy. Specifically, given a shape database with classification, the descriptors of the shapes in the database consist of a point set in a high dimensional space. First, as a high dimensional data set, the set of shape descriptors are projected to a 2-dimensional plane using the dimensionality reduction (DR) methods. Moreover, an attraction and repulsion model is devised to reinforce the classification results in the plane, by enlarging the inter-class margin and reducing the intra-class variance. Then, taking the original shape descriptors as input, a deep convolutional residual network (ResNet) is constructed and trained. To improve the fitting precision of the ResNet, each component of the projected 2-dimensional shape descriptors is transformed into a vector, and taken as a label. Using the ResNet, a short shape descriptor with just two real numbers, named 2-descriptor, can be extracted from lengthy or high dimensional shape descriptors. In our implementation, the shapeDNA (one-dimensional lengthy descriptor) and the Fourier shape descriptor (three-dimensional descriptor) are taken as the original shape descriptors, respectively, and two kinds of ResNets are devised. It is shown by experiments that, the 2-descriptor generated by our method not only accelerated the retrieval speed significantly, but the retrieval precision is also improved as well, compared with the original lengthy or high dimensional descriptors.

The structure of this paper is as follows. In Section 2, some related work is briefly reviewed. In Section 3, the generation method of the 2-descriptor is introduced in detail, including the attraction and repulsion model and the construction and training of the ResNets. Moreover, the implementation details and results are presented in Section 4. Finally, Section 5 concludes the paper.

Section snippets

Related work

In this section, we briefly review the related work on 3 dimensional (3D) shape descriptors, dimension reduction (DR) methods, and descriptor learning methods by deep networks, respectively.

Method

The goal of our work is to find a high discriminative low-dimensional representation of 3D shapes from existing shape descriptors.

Mathematically, for each kind of shape descriptor, we aim to find a function f that takes any descriptor Xs extracted from shape s as input and output a feature Ys=f(Xs)Rd of much lower dimensionality as the new representation of shape s. Through this function, descriptors of 3D shapes can be quickly mapped to low-dimensional features.

Our architecture (refer to

Datasets

We carry out several experiments for shape retrieval to assess the performance of our dimensionality reduction method. The Princeton ModelNet dataset is a collection of 3D CAD models for objects which contains 127,915 3D models from 662 categories. In our implementation, modelNet40, a clean and well annotated subset of modelNet, is employed to evaluate the effect of our DR method on the Fourier descriptor and CSD. ModelNet40 is provided on the ModelNet website [43] with 12311 shapes from 40

Conclusion

In this paper, we proposed a new architecture for shape retrieval. Our architecture combines the advantages of dimensionality reduction and deep learning. Taking a lengthy shape descriptor as input, our method can generate a 2-descriptor with only two components effectively and efficiently. By the 2-descriptors, not only is the retrieval speed accelerated significantly, but the retrieval precision is improved as well, compared with the original lengthy shape descriptors. The 2-descriptors

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant no. 61872316, and the National Key R&D Plan of China under Grant no. 2016YFB1001501.

References (47)

  • B. Raducanu et al.

    A supervised non-linear dimensionality reduction approach for manifold learning

    Pattern Recognit

    (2012)
  • D. Smeets et al.

    Isometric deformation modelling for object recognition

    Proceedings of the international conference on computer analysis of images and patterns

    (2009)
  • LianZ. et al.

    Non-rigid 3d shape retrieval using multidimensional scaling and bag-of-features

    Proceedings of the ICIP

    (2010)
  • A. Giachetti et al.

    Radial symmetry detection and shape characterization with the multiscale area projection transform

    Proceedings of the computer graphics forum

    (2012)
  • R. Marimont et al.

    Nearest neighbour searches and the curse of dimensionality

    IMA J Appl Math

    (1979)
  • S. Har-Peled et al.

    Approximate nearest neighbor: towards removing the curse of dimensionality

    Theory Comput

    (2012)
  • SunJ. et al.

    A concise and provably informative multi-scale signature based on heat diffusion

    Proceedings of the computer graphics forum

    (2009)
  • A. Bronstein et al.

    Shape Google: geometric words and expressions for invariant shape retrieval

    ACM Trans Graph

    (2011)
  • W. Wohlkinger et al.

    Ensemble of shape functions for 3d object classification

    Proceedings of the 2011 IEEE international conference on robotics and biomimetics (ROBIO)

    (2011)
  • LianZ. et al.

    Rectilinearity of 3d meshes

    Int J Comput Vis

    (2010)
  • J. Zunic et al.

    A new convexity measure for polygons

    IEEE Trans Pattern Anal Mach Intell

    (2004)
  • LianZ. et al.

    A new convexity measurement for 3d meshes

    Proceedings of the 2012 IEEE conference on computer vision and pattern recognition (CVPR)

    (2012)
  • ChenD. et al.

    On visual similarity based 3d model retrieval

    Proceedings of the computer graphics forum

    (2003)
  • Cited by (5)

    • Multi-scale and multi-level shape descriptor learning via a hybrid fusion network

      2022, Graphical Models
      Citation Excerpt :

      MeshNet [9] regards faces as the unit and captures the spatial feature and structural feature separately. Wang et al. [35] develop a method for extracting short shape descriptors from lengthy descriptors by exploiting the capabilities of the dimensionality reduction methods and the deep convolutional residual network. Fusion methods There are a few existing fusion methods exploiting multiple data types.

    • A Comparative Dimensionality Reduction Study in Diabetes Patients using LDA and PCA

      2022, 3rd Information Technology to Enhance e-Learning and Other Application, IT-ELA 2022
    View full text