Special Section on CAD & Graphics 2019A dimensional reduction guiding deep learning architecture for 3D shape retrieval
Graphical abstract
Introduction
3D shape retrieval is a crucial topic in computer vision and pattern recognition, which aims to retrieve the most relevant shapes to the query shape, based on the shape descriptors. In the field of shape retrieval, there are two competing ingredients: retrieval accuracy and retrieval speed. While high retrieval accuracy requires long shape descriptors, quick retrieval speed needs short descriptors. The state-of-the-art studies on shape retrieval mainly focus on improving accuracy, thus producing lots of lengthy shape descriptors, sometimes with high dimensions [1], [2], [3]. Although the accuracies of the retrieval algorithms based on these lengthy descriptors are acceptable, the retrieval speeds are usually very slow in huge shape database. More seriously, when the traditional index structures are employed for the fast query, the space and time requirements grow exponentially in the dimension [4]. Although there are indexing methods that avoid the curse of dimensionality [5], they do not support the exact search, and the dimension is also an important factor especially when facing a huge shape database. Therefore, due to the rapidly growing number of 3D shapes, it has become an urgent task to develop high-speed shape retrieval algorithms for the huge shape database, while keeping high retrieval accuracy.
In this paper, we develop an effective method for extracting short shape descriptors from lengthy or high dimensional descriptors, while keeping or even improving the retrieval accuracy. Specifically, given a shape database with classification, the descriptors of the shapes in the database consist of a point set in a high dimensional space. First, as a high dimensional data set, the set of shape descriptors are projected to a 2-dimensional plane using the dimensionality reduction (DR) methods. Moreover, an attraction and repulsion model is devised to reinforce the classification results in the plane, by enlarging the inter-class margin and reducing the intra-class variance. Then, taking the original shape descriptors as input, a deep convolutional residual network (ResNet) is constructed and trained. To improve the fitting precision of the ResNet, each component of the projected 2-dimensional shape descriptors is transformed into a vector, and taken as a label. Using the ResNet, a short shape descriptor with just two real numbers, named 2-descriptor, can be extracted from lengthy or high dimensional shape descriptors. In our implementation, the shapeDNA (one-dimensional lengthy descriptor) and the Fourier shape descriptor (three-dimensional descriptor) are taken as the original shape descriptors, respectively, and two kinds of ResNets are devised. It is shown by experiments that, the 2-descriptor generated by our method not only accelerated the retrieval speed significantly, but the retrieval precision is also improved as well, compared with the original lengthy or high dimensional descriptors.
The structure of this paper is as follows. In Section 2, some related work is briefly reviewed. In Section 3, the generation method of the 2-descriptor is introduced in detail, including the attraction and repulsion model and the construction and training of the ResNets. Moreover, the implementation details and results are presented in Section 4. Finally, Section 5 concludes the paper.
Section snippets
Related work
In this section, we briefly review the related work on 3 dimensional (3D) shape descriptors, dimension reduction (DR) methods, and descriptor learning methods by deep networks, respectively.
Method
The goal of our work is to find a high discriminative low-dimensional representation of 3D shapes from existing shape descriptors.
Mathematically, for each kind of shape descriptor, we aim to find a function f that takes any descriptor Xs extracted from shape s as input and output a feature of much lower dimensionality as the new representation of shape s. Through this function, descriptors of 3D shapes can be quickly mapped to low-dimensional features.
Our architecture (refer to
Datasets
We carry out several experiments for shape retrieval to assess the performance of our dimensionality reduction method. The Princeton ModelNet dataset is a collection of 3D CAD models for objects which contains 127,915 3D models from 662 categories. In our implementation, modelNet40, a clean and well annotated subset of modelNet, is employed to evaluate the effect of our DR method on the Fourier descriptor and CSD. ModelNet40 is provided on the ModelNet website [43] with 12311 shapes from 40
Conclusion
In this paper, we proposed a new architecture for shape retrieval. Our architecture combines the advantages of dimensionality reduction and deep learning. Taking a lengthy shape descriptor as input, our method can generate a 2-descriptor with only two components effectively and efficiently. By the 2-descriptors, not only is the retrieval speed accelerated significantly, but the retrieval precision is improved as well, compared with the original lengthy shape descriptors. The 2-descriptors
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant no. 61872316, and the National Key R&D Plan of China under Grant no. 2016YFB1001501.
References (47)
- et al.
Efficient 3d shape matching and retrieval using a concrete radialized spherical projection representation
Pattern Recognit
(2007) - et al.
Reeb graphs for shape analysis and applications
Theor Comput Sci
(2008) - et al.
Laplace-Beltrami spectra as “Shape-DNA” of surfaces and solids
Comput Aided Des
(2006) - et al.
A 3d shape descriptor based on spectral analysis of medial axis
Comput Aided Geom Des
(2015) - et al.
Intrinsic shape matching via tensor-based optimization
Comput Aided Des
(2019) - et al.
Shape similarity assessment based on partial feature aggregation and ranking lists
Pattern Recognit Lett
(2016) - et al.
Learning local shape descriptors from part correspondences with multiview convolutional networks
ACM Trans Graph (TOG)
(2018) An easy measure of compactness for 2d and 3d shapes
Pattern Recognit
(2008)- et al.
A robust and accurate algorithm for estimating the complexity of the cortical surface
J Neurosci Methods
(2008) - et al.
Kernel ridge regression for out-of-sample mapping in supervised manifold learning
Expert Syst Appl
(2012)
A supervised non-linear dimensionality reduction approach for manifold learning
Pattern Recognit
Isometric deformation modelling for object recognition
Proceedings of the international conference on computer analysis of images and patterns
Non-rigid 3d shape retrieval using multidimensional scaling and bag-of-features
Proceedings of the ICIP
Radial symmetry detection and shape characterization with the multiscale area projection transform
Proceedings of the computer graphics forum
Nearest neighbour searches and the curse of dimensionality
IMA J Appl Math
Approximate nearest neighbor: towards removing the curse of dimensionality
Theory Comput
A concise and provably informative multi-scale signature based on heat diffusion
Proceedings of the computer graphics forum
Shape Google: geometric words and expressions for invariant shape retrieval
ACM Trans Graph
Ensemble of shape functions for 3d object classification
Proceedings of the 2011 IEEE international conference on robotics and biomimetics (ROBIO)
Rectilinearity of 3d meshes
Int J Comput Vis
A new convexity measure for polygons
IEEE Trans Pattern Anal Mach Intell
A new convexity measurement for 3d meshes
Proceedings of the 2012 IEEE conference on computer vision and pattern recognition (CVPR)
On visual similarity based 3d model retrieval
Proceedings of the computer graphics forum
Cited by (5)
Multi-scale and multi-level shape descriptor learning via a hybrid fusion network
2022, Graphical ModelsCitation Excerpt :MeshNet [9] regards faces as the unit and captures the spatial feature and structural feature separately. Wang et al. [35] develop a method for extracting short shape descriptors from lengthy descriptors by exploiting the capabilities of the dimensionality reduction methods and the deep convolutional residual network. Fusion methods There are a few existing fusion methods exploiting multiple data types.
Foreword to the special section on the international conference on computer-aided design and computer graphics (CAD/Graphics) 2019
2020, Computers and Graphics (Pergamon)A Comparative Dimensionality Reduction Study in Diabetes Patients using LDA and PCA
2022, 3rd Information Technology to Enhance e-Learning and Other Application, IT-ELA 2022Mathematical Modeling for Ceramic Shape 3D Image Based on Deep Learning Algorithm
2021, Advances in Mathematical PhysicsCondition monitoring systems: a systematic literature review on machine-learning methods improving offshore-wind turbine operational management
2021, International Journal of Sustainable Energy