Intrinsic shape matching via tensor-based optimization☆
Introduction
Seeking correspondences between a set of discrete points on two different models is a fundamental problem in both computer graphics and computer vision communities. The most crucial property of these correspondences is that they allow the transfer of information from one model to another. It is therefore not surprising that computing maps lies at the essence of a wide spectrum of geometry processing applications as diverse as attribute transfer, shape morphing and shape matching among many others [1].
As an extensive direction, intrinsic surface correspondence computation between two nearly isometric models is well studied in last decades. Since many real-world deformations are approximately isometric (e.g., bending limbs or parts of humans or animals and varying facial expressions), finding near isometric maps possesses practical significance in non-rigid shape analysis. While several solutions to rigid matching are well established, non-rigid shape matching remains difficult even when the space of deformations is limited. Unlike the rigid case, the main challenge for non-rigid shape matching is that the search space of intrinsic correspondences is too large to be computationally tractable. A well-established approach is to reduce the search space by extracting a set of distinctive features from both models and detect correspondences between these features only [2], [3]. For example, isometric matching techniques try to find correspondences that preserve the pairwise geodesic distances [4], [5]. In this paper, we present an efficient algorithm that is able to automatically find sparse sets of feature points between two isometric or nearly isometric models.
The correspondence problem between isometric models is more difficult for a number of reasons: (1) Most of the isometric correspondence methods try to use pairwise geometric relations to measure the similarity between pairs of feature points. For example, the Euclidean distance between pairs of points is used in [6], leading to a matching criterion that is invariant to rotation. Berg et al. [7] used a combination of rotation and scale invariant potentials such as distances and angles between feature points. Zheng et al. [8] proposed a point matching problem based on the notion of neighborhood structure, where each point is a node in the graph, and two nodes are connected by an edge if they are neighbors. However, these pairwise based approaches may fail in the presence of ambiguities such as repeated patterns or non discriminative local appearances. (2) Concretely, the search for correspondences between two isometric or nearly isometric shapes is solved using a distance-preserving mapping such as the geodesic distances, where the distance between any two points on one shape is exactly the same as the distance between their correspondences on the other one. However, these distance-preserving mappings are low-dimensional, and many feature points may lead to similar descriptors, which make the matching procedure ambiguous. (3) Several non-rigid shape matching methods try to employ the local information around the feature points to solve the correspondence problem. Despite the fact that using the local signatures such as curvature or local patches is well-suited for establishing correspondences between two non-rigid shapes, these local signatures may cause many false matches on a shape as they rely on local geometry cues only.
To alleviate the first problem, feature matching consistency is enforced by using third order potentials that are defined over triplets of points, instead of unary and pairwise ones used in classical methods. Note that Duchenne et al. [9] also proposed to use high-order potentials. However, these potentials are not applicable to non-rigid shape matching. As for the second and third problems, the quality of a correspondence is measured in two ways: how similar a feature point from one shape is to its corresponding feature point from the other one (a local criterion), and how much their spatial distance is changed (a global criterion). Therefore, we propose to use shape priors as they received much attention in the last few years [10], [11], [12], and favor correspondences that are locally and isometrically consistent and can deal with deformations to some extent.
This paper builds a framework that can accommodate both local constraints (3D shape priors sampled from the feature points) and global constraints (the normalized geodesic distances). The first one is referred to as the prior-based potential, and the second one as the geodesic-based potential. To efficiently match two non-rigid shapes, our method first extracts a set of feature points residing on the prominent parts of the two models and constructs 3D local shape priors around these feature points. Next, we provide feature vectors for triplets of feature points in both shapes using both prior-based and geodesic-based potentials. These feature vectors define a similarity tensor representing the affinity between feature triangles. Inspired by the work [9], a high order graph matching problem that integrates third-order constraints formulated to guide the search for correspondences, which is solved with a tensor power iteration technique. Note that our task is different from [9] in the fact that our optimization involves row/column unit normalizations and it is designed for non-rigid shape matching.
To evaluate the proposed algorithm, we test it along with state-of-the-art methods on the Surface Correspondence Benchmark [13]. The presented method is easy to implement, fully automatic and it generates quality correspondences for a wide variety of model pairs. Our paper makes the following two main contributions:
- •
We propose a fully automatic non-rigid shape correspondence method, which incorporates both the prior-based (local) and the geodesic-based (global) potentials into a similarity tensor to avoid the problem of matching inconsistency.
- •
We devise a tensor-based optimization technique to solve the third-order graph matching problem, which involves row/column normalizations.
Section snippets
Feature point extraction
The first step of our method is to select a set of feature points from the vertices of triangular meshes on the surface. Given a 3D shape model represented by triangular meshes, a variety of methods can be applied to generate feature points. A common approach is to extract feature points in shape extremities. For example, Zhang et al. [4] detected feature points at the local minima of the average squared geodesic distance field defined over the given mesh. Lipman et al. [2] took all points at
Overview
Given two shapes as input, both represented as triangular meshes, the proposed framework aims to establish a set of correspondences that relate points on shape 1 to points on shape 2 (see problem formulation in Section 4.1). The main idea of our method is illustrated in Fig. 1, and outlined in more detail in the following sections. Specifically, our algorithm proceeds in the following two steps: feature extraction and tensor based optimization.
We first approach the correspondence problem by
Algorithm
In this section, every phase of the proposed framework is discussed in detail. First, the problem of finding correspondences between two input shapes is formulated . Next, the feature points are extracted from the surfaces and the geodesic-based and prior-based distances are computed for these points on both shapes. Finally, the set of sparse correspondences between the two sets of feature points is found by solving an optimization problem using tensor power iteration technique.
Results
In this section, the results of our proposed framework are reported. The effectiveness of our matching algorithm is evaluated on a variety of 3D shapes, and compared to several state-of-the-art methods for finding inter-surface maps and correspondences.
Conclusion and future work
In this paper, we describe a fully automatic algorithm for finding a set of sparse correspondences between two sets of feature points extracted from two shapes. To avoid the inconsistency problem, both the local constraints, in the form of 3D shape priors sampled around the feature points, and the global constraints, in the form of normalized geodesic distances, are incorporated into a similarity tensor. The sparse set of correspondences is then obtained by solving a tensor-based optimization
Acknowledgments
We thank the anonymous reviewers for their valuable comments and suggestions. The work was supported in part by National Natural Science Foundation of China (61402224), the Fundamental Research Funds for the Central Universities, China (NE2014402, NE2016004), the Natural Science Foundation of Jiangsu Province, China (BK2014833), the NUAA, China Fundamental Research Funds (NS2015053), and Jiangsu Specially-Appointed Professorship.
References (43)
- et al.
Surface reconstruction with data-driven exemplar priors
Comput-Aided Des
(2017) - et al.
A survey on shape correspondence
Comput Graph Forum
(2011) - et al.
Möbius Voting for Surface Correspondence
ACM Trans Graph (Proc SIGGRAPH)
(2009) - et al.
One point isometric matching with the heat kernel
Comput Graph Forum
(2010) - et al.
Deformation-Driven shape correspondence
Comput Graph Forum
(2008) - et al.
Non-rigid registration under isometric deformations
Comput Graph Forum
(2008) - et al.
A spectral technique for correspondence problems using pairwise constraints
- et al.
Shape matching and object recognition using low distortion correspondences
- et al.
Robust point matching for nonrigid shapes by preserving local neighborhood structures
IEEE Trans Pattern Anal Mach Intell
(2006) - et al.
A tensor-based algorithm for high-order graph matching
IEEE Trans Pattern Anal Mach Intell
(2011)
Self-similarity for accurate compression of point sampled surfaces
Comput Graph Forum
Data-driven sparse priors of 3D shapes
Comput Graph Forum
Blended intrinsic maps
ACM Trans Graph
A concise and provably informative multi-scale signature based on heat diffusion
Comput Graph Forum
Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching
Proc Natl Acad Sci (PNAS)
Functional maps: A flexible representation of maps between shapes
ACM Trans Graph
Integral invariants for shape matching
IEEE Trans Pattern Anal Mach Intell
Shape matching using multiscale integral invariants
IEEE Trans Pattern Anal Mach Intell
Partial functional correspondence
Comput Graph Forum
Cited by (4)
Discrete artificial electric field algorithm for high-order graph matching
2020, Applied Soft Computing JournalCitation Excerpt :Graph matching is a problem of finding the correspondence between two feature sets and mainly arises in the field of computer science such as shape matching (i.e. to find the geometrical transformation between two shapes and measure the similarity between them) [1], image retrieval which is a process of retrieving or searching an image of user interest from the large database [2], image classification (i.e. classifying or extracting an image according to its visual content) [3], object tracking which is a problem of estimating the trajectory of a moving object in the image) [4], object detection (i.e. locating and identifying the object of certain classes in an image) [5].
A dimensional reduction guiding deep learning architecture for 3D shape retrieval
2019, Computers and Graphics (Pergamon)Citation Excerpt :Although the retrieval accuracy is improved by these lengthy global shape descriptors, the retrieval speed is very slow, especially in the huge shape database. On the other hand, local shape descriptors capture important geometric information on local regions of 3D shapes, and are frequently employed in local shape analysis such as partial shape retrieval [17,18], and shape correspondence [19,20]. Representative work on local shape descriptors includes spin images [21], intrinsic shape signatures [22], heat kernel signature [6], wave kernel signature [19], etc.
A Comprehensive Survey on Artificial Electric Field Algorithm: Theories and Applications
2024, Archives of Computational Methods in Engineering3D Symmetric Shape Matching Algorithm Using Heat Kernel
2020, Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics
- ☆
This paper has been recommended for acceptance by L.B. Kara.