Discrete curve model for non-elastic shape analysis on shape manifold

doi:10.1016/j.patcog.2022.108760

Pattern Recognition

Volume 130, October 2022, 108760

https://doi.org/10.1016/j.patcog.2022.108760 Get rights and content

Highlights

•
We construct a shape manifold and its tangent space for the discrete curve model.
•
We design a geodesic distance that significantly improves the retrieval score.
•
Our method overcomes some limitations in bending-only model.
•
We propose a novel application named shape arithmetic.

Abstract

In this paper, we construct a novel finite dimensional shape manifold for shape analyses. Elements of the shape manifold are a set of discrete, planar, and closed curves, which stand for object boundaries and are represented by direction function. On this manifold, we use a set of N-dimensional Fourier basis to construct the tangent space of the shape manifold as a finite dimensional space. Furthermore, we construct the shape manifold as a Riemannian manifold, in which the Riemannian metric is interpreted as an $l^{2}$ metric. Our method improves the performance of bending-only models in the issues of shape analysis including the shape synthesis, comparison, and statistic analysis. We evaluate the performance of the manifold via the following applications: 1) shape interpolation and extrapolation between curves, 2) shape retrieval on the Flavia leaf database, 3) shape synthesis using an estimated probability distribution on the manifold, and 4) a novel application named shape arithmetic. All the above experiments clearly demonstrate our approach achieves superior performance to state-of-the-art methods.

Introduction

Shape analysis aims to compare, synthesize and statistically analyze the object boundaries under one mathematical framework. Methods for shape analysis usually originate from mathematics [1], and are widely applied in computer vision [2], medical imaging [3], and biomedical engineering [4]. Furthermore, the related techniques are the foundations for the $3 D$ surface analysis [5] and $3 D$ facial recognition [6].

Klassen et al. [7] proposed one classical framework that achieves the tasks of shape analysis using the analytic and statistic methods on shape manifold. The tasks of comparison, synthesis, and statistical analysis are formulated as the mathematical notions, such as the geodesic, the geodesic distance, and the probability distribution on manifold. The framework, as shown in the top row of Fig. 1, includes the boundary formulation, the shape representation, and the shape manifold construction. The shapes of object boundaries are formulated as the planar, closed curves and represented by the direction functions. The shape manifold, which contains all the curves, is constructed as an infinite dimensional manifold and equips an explicit Sobolev type metric as the Riemannian metric [8]. At the end of the framework, there is a numerical algorithm to approximate the infinite dimensional manifold as a finite dimensional manifold.

However, this method fails to find the geodesic or the self-intersecting curves, even when these curves are elements on the shape manifold. In this paper, we remove the limitation in the boundary formulation and the shape manifold construction to improve bending-only model performance. We still use the direction function to represent shape, because the direction function completely describes the planar curves’ behavior [9].

To solve the above limitations, we propose a finite dimensional shape manifold that is derived from the discrete curve. The discrete curve extends the bending-only model that does not impose smoothness constraints while leading to an explicit and robust geodesic equation.

The tangent space of the shape manifold is constructed by a set of $N$ -dimensional Fourier basis. In addition, we equip an $l^{2}$ metric as the Riemannian metric on our shape manifold. Finally, a shooting approach is adopted on our shape manifold to find the geodesics.

Our framework is illustrated in the bottom row of Fig. 1 to compare with the existing framework in [7]. Our contributions and improvements in boundary formulation and shape manifold construction are summarized as the follows:

1.
The discrete curve model represented by direction function avoids a pre-smoothing stage for the curve model. Such pre-smoothing will filter legitimate features for shapes.
2.
Our method overcomes some limitations in bending-only model and extends the associated applications, which are summarized in Tab. 1. The shape arithmetics aims to solve the equations about addition, subtraction and scale multiplication for curves.
3.
Our metric derived from the discrete curve model is effective in the shape retrieval, which is verified in the Flavia database. The comparison results show that our approach significantly improves the retrieval score for the non-elastic shape analysis.

The remaining of the paper is organized as follows: Section 2 discusses related works. In Section 3, we introduce shape representation and shape registration for the discrete curve model. In addition, we discuss some fundamental concepts for the shape manifold construction, such as the tangent space and the definition of geodesic distance. In Section 4, we adopt the shooting technology on our shape manifold. Some applications are presented in Section 5 to demonstrate the performance of our method. The paper ends with conclusion in Section 6.

Section snippets

Related works

A recent review for shape analysis is presented in [14]. Approaches that explicitly study the formulation of object boundaries and the construction of shape space can be divided into three categories, i.e., the landmarks model-based [15], the template model-based [16], and the continuous curve model-based [17].

The landmark model formulates shapes as a collection of labelled points [15]. The following work [18] extend and develop the statistical techniques on the associated shape space for

Shape representation for the discrete curve model

The shapes are the objects’ boundaries that are formulated by continuous, closed curves $α : [0, 1] \to R^{2}$ with a parameter $s \in [0, 1]$ . In practice, these shape data are stored in digital form so that the shape is represented as a set of $N + 1$ points ${p_{0}, \dots, p_{N}}$ , $p_{m} \in R^{2}$ . Since the boundary is closed curve, we have $p_{0} = p_{N}$ . To model the set of points, we use a discrete curve model instead of continuous curve, in which the parameter $s$ is replaced by $\frac{m}{N}$ . The discrete curves formulation is denoted as $α [m] = (α_{x} [m], α_{y} [m]$

Finding the geodesic

Let $θ_{1}$ and $θ_{2}$ are respectively located at the $x_{1}$ and $x_{2}$ on shape manifold $S$ . We aim to find the geodesic between them that is the shortest path between the points $x_{1}$ and $x_{2}$ on shape manifold.

Since we use the shooting method to find geodesic, the geodesic $G (τ)$ will be generated by a flow $Ψ (θ_{1}, t, f)$ starting from $θ_{1}$ and with a tangent vector (the shooting direction) $f$ at $θ_{1}$ , where the $t$ is the time parameter. For a small enough $ε > 0$ , $Ψ (θ_{1}, ε, f)$ is computed as $Ω (θ_{1} + ε f)$ , where the projection $Ω$ is

Applications

In this section, we use our construct shape manifold to systematic study the classical applications in shape analysis, such as the shape interpolation, extrapolation, shape retrieval and statistics shape analysis. Moreover, we develop a novel application about the shape arithmetics for bending-only model.

Before we discuss the applications, we introduce a technique to normalize the scales of recovered curves. This technique will be used in the applications of shape synthesis. Since the

Conclusion

In this paper, we propose a discrete curve model for non-elastic shape analysis. Our approach overcomes the limitation in non-elastic shape analysis that the obtained geodesics are often far from the optimal, especially for the case of the geodesic connecting a simple curve and a self-intersecting curve. The experiment of shape interpolation demonstrates that the finite-dimensional tangent space enables us to improve the performance of finding the geodesic. Compared with existing approaches,

Declaration of Competing Interest

The authors declare no conflict of interest.

Acknowledgements

This work was supported in part by the National Key R&D Program of China (2019YFB1312000), the National Natural Science Foundation of China (61471229, 62022030 and 62033005), the Natural Science Foundation of Guangdong Province (2019A1515011950), the Key Field Projects of Colleges and Universities of Guangdong Province (2020ZDZX3065), the Degree & Postgraduate Education Reform Project of Harbin Institute of Technology (21MS003) and the Heilongjiang Provincial Natural Science Foundation of China

Peng Chen received the BS degree in Electronic Engineering from Shantou University, Shantou, China in 2011; the ME degree in Information and Communication Engineering from Shantou University, Shantou, China in 2015. He is currently pursuing the PhD degree with the Harbin Institute of Technology, Harbin, China. His current research interests include shape analysis, shape clustering and their applications to computer vision.

References (43)

M.J. Flores et al.
Intrinsic sample mean in the space of planar shapes
Pattern Recognit.
(2016)
M. Devanne et al.
Motion segment decomposition of RGB-D sequences for human behavior understanding
Pattern Recognit.
(2017)
R. Slama et al.
Accurate 3D action recognition using learning on the Grassmann manifold
Pattern Recognit.
(2015)
C. Lv et al.
Nasal similarity measure of 3D faces based on curve shape space
Pattern Recognit
(2019)
L. Younes
Spaces and manifolds of shapes in computer vision: an overview
Image Vis. Comput.
(2012)
L. Younes
Optimal matching between shapes via elastic deformations
Image Vis. Comput.
(1999)
H. Laga et al.
Landmark-free statistical analysis of the shape of plant leaves
J. Theor. Biol.
(2014)
L. Tu et al.
Skeletal shape correspondence through entropy
IEEE Trans. Med. Imaging
(2018)
H. Laga et al.
Numerical inversion of SRNF maps for elastic shape analysis of genus-zero surfaces
IEEE Trans. Pattern Anal. Mach. Intell.
(2017)
E. Klassen et al.
Analysis of planar shapes using geodesic paths on shape spaces
IEEE Trans. Pattern Anal. Mach. Intell.
(2004)

M.P.D. Carmo

Differential Geometry of Curves and Surfaces

(1976)

L. Younes et al.

A metric on shape space with explicit geodesics

Matematica e Applicazioni

(2008)

W. Mio et al.

On shape of plane elastic curves

Int. J. Comput. Vis.

(2007)

A. Srivastava et al.

Shape analysis of elastic curves in euclidean spaces

IEEE Trans. Pattern Anal. Mach. Intell.

(2011)

G.G. Demisse et al.

Deformation based curved shape representation

IEEE Trans. Pattern Anal. Mach. Intell.

(2018)

K. Bharath et al.

Analysis of shape data: from landmarks to elastic curves

Wiley Interdiscip. Rev. Comput. Stat.

(2020)

D.G. Kendall

Shape manifolds, procrustean metrics, and complex projective spaces

Bull. London Math. Soc.

(1984)

Y. Amit et al.

Structural image restoration through deformable templates

J. Am. Stat. Assoc.

(1991)

L. Younes

Computable elastic distances between shapes

SIAM J. Appl. Math.

(1998)

H. Le et al.

The Riemannian structure of euclidean shape spaces: a novel environment for statistics

Ann. Stat.

(1993)

J.Q. Campbell et al.

An automated method for landmark identification and finite-element modeling of the lumbar spine

IEEE Trans. Biomed. Eng.

(2015)

Cited by (1)

A gradient optimization and manifold preserving based binary neural network for point cloud
2023, Pattern Recognition
With significant progress of deep learning on 3D point cloud, the demand for deployment of point cloud neural network on the edge devices is growing. Binary neural network, a type of quantization compression method, with extreme low bit and fast inference speed, attracts more attention. It is more challenging, but has greater potentiality. Most of the researches on binary networks focus on images rather than point cloud. Considering the particularity of point cloud neural network, this paper presents a novel binarization framework, which includes two main contributions. Firstly, a gradient optimization method is proposed to overcome the shortcomings of Straight Through Estimator (STE) commonly used in the back propagation of binary network training. Secondly, based on the analysis of manifold distortion caused by the binary convolution and pooling operations, we propose an optimized scaling recovery method to restore manifold for the convoluted feature, and also, a pooling correction method to improve the pooled feature's fidelity. Manifold distortion leads to the severe feature homogeneity problem, which brings trouble in generating features with sufficient discrimination for classification and segmentation. The manifold preserving optimizations are designed to introduce minimum extra parameters to balance the accuracy with the computation and storage consumption. Experiments show that the proposed method outperforms state-of-the-art in accuracy with ignored overhead, and also has good scalability.

Xutao Li received the PhD degree in Electronics Engineering from Huazhong University of Science and Technology, in 2006. From 2006 to 2008, he was a postdoctoral fellow of South China University of technology. Since 2013, he has been a professor with the Department of Electronic Engineering, Shantou University. He has authored or coauthored more than 20 papers published in refereed journals. His research interests include Array Signal Processing, Radar Systems, Computer Vision and nonlinear presentation to signals. Dr. Li is a member of IEEE.

Changxing Ding received the PhD degree from the University of Technology Sydney, Australia, in 2016. He is now with the School of Electronic and Information Engineering, South China University of Technology. His research interests include computer vision and deep learning. He has published about 20 papers in refereed journals and proceedings including IEEE T-PAMI, IEEE T-IP,T-MM etc.

Jianxing Liu received the BS degree in mechanical engineering in 2008, the ME degree in control science and engineering in 2010, both from Harbin Institute of Technology, Harbin, China and the PhD degree in Automation from the Technical University of Belfort-Montbeliard (UTBM), France, in 2014. Since 2014, he joined Harbin Institute of Technology, China, as an Assistant Professor, and was then promoted to an Associate Professor in 2017. His current research interests include nonlinear control algorithms, sliding mode control, fuel cell systems, hybrid electric or fuel cell vehicles, control of power electronics and converters, wind-driven generator systems, energy management for micro-grids, nonlinear control and observation methods, and their applications in industrial electronics systems and renewable energy systems.

Ligang Wu received the BS degree in Automation from Harbin University of Science and Technology, China in 2001; the M.E. degree in Navigation Guidance and Control from Harbin Institute of Technology, China in 2003; the PhD degree in Control Theory and Control Engineering from Harbin Institute of Technology, China in 2006. From January 2006 to April 2007, he was a Research Associate in the Department of Mechanical Engineering, The University of Hong Kong, Hong Kong. From September 2007 to June 2008, he was a Senior Research Associate in the Department of Mathematics, City University of Hong Kong, Hong Kong. From December 2012 to December 2013, he was a Research Associate in the Department of Electrical and Electronic Engineering, Imperial College London, London, UK. In 2008, he joined the Harbin Institute of Technology, China, as an Associate Professor, and was then promoted to a Full Professor in 2012. Dr. Wu was the winner of the National Science Fund for Distinguished Young Scholars in 2015, and received China Young Five Four Medal in 2016. He was named as the Distinguished Professor of Chang Jiang Scholar in 2017, and was named as the Highly Cited Researcher in 2015, 2016, 2017 and 2018. Dr. Wu currently serves as an Associate Editor for a number of journals, including IEEE Transactions on Automatic Control, IEEE/ASME Transactions on Mechatronics, Information Sciences, Signal Processing, and IET Control Theory and Applications. He is also an Associate Editor for the Conference Editorial Board, IEEE Control Systems Society. Dr. Wu has published 6 research monographs and more than 150 research papers in international referred journals. His current research interests include switched systems, stochastic systems, computational and intelligent systems, sliding mode control, and advanced control techniques for power electronic systems.

View full text

Discrete curve model for non-elastic shape analysis on shape manifold

Highlights

Abstract

Introduction

Section snippets

Related works

Shape representation for the discrete curve model

Finding the geodesic

Applications

Conclusion

Declaration of Competing Interest

Acknowledgements

Pattern Recognit.

Pattern Recognit.

Pattern Recognit.

Pattern Recognit

Image Vis. Comput.

Image Vis. Comput.

J. Theor. Biol.

Skeletal shape correspondence through entropy

IEEE Trans. Med. Imaging

Numerical inversion of SRNF maps for elastic shape analysis of genus-zero surfaces

IEEE Trans. Pattern Anal. Mach. Intell.

Analysis of planar shapes using geodesic paths on shape spaces

IEEE Trans. Pattern Anal. Mach. Intell.

Differential Geometry of Curves and Surfaces

A metric on shape space with explicit geodesics

Matematica e Applicazioni

On shape of plane elastic curves

Int. J. Comput. Vis.

Shape analysis of elastic curves in euclidean spaces

IEEE Trans. Pattern Anal. Mach. Intell.

Deformation based curved shape representation

IEEE Trans. Pattern Anal. Mach. Intell.

Analysis of shape data: from landmarks to elastic curves

Wiley Interdiscip. Rev. Comput. Stat.

Shape manifolds, procrustean metrics, and complex projective spaces

Bull. London Math. Soc.

Structural image restoration through deformable templates

J. Am. Stat. Assoc.

Computable elastic distances between shapes

SIAM J. Appl. Math.

The Riemannian structure of euclidean shape spaces: a novel environment for statistics

Ann. Stat.

An automated method for landmark identification and finite-element modeling of the lumbar spine

IEEE Trans. Biomed. Eng.