Statistical initialization of intrinsic K-means clustering on homogeneous manifolds

Abstract

The K-means algorithm is widely applied for clustering, and its clustering effect is influenced by its initialization. However, most existing works focus on the initialization of K and centers in Euclidean spaces, but few works in the literature deal with the initialization of K-means clustering on Riemannian manifolds. In this paper, we propose a unified scheme for learning K and selecting the initial centers for intrinsic K-means clustering on homogeneous manifolds, which can also be generalized to other types of manifolds. First, geodesic verticality is presented based on the geometric properties abstracted from the definition of orthogonality in Euclidean spaces. Then, geodesic projection on Riemannian manifolds is proposed for learning K, which achieves nonlinear dimensionality reduction and improves the computing efficiency. Additionally, the Riemannian metric of $\mathbb {S}^{n}$ is derived for the statistical initialization of the centers to improve the clustering accuracy. Finally, the intrinsic K-means algorithm for clustering on homogeneous manifolds based on the Karcher mean is given by applying the proposed manifold initialization, which improves the clustering effect. Simulations and experimental studies are conducted to show the effectiveness and accuracy of the proposed K-means scheme on manifolds.

Appendix:: Convergence of intrinsic K-means

The convergence of the proposed intrinsic K-means algorithm is crucial. The calculation to minimize objective function (6) is an NP-hard problem. This can be proven as follows.

Loss function

$$ L(\mathbf{\mu},\mathbf{P},D) = \sum\limits_{n = 1}^{N} {\sum\limits_{k = 1}^{K} {{D_{nk}}{{|| {{{\text{Log}}_{{\mathbf{\mu}_{k}}}}{\mathbf{p}_{n}}} ||}^{2}}} } $$

(19)

where D_nk = 1 if p_n ∈ D_k; otherwise, D_nk = 0 and ${{|| {{{\text {Log}}_{{\mathbf {\mu }_{k}}}}{\mathbf {p}_{n}}} ||}} = {||{\text {Log}}_{\mathbf {e}} \mathbf {\mathcal {A}}_{\mathbf {e}}^{\mathbf {\mu }_{k}} \mathbf {p}_{n} ||_{\mathbf {\mu }_{k}}}$.

E-step

When we update D from D^t− 1 to D^t, the distance between points on the manifolds is determined by the Riemannian geodesic, and can be calculated by (5).

$$ {D_{nk}} = \left\{ {\begin{array}{*{20}{c}} {1, \text{if} k = \arg {{\min }_{j}}{{|| {{{\text{Log}}_{{\mathbf{\mu}_{j}}}}{\mathbf{p}_{n}}} ||}^{2}}} \\ {0, \text{otherwise} } \end{array}} \right. $$

(20)

where ${|| {\text {Log}}_{{\mathbf {\mu }_{j}}}{\mathbf {p}_{n}} ||} = {||{\text {Log}}_{\mathbf {e}} \mathbf {\mathcal {A}}_{\mathbf {e}}^{\mathbf {\mu }_{j}} \mathbf {p}_{n} ||_{\mathbf {\mu }_{j}}}$; thus, we can obtain

$$ L({\mathbf{\mu}^{(t - 1)}}, \mathbf{P}, {D^{(t)}}) \leqslant L({\mathbf{\mu}^{(t - 1)}}, \mathbf{P}, {D^{(t - 1)}}) $$

(21)

where ${D^{(t)}} = \arg {\min \limits _{D}}L({\mathbf {\mu }^{(t - 1)}}, \mathbf {P}, D)$.

M-step

When we update μ from μ^(t− 1) to μ^(t), the current mean is determined by Karcher mean (Algorithm 3).

$$ \mathbf{\mu}_{k}^{(t)} = mean(D_{k}^{(t)}) = \arg \mathop {\min }\limits_{\mathbf{\mu} \in \mathcal{M}} {\mathbf{E}}\left[ {{{|| {{{\text{Log}}_{\mathbf{\mu}} }{\mathbf{p}^{(k)}}} ||}^{2}}} \right] $$

(22)

This is a decreasing process; thus, we can obtain

$$ L({\mathbf{\mu}^{(t)}},\mathbf{P},{D^{(t)}}) \leqslant L({\mathbf{\mu}^{(t - 1)}},\mathbf{P},{D^{(t)}}) $$

(23)

where ${\mathbf {\mu }^{(t)}} = \arg {\min \limits _{\mathbf {\mu }} }L(\mathbf {\mu } ,\mathbf {P},{D^{(t)}})$. Each iteration of the algorithm decreases the otherwise positive quantization error until the error reaches a fixed point.