Hybrid Manifold Clustering with Evolutionary Tuning

Abstract

Manifold clustering, also known as submanifold learning, is the task to embed patterns in submanifolds with different characteristics. This paper proposes a hybrid approach of clustering the data set, computing a global map of cluster centers, embedding each cluster, and then merging the scaled submanifolds with the global map. We introduce various instantiations of cluster and embedding algorithms based on hybridization of $k$-means, principal component analysis, isometric mapping, and locally linear embedding. A (1$+$1)-ES is employed to tune the submanifolds by rotation and scaling. The submanifold learning algorithms are compared w.r.t. the nearest neighbor classification performance on various experimental data sets.

A Benchmark Problems

MakeClass is a classification data set generated with the scikit-learn [19] method make_classification with $d$ dimensions and two centers. The UCI Digits data set [20] comprises handwritten digits with $d=64$. It is a frequent reference problem related to the recognition of handwritten characters and digits. The Faces data set is called Labeled Faces in the Wild [21] and has been introduced for studying the face recognition problem. The data set source is http://vis-www.cs.umass.edu/lfw/. It contains JPEG images of famous people collected from the internet. The Gaussian blobs data set is generated with the scikit-learn [19] method make_blobs and the following settings. Two centers, i.e., two classes are generated, each with a standard deviation of $\sigma = 10.0$ and variable $d$. Friedman 1 is a regression data set generated with the scikit-learn [19] method make_friedman1. The regression problem has been introduced in [22], where Friedman introduces multivariate adaptive regression splines. Friedman 2 is also a regression data set of scikit-learn [19] and can be generated with make_friedman2. The wind data set is based on spatio-temporal time series data from the National Renewable Energy Laboratory (NREL) western wind data set. The whole data set comprises time series of 32,043 wind turbines, each holding ten $3$ MW turbines over a timespan of three years in a $10$-minute resolution. The dimensionality is $d=22$. Fitness is data set based on an optimization run of a (15+100)-ES [4] on the Sphere function $f(\mathbf {z}) = \mathbf {z}^T \mathbf {z}$ with $d=20$ dimensions and 21000 fitness function evaluations. The patterns are the objective variable values of the best candidate solution in each generation, the labels are the fitness function values in each generation. The data set Photos contains thirty JPEG photos with resolution $320 \times 214$ taken with a SONY DSLR-A300. The Iris data sets consists of 150 4-dimensional patterns of three different types of irises.