Analysis of k-Means++ for Separable Data

Abstract

k-means++ [5] seeding procedure is a simple sampling based algorithm that is used to quickly find k centers which may then be used to start the Lloyd’s method. There has been some progress recently on understanding this sampling algorithm. Ostrovsky et al. [10] showed that if the data satisfies the separation condition that $\frac{\Delta_{k-1}(P)}{\Delta_k(P)} \geq c$ (Δ _i (P) is the optimal cost w.r.t. i centers, c > 1 is a constant, and P is the point set), then the sampling algorithm gives an O(1)-approximation for the k-means problem with probability that is exponentially small in k. Here, the distance measure is the squared Euclidean distance. Ackermann and Blömer [2] showed the same result when the distance measure is any μ-similar Bregman divergence. Arthur and Vassilvitskii [5] showed that the k-means++ seeding gives an O(logk) approximation in expectation for the k-means problem. They also give an instance where k-means++ seeding gives Ω(logk) approximation in expectation. However, it was unresolved whether the seeding procedure gives an O(1) approximation with probability $\Omega \left(\frac{1}{poly(k)}\right)$, even when the data satisfies the above-mentioned separation condition. Brunsch and Röglin [8] addressed this question and gave an instances on which k-means++ achieves an approximation ratio of (2/3 − ε) ·logk only with exponentially small probability. However, the instances that they give satisfy $\frac{\Delta_{k-1}(P)}{\Delta_k(P)} = 1+o(1)$. In this work, we show that the sampling algorithm gives an O(1) approximation with probability $\Omega\left(\frac{1}{k}\right)$ for any k-means problem instance where the point set satisfy separation condition $\frac{\Delta_{k-1}(P)}{\Delta_k(P)} \geq 1 + \gamma$, for some fixed constant γ. Our results hold for any distance measure that is a metric in an approximate sense. For point sets that do not satisfy the above separation condition, we show O(1) approximation with probability Ω(2^− 2k).