EM Algorithm with PIP Initialization and Temperature-Based Selection

Abstract

The EM algorithm is an efficient algorithm to obtain the ML estimate for incomplete data, but has the local optimality problem. The deterministic annealing EM (DAEM) algorithm was once proposed to solve this problem, which begins a search from the primitive initial point (PIP). Then, proposed was the mes-EM algorithm, which runs the EM repeatedly in many various directions from the PIP, and achieves good solution quality with high computing cost. This paper proposes a variant of the mes-EM, called mes-EM(β), which uses the temperature to select a small promising portion of the mes-EM runs. Our experiments for the Gaussian mixture estimation showed the proposed algorithm was much faster than the original mes-EM without degrading its solution quality.