Accuracy-Based Classification EM: Combining Clustering with Prediction

Abstract

Expectation-Maximization (EM) is well-known for its use in clustering. During operation, EM makes a “soft” assignment of each row to multiple clusters in proportion to the likelihood of each cluster. Classification EM (CEM) is a variant of EM that makes a “hard” assignment of each row to its most likely class. This paper presents a variant of CEM, which we call Accuracy-Based CEM (ABCEM), where the goal is prediction rather clustering. ABCEM first assigns each row to the most likely class based on the input columns, and then estimates performance of this assignment by evaluating the mean squared prediction error (MSPE) on the output columns, and proceeds as in CEM to update clusters and re-assign each row to the new clusters. Finally, the optimal clustering is selected to minimize the MSPE, selecting a local optimum from the left, and thus the procedure can also be viewed as a principled version of early stopping which uses only the training set. Our results show that ABCEM is nearly 40% more accurate than CEM.