In conventional algorithms a data point can be assigned to only a single cluster. However, in reality there are several types of data that a data point belongs to multiple categories and causes ground-truth clusters overlap. In this circumstance, conventional clustering cannot work effectively. To handle this problem, several algorithms are proposed and referred as “overlapping clustering”. One of state-of-the-art partition-based overlapping clustering technique is “Non-exhaustive, Overlapping K-Means” or “NEO-K-Means” in short, which is an extension of K-Means clustering algorithm. Although NEO-K-Means works effectively for most real-world multi-category data, however, distance between clusters that is essential parameter for overlapping clustering is not included in the process of algorithm. This is a huge drawback of NEO-K-Means that makes clustering accuracy lower than it should be. In this paper, we aim to overcome this limitation in NEO-K-Means with our assumption that closer clusters should have higher probability to overlap than others that are faraway. To achieve the goal, the parameter that represents distances between clusters centroids is included in objective function before prioritizing and assigning data points to the overlapped regions of clusters. The experimental results show that our method significantly outperforms NEO-K-Means on all nine real multi-category data sets in terms of F₁.