Rough K-means algorithm and its extensions, such as Rough K-means Clustering Algorithm with Weight Based on Density have been useful in situations where clusters do not necessarily have crisp boundaries. Nevertheless, there are flaws of selecting the weight of upper and lower approximation, defining the density of samples and searching the center in the Rough K-means Clustering Algorithm with Weight Based on Density. Aiming at the flaws, this paper proposes a solution to search initial central points and combines it with a distance measure with weight which is based on attribute reduction of rough set to achieve the algorithm. This improved algorithm decreases the level of interference brought by the isolated points to the k-means algorithm, and makes the clustering analysis more effective and objective. This experiment was performed by testing the true data sets. The results showed that the improved algorithm is effective, especially to those data sets with huge redundance.