Expectation maximization (EM) is a clustering-based machine learning algorithm that is widely used in many areas of science (e.g., bioinformatics and computer vision) to find maximum likelihood and maximum a posteriori estimates for models with latent variables. To deploy such an algorithm in cloud environments, security and privacy issues need be considered to avoid data breaches or abuses by external malicious parties or even by cloud service providers. However, the processing performance of the EM algorithm poses a challenge in terms of building a secure environment. This article describes an innovative and practical privacy-preserving EM algorithm for cloud systems that addresses this challenge, and estimates the EM parameters in an accurate and secure manner. Fully homomorphic encryption (FHE) is used to ensure the privacy of both the EM algorithm computations and the users' sensitive data in the cloud. A distributed-based approach is also proposed to overcome the overheads of FHE computations and ensure a fast convergence of the EM algorithm. The conducted experiments demonstrate a significant improvement in the convergence time of the distributed EM algorithm, while achieving a high level of accuracy and reducing the associated computational FHE overheads.