The cloud-based Internet-of-Things (IoT) has been applied to support ubiquitous data collection and centralized data processing among various applications. Equipped with powerful resources, a semi-trusted cloud is able to deduce private information by launching inference attack. Homomorphic Encryption (HE) has been proposed as an effective way to preserve privacy from inference attack while allowing certain computation over ciphertext. However, HE leads to longer latency due to additional communication and computation overheads. In this paper, we propose an optimization framework in privacy-preserving access control under cloud-fog computing systems. The optimization goal is to maximize the average user satisfaction in the system, where cost and latency serve as key metrics measuring user satisfaction. Due to the NP-hardness of the formulated problem, we propose a low-complexity suboptimal algorithm to solve it, where the access offloading decision making, user cooperation, and resource allocation are considered. Simulation results are presented to show the advantages of our proposed algorithm in terms of the average USI (User Satisfaction Index) and the number of users with zero USI.