As a versatile technique, cloud-fog computing extends the traditional cloud server to offer various on-demand data services. Maintaining data confidentiality is one of the most crucial requirements for data services, many cryptosystems have been proposed to reserve information privacy against such an untrusted environment. However, in cloud-fog computing, how to confidentially and efficiently share data and fetch desirable data without expensive data decryption for resource-constrained end-devices is challenging. In this paper, we propose a cloud-fog system for the Internet-of-Things (IoT) ecosystem by introducing a cryptographic primitive called server-aided revocable bilateral attribute-based encryption (SRB-ABE). Our solution is a secure and lightweight bilateral access control system with dynamic user groups, including (1) fine-grained data user and data owner access control simultaneously; (2) outsourced data source identification; (3) server-aided user revocation with publicly updatable ciphertexts; and (4) lightweight data decryption mechanism with one exponentiation computation. We present the formal definition and concrete construction of SRB-ABE with security proofs to build cloud-fog systems. The extensive comparison and experimental analysis demonstrate that our construction has superior functionality and comparable performance than the most relevant solutions.