Differentially private data fusion and deep learning Framework for Cyber–Physical–Social Systems: State-of-the-art and perspectives

Highlights

• We review the design requirements of differential privacy for CPSS applications.

• We review the data fusion in CPSS, differentially private deep learning in CPSS.

• We propose Differentially Private Data Fusion and Deep Learning Framework for CPSS.

• We discuss the future research directions for CPSSs.

Abstract

The modern technological advancement influences the growth of the cyber–physical system and cyber–social system to a more advanced computing system cyber–physical–social system (CPSS). Therefore, CPSS leads the data science revolution by promoting tri-space information resource from a single space. The establishment of CPSSs increases the related privacy concerns. To provide privacy on CPSSs data, various privacy-preserving schemes have been introduced in the recent past. However, technological advancement in CPSSs requires the modifications of previous techniques to suit its dynamics. Meanwhile, differential privacy has emerged as an effective method to safeguard CPSSs data privacy. To completely comprehend the state-of-the-art developments and learn the field’s research directions, this article provides a comprehensive review of differentially private data fusion and deep learning in CPSSs. Additionally, we present a novel differentially private data fusion and deep learning Framework for Cyber–Physical–Social Systems , and various future research directions for CPSSs.

Introduction

Cyber–Physical–Social Systems(CPSS), as a computing system originates from the technological growth of the CPS (Cyber–Physical System) and CSS (Cyber–Social System) and provides a new platform of growth for pervasive computing [1]. Certainly, CPS provides a mapping of physical space objects into cyber space, whereas CSS provides a mapping of social space objects into cyber space [2]. Additionally, social behaviors and interactions are encompassed into the CPS because of the ever-increasing human-centric computing [3], [4]. With the technological advancement in information, computing and communication, CPSS has proved to be among the fundamental shifts to deliver personalized and proactive human being services through meeting peoples social interaction demands and their reaction to the physical environment [3], [5]. Therefore, helping to offer people from various communities improved customized services. Additional introductory details related to CPSS concepts can be referred in [6], [7].

The ever-increasing popularity of CPSS implementation leads to the generation of a large scale of heterogeneous data from numerous sources, such as user response, sensor observation, social network, and measuring instrument. Data partaking and association proves to be a pivotal component to sustain the quality of services in a pervasive environment like smart homes and societies [8]. Thus, through data fusion, CPSS provides full integration of CPS and CSS capabilities by fusing numerous data from physical space, cyber space, and social space to enhances the quality of data henceforth promoting quality decision making [3].

Typically, the usual data fusion processes fuse data originating from physical space. However, CPSS data exists in three spaces (i.e, cyber, physical and social spaces), which makes it a challenging task to merge the data. Precisely, all facets of data in social networks should be taken into account, which implies data fusion in CPSS needs algorithms and data processing techniques that are sophisticated [1], [6], [7], [9]. In this perspective, a human in a data fusion based system becomes both a data provider as well as a data consumer, which provides valued inferences that are missing from the customary physical sensors [3], [10].

Deep learning represents a promising method for precise mining of information in CPSS. The last few years have seen the significant achievement of deep learning methods in numerous machine learning/data mining tasks ranging from data analytics, and autonomous systems to signal and information processing [16]. The success greatly depends on the substantial collection of CPSS data making deep learning widely accepted in healthcare applications such as phenotype extortion, critical disease development prediction, and others [17], [18]. Therefore, there is an obvious danger posed to the privacy of deep learning models made on users sensitive data, like healthcare records. For instance, [19] proves that the private information of an individual in the training dataset can be recuperated by repetitively querying the output probabilities of a classifier for disease recognition built upon a CNN. Existing privacy worries discourage users from partaking their data and thereby hinder the future growth of deep learning itself.

However, the architectural complexity of CPSS makes the assessment of privacy threats challenging, and new privacy concerns arise [20]. Moreover, CPSSs depends on numerous data centers and sensors comprising of a vast amount of private and personal data. For instance, wearable patient devices are reporting real-time data continuously to doctors for consultation [21]. However, without proper privacy preservation approach, individual privacy of CPSS users may be compromised [22].

Attacks on CPSS can be categorized as security-based (active) and privacy based (passive). Passive attacks have the objective of accessing private and individual data being shared from the public datasets [23], [24]. There exist researches that propose various cryptographic methods to protect the privacy of data [11]. However, cryptographic methods are naturally computationally expensive. Furthermore, in scenarios requiring public sharing of data, it becomes more challenging to guarantee privacy [12]. Likewise, anonymization methods like k-anonymity [13] are being proposed to tackle privacy concerns. However, anonymization methods fail to ensure the required level of privacy [25]. Especially as the number of attributes datasets increases, they become vulnerable to re-identification [14].

As a concrete privacy notion, differential privacy [15] is considered as the future of privacy. It is a promising scheme for preserving privacy in deep learning and data fusion. Differential privacy safeguards real-time or statistical data by inserting the appropriate quantity of noise while retaining the healthy trade-off between accuracy and privacy. Especially, differential privacy offers individuals an attestable privacy assurance, which is benefiting from a concrete theoretical foundation compared to other privacy-preserving schemes [14], [26], [27].

Additionally, through the adjustment of privacy budget value, differential privacy can compromise gracefully between utility and privacy, where larger privacy budgets imply weak privacy guarantee offered. Furthermore, it ensures data owners that adversaries are not capable of inferring any information related to a single record confidently from the released deep learning resulting models, even if all the remaining records in the dataset are known to the adversary [28].

Differential privacy proves to be capable of preserving privacy of a large quantity of data from real-time data and databases [29]. It realizes privacy-preservation through the addition of a carefully calibrated quantity of noise to the output results or model according to the robust mechanisms rather than merely individual data anonymization. This is completely achieved due to differential privacy capability to provide efficient and effectual schemes solving privacy preservation problems by using the basic methods like functional mechanism [30], exponential mechanism [31], Gaussian mechanism [32] and Laplace mechanism [11]. These mechanisms satisfy the need for privacy preservation by combining differential privacy strength for a wide range of non-private deep learning prototypes (see Table 1).

However, there is very little research on differential privacy deep learning and data fusion for CPSSs. Dwork [33], Ji [34] has their focus on a few basic differential privacy based machine learning and they do not focus on differentially privacy data fusion and deep learning. Therefore, at present, the community lacks a comprehensive synopsis of state-of-the-art methods on differential privacy-preserving deep learning and data fusion for CPSSs. Inspired by this, we devise a comprehensive survey on differential privacy for deep learning and data fusion on CPSS. Specifically, the survey offers the following major contribution.

• We propose a novel differentially private data fusion and deep learning framework for CPSS.

• We offer a thorough and comprehensive analysis of the state-of-the-art techniques. Consequently, the survey put forward new perspectives for clear understanding of the existing works and thus, improve the privacy levels of deep learning and data fusion techniques.

• To enable timely and impending research in the area of privacy-preserving data fusion and learning, we suggest various promising future research directions.

We organize this monograph as follows. Section 2 provides a brief introduction of the key CPSS concepts, including system architectures, applications, and the motives to use differential privacy in CPSS. Section 3 surveys key concepts on differential privacy, including privacy attacks on deep learning, privacy budget, and various differential privacy mechanisms. Section 4 reviews the data fusion in CPSS by focusing on CPSS data requirements for data fusion, data fusion techniques, and differential privacy data fusion. Section 5 describes differential privacy deep learning for CPSS. We propose CPSS data fusion and deep learning framework design and provide the future work and challenges of CPSS data fusion and deep learning in Section 6. Conclusions are included in Section 7.