Skip to main content
SpringerLink
Log in

A Framework for Privacy Quantification: Measuring the Impact of Privacy Techniques Through Mutual Information, Distance Mapping, and Machine Learning

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

In this paper, we propose to investigate how the effects of privacy techniques can be practically assessed in the specific context of data anonymization, and present some possible tools for measuring the effects of such anonymization. We develop an approach using mutual information for measuring the information content in any dataset, including over non-Euclidean data spaces, by means of mapping non-Euclidean distances to a Euclidean space. We further evaluate the proposed approach over toy datasets composed of timestamped GPS traces, and attempt to quantify the information content loss created by three state-of-the-art anonymization approaches. The results allow for an objective quantification of the effects of the k-anonymity and differential privacy algorithms, and illustrate on the toy data used, that such privacy techniques have very non-linear effects on the information content of the data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abramowitz M. Handbook of mathematical functions, with formulas, graphs, and mathematical tables. New York: Dover Publications; 1974.

    Google Scholar 

  2. Asgarian E, Kahani M, Sharifi S. The impact of sentiment features on the sentiment polarity classification in Persian reviews. Cogn Comput 2018;10(1):117–35. 00001.

    Article  Google Scholar 

  3. Auer P, Burgsteiner H, Maass W. A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Netw 2008;21(5):786–95.

    Article  PubMed  Google Scholar 

  4. Belghazi M I, Baratin A, Rajeswar S, Ozair S, Bengio Y, Courville A, Hjelm RD. 2018. MINE: mutual information neural estimation. arXiv:1801.04062 [cs, stat]. 00003.

  5. Cambria E, Huang G-B, Kasun L L C, Zhou H, Vong C M, Lin J, Yin J, Cai Z, Liu Q, Li K, et al. Extreme learning machines [trends & controversies]. IEEE Intell Syst 2013;28(6):30–59.

    Article  Google Scholar 

  6. European Commission. 2012. European Commission’s press release announcing the proposed comprehensive reform of data protection rules, 25 January.

  7. Cover TM, Thomas JA. Elements of information theory (Wiley series in telecommunications and signal processing). New York: Wiley-Interscience; 2006.

    Google Scholar 

  8. Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signals Syst (MCSS) 1989; 2(4):303–14.

    Article  Google Scholar 

  9. Dashtipour K, Poria S, Hussain A, Cambria E, Hawalah AY A, Gelbukh A, Zhou Q. Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cogn Comput 2016;8(4): 757–71. 00025.

    Article  Google Scholar 

  10. Ding S, Zhao H, Zhang Y, Xu X, Nie R. Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 2015;44(1):103–15.

    Article  Google Scholar 

  11. Dwork C. Differential privacy. Berlin: Springer; 2006, pp. 1–12.

    Google Scholar 

  12. Dwork C. Differential privacy: a survey of results. Theory and applications of models of computation, volume 4978 of Lecture Notes in Computer Science. Berlin: Springer; 2008. p. 1–19.

  13. Dwork C, McSherry F, Nissim K, Smith A. Calibrating noise to sensitivity in private data analysis. Berlin: Springer; 2006, pp. 265–84.

    Google Scholar 

  14. EU. 2000. 2000/520/EC: Commission Decision of 26 July 2000 pursuant to Directive 95/46/EC of the European Parliament and of the Council on the adequacy of the protection provided by the safe harbour privacy principles and related frequently asked questions issued by the US Department of Commerce (notified under document number C(2000) 2441) (Text with EEA relevance.)

  15. François D. 2008. High-dimensional data analysis: optimal metrics and feature selection. VDM Verlag, 01.

  16. Goss R N. Information theory with applications (silviu guiaşu). SIAM Rev 1979;21(4):579–80.

    Article  Google Scholar 

  17. Hafiz M. A collection of privacy design patterns. Proceedings of the 2006 conference on pattern languages of programs, PLoP ’06. New York: ACM; 2006. p. 7:1–13.

  18. Holmes C, Nemenman I. Progress in estimation of mutual information for real-valued data. Bulletin of the American Physical Society; 2018.

  19. Huang G-B, Chen L, Siew C K, et al. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 2006;17(4):879–92.

    Article  PubMed  Google Scholar 

  20. Huang G-B, Zhu Q-Y, Siew C-K. Extreme learning machine: theory and applications. Neurocomputing 2006;70(1):489–501.

    Article  Google Scholar 

  21. The Information Commissioner’s Office (UK). Direct marketing: data protection act privacy and electronic communications regulations, 24 November 2013. Version 1.1.

  22. Kraskov A, Stögbauer H, Grassberger P. Estimating mutual information. Phys Rev E 2004;69(6): 066138.

    Article  CAS  Google Scholar 

  23. Lauren P, Qu G, Yang J, Watta P, Huang G-B, Lendasse A. 2018. Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cognit Comput. 1–14. 00000.

  24. Li N, Li T. t-closeness: privacy beyond κ-anonymity and ℓ-diversity. Proceedings of IEEE 23rd international conference on data engineering (ICDE’07); 2007.

  25. Ma Y, Peng H, Khan T, Cambria E, Hussain A. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cognit Comput. 2018;10(4):639–650. https://doi.org/10.1007/s12559-018-9549-x.

    Article  Google Scholar 

  26. Machanavajjhala A, Gehrke J, Kifer D, Venkitasubramaniam M. ℓ-diversity: privacy beyond κ-anonymity. 2013 IEEE 29th international conference on data engineering (ICDE); 2006. p. 24.

  27. Mahmud M, Kaiser M S, Hussain A, Vassanelli S. 2017. Applications of deep learning and reinforcement learning to biological data. IEEE Trans Neural Netw Learn Syst. PP. 00004.

  28. Miche Y, Oliver I, Holtmanns S, Akusok A, Lendasse A, Björk K-M. On mutual information over non-Euclidean Spaces, data mining and data privacy levels. Cham: Springer International Publishing; 2016, pp. 371–83.

    Google Scholar 

  29. Miche Y, Oliver I, Holtmanns S, Kalliola A, Akusok A, Lendasse A, Björk K-M. Data anonymization as a vector quantization problem: control over privacy for health data. Availability, reliability, and security in information systems, Lecture Notes in Computer Science. Cham: Springer; 2016. p. 193– 203.

  30. Miche Y, Oliver I, Ren W, Holtmanns S, Akusok A, Lendasse A. Practical estimation of mutual information on non-Euclidean spaces. Machine learning and knowledge extraction. Cham: Springer; 2017. p. 123–36.

  31. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A. Op-elm: optimally pruned extreme learning machine. IEEE Trans Neural Netw 2010;21(1):158–62.

    Article  PubMed  Google Scholar 

  32. Miche Y, Van Heeswijk M, Bas P, Simula O, Lendasse A. Trop-elm: a double-regularized elm using lars and tikhonov regularization. Neurocomputing 2011;74(16):2413–21.

    Article  Google Scholar 

  33. Molina D, LaTorre A, Herrera F. An insight into bio-inspired and evolutionary algorithms for global optimization: review, analysis, and lessons learnt over a decade of competitions. Cognit Comput 2018;10(4):517–544. https://doi.org/10.1007/s12559-018-9554-0.

    Article  Google Scholar 

  34. Nissenbaum H. A contextual approach to privacy online. Daedalus 2011;140(4):32–48.

    Article  Google Scholar 

  35. Oliver I. Privacy engineering: a data flow and ontological approach. CreateSpace Independent Publishing, July 2014. 978-1497569713.

  36. Pál D, Póczos B, Szepesvári C. Estimation of rényi entropy and mutual information based on generalized nearest-neighbor graph. Advances in neural information processing systems; 2010. p. 1849–57.

  37. Rao C R, Mitra S K. 1971. Generalized inverse of matrices and its applications.

  38. Reed J, Pierce BC. Distance makes the types grow stronger: a calculus for differential privacy. ACM SIGPLAN international conference on functional programming (ICFP), Baltimore; 2010.

  39. Savitha R, Suresh S, Kim H J. A meta-cognitive learning algorithm for an extreme learning machine classifier. Cogn Comput 2014;6(2):253–63. 00048.

    Article  Google Scholar 

  40. Schneier B. Architecture of privacy. IEEE Secur Priv 2009;7(1):88.

    Article  Google Scholar 

  41. Singh P K. Similar vague concepts selection using their euclidean distance at different granulation. Cogn Comput 2018;10(2):228–41. 00001.

    Article  Google Scholar 

  42. Sweeney L. κ-anonymity: a model for protecting privacy. Int J Uncertain Fuzziness Knowl-Based Syst 2002;10(5):557–70.

    Article  Google Scholar 

  43. Ustaran E, editor. European Privacy: Law and Practice for Data Protection Professionals. An IAPP Publication, 2012. 978-0-9795901-5-3.

  44. Van Heeswijk M, Miche Y, Oja E, Lendasse A. Gpu-accelerated and parallelized elm ensembles for large-scale regression. Neurocomputing 2011;74(16):2430–7.

    Article  Google Scholar 

  45. Wang H, Zhang Y, Waytowich N R, Krusienski D J, Zhou G, Jin J, Wang X, Cichocki A. Discriminative feature extraction via multivariate linear regression for SSVEP-based BCI. IEEE Trans Neural Syst Rehabil Eng 2016;24(5):532–41.

    Article  CAS  PubMed  Google Scholar 

  46. Wang R, Zhang Y, Zhang L. An adaptive neural network approach for operator functional state prediction using psychophysiological data. Integrated Computer Aided Eng 2015;23:81–97. 00006.

    Article  Google Scholar 

  47. Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cognit Comput 2018;10(2):359–67. 00001.

    Article  Google Scholar 

  48. Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. Int J Neural Syst. 2017;27(02):1650032. https://doi.org/10.1142/S0129065716500325.

    Article  PubMed  Google Scholar 

  49. Zhang Y, Zhou G, Jin J, Zhao Q, Wang X, Cichocki A. Sparse Bayesian classification of EEG for brain-computer interface. IEEE Trans Neural Netw Learn Syst 2015;27:1–1. 00058.

    Google Scholar 

Download references

Funding

This work was supported by the research from SCOTT project. SCOTT (http://www.scott-project.eu) has received funding from the Electronic Component Systems for European Leadership Joint Undertaking under grant agreement no. 737422. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and Austria, Spain, Finland, Ireland, Sweden, Germany, Poland, Portugal, Netherlands, Belgium, and Norway.

Author information

Authors and Affiliations

  1. Nokia Bell Labs, Espoo, Finland

    Yoan Miche, Wei Ren, Ian Oliver & Silke Holtmanns

  2. Information and Logistics Technology Department, College of Technology, University of Houston, Houston, USA

    Amaury Lendasse

Authors
  1. Yoan Miche
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ian Oliver
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Silke Holtmanns
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amaury Lendasse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoan Miche.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miche, Y., Ren, W., Oliver, I. et al. A Framework for Privacy Quantification: Measuring the Impact of Privacy Techniques Through Mutual Information, Distance Mapping, and Machine Learning. Cogn Comput 11, 241–261 (2019). https://doi.org/10.1007/s12559-018-9604-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-018-9604-7

Keywords

Search

Navigation