skip to main content
10.1145/2582112.2582123acmotherconferencesArticle/Chapter ViewAbstractPublication PagessocgConference Proceedingsconference-collections
tutorial

Using Convex Relaxations for Efficiently and Privately Releasing Marginals

Published: 08 June 2014 Publication History

Abstract

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar [26], focusing on the problem of marginal release. Here, a database is a collection of the data of n individuals, each characterized by d binary attributes. A k-way marginal query is specified by a subset S of k attributes, together with a |S|-dimensional binary vector β specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to S agrees with β.
Information theoretically, the error complexity of marginal queries -- how "wrong" do the answers have to be in order to preserve differential privacy -- is well-understood: the perquery additive error is known to be at least Ω(min{√n,dk/2}) and at most Õ(√nd1/4,dk/2). However, no polynomial time algorithm with error complexity as low as the information theoretic upper bound is known for small n.
We present a polynomial time algorithm that matches the best known information-theoretic bounds when k = 2; more generally, by reducing to the case k = 2, for any distribution on marginal queries, our algorithm achieves average error at most Õ(√nd[k/2]/4), an improvement over previous work on when k is small and when error o(n) is desirable. Using private boosting we are also able to give nearly matching worst-case error bounds.
Our algorithms are based on the geometric techniques of Nikolov, Talwar, and Zhang [29], wherein a vector of "sufficiently noisy" answers is projected onto a particular convex body. We reduce projection, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body K, find a containing convex body L that one can efficiently optimize over, while keeping the Gaussian width of L small. This reduction is achieved by a careful use of the Frank-Wolfe algorithm.

References

[1]
N. Alon and A. Naor. Approximating the cut-norm via Grothendieck's inequality. In ACM Symposium on Theory of Computing, pages 72--80, 2004.
[2]
B. Barak, K. Chaudhuri, C. Dwork, S. Kale, F. McSherry, and K. Talwar. Privacy, accuracy, and consistency too: a holistic solution to contingency table release. In L. Libkin, editor, Proceedings of ACM PODS, pages 273--282. ACM, 2007.
[3]
A. Blum, K. Ligett, and A. Roth. A learning theory approach to non-interactive database privacy. In STOC '08: Proceedings of the 40th annual ACM symposium on Theory of computing, pages 609--618, New York, NY, USA, 2008. ACM.
[4]
M. Bun, J. Ullman, and S. Vadhan. Fingerprinting codes and the price of approximate differential privacy. arXiv preprint arXiv:1311.3158, 2013.
[5]
S. R. Buss, D. Grigoriev, R. Impagliazzo, and T. Pitassi. Linear gaps between degrees for the polynomial calculus modulo distinct primes. J. Comput. Syst. Sci., 62(2):267--289, 2001.
[6]
K. Chandrasekaran, J. Thaler, J. Ullman, and A. Wan. Faster private release of marginals on small databases. CoRR, abs/1304.3754, 2013.
[7]
M. Cheraghchi, A. Klivans, P. Kothari, and H. K. Lee. Submodular functions are noise stable. In SODA, pages 1586--1592, 2012.
[8]
K. Clarkson. Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Transactions on Algorithms (TALG), 6(4):63, 2010.
[9]
S. Dasgupta and A. Gupta. An elementary proof of a theorem of Johnson and Lindenstrauss. Random Structures and Algorithms, 22:60--65, 2003.
[10]
I. Dinur and K. Nissim. Revealing information while preserving privacy. In Proc. 22nd ACM Symposium on Principles of Database Systems, pages 202--210, 2003.
[11]
D. P. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, 2009.
[12]
C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation, 2006.
[13]
C. Dwork, F. Mcsherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006.
[14]
C. Dwork, M. Naor, O. Reingold, G. N. Rothblum, and S. Vadhan. On the complexity of differentially private data release: efficient algorithms and hardness results. In Proceedings of the 41st ACM Symposium on Theory of Computing, 2009.
[15]
C. Dwork and K. Nissim. Privacy-preserving datamining on vertically partitioned databases. In Advances in Cryptology -- CRYPTO'04, pages 528--544, 2004.
[16]
C. Dwork and A. Roth. The Algorithmic Foundations of Differential Privacy. 2014.
[17]
C. Dwork, G. N. Rothblum, and S. Vadhan. Boosting and differential privacy. In Foundations of Computer Science (FOCS), 2010 51st Annual IEEE Symposium on, pages 51--60. IEEE, 2010.
[18]
M. Frank and P. Wolfe. An algorithm for quadratic programming. Naval research logistics quarterly, 3(1-2):95--110, 1956.
[19]
D. Grigoriev. Linear lower bound on degrees of positivstellensatz calculus proofs for the parity. Theor. Comput. Sci., 259(1-2):613--622, 2001.
[20]
A. Grothendieck. Résumé de la théorie métrique des produits tensoriels topologiques. Bol. Soc. Mat. Sao Paulo, 8(1-79):88, 1953.
[21]
M. Grötschel, L. Lovász, and A. Schrijver. The ellipsoid method and its consequences in combinatorial optimization. Combinatorica, 1(2):169--197, 1981.
[22]
A. Gupta, M. Hardt, A. Roth, and J. Ullman. Privately releasing conjunctions and the statistical query barrier. In STOC, pages 803--812, 2011.
[23]
M. Hardt, K. Ligett, and F. McSherry. A simple and practical algorithm for differentially private data release. In NIPS, 2012. To appear.
[24]
M. Hardt and G. Rothblum. A multiplicative weights mechanism for privacy-preserving data analysis. Proc. 51st Foundations of Computer Science (FOCS). IEEE, 2010.
[25]
M. Hardt, G. N. Rothblum, and R. A. Servedio. Private data release via learning thresholds. In Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '12, pages 168--187. SIAM, 2012.
[26]
M. Hardt and K. Talwar. On the geometry of differential privacy. In Proceedings of the 42nd ACM symposium on Theory of computing, STOC '10, pages 705--714, New York, NY, USA, 2010. ACM.
[27]
S. Kasiviswanathan, M. Rudelson, A. Smith, and J. Ullman. The price of privately releasing contingency tables and the spectra of random matrices with correlated rows. In Proceedings of the 42nd ACM symposium on Theory of computing, pages 775--784. ACM, 2010.
[28]
J. Lindenstrauss and A. Pełczyński. Absolutely summing operators in --{p}-spaces and their applications. Studia Mathematica, 29(3):275--326, 1968.
[29]
A. Nikolov, K. Talwar, and L. Zhang. The geometry of differential privacy: the sparse and approximate cases. In Proceedings of the 45th annual ACM symposium on Symposium on theory of computing, STOC '13, pages 351--360, New York, NY, USA, 2013. ACM.
[30]
R. O'Donnell and Y. Zhou. Approximability and proof complexity. In SODA, pages 1537--1556, 2013.
[31]
A. Roth and T. Roughgarden. Interactive privacy via the median mechanism. In Proceedings of the 42nd ACM symposium on Theory of computing, STOC '10, pages 765--774, New York, NY, USA, 2010. ACM.
[32]
J. Thaler, J. Ullman, and S. P. Vadhan. Faster algorithms for privately releasing marginals. In ICALP (1), pages 810--821, 2012.
[33]
J. Ullman and S. Vadhan. Pcps and the hardness of generating private synthetic data. In Proceedings of the 8th conference on Theory of Cryptography, 2011.

Cited By

View all
  • (2020) Structure and Sensitivity in Differential Privacy: Comparing K -Norm Mechanisms Journal of the American Statistical Association10.1080/01621459.2020.1773831116:534(935-954)Online publication date: 20-Jul-2020
  • (2020)PCPs and the Hardness of Generating Synthetic DataJournal of Cryptology10.1007/s00145-020-09363-yOnline publication date: 31-Jul-2020
  • (2019)Towards instance-optimal private query releaseProceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3310435.3310587(2480-2497)Online publication date: 6-Jan-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SOCG'14: Proceedings of the thirtieth annual symposium on Computational geometry
June 2014
588 pages
ISBN:9781450325943
DOI:10.1145/2582112
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. convex relaxation
  2. differential privacy
  3. marginals

Qualifiers

  • Tutorial
  • Research
  • Refereed limited

Conference

SOCG'14

Acceptance Rates

SOCG'14 Paper Acceptance Rate 60 of 175 submissions, 34%;
Overall Acceptance Rate 625 of 1,685 submissions, 37%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020) Structure and Sensitivity in Differential Privacy: Comparing K -Norm Mechanisms Journal of the American Statistical Association10.1080/01621459.2020.1773831116:534(935-954)Online publication date: 20-Jul-2020
  • (2020)PCPs and the Hardness of Generating Synthetic DataJournal of Cryptology10.1007/s00145-020-09363-yOnline publication date: 31-Jul-2020
  • (2019)Towards instance-optimal private query releaseProceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3310435.3310587(2480-2497)Online publication date: 6-Jan-2019
  • (2018)Semantic Security for Sharing Computing Knowledge/InformationCyber Security10.1007/978-981-10-8536-9_45(475-481)Online publication date: 28-Apr-2018
  • (2018)Hardness of Non-interactive Differential Privacy from One-Way FunctionsAdvances in Cryptology – CRYPTO 201810.1007/978-3-319-96884-1_15(437-466)Online publication date: 25-Jul-2018
  • (2016)The Geometry of Differential Privacy: The Small Database and Approximate CasesSIAM Journal on Computing10.1137/13093894345:2(575-616)Online publication date: Jan-2016
  • (2016)Order-Revealing Encryption and the Hardness of Private LearningProceedings, Part I, of the 13th International Conference on Theory of Cryptography - Volume 956210.1007/978-3-662-49096-9_8(176-206)Online publication date: 10-Jan-2016
  • (2015)An Improved Private Mechanism for Small DatabasesAutomata, Languages, and Programming10.1007/978-3-662-47672-7_82(1010-1021)Online publication date: 20-Jun-2015
  • (2015)Related WorkProtecting Privacy in Data Release10.1007/978-3-319-16109-9_2(11-33)Online publication date: 2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media