Skip to main content
SpringerLink
Log in

Common Information and Unique Disjointness

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

We provide an information-theoretic framework for establishing strong lower bounds on the nonnegative rank of matrices by means of common information, a notion previously introduced in Wyner (IEEE Trans Inf Theory 21(2):163–179, 1975). The framework is a generalization of the one in Braverman and Moitra (Proceedings of the forty-fifth annual ACM symposium on theory of computing, pp 161–170, 2013) for the shifted uniqe disjointness (UDISJ) matrix to arbitrary nonnegative matrices. Common information is a natural lower bound for the nonnegative rank of a matrix and by combining it with Hellinger distance estimations we compute the (almost) exact common information of UDISJ partial matrix. The bounds are obtained very naturally. We also establish robustness of this estimation under random or adversarial removal of rows and columns of the UDISJ partial matrix. This robustness translates, via a variant of Yannakakis’s factorization theorem, to lower bounds on the average case and adversarial approximate extension complexity of removals. We present the first family of polytopes, the hard pair introduced in Braun et al. (Math Oper Res 40(3):756–772, 2015) related to the CLIQUE problem, with high average case and adversarial approximate extension complexity of removals. The framework relies on a strengthened version of the link between information theory and Hellinger distance from Bar-Yossef (J Comput Syst Sci 68(4):702–732, 2004). We also provide an information theoretic variant of the fooling set method that allows us to extend fooling set lower bounds from extension complexity to approximate extension complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Arora, S., Ge, R., Kannan, R., Moitra, A.: Computing a nonnegative matrix factorization-provably. In: Proceedings of the 44th Symposium on Theory of Computing, pp. 145–162. ACM (2012)

  2. Avis, D., Tiwary, H.R.: On the extension complexity of combinatorial polytopes. Math. Program. Ser. B 153(1), 95–115 (2015). doi:10.1007/s10107-014-0764-2

  3. Bar-Yossef, Z., Jayram, T., Kumar, R., Sivakumar, D.: An information statistics approach to data stream and communication complexity. J. Comput. Syst. Sci. 68(4), 702–732 (2004). doi:10.1016/j.jcss.2003.11.006

  4. Ben-Tal, A., Nemirovski, A.: On polyhedral approximations of the second-order cone. Math. Oper. Res. 26, 193–205 (2001). doi:10.1287/moor.26.2.193.10561

    Article  MathSciNet  MATH  Google Scholar 

  5. Braun, G., Pokutta, S.: The matching polytope does not admit fully-polynomial size relaxation schemes. In: Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 837–846 (2015)

  6. Braun, G., Fiorini, S., Pokutta, S., Steurer, D.: Approximation limits of linear programs (beyond hierarchies). Math. Oper. Res. 40(3), 756–772 (2015). doi:10.1287/moor.2014.0694

  7. Braun, G., Fiorini, S., Pokutta, S.: Average case polyhedral complexity of the maximum stable set problem. Math. Program. (2013). doi:10.1007/s10107-016-0989-3

  8. Braun, G., Jain, R., Lee, T., Pokutta, S.: Information-theoretic approximations of the nonnegative rank. Electron. Colloq. Comput. Complex. (ECCC) 13(158) (2013). http://eccc.hpi-web.de/report/2013/158/

  9. Braverman, M., Moitra, A.: An information complexity approach to extended formulations. In: Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing, pp. 161–170 (2013). doi:10.1145/2488608.2488629

  10. Braverman, M., Garg, A., Pankratov, D., Weinstein, O.: Information lower bounds via self-reducibility. In: 8th International Computer Science Symposium in Russia, CSR 2013, pp. 183–194 (2013). doi:10.1007/978-3-642-38536-0_16

  11. Braverman, M., Garg, A., Pankratov, D., Weinstein, O.: From information to exact communication. In: Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing, pp. 151–160 (2013). doi:10.1145/2488608.2488628

  12. Chan, S., Lee, J., Raghavendra, P., Steurer, D.: Approximate constraint satisfaction requires large LP relaxations. In: IEEE 54th Annual Symposium on Foundations of Computer Science (2013). doi:10.1109/FOCS.2013.45

  13. Conforti, M., Cornuéjols, G., Zambelli, G.: Extended formulations in combinatorial optimization. 4OR 8, 1–48 (2010). doi:10.1007/s10288-010-0122-z

    Article  MathSciNet  MATH  Google Scholar 

  14. Cover, T., Thomas, J.: Elements of Information Theory. Wiley-interscience, Hoboken, NJ (2006)

  15. Faenza, Y., Fiorini, S., Grappe, R., Tiwary, H.R.: Extended formulations, non-negative factorizations and randomized communication protocols. In: Proceedings of the Second International Conference on Combinatorial Optimization (ISCO 2012), pp. 129–140 (2012)

  16. Fiorini, S., Massar, S., Pokutta, S., Tiwary, H.R., de Wolf, R.: Linear vs. semidefinite extended formulations: exponential separation and strong lower bounds. In: Proceedings of STOC 2012 (2012)

  17. Fiorini, S., Rothvoß, T., Tiwary, H.: Extended formulations for polygons. J. Discrete Comput. Geom. 48(3), 658–668 (2012)

  18. Fiorini, S., Kaibel, V., Pashkovich, K., Theis, D.O.: Combinatorial bounds on nonnegative rank and extended formulations. Discrete Math. 313, 67–83 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gillis, N.: Sparse and unique nonnegative matrix factorization through data preprocessing. J. Mach. Learn. Res. 13, 3349–3386 (2012)

    MathSciNet  MATH  Google Scholar 

  20. Gillis, N., Glineur, F., Theis, D.O.: Personal communication. 2 (2013)

  21. Håstad, J.: Clique is hard to approximate within \(1-\varepsilon \). Acta Math. 182(1), 105–142 (1999)

    Article  MathSciNet  Google Scholar 

  22. Jain, R., Shi, Y., Wei, Z., Zhang, S.: Efficient protocols for generating bipartite classical distributions and quantum states. In: Proceedings of SODA 2013 (2013)

  23. Kaibel, V.: Extended formulations in combinatorial optimization. Optima 85, 2–7 (2011)

    Google Scholar 

  24. Lin, J.: Divergence measures based on the shannon entropy. IEEE Trans. Inf. Theory 37(I), 145–151 (1991). doi:10.1109/18.61115

    Article  MathSciNet  MATH  Google Scholar 

  25. Moitra, A.: An almost optimal algorithm for computing nonnegative rank. In: Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2013), pp. 1454–1464 (2013)

  26. Pashkovich, K.: Extended formulations for combinatorial polytopes. PhD thesis, Magdeburg Universität (2012)

  27. Pokutta, S., Van Vyve, M.: A note on the extension complexity of the knapsack polytope. Oper. Res. Lett. 41(4), 347–350 (2013)

  28. Razborov, A.A.: On the distributional complexity of disjointness. Theor. Comput. Sci. 106(2), 385–390 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Razborov, A.A.: Personal communication (2012)

  30. Rothvoß, T.: Some 0/1 polytopes need exponential size extended formulations, (2011). arXiv:1105.0036

  31. Rothvoß, T.: The matching polytope has exponential extension complexity. In: Proceedings of STOC, pp. 263–272 (2014). doi:10.1145/2591796.2591834

  32. Shitov, Y.: An upper bound for nonnegative rank. J. Comb. Theory 122, 126–132 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  33. Witsenhausen, H.S.: Values and bounds for the common information of two discrete random variables. SIAM J. Appl. Math. 31(2), 313–333 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  34. de Wolf, R.: Nondeterministic quantum query and communication complexities. SIAM J. Comput. 32(3), 681–699 (2003)

  35. Wyner, A.: The common information of two dependent random variables. IEEE Trans. Inf. Theory 21(2), 163–179 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  36. Yannakakis, M.: Expressing combinatorial optimization problems by linear programs (extended abstract). In: Proceedings of STOC 1988, pp. 223–228 (1988)

  37. Yannakakis, M.: Expressing combinatorial optimization problems by linear programs. J. Comput. Syst. Sci. 43(3), 441–466 (1991). doi:10.1016/0022-0000(91)90024-Y

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhang, S.: Quantum strategic game theory. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 39–59. ACM (2012)

Download references

Acknowledgments

We are indebted to Samuel Fiorini for providing extensive feedback through the various stages of this work and who helped to significantly improve the presentation. We would also like to thank Daniel Dadush, Kostya Pashkovich, Santosh Vempala, and Ronald de Wolf for extremely valuable feedback as well as Jérémie Roland for pointing us to [22]. Part of this work was conducted at the Dagstuhl Seminar 13082 on Communication Complexity, Linear Optimization, and lower bounds for the nonnegative rank of matrices (http://www.dagstuhl.de/13082). The authors would like to thank the organizers for providing such a stimulating environment. Research reported in this paper was partially supported by NSF Grant CMMI-1300144.

Author information

Authors and Affiliations

  1. ISyE, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Gábor Braun & Sebastian Pokutta

Authors
  1. Gábor Braun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sebastian Pokutta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gábor Braun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Braun, G., Pokutta, S. Common Information and Unique Disjointness. Algorithmica 76, 597–629 (2016). https://doi.org/10.1007/s00453-016-0132-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-016-0132-0

Keywords

Search

Navigation