skip to main content
research-article

Information Complexity and the Quest for Interactive Compression

Published: 04 June 2015 Publication History

Abstract

Information complexity is the interactive analogue of Shannon's classical information theory. In recent years this field has emerged as a powerful tool for proving strong communication lower bounds, and for addressing some of the major open problems in communication complexity and circuit complexity. A notable achievement of information complexity is the breakthrough in understanding of the fundamental direct sum and direct product conjectures, which aim to quantify the power of parallel computation. This survey provides a brief introduction to information complexity, and overviews some of the recent progress on these conjectures and their tight relationship with the fascinating problem of compressing interactive protocols.

References

[1]
Boaz Barak, Mark Braverman, Xi Chen, and Anup Rao. How to compress interactive communication. In Proceedings of the 2010 ACM International Symposium on Theory of Computing, pages 67--76, 2010.
[2]
Joshua Brody, Harry Buhrman, Michal Kouck_y, Bruno Lo_, Florian Speelman, and Nikolay K. Vereshchagin. Towards a reverse newman's theorem in interactive information complexity. In Proceedings of the 28th Conference on Computational Complexity, CCC 2013, K.lo Alto, California, USA, 5-7 June, 2013, pages 24--33, 2013.
[3]
Eric Blais, Joshua Brody, and Kevin Matulef. Property testing lower bounds via communication complexity. Computational Complexity, 21(2):311--358, 2012.
[4]
Mark Braverman, Faith Ellen, Rotem Oshman, Toniann Pitassi, and Vinod Vaikuntanathan. Tight bounds for set disjointness in the message passing model. CoRR, abs/1305.4696, 2013.
[5]
Mark Braverman, Ankit Garg, Denis Pankratov, and Omri Weinstein. From information to exact communication. In Proceedings of the Forty-_fth Annual ACM Symposium on Theory of Computing, STOC '13, pages 151--160, New York, NY, USA, 2013. ACM.
[6]
Mark Braverman and Ankur Moitra. An information complexity approach to extended formulations. Electronic Colloquium on Computational Complexity (ECCC), 19:131, 2012.
[7]
Balthazar Bauer, Shay Moran, and Amir Yehudayo_. Internal compression of protocols to entropy. Electronic Colloquium on Computational Complexity (ECCC), 21:101, 2014.
[8]
Gábor Braun and Sebastian Pokutta. Common information and unique disjointness. In 54th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2013, 26-29 October, 2013, Berkeley, CA, USA, pages 688--697, 2013.
[9]
Mark Braverman and Anup Rao. Information equals amortized communication. In Rafail Ostrovsky, editor, FOCS, pages 748--757. IEEE, 2011.
[10]
Mark Braverman. Interactive information complexity. In Proceedings of the 44th symposium on Theory of Computing, STOC '12, pages 505--524, New York, NY, USA, 2012. ACM.
[11]
Mark Braverman. A hard-to-compress interactive task? In 2013 51st Annual Allerton Conference on Communication, Control, and Computing, Allerton Park & Retreat Center, Monticello, IL, USA, October 2-4, 2013, pages 8--12, 2013.
[12]
Mark Braverman. Interactive information and coding theory. 2014.
[13]
Mark Braverman, Anup Rao, Omri Weinstein, and Amir Yehudayo_. Direct products in communication complexity. Electronic Colloquium on Computational Complexity (ECCC), 19:143, 2012.
[14]
Mark Braverman, Anup Rao, Omri Weinstein, and Amir Yehudayo_. Direct product via round-preserving compression. Electronic Colloquium on Computational Complexity (ECCC), 20:35, 2013.
[15]
Richard Beigel and Jun Tarui. On acc. In FOCS, pages 783--792, 1991.
[16]
Mark Braverman and Omri Weinstein. A discrepancy lower bound for information complexity. In APPROX-RANDOM, pages 459--470, 2012.
[17]
Mark Braverman and Omri Weinstein. An interactive information odometer with applications. Electronic Colloquium on Computational Complexity (ECCC), 21:47, 2014.
[18]
Ziv Bar-Yossef, T. S. Jayram, Ravi Kumar, and D. Sivakumar. An information statistics approach to data stream and communication complexity. Journal of Computer and System Sciences, 68(4):702--732, 2004.
[19]
Amit Chakrabarti, Subhash Khot, and Xiaodong Sun. Near-optimal lower bounds on the multi-party communication complexity of set disjointness. In IEEE Conference on Computational Complexity, pages 107--117, 2003.
[20]
Amit Chakrabarti, Yaoyun Shi, Anthony Wirth, and Andrew Yao. Informational complexity and the direct sum problem for simultaneous message complexity. In Proceedings of the 42nd Annual IEEE Symposium on Foundations of Computer Science, pages 270--278, 2001.
[21]
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley series in telecommunications. J. Wiley and Sons, New York, 1991.
[22]
Shahar Dobzinski and Noam Nisan. Limitations of vcg-based mechanisms. Combinatorica, 31(4):379--396, 2011.
[23]
Tom_as Feder, Eyal Kushilevitz, Moni Naor, and Noam Nisan. Amortized communication complexity. SIAM Journal on Computing, 24(4):736--750, 1995. Prelim version by Feder, Kushilevitz, Naor FOCS 1991.
[24]
Uriel Feige, David Peleg, Prabhakar Raghavan, and Eli Upfal. Computing with noisy information. SIAM Journal on Computing, 23(5):1001--1018, 1994.
[25]
Anat Ganor, Gillat Kol, and Ran Raz. Exponential separation of information and communication. Electronic Colloquium on Computational Complexity (ECCC), 21:49, 2014.
[26]
Dmitry Gavinsky, Or Meir, Omri Weinstein, and Avi Wigderson. Toward better formula lower bounds: An information complexity approach to the krw composition conjecture. In Proceedings of the 46th Annual ACM Symposium on Theory of Computing, STOC '14, pages 213--222, New York, NY, USA, 2014. ACM.
[27]
Venkatesan Guruswami and Krzysztof Onak. Superlinear lower bounds for multipass graph processing. In IEEE Conference on Computational Complexity, pages 287--298, 2013.
[28]
Prahladh Harsha, Rahul Jain, David A. McAllester, and Jaikumar Radhakrishnan. The communication complexity of correlation. In IEEE Conference on Computational Complexity, pages 10--23. IEEE Computer Society, 2007.
[29]
Thomas Holenstein. Parallel repetition: Simpli_cations and the no-signaling case. In Proceedings of the 39th Annual ACM Symposium on Theory of Computing, 2007.
[30]
Zengfeng Huang, Bozidar Radunovic, Milan Vojnovic, and Qin Zhang. Communication complexity of approximate maximum matching in distributed graph data. Technical Report MSR-TR-2013-35, April 2013.
[31]
D.A. Hu_man. A method for the construction of minimum-redundancy codes. Proceedings of the IRE, 40(9):1098--1101, 1952.
[32]
Rahul Jain. New strong direct product results in communication complexity. 2011.
[33]
T. S. Jayram, Swastik Kopparty, and Prasad Raghavendra. On the communication complexity of read-once ac0 formulae. In IEEE Conference on Computational Complexity, pages 329--340, 2009.
[34]
Rahul Jain, Attila Pereszlenyi, and Penghui Yao. A direct product theorem for the two-party bounded-round public-coin communication complexity. In Foundations of Computer Science (FOCS), 2012 IEEE 53rd Annual Symposium on, pages 167--176. IEEE, 2012.
[35]
Rahul Jain and Penghui Yao. A strong direct product theorem in terms of the smooth rectangle bound. CoRR, abs/1209.0263, 2012.
[36]
Hartmut Klauck. A strong direct product theorem for disjointness. In STOC, pages 77--86, 2010.
[37]
Iordanis Kerenidis, Sophie Laplante, Virginie Lerays, J_er_emie Roland, and David Xiao. Lower bounds on information complexity via zero-communication protocols and applications. CoRR, abs/1204.1505, 2012.
[38]
Mauricio Karchmer, Ran Raz, and Avi Wigderson. Super-logarithmic depth lower bounds via the direct sum in communication complexity. Computational Complexity, 5(3/4):191--204, 1995. Prelim version CCC 1991.
[39]
Mauricio Karchmer and Avi Wigderson. Monotone circuits for connectivity require super-logarithmic depth. In STOC, pages 539--550, 1988.
[40]
Nikos Leonardos and Michael Saks. Lower bounds on the randomized communication complexity of read-once functions. Computational Complexity, 19(2):153--181, 2010.
[41]
Troy Lee, Adi Shraibman, and Robert Spalek. A direct product theorem for discrepancy. In CCC, pages 71--80, 2008.
[42]
Marco Molinaro, David Woodru_, and Grigory Yaroslavtsev. Beating the direct sum theorem in communication complexity with implications for sketching. In SODA, page to appear, 2013.
[43]
Itzhak Parnafes, Ran Raz, and Avi Wigderson. Direct product results and the GCD problem, in old and new communication models. In Proceedings of the 29th Annual ACM Symposium on the Theory of Computing (STOC '97), pages 363--372, New York, May 1997. Association for Computing Machinery.
[44]
Mihai Patrascu and Ryan Williams. On the possibility of faster sat algorithms. In Moses Charikar, editor, SODA, pages 1065--1075. SIAM, 2010.
[45]
Anup Rao. Parallel repetition in projection games and a concentration bound. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing, 2008.
[46]
Ran Raz. A parallel repetition theorem. SIAM Journal on Computing, 27(3):763--803, June 1998. Prelim version in STOC '95.
[47]
Alexander Razborov. A simple proof of bazzi's theorem. Technical Report TR08-081, ECCC: Electronic Colloquium on Computational Complexity, 2008.
[48]
Claude E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27, 1948. Monograph B-1598.
[49]
Ronen Shaltiel. Towards proving strong direct product theorems. Computational Complexity, 12(1-2):1--22, 2003. Prelim version CCC 2001.
[50]
Alexander A. Sherstov. Strong direct product theorems for quantum communication and query complexity. SIAM J. Comput., 41(5):1122--1165, 2012.
[51]
Mert Saglam and G_abor Tardos. On the communication complexity of sparse set disjointness and exists-equal problems. CoRR, abs/1304.1217, 2013.
[52]
Juraj Waczulk. Area time squared and area complexity of vlsi computations is strongly unclosed under union and intersection. In Jrgen Dassow and Jozef Kelemen, editors, Aspects and Prospects of Theoretical Computer Science, volume 464 of Lecture Notes in Computer Science, pages 278--287. Springer Berlin Heidelberg, 1990.
[53]
Virginia Vassilevska Williams. Multiplying matrices faster than coppersmithwinograd. In Proceedings of the Forty-fourth Annual ACM Symposium on Theory of Computing, STOC '12, pages 887--898, New York, NY, USA, 2012. ACM.
[54]
David P. Woodru_ and Qin Zhang. An optimal lower bound for distinct elements in the message passing model. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2014, Portland, Oregon, USA, January 5-7, 2014, pages 718--733, 2014.
[55]
Andrew Chi-Chih Yao. Some complexity questions related to distributive computing. In STOC, pages 209--213, 1979.
[56]
Andrew Chi-Chih Yao. Theory and applications of trapdoor functions (extended abstract). In FOCS, pages 80--91. IEEE, 1982.
[57]
Yuchen Zhang, John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. Information-theoretic lower bounds for distributed statistical estimation with communication constraints. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States., pages 2328--2336, 2013.

Cited By

View all
  • (2024)O(log log n) Passes Is Optimal for Semi-streaming Maximal Independent SetProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649763(847-858)Online publication date: 10-Jun-2024
  • (2024)Common Information in Well-Mixing Graphs and Applications to Information-Theoretic Cryptography2024 IEEE Information Theory Workshop (ITW)10.1109/ITW61385.2024.10806994(181-186)Online publication date: 24-Nov-2024
  • (2023)Recent Advances in Multi-Pass Graph Streaming Lower BoundsACM SIGACT News10.1145/3623800.362380854:3(48-75)Online publication date: 7-Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGACT News
ACM SIGACT News  Volume 46, Issue 2
June 2015
103 pages
ISSN:0163-5700
DOI:10.1145/2789149
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2015
Published in SIGACT Volume 46, Issue 2

Check for updates

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)O(log log n) Passes Is Optimal for Semi-streaming Maximal Independent SetProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649763(847-858)Online publication date: 10-Jun-2024
  • (2024)Common Information in Well-Mixing Graphs and Applications to Information-Theoretic Cryptography2024 IEEE Information Theory Workshop (ITW)10.1109/ITW61385.2024.10806994(181-186)Online publication date: 24-Nov-2024
  • (2023)Recent Advances in Multi-Pass Graph Streaming Lower BoundsACM SIGACT News10.1145/3623800.362380854:3(48-75)Online publication date: 7-Sep-2023
  • (2021)New separations results for external informationProceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing10.1145/3406325.3451044(248-258)Online publication date: 15-Jun-2021
  • (2019)Communication for Generating Correlation: A Unifying SurveyIEEE Transactions on Information Theory10.1109/TIT.2019.294636466:1(5-37)Online publication date: 20-Dec-2019
  • (2019)Information complexity and applications?Japanese Journal of Mathematics10.1007/s11537-018-1727-9Online publication date: 5-Mar-2019
  • (2019)Some lower bounds in dynamic networks with oblivious adversariesDistributed Computing10.1007/s00446-019-00360-433:1(1-40)Online publication date: 7-Aug-2019
  • (2018)On Slepian---Wolf Theorem with InteractionTheory of Computing Systems10.1007/s00224-016-9741-x62:3(583-599)Online publication date: 1-Apr-2018
  • (2017)Interactive Information ComplexitySIAM Review10.1137/17M113925459:4(803-846)Online publication date: 6-Nov-2017
  • (2016)A Concise Tour of Communication Complexity TheoryIEICE ESS Fundamentals Review10.1587/essfr.10.1_4610:1(46-56)Online publication date: 2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media