Vitaly Feldman
ABSTRACT
Producing a small DNF expression consistent with given data is a classical problem in computer science that occurs in a number of forms and has numerous applications. We consider two standard variants of this problem. The first one is two-level logic minimization or finding a minimal DNF formula consistent with a given complete truth table (TT-MinDNF. This problem was formulated by Quine in 1952 and has been since one of the key problems in logic design. It was proved NP-complete by Masek in 1979. The best known polynomial approximation algorithm is based on a reduction to the SET-COVER problem and produces a DNF formula of size O(d ∙ OPT), where d is the number of variables. We prove that TT-MinDNF is NP-hard to approximate within dγ for some constant γ > 0, establishing the first inapproximability result for the problem.The other DNF minimization problem we consider is PAC learning of DNF expressions when the learning algorithm must output a DNF expression as its hypothesis (referred to as proper learning). We prove that DNF expressions are NP-hard to PAC learn properly even when the learner has access to membership queries, thereby answering a long-standing open question due to Valiant [40]. Finally, we show that inapproximability of TT-MinDNF implies hardness results for restricted proper learning of DNF expressions with membership queries even when learning with respect to the uniform distribution only.
- R. Agrawal, J. Gehrke, D. Gunopulos, and P. Raghavan. Automatic subspace clustering of high dimensional data. Data Mining and Knowledge Discovery, 11(1):5--33, July 2005.]] Google Scholar
Digital Library
- M. Alekhnovich, M. Braverman, V. Feldman, A. Klivans, and T. Pitassi. Learnability and automizability. In Proceeding of FOCS '04, pages 621--630, 2004.]] Google ScholarDigital Library
- E. Allender, L. Hellerstein, P. McCabe, T. Pitassi, and M.Saks. Minimizing DNF Formulas and AC0 Circuits Given a Truth Table. Electronic Colloquium on Computational Complexity (ECCC), (126), Nov, 2005.]]Google Scholar
- E. Allender, L. Hellerstein, T. Pitassi, and M.Saks. On the complexity of finding minimal representations of boolean functions. 2004. Unpublished.]]Google Scholar
- D. Angluin and M. Kharitonov. When won't membership queries help? Journal of Computer and System Sciences, 50(2):336--355, 1995.]] Google ScholarDigital Library
- M. Bellare, S. Goldwasser, C. Lund, and A. Russell. Efficient probabilistically checkable proofs and applications to approximations. In STOC '93, pages 294--304, 1993.]] Google ScholarDigital Library
- A. Blum and S. Rudich. Fast learning of k-term DNF formulas with queries. Journal of Computer and System Sciences, 51(3):367--373, 1995.]] Google ScholarDigital Library
- A. L. Blum and R. L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5(1):117--127, 1992.]] Google ScholarDigital Library
- O. Coudert and J. Madre. METAPRIME, an Interactive Fault-Tree Analyser. IEEE Transactions on Reliability, 43(1):121--127, 1994.]]Google ScholarCross Ref
- O. Coudert and T. Sasao. Two-level logic minimization. Kluwer Academic Publishers, 2001.]]Google Scholar
- S. Czort. The complexity of minimizing disjunctive normal form formulas. Master's thesis, University of Aarhus, 1999.]]Google Scholar
- I. Dinur, V. Guruswami, S. Khot, and O. Regev. A new multilayered pcp and the hardness of hypergraph vertex cover. SIAM Journal of Computing, 34(5):1129--1146, 2005.]] Google ScholarDigital Library
- P. Erdös, P. Frankl, and Z. Furedi. Families of finite sets in which no set is covered by the union of r others. Israel Journal of Mathematics, 51:79--89, 1985.]]Google ScholarCross Ref
- U. Feige. A threshold of ln n for approximating set cover. Journal of the ACM, 45(4):634--652, 1998.]] Google ScholarDigital Library
- U. Feige and J. Kilian. Zero knowledge and the chromatic number. In Proceedings of Conference on Computational Complexity (CCC-96), pages 278--289, May 24--27 1996.]] Google ScholarDigital Library
- J.F. Gimpel. A method for producing a boolean function having an arbitrary prescribed prime implicant table. IEEE Transactions on Computers, 14:485--488, 1965.]]Google ScholarCross Ref
- E. A. Gold. Complexity of automaton identification from given data. Information and Control, 37:302--320, 1978.]]Google ScholarCross Ref
- Johan Håstad and Subhash Khot. Query Efficient PCPs with Perfect Completeness. Theory of Computing, 1(7):119--148, 2005.]]Google ScholarCross Ref
- Lisa Hellerstein and Vijay Raghavan. Exact learning of DNF formulas using DNF hypotheses. In ACM, editor, Proceedings of STOC '02, pages 465--473, 2002.]] Google ScholarDigital Library
- J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55:414--440, 1997.]] Google ScholarDigital Library
- W. Kautz and R. Singleton. Nonrandom binary superimposed codes. IEEE Trans. Inform. Theory, 10:363--377, 1964.]]Google ScholarDigital Library
- M. Kearns, M. Li, L. Pitt, and L. Valiant. On the learnability of boolean formulae. In Proceedings of the Nineteenth Annual Symposium on Theory of Computing, pages 285--295, 1987.]] Google ScholarDigital Library
- A. Klivans and R. Servedio. Learning DNF in time 2Õ(n1/3). In Proceedings of the Thirty-Third Annual Symposium on Theory of Computing, pages 258--265, 2001.]] Google ScholarDigital Library
- H. Liu. Routing table compaction in ternary cam. IEEE Micro, 22(1):58--64, 2002.]] Google ScholarDigital Library
- C. Lund and M. Yannakakis. On the hardness of approximating minimization problems. Journal of the ACM, 41(5):960--981, 1994.]] Google ScholarDigital Library
- W. Masek. Some NP-complete set covering problems. unpublished, 1979.]]Google Scholar
- E. L. Jr. McCluskey. Minimization of Boolean Functions. Bell Sys. Tech. Jour., 35:1417--1444, 1956.]]Google ScholarCross Ref
- N. Nisan and A. Wigderson. Hardness versus randomness. Journal of Computer and System Sciences, 49:149--167, 1994.]] Google ScholarDigital Library
- R. Nock, P. Jappy, and J. Sallantin. Generalized graph colorability and compressibility of boolean formulae. In Proceedings of the 9th International Symposium on Algorithms and Computation (ISAAC), 1998.]] Google ScholarDigital Library
- W. J. Paul. Boolesche minimalpolynome und überdeckungsprobleme. Acta Informatica, 4:321--336, 1974.]]Google ScholarDigital Library
- L. Pitt and L. Valiant. Computational limitations on learning from examples. Journal of the ACM, 35(4):965--984, 1988.]] Google ScholarDigital Library
- W.V. Quine. The problem of simplifying truth functions. Americal Mathematical Monthly, 59:521--531, 1952.]]Google ScholarCross Ref
- W.V. Quine. A way to simplify truth functions. Americal Mathematical Monthly, 62:627--631, 1956.]]Google ScholarCross Ref
- R. Raz and S. Safra. A sub-constant error-probability low-degree test, and a sub-constant error-probability PCP characterization of NP. In Proceedings of STOC '97, pages 475--484, 1997.]] Google ScholarDigital Library
- A. Ta-Shma. A Note on PCP vs. MIP. Information Processing Letters, 58(3):135--140, 1996.]] Google ScholarDigital Library
- L. Trevisan. Non-approximability results for optimization problems on bounded degree instances. In Proceedings of STOC '01, pages 453--461, 2001.]] Google ScholarDigital Library
- C. Umans. The minimum equivalent DNF problem and shortest implicants. J. Comput. Syst. Sci., 63(4):597--611, 2001.]] Google ScholarDigital Library
- C. Umans, T. Villa, and A. L. Sangiovanni-Vincentelli. Complexity of two-level logic minimization. Technical Report UCB/ERL M04/45, UC Berkeley, October 2004.]]Google Scholar
- S. Vadhan. Lecture notes on pseudorandomness. Available at http://www.courses.fas.harvard.edu/~cs225/, 2004.]]Google Scholar
- L. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134--1142, 1984.]] Google ScholarDigital Library
- L. Valiant. Learning disjunctions of conjunctions. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pages 560--566, 1985.]]Google ScholarDigital Library
Index Terms
- Hardness of approximate two-level logic minimization and PAC learning with membership queries
Recommendations
Hardness of approximate two-level logic minimization and PAC learning with membership queries
Producing a small DNF expression consistent with given data is a classical problem in computer science that occurs in a number of forms and has numerous applications. We consider two standard variants of this problem. The first one is two-level logic ...
The Minimum Equivalent DNF Problem and Shortest Implicants
We prove that the Minimum Equivalent DNF problem is P2-complete, resolving a conjecture due to Stockmeyer. We also consider the complexity and approximability of a related optimization problem in the second level of the polynomial hierarchy, that of ...
Hardness of Learning Halfspaces with Noise
Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noise-free case, when a halfspace consistent with all the training examples exists, the problem can be solved in ...
Comments