Skip to main content
SpringerLink
Log in

Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

We study the runtime distributions of backtrack procedures for propositional satisfiability and constraint satisfaction. Such procedures often exhibit a large variability in performance. Our study reveals some intriguing properties of such distributions: They are often characterized by very long tails or “heavy tails”. We will show that these distributions are best characterized by a general class of distributions that can have infinite moments (i.e., an infinite mean, variance, etc.). Such nonstandard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We also show how random restarts can effectively eliminate heavy-tailed behavior. Furthermore, for harder problem instances, we observe long tails on the left-hand side of the distribution, which is indicative of a non-negligible fraction of relatively short, successful runs. A rapid restart strategy eliminates heavy-tailed behavior and takes advantage of short runs, significantly reducing expected solution time. We demonstrate speedups of up to two orders of magnitude on SAT and CSP encodings of hard problems in planning, scheduling, and circuit synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adler, R., Feldman, R. and Taggu, M. (eds.): A Practical Guide to Heavy Tails, Birkhäuser, 1998

  2. Alt, H., Guibas, L., Mehlhorn, K., Karp, R. and Wigderson, A.: A method for obtaining randomized algorithms with small tail probabilities, Algorithmica 16(4–5) (1996), 543–547.

    Google Scholar 

  3. Anderson, L.: Completing partial latin squares, Math. Fys. Meddelelser 41 (1985), 23–69.

    Google Scholar 

  4. Bayardo, R.: Personal communications, 1999.

  5. Bayardo, R. and Miranker, D.: A complexity analysis of space-bounded learning algorithms for the constraint satisfaction problem, in Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96), Portland, OR, 1996, pp. 558–562.

  6. Bayardo, R. and Schrag, R.: Using CSP look-back techniques to solve real-world SAT instances, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), New Providence, RI, 1997, pp. 203–208.

  7. Brelaz, D.: New methods to color the vertices of a graph, Comm. ACM 22(4) (1979), 251–256.

    Google Scholar 

  8. Colbourn, C.: Embedding partial Steiner triple systems is NP-complete, J. Combin. Theory A 35 (1983), 100–105.

    Google Scholar 

  9. Cook, S. and Mitchell, D.: Finding hard instances of the satisfiability problem: A survey, in Du, Gu, and Pardalos (eds), Satisfiability Problem: Theory and Applications, Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 35, 1998.

  10. Crato, N., Gomes, C. and Selman, B.: On estimating the index of stability with truncated data, Manuscript in preparation.

  11. Crawford, J. and Baker, A.: Experimental results on the application of satisfiability algorithms to scheduling problems, in Proceedings of The Tenth National Conference on Artificial Intelligence (AAAI-94), Austin, TX, 1994, pp. 1092–1098.

  12. Davis, M., Logemann, G. and Loveland, D.: A machine program for theorem proving, Comm. ACM 5 (1979), 394–397.

    Google Scholar 

  13. Davis, M. and Putman, H.: A computing procedure for quantification theory, J. ACM 7 (1960), 201–215.

    Google Scholar 

  14. Denes, J. and Keedwell, A.: Latin Squares and Their Applications, Akademiai Kiado, Budapest, and English Universities Press, 1974.

    Google Scholar 

  15. Ertel, W. and Luby, M.: Optimal parallelization of Las Vegas algorithms, in Symp. on Theoretical Aspects of Computer Science, Lecture Notes in Comput. Sci. 775, 1994, pp. 463–475.

  16. Feller,W.: An Introduction to Probability Theory and Its Applications, Vol. I, Wiley, New York, 1968.

    Google Scholar 

  17. Feller, W.: An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, New York, 1971.

    Google Scholar 

  18. Frost, D., Rish, I. and Vila, L.: Summarizing CSP hardness with continuous probability distributions, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), New Providence, RI, 1997, pp. 327–334.

  19. Fujita, M., Slaney, J. and Bennett, F.: Automatic generation of some results in finite algebra, in Proceedings of the International Joint Conference on Artificial Intelligence, France, 1993, pp. 52–57.

  20. Gent, I. and Walsh, T.: Easy problems are sometimes hard, Artif. Intell. 70 (1993), 335–345.

    Google Scholar 

  21. Gomes, C. P. and Selman, B.: Algorit portfolio design: Theory vs. practice, in Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI-97), Providence, RI, 1997, pp. 190–197.

  22. Gomes, C. P. and Selman, B.: Problem structure in the presence of perturbations, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), New Providence, RI, 1997, pp. 221–227.

  23. Gomes, C. P., Selman, B. and Crato, N.: Heavy-tailed distributions in combinatorial search, in G. Smolka (ed.), Principles and Practice of Constraint Programming (CP97), Lecture Notes in Comput. Sci., Linz, Austria, 1997, pp. 121–135.

    Google Scholar 

  24. Hall, P.: On some simple estimates of an exponent of regular variation, J. Roy. Statist. Soc. 44 (1982), 37–42.

    Google Scholar 

  25. Hayes, B.: Computer science: Can't get no satisfaction, American Scientist 85(2) (1996), 108–112.

    Google Scholar 

  26. Hill, B.: A simple general approach to inference about the tail of a distribution, Ann. Statist. 3 (1975), 1163–1174.

    Google Scholar 

  27. Hogg, T., Huberman, B. and Williams, C. (eds.): Phase transitions and complexity (Special Issue), Artif. Intell. 81(1–2) (1996).

  28. Hogg, T. and Williams, C.: Expected gains from parallelizing constraint solving for hard problems, in Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94), Seattle, WA, 1994, pp. 1310–1315.

  29. Hoos, H.: Stochastic local search-methods, models, applications, Ph.D. Thesis, TU Darmstadt, 1998.

  30. Huberman, B., Lukose, R. and Hogg, T.: An economics approach to hard computational problems, Science 275 (1993), 51–54.

    Google Scholar 

  31. Johnson, D.: A theoretician's guide to the experimental analysis of algorithms, Preliminary manuscript. Invited talk presented at AAAI-96, Portland, OR, 1996.

  32. Johnson, D. and Trick,M.: Cliques, coloring, and satisfiability: Second dimacs implementation challenge, in Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 36, 1996.

  33. Kamath, A., Karmarkar, N., Ramakrishnan, K. and Resende, M.: Computational experience with an interior point algorithm on the satisfiability problem, in Proceedings of Integer Programming and Combinatorial Optimization, Waterloo, Canada, 1990, pp. 333–349.

  34. Kautz, H. and Selman, B.: Pushing the envelope: Planning, propositional logic, and stochastic search, in Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96), Portland, OR, 1996, pp. 1188–1194.

  35. Kirkpatrick, S. and Selman, B.: Critical behavior in the satisfiability of random Boolean expressions, Science 264 (1994), 1297–1301.

    Google Scholar 

  36. Kwan, A.: Validity of normality assumption in CSP research, in Proceedings of the Pacific Rim International Conference on Artificial Intelligence, 1995, pp. 459–465.

  37. Lam, C., Thiel, L. and Swiercz, S.: The non-existence of finite projective planes of order 10, Canad. J. Math. XLI(6) (1994), 1117–1123.

    Google Scholar 

  38. Li, C. M.: Personal communications, 1999.

  39. Li, C. M. and Anbulagan: Heuristics based on unit propagation for satisfiability problems, in Proceedings of the International Joint Conference on Artificial Intelligence, 1997, pp. 366–371.

  40. Luby, M., Sinclair, A. and Zuckerman, D.: Optimal speedup of Las Vegas algorithms, Inform. Process. Lett., 1993, pp. 173–180.

  41. Macready, W., Siapas, A. and Kauffman, S.: Criticality and parallelism in combinatorial optimization, Science 271 (1996), 56–59.

    Google Scholar 

  42. Mandelbrot, B.: The Pareto-Lévy law and the distribution of income, Internat. Econom. Rev. 1 (1960), 79–106.

    Google Scholar 

  43. Mandelbrot, B.: The variation of certain speculative prices, J. Business 36 (1963), 394–419.

    Google Scholar 

  44. Mandelbrot, B.: The Fractal Geometry of Nature, Freeman, New York, 1983

    Google Scholar 

  45. McAloon, K., Tretkoff, C. and Wetzel, G.: Sports league scheduling, in Proceedings of Third Ilog International Users Meeting, 1997.

  46. Mitchell, D. and Levesque, H.: Some pitfalls for experimenters with random SAT, Artif. Intell. 81(1–2) (1996), 111–125.

    Google Scholar 

  47. Monasson, R., Zecchina, R., Kirkpatrick, S., Selman, B. and Troyansky, L.: Determining computational complexity from characteristic 'phase transitions', Nature 400(8) (1999), 133–137.

    Google Scholar 

  48. Nemhauser, G. and Trick, M.: Scheduling a major college basketball conference, Oper. Res. 46(1) (1998), 1–8.

    Google Scholar 

  49. Niemela, I.: Logic programs with stable model semantics as a constraint programming paradigm. Ext. version of paper presented at the Workshop on Computational Aspects of Nonmonotonic Reasoning, 1998.

  50. Niemela, I.: Personal communications, 1999.

  51. Nolan, J.: Stable distributions, Manuscript, in preparation, 1999.

  52. Prosser, P.: Hybrid algorithms for the constraint satisfaction problem, Comput. Intell. 9(3) (1993), 268–299.

    Google Scholar 

  53. Puget, J. F. and Leconte, M.: Beyond the black box: Constraints as objects, in Proceedings of ILPS'95, Lisbon, Portugal, 1995, pp. 513–527.

  54. Regin, J. C.: A filtering algorithm for constraints of difference in CSP, in Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94), Seattle, WA, 1994, pp. 362–367.

  55. Rish, I. and Frost, D.: Statistical analysis of backtracking on inconsistent CSPs, in Proceedings of Third International Conference on Principles and Practices of Constraint Programming, 1997, pp. 150–162.

  56. Samorodnitsky, G. and Taqqu, M.: Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance, Chapman and Hall, 1994.

  57. Selman, B., Kautz, H. and Cohen, B.: Local search strategies for satisfiability testing, in D. Johnson and M. Trick (eds), Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 26, 1993, pp. 521–532.

  58. Selman, B. and Kirkpatrick, S.: Finite-size scaling of the computational cost of systematic search, Artif. Intell. 81(1–2) (1996), 273–295.

    Google Scholar 

  59. Shaw, P., Stergiou, K. and Walsh, T.: Arc consistency and quasigroup completion, in Proceedings of ECAI-98, Workshop on Binary Constraints, 1998.

  60. Shonkwiler, R., E. and van Vleck, E.: Parallel speedup of Monte Carlo methods for global optimization, J. Complexity 10 (1994), 64–95.

    Google Scholar 

  61. Slaney, J., Fujita, M. and Stickel, M.: Automated reasoning and exhaustive search: Quasigroup existence problems, Comput. Math. Appl. 29 (1995), 115–132.

    Google Scholar 

  62. Smith, B. and Grant, S.: Sparse constraint graphs and exceptionally hard problems, in Proceedings of the International Joint Conference on Artificial Intelligence, 1995, pp. 646–651.

  63. Stickel, M.: Personal communications, 1998.

  64. Vandengriend and Culberson: The Gnm phase transition is not hard for the Hamiltonian cycle problem, J. Artif. Intell. Res. 9 (1999), 219–245.

    Google Scholar 

  65. Walsh, T.: Search in a small world, in Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 1999

  66. Zolotarev, V.: One-Dimensional Stable Distributions, Transl. Math. Monographs 65, Amer. Math. Soc., 1986. Translation from the original 1983 Russian ed.

Download references

Authors
  1. Carla P. Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bart Selman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nuno Crato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Henry Kautz
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gomes, C.P., Selman, B., Crato, N. et al. Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems. Journal of Automated Reasoning 24, 67–100 (2000). https://doi.org/10.1023/A:1006314320276

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1006314320276

Search

Navigation