Skip to main content
SpringerLink
Log in
Book cover

Data Mining and Constraint Programming pp 113–146Cite as

Learning Modulo Theories

  • Chapter
  • First Online:

  • 1558 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10101))

Abstract

Many real-world applications require reasoning over hybrid domains involving combinations of continuous and discrete variables and their relationships. Being able to precisely specify all constraints and their respective importance beforehand is often infeasible for the most experienced designer, let alone for a typical decision maker. In this chapter we discuss Learning Modulo Theories (LMT), a learning framework capable of dealing with hybrid domains by combining structured learning with Satisfiability Modulo Theory (SMT) techniques. LMT incorporates SMT solvers and their extensions for optimization as inference engines within learning algorithms. The learning stage automatically identifies the relevant constraints and their respective weights among a set of candidates. The framework can be cast in the structured-output learning paradigm, where the task is learning the structure of the problem from a set of noisy instances, or as a preference elicitation task, where a decision maker is involved in an interactive optimization loop aimed at generating the most preferred solution. We report experimental results highlighting the potential of the method in automated design and recommendation scenarios.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    We depart from the conventional x/y notation for indicating input/output pairs to avoid name clashes with the x-y coordinate variables.

  2. 2.

    While we write Boolean variables using a first-order syntax for readability, the OMT solver currently requires the grounding of all Boolean predicates.

  3. 3.

    Dataset taken from http://cs.nyu.edu/~roweis/data.html.

  4. 4.

    Our preliminary experimental studies showed that Church is incapable of solving in reasonable time the simple task of generating a pair of blocks conditioned on the fact that they touch somewhere.

  5. 5.

    Unassigned features are catalogue features not appearing in any hard constraint or non-zero weight soft constraint.

  6. 6.

    DM utility functions involving nine complex constraints are quite unrealistic and are considered here just for testing the scalability of the algorithm.

References

  1. Ahmadi, B., Kersting, K., Sanner, S.: Multi-evidence lifted message passing, with application to pagerank and the kalman filter. In: Proceedings of IJCAI 2011, pp. 1152–1158 (2011)

    Google Scholar 

  2. Bakir, G.H., Hofmann, T., Schölkopf, B., Smola, A.J., Taskar, B., Vishwanathan, S.V.N.: Predicting Structured Data (Neural Information Processing). The MIT Press, Cambridge (2007)

    Google Scholar 

  3. Barrett, C., Sebastiani, R., Seshia, S.A., Tinelli, C.: Satisfiability modulo theories, chap. 26, Frontiers in Artificial Intelligence and Applications, pp. 825–885. IOS Press, February 2009

    Google Scholar 

  4. Battiti, R., Brunato, M.: Reactive search optimization: learning while optimizing. In: Gendreau, M., Potvin, J.-Y. (eds.) Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 146, pp. 543–571. Springer, New York (2010)

    Chapter  Google Scholar 

  5. Baumgartner, P., Tinelli, C.: Model evolution with equality modulo built-in theories. In: Bjørner, N., Sofronie-Stokkermans, V. (eds.) CADE 2011. LNCS (LNAI), vol. 6803, pp. 85–100. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22438-6_9

    Chapter  Google Scholar 

  6. Beldiceanu, N., Simonis, H.: A model seeker extracting global constraint models from positive examples. In: Milano, M. (ed.) CP 2012. LNCS, vol. 7514, pp. 141–157. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33558-7_13

    Chapter  Google Scholar 

  7. Belle, V., Passerini, A., Van den Broeck, G.: Probabilistic inference in hybrid domains by weighted model integration. In: Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI) (2015)

    Google Scholar 

  8. Belle, V., Van den Broeck, G., Passerini, A.: Hashing-based approximate probabilistic inference in hybrid domains. In: Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence (UAI) (2015)

    Google Scholar 

  9. Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bessiere, C., Coletta, R., Hebrard, E., Katsirelos, G., Lazaar, N., Narodytska, N., Quimper, C.-G., Walsh, T.: Constraint acquisition via partial queries. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI 2013, pp. 475–481. AAAI Press (2013)

    Google Scholar 

  11. Birlutiu, A., Groot, P., Heskes, T.: Efficiently learning the preferences of people. Mach. Learn. 90, 1–28 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bonilla, E., Guo, S., Sanner, S.: Gaussian process preference elicitation. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems, pp. 262–270 (2010)

    Google Scholar 

  13. Boutilier, C., Patrascu, R., Poupart, P., Schuurmans, D.: Constraint-based optimization and utility elicitation using the minimax decision criterion. Artif. Intell. 170(8–9), 686–713 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  14. Boutilier, C., Regan, K., Viappiani, P.: Simultaneous elicitation of preference features and utility. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, GA, USA. AAAI Press, pp. 1160–1167, July 2010

    Google Scholar 

  15. Braziunas, D.: Computational approaches to preference elicitation. Technical report, Department of Computer Science, University of Toronto (2006)

    Google Scholar 

  16. Braziunas, D., Boutilier, C.: Minimax regret based elicitation of generalized additive utilities. In: Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence (UAI 2007), Vancouver, pp. 25–32 (2007)

    Google Scholar 

  17. Broecheler, M., Mihalkova, L., Getoor, L.: Probabilistic similarity logic. In: Uncertainty in Artificial Intelligence (UAI), pp. 73–82 (2010)

    Google Scholar 

  18. Burges. C.: A tutorial on support vector machines for pattern recognition. In: Data Mining and Knowledge Discovery, vol. 2. Kluwer Academic Publishers, Boston (1998)

    Google Scholar 

  19. Campigotto, P., Battiti, R., Passerini, A.: Learning modulo theories for preference elicitation in hybrid domains. arXiv (2015)

    Google Scholar 

  20. Chavira, M., Darwiche, A.: On probabilistic inference by weighted model counting. Artif. Intell. 172(6–7), 772–799 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Choi, J., Amir, E.: Lifted relational variational inference. In: de Freitas, N., Murphy, K.P. (eds.) UAI 2012: Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, pp. 196–206. AUAI Press (2012)

    Google Scholar 

  22. Choi, J., Guzmn-Rivera, A., Amir, E.: Lifted relational kalman filtering. In: Proceedings of IJCAI 2011, pp. 2092–2099 (2011)

    Google Scholar 

  23. Choi, J., Hill, D., Amir, E.: Lifted inference for relational continuous models. In: UAI 2010: Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (2010)

    Google Scholar 

  24. Cimatti, A., Franzén, A., Griggio, A., Sebastiani, R., Stenico, C.: Satisfiability modulo the theory of costs: foundations and applications. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 99–113. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12002-2_8

    Chapter  Google Scholar 

  25. Cimatti, A., Griggio, A., Schaafsma, B.J., Sebastiani, R.: A modular approach to MaxSAT modulo theories. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 150–165. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39071-5_12

    Chapter  Google Scholar 

  26. Diligenti, M., Gori, M., Maggini, M., Rigutini, L.: Bridging logic and kernel machines. Mach. Learn. 86(1), 57–88 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Domshlak, C., Hüllermeier, E., Kaci, S., Prade, H.: Preferences in AI: an overview. Artif. Intell. 175(7–8), 1037–1052 (2011)

    Article  MathSciNet  Google Scholar 

  28. Fränzle, M., Herde, C., Teige, T., Ratschan, S., Schubert, T.: Efficient solving of large non-linear arithmetic constraint systems with complex boolean structure. JSAT 1(3–4), 209–236 (2007)

    MATH  Google Scholar 

  29. Friedman, J., Hastie, T., Rosset, S., Tibshirani, R.: Discussion of boosting papers. Ann. Stat. 32, 102–107 (2004)

    MATH  Google Scholar 

  30. Gelain, M., Pini, M., Rossi, F., Venable, K., Wilson, N.: Interval-valued soft constraint problems. Ann. Math. Artif. Intell. 58, 261–298 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Gelain, M., Pini, M.S., Rossi, F., Venable, K.B., Walsh, T.: Elicitation strategies for soft constraint problems with missing preferences: properties, algorithms and experimental studies. Artif. Intell. J. 174(3–4), 270–294 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  32. Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning). The MIT Press, Cambridge (2007)

    MATH  Google Scholar 

  33. Goodman, N.D., Mansinghka, V.K., Roy, D.M., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: McAllester, D.A., Myllymäki, P. (eds.) UAI, pp. 220–229. AUAI Press (2008)

    Google Scholar 

  34. Griggio, A., Phan, Q.-S., Sebastiani, R., Tomasi, S.: Stochastic local search for SMT: combining theory solvers with WalkSAT. In: Tinelli, C., Sofronie-Stokkermans, V. (eds.) FroCoS 2011. LNCS (LNAI), vol. 6989, pp. 163–178. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24364-6_12

    Chapter  Google Scholar 

  35. Guo, S., Sanner, S.: Real-time multiattribute Bayesian preference elicitation with pairwise comparison queries. J. Mach. Learn. Res. - Proc. Track 9, 289–296 (2010)

    Google Scholar 

  36. Gutmann, B., Jaeger, M., Raedt, L.: Extending ProbLog with continuous distributions. In: Frasconi, P., Lisi, F.A. (eds.) ILP 2010. LNCS (LNAI), vol. 6489, pp. 76–91. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21295-6_12

    Chapter  Google Scholar 

  37. Hausner, A.: Simulating decorative mosaics. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2001, pp. 573–580. ACM, New York (2001)

    Google Scholar 

  38. Islam, M.A., Ramakrishnan, C.R., Ramakrishnan, I.V.: Inference in probabilistic logic programs with continuous random variables. Theory Pract. Log. Program. 12(4–5), 505–523 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  39. Joachims, T.: Optimizing search engines using click through data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002, pp. 133–142. ACM, New York (2002)

    Google Scholar 

  40. Joachims, T., Finley, T., Chun-Nam John, Y.: Cutting-plane training of structural SVMs. Mach. Learn. 77(1), 27–59 (2009)

    Article  MATH  Google Scholar 

  41. Jovanović, D., Moura, L.: Solving non-linear arithmetic. In: Gramlich, B., Miller, D., Sattler, U. (eds.) IJCAR 2012. LNCS (LNAI), vol. 7364, pp. 339–354. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31365-3_27

    Chapter  Google Scholar 

  42. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. The MIT Press, Cambridge (2009)

    Google Scholar 

  43. Kruglov, E.: Superposition modulo theory. Ph.D. thesis, Universität des Saarlandes, Postfach 151141, 66041 Saarbrücken (2013)

    Google Scholar 

  44. Kuželka, O., Szabóová, A., Holec, M., Železný, F.: Gaussian logic for predictive classification. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 277–292. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23783-6_18

    Chapter  Google Scholar 

  45. Lallouet, A., Lopez, M., Martin, L., Vrain, C.: On learning constraint problems. In: ICTAI (1) 2010, pp. 45–52 (2010)

    Google Scholar 

  46. Larraz, D., Oliveras, A., Rodríguez-Carbonell, E., Rubio, A.: Minimal-model-guided approaches to solving polynomial constraints and extensions. In: Sinz, C., Egly, U. (eds.) SAT 2014. LNCS, vol. 8561, pp. 333–350. Springer, Heidelberg (2014). doi:10.1007/978-3-319-09284-3_25

    Google Scholar 

  47. LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., Huang, F.-J.: A tutorial on energy-based learning. In: Bakir, G., Hofman, T., Schölkopf, B., Smola, A., Taskar, B. (eds.) Predicting Structured Data. MIT Press, Cambridge (2006)

    Google Scholar 

  48. Leenen, L., Anbulagan, A., Meyer, T., Ghose, A.: Modeling and solving semiring constraint satisfaction problems by transformation to weighted semiring Max-SAT. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 202–212. Springer, Heidelberg (2007). doi:10.1007/978-3-540-76928-6_22

    Chapter  Google Scholar 

  49. Li, Y., Albarghouthi, A., Kincad, Z., Gurfinkel, A., Chechik, M.: Symbolic optimization with SMT solvers. In: Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2014, pp. 607–618. ACM, New York (2014)

    Google Scholar 

  50. Lippi, M., Frasconi, P.: Prediction of protein-residue contacts by markov logic networks with grounding-specific weights. Bioinformatics 25(18), 2326–2333 (2009)

    Article  Google Scholar 

  51. March, J.G.: Bounded rationality, ambiguity, and the engineering of choice. Bell J. Econ. 9, 587–608 (1978)

    Article  Google Scholar 

  52. McCormack, J., d’Inverno, M. (eds.): Computers and Creativity. Springer, Heidelberg (2012)

    Google Scholar 

  53. Nieuwenhuis, R., Oliveras, A.: On SAT modulo theories and optimization problems. In: Biere, A., Gomes, C.P. (eds.) SAT 2006. LNCS, vol. 4121, pp. 156–169. Springer, Heidelberg (2006). doi:10.1007/11814948_18

    Chapter  Google Scholar 

  54. Nrman, P., Buschle, M., Knig, J., Johnson, P.: Hybrid probabilistic relational models for system quality analysis. In: EDOC, pp. 57–66. IEEE Computer Society (2010)

    Google Scholar 

  55. Ravkic, I., Ramon, J., Davis, J.: Learning relational dependency networks in hybrid domains. Mach. Learn. 100(2), 217–254 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  56. Ricci, F., Rokach, L., Shapira, B., Kantor, P.B.: Recommender Systems Handbook, 1st edn. Springer, New York (2010)

    MATH  Google Scholar 

  57. Rümmer, P.: A constraint sequent calculus for first-order logic with linear integer arithmetic. In: Cervesato, I., Veith, H., Voronkov, A. (eds.) LPAR 2008. LNCS (LNAI), vol. 5330, pp. 274–289. Springer, Heidelberg (2008). doi:10.1007/978-3-540-89439-1_20

    Chapter  Google Scholar 

  58. Savage, L.J.: The theory of statistical decision. J. Am. Stat. Assoc. 46(253), 55–67 (1951)

    Article  MATH  Google Scholar 

  59. Sebastiani, R.: Lazy satisfiability modulo theories. J. Satisfiability, Boolean Model. Comput., JSAT 3(3–4), 141–224 (2007)

    MathSciNet  MATH  Google Scholar 

  60. Sebastiani, R., Tomasi, S.: Optimization in SMT with \({\cal{LA}}\)(\(\mathbb{Q}\)) cost functions. In: Gramlich, B., Miller, D., Sattler, U. (eds.) IJCAR 2012. LNCS (LNAI), vol. 7364, pp. 484–498. Springer, Heidelberg (2012). doi:10.1007/978-3-642-31365-3_38

    Chapter  Google Scholar 

  61. Sebastiani, R., Tomasi, S.: Optimization modulo theories with linear rational costs. ACM Trans. Comput. Logic 16(2), 12:1–12:43 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  62. Sebastiani, R., Trentin, P.: OptiMathSAT: a tool for optimization modulo theories. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 447–454. Springer, Heidelberg (2015). doi:10.1007/978-3-319-21690-4_27

    Chapter  Google Scholar 

  63. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)

    Book  MATH  Google Scholar 

  64. Teso, S., Sebastiani, R., Passerini, A.: Structured learning modulo theories. Artif. Intell. (2015)

    Google Scholar 

  65. Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005)

    MathSciNet  MATH  Google Scholar 

  66. Vanderplaats, G.N.: Multidiscipline Design Optimization: Textbook. Vanderplaats Research & Development, Incorporated, Colorado Springs (2007)

    Google Scholar 

  67. Viappiani, P.: Monte Carlo methods for preference learning. In: Hamadi, Y., Schoenauer, M. (eds.) LION 6. LNCS, vol. 7219, pp. 503–508. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34413-8_52

    Chapter  Google Scholar 

  68. Viappiani, P., Boutilier, C.: Optimal Bayesian recommendation sets and myopically optimal choice query sets. In: Advances in Neural Information Processing Systems 23 (NIPS), Vancouver, pp. 2352–2360 (2010)

    Google Scholar 

  69. Wang, J., Domingos, P.: Hybrid Markov logic networks. In: Proceedings of the 23rd National Conference on Artificial Intelligence, vol. 2, AAAI 2008, pp. 1106–1111. AAAI Press (2008)

    Google Scholar 

  70. Yang, Y.-L., Wang, J., Vouga, E., Wonka, P.: Urban pattern: layout design by hierarchical domain splitting. ACM Trans. Graph. 32(6), 181:1–181:12 (2013)

    Article  Google Scholar 

  71. Yu, C.-N.J., Joachims, T.: Learning structural SVMs with latent variables. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 1169–1176. ACM, New York (2009)

    Google Scholar 

  72. Yu, L.-F., Yeung, S.-K., Tang, C.-K., Terzopoulos, D., Chan, T.F., Osher, S.J.: Make it home: automatic optimization of furniture arrangement. ACM Trans. Graph. 30(4), 86:1–86:12 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

This chapter builds on a body of work done in collaboration with Paolo Campigotto, Roberto Battiti, Stefano Teso and Roberto Sebastiani.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, University of Trento, Trento, Italy

    Andrea Passerini

Authors
  1. Andrea Passerini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Passerini .

Editor information

Editors and Affiliations

  1. Université Montpellier 2, Montpellier, France

    Christian Bessiere

  2. KU Leuven, Heverlee, Belgium

    Luc De Raedt

  3. University of British Columbia, Vancouver, British Columbia, Canada

    Lars Kotthoff

  4. Université Catholique de Louvain, Louvain-la-Neuve, Belgium

    Siegfried Nijssen

  5. University College Cork, Cork, Ireland

    Barry O'Sullivan

  6. University of Pisa, Pisa, Italy

    Dino Pedreschi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this chapter

Cite this chapter

Passerini, A. (2016). Learning Modulo Theories. In: Bessiere, C., De Raedt, L., Kotthoff, L., Nijssen, S., O'Sullivan, B., Pedreschi, D. (eds) Data Mining and Constraint Programming. Lecture Notes in Computer Science(), vol 10101. Springer, Cham. https://doi.org/10.1007/978-3-319-50137-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50137-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50136-9

  • Online ISBN: 978-3-319-50137-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation