Skip to main content
SpringerLink
Log in

A Triple Uniqueness of the Maximum Entropy Approach

  • Conference paper
  • First Online:
Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12897))

Abstract

Inductive logic is concerned with assigning probabilities to sentences given probabilistic constraints. The Maximum Entropy Approach to inductive logic I here consider assigns probabilities to all sentences of a first order predicate logic. This assignment is built on an application of the Maximum Entropy Principle, which requires that probabilities for uncertain inference have maximal Shannon Entropy. This paper puts forward two different modified applications of this principle to first order predicate logic and shows that the original and the two modified applications agree in many cases. A third promising modification is studied and rejected.

I gratefully acknowledge funding from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 432308570 and 405961989.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the equivocator function is the unique probability function in \(\mathbb P\) which is uniform over all \(\varOmega _n\), \(P_=(\omega _n)=\frac{1}{|\varOmega _n|}\) for all n and all \(\omega _n\in \varOmega _n\). The name for this function is derived from the fact that it is maximally equivocal. The function has also been given other names. In Pure Inductive Logic it is known as the completely independent probability function and is often denoted by \(c_\infty \) in reference to the role it plays in Carnap’s famous continuum of inductive methods [3].

References

  1. Barnett, O., Paris, J.B.: Maximum entropy inference with quantified knowledge. Logic J. IGPL 16(1), 85–98 (2008). https://doi.org/10.1093/jigpal/jzm028

  2. Carnap, R.: The two concepts of probability: the problem of probability. Philos. Phenomenological Res. 5(4), 513–532 (1945). https://doi.org/10.2307/2102817

  3. Carnap, R.: The Continuum of Inductive Methods. Chicago University of Chicago Press, Chicago (1952)

    MATH  Google Scholar 

  4. Crupi, V.: Inductive logic. J. Philos. Logic 44(6), 641–650 (2015). https://doi.org/10.1007/s10992-015-9348-8

  5. Crupi, V., Nelson, J., Meder, B., Cevolani, G., Tentori, K.: Generalized information theory meets human cognition: introducing a unified framework to model uncertainty and information search. Cogn. Sci. 42, 1410–1456 (2018). https://doi.org/10.1111/cogs.12613

  6. Csiszár, I.: Axiomatic characterizations of information measures. Entropy 10(3), 261–273 (2008). https://doi.org/10.3390/e10030261

  7. Cui, H., Liu, Q., Zhang, J., Kang, B.: An improved deng entropy and its application in pattern recognition. IEEE Access 7, 18284–18292 (2019). https://doi.org/10.1109/access.2019.2896286

  8. Gaifman, H.: Concerning measures in first order calculi. Isr. J. Math. 2(1), 1–18 (1964). https://doi.org/10.1007/BF02759729

  9. Groves, T.: Lakatos’s criticism of Carnapian inductive logic was mistaken. J. Appl. Logic 14, 3–21 (2016). https://doi.org/10.1016/j.jal.2015.09.014

    Article  MathSciNet  MATH  Google Scholar 

  10. Grünwald, P.D., Dawid, A.P.: Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. Ann. Stat. 32(4), 1367–1433 (2004). https://doi.org/10.1214/009053604000000553

    Article  MathSciNet  MATH  Google Scholar 

  11. Haenni, R., Romeijn, J.W., Wheeler, G., Williamson, J.: Probabilistic Argumentation, Synthese Library, vol. 350. Springer, Dordrecht (2011). https://doi.org/10.1007/978-94-007-0008-6_3

  12. Halpern, J.Y., Koller, D.: Representation dependence in probabilistic inference. J. Artif. Intell. Res. 21, 319–356 (2004). https://doi.org/10.1613/jair.1292

    Article  MathSciNet  MATH  Google Scholar 

  13. Hanel, R., Thurner, S., Gell-Mann, M.: Generalized entropies and the transformation group of superstatistics. Proc. Nat. Acad. Sci. 108(16), 6390–6394 (2011). https://doi.org/10.1073/pnas.1103539108

    Article  MathSciNet  MATH  Google Scholar 

  14. Howarth, E., Paris, J.B.: Pure inductive logic with functions. J. Symbolic Logic, 1–22 (2019). https://doi.org/10.1017/jsl.2017.49

  15. Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  16. Kließ, M.S., Paris, J.B.: Second order inductive logic and Wilmers’ principle. J. Appl. Logic 12(4), 462–476 (2014). https://doi.org/10.1016/j.jal.2014.07.002

    Article  MathSciNet  MATH  Google Scholar 

  17. Landes, J.: The Principle of Spectrum Exchangeability within Inductive Logic. Ph.D. thesis, Manchester Institute for Mathematical Sciences (2009). https://jlandes.files.wordpress.com/2015/10/phdthesis.pdf

  18. Landes, J.: Probabilism, entropies and strictly proper scoring rules. Int. J. Approximate Reason. 63, 1–21 (2015). https://doi.org/10.1016/j.ijar.2015.05.007

    Article  MathSciNet  MATH  Google Scholar 

  19. Landes, J.: The entropy-limit (Conjecture) for \(\Sigma _2\)-premisses. Stud. Logica. 109, 423–442 (2021). https://doi.org/10.1007/s11225-020-09912-3

    Article  MathSciNet  MATH  Google Scholar 

  20. Landes, J., Masterton, G.: Invariant equivocation. Erkenntnis 82, 141–167 (2017). https://doi.org/10.1007/s10670-016-9810-1

    Article  MathSciNet  MATH  Google Scholar 

  21. Landes, J., Paris, J., Vencovská, A.: Language invariance and spectrum exchangeability in inductive logic. In: Mellouli, K. (ed.) ECSQARU 2007. LNCS (LNAI), vol. 4724, pp. 151–160. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75256-1_16

    Chapter  MATH  Google Scholar 

  22. Landes, J., Paris, J.B., Vencovská, A.: Some aspects of polyadic inductive logic. Stud. Logica 90(1), 3–16 (2008). https://doi.org/10.1007/s11225-008-9140-7

    Article  MathSciNet  MATH  Google Scholar 

  23. Landes, J., Paris, J.B., Vencovská, A.: Representation theorems for probability functions satisfying spectrum exchangeability in inductive logic. Int. J. Approximate Reason. 51(1), 35–55 (2009). https://doi.org/10.1016/j.ijar.2009.07.001

    Article  MathSciNet  MATH  Google Scholar 

  24. Landes, J., Paris, J.B., Vencovská, A.: A survey of some recent results on spectrum exchangeability in polyadic inductive logic. Synthese 181, 19–47 (2011). https://doi.org/10.1007/s11229-009-9711-9

    Article  MATH  Google Scholar 

  25. Landes, J., Rafiee Rad, S., Williamson, J.: Towards the entropy-limit conjecture. Ann. Pure Appl. Logic 172, 102870 (2021). https://doi.org/10.1016/j.apal.2020.102870

    Article  MathSciNet  MATH  Google Scholar 

  26. Landes, J., Rafiee Rad, S., Williamson, J.: Determining maximal entropy functions for objective Bayesian inductive logic (2022). Manuscript

    Google Scholar 

  27. Landes, J., Williamson, J.: Objective Bayesianism and the maximum entropy principle. Entropy 15(9), 3528–3591 (2013). https://doi.org/10.3390/e15093528

    Article  MathSciNet  MATH  Google Scholar 

  28. Landes, J., Williamson, J.: Justifying objective Bayesianism on predicate languages. Entropy 17(4), 2459–2543 (2015). https://doi.org/10.3390/e17042459

    Article  MathSciNet  Google Scholar 

  29. Niiniluoto, I.: The development of the Hintikka program. In: Gabbay, D.M., Hartmann, S., Woods, J. (eds.) Handbook of the History of Logic, pp. 311–356. Elsevier, Kidlington (2011). https://doi.org/10.1016/b978-0-444-52936-7.50009-4

  30. Ognjanović, Z., Rašković, M., Marković, Z.: Probability Logics. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47012-2

    Book  MATH  Google Scholar 

  31. Ortner, R., Leitgeb, H.: Mechanizing induction. In: Gabbay, D.M., Hartmann, S., Woods, J. (eds.) Handbook of the History of Logic, pp. 719–772. Elsevier (2011). https://doi.org/10.1016/b978-0-444-52936-7.50018-5

  32. Paris, J.B.: Common sense and maximum entropy. Synthese 117, 75–93 (1998). https://doi.org/10.1023/A:1005081609010

    Article  MathSciNet  MATH  Google Scholar 

  33. Paris, J.B.: The Uncertain Reasoner’s Companion: A Mathematical Perspective, 2 edn. Cambridge Tracts in Theoretical Computer Science, vol. 39. Cambridge University Press, Cambridge (2006)

    Google Scholar 

  34. Paris, J.B.: What you see is what you get. Entropy 16(11), 6186–6194 (2014). https://doi.org/10.3390/e16116186

    Article  Google Scholar 

  35. Paris, J.B., Rad, S.R.: A note on the least informative model of a theory. In: Ferreira, F., Löwe, B., Mayordomo, E., Mendes Gomes, L. (eds.) CiE 2010. LNCS, vol. 6158, pp. 342–351. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13962-8_38

    Chapter  Google Scholar 

  36. Paris, J.B., Vencovská, A.: On the applicability of maximum entropy to inexact reasoning. Int. J. Approximate Reason. 3(1), 1–34 (1989). https://doi.org/10.1016/0888-613X(89)90012-1

    Article  MathSciNet  MATH  Google Scholar 

  37. Paris, J.B., Vencovská, A.: A note on the inevitability of maximum entropy. Int. J. Approximate Reason. 4(3), 183–223 (1990). https://doi.org/10.1016/0888-613X(90)90020-3

    Article  MathSciNet  MATH  Google Scholar 

  38. Paris, J.B., Vencovská, A.: In defense of the maximum entropy inference process. Int. J. Approximate Reason. 17(1), 77–103 (1997). https://doi.org/10.1016/S0888-613X(97)00014-5

    Article  MathSciNet  MATH  Google Scholar 

  39. Paris, J.B., Vencovská, A.: Common sense and stochastic independence. In: Corfield, D., Williamson, J. (eds.) Foundations of Bayesianism, pp. 203–240. Kluwer, Dordrecht (2001)

    Chapter  Google Scholar 

  40. Paris, J.B., Vencovská, A.: Pure Inductive Logic. Cambridge University Press, Cambridge (2015)

    Book  Google Scholar 

  41. Paris, J.B., Vencovská, A.: Six problems in pure inductive logic. J. Philos. Logic (2019). https://doi.org/10.1007/s10992-018-9492-z

  42. Rafiee Rad, S.: Inference processes for first order probabilistic languages. Ph.D. thesis, Manchester Institute for Mathematical Sciences (2009). http://www.rafieerad.org/manthe.pdf

  43. Rafiee Rad, S.: Equivocation axiom on first order languages. Stud. Logica. 105(1), 121–152 (2017). https://doi.org/10.1007/s11225-016-9684-x

    Article  MathSciNet  MATH  Google Scholar 

  44. Rafiee Rad, S.: Maximum entropy models for \(\Sigma _1\) sentences. J. Logics Appl. 5(1), 287–300 (2018). http://www.collegepublications.co.uk/admin/download.php?ID=ifcolog00021

  45. Williamson, J.: Objective Bayesian probabilistic logic. J. Algorithms 63(4), 167–183 (2008). https://doi.org/10.1016/j.jalgor.2008.07.001

    Article  MathSciNet  MATH  Google Scholar 

  46. Williamson, J.: Objective Bayesianism with predicate languages. Synthese 163(3), 341–356 (2008). https://doi.org/10.1007/s11229-007-9298-y

    Article  MathSciNet  MATH  Google Scholar 

  47. Williamson, J.: In Defence of Objective Bayesianism. Oxford University Press, Oxford (2010)

    Google Scholar 

  48. Williamson, J.: Lectures on Inductive Logic. Oxford University Press, Oxford (2017)

    Book  Google Scholar 

Download references

Acknowledgements

Many thanks to Jeff Paris, Soroush Rafiee Rad, Alena Vencovská and Jon Williamson for continued collaboration on maximum entropy inference. I’m also indebted to anonymous referees who helped me improve this paper.

Author information

Authors and Affiliations

  1. Munich Center for Mathematical Philosophy, LMU Munich, Munich, Germany

    Jürgen Landes

Authors
  1. Jürgen Landes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jürgen Landes .

Editor information

Editors and Affiliations

  1. Institute of Information Theory and Automation, Prague, Czech Republic

    Jiřina Vejnarová

  2. Insight Centre for Data Analytics, School of Computer Science and IT University College Cork, Cork, Ireland

    Nic Wilson

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Landes, J. (2021). A Triple Uniqueness of the Maximum Entropy Approach. In: Vejnarová, J., Wilson, N. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2021. Lecture Notes in Computer Science(), vol 12897. Springer, Cham. https://doi.org/10.1007/978-3-030-86772-0_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86772-0_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86771-3

  • Online ISBN: 978-3-030-86772-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation