Skip to main content
SpringerLink
Log in

A Model of Agential Learning Using Active Inference

  • Conference paper
  • First Online:
Active Inference (IWAI 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1915))

Included in the following conference series:

  • 196 Accesses

Abstract

Agential learning refers to the process of forming beliefs regarding one’s degree of control over actions and outcomes in their environment. We first provide an overview and evaluation of associative, statistical, and Bayesian models of agential learning. We then argue that the existing models have limitations in explaining the process of agential learning. Finally, we introduce an active inference account of agential learning, and present results from simulations. We propose that the active inference framework may provide a comprehensive model of agential learning describing three fundamental processes: (i) perception, (ii) learning, and (iii) action.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gallagher, S.: Philosophical conceptions of the self: implications for cognitive science. Trends Cogn. Sci. 4(1), 14–21 (2000)

    Article  Google Scholar 

  2. Haggard, P.: Sense of agency in the human brain. Nat. Rev. Neurosci. 18(4), 196–207 (2017)

    Article  Google Scholar 

  3. Albarracin, M., Pitliya, R.J.: The nature of beliefs and believing. Front. Psychol. 3 (2022)

    Google Scholar 

  4. Verschure, P.F., Pennartz, C.M., Pezzulo, G.: The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Philos. Trans. R. Soc. B Biol. Sci. 369(1655), 20130483 (2014)

    Google Scholar 

  5. Ferster, C.B.: The use of the free operant in the analysis of behavior. Psychol. Bull. 50(4), 263 (1953)

    Article  Google Scholar 

  6. Allan, L.G., Jenkins, H.M.: The judgment of contingency and the nature of the response alternatives. Can. J. Psychol./Revue canadienne de psychologie 34(1), 1 (1980)

    Article  Google Scholar 

  7. Shanks, D.R., Dickinson, A.: Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Memory Cogn. 19, 353–360 (1991)

    Article  Google Scholar 

  8. Wasserman, E.A., Chatlosh, D., Neunaber, D.: Perception of causal relations in humans: factors affecting judgments of response-outcome contingencies under free-operant procedures. Learn. Motiv. 14(4), 406–432 (1983)

    Article  Google Scholar 

  9. Wasserman, E.A., Elek, S.M., Chatlosh, D.L., Baker, A.G.: Rating causal relations: role of probability in judgments of response-outcome contingency. J. Exp. Psychol. Learn. Mem. Cogn. 19(1), 174 (1993)

    Article  Google Scholar 

  10. Vallée-Tourangeau, F., Murphy, R.A., Baker, A.: Contiguity and the outcome density bias in action-outcome contingency judgements. Q. J. Exp. Psychol. Sect. B 58(2b), 177–192 (2005)

    Google Scholar 

  11. Vallee-Tourangeau, F., Murphy, R.: Action-effect contingency judgment tasks foster normative causal reasoning. In: Proceedings of the Twenty First Annual Conference of the Cognitive Science Society, pp. 820–820 (1999)

    Google Scholar 

  12. Msetfi, R.M., Murphy, R.A., Simpson, J., Kornbrot, D.E.: Depressive realism and outcome density bias in contingency judgments: the effect of the context and intertrial interval. J. Exp. Psychol. Gen. 134(1), 10 (2005)

    Article  Google Scholar 

  13. Cheng, P.W.: From covariation to causation: a causal power theory. Psychol. Rev. 104(2), 367 (1997)

    Article  Google Scholar 

  14. Hume, D.: A treatise of human nature: Volume 1: Texts (1739)

    Google Scholar 

  15. Kant, I.: Critique of pure reason. 1781, Modern Classical Philosophers, Cambridge, MA, Houghton Mifflin, pp. 370–456 (1908)

    Google Scholar 

  16. Michotte, A.: The perception of causality. Routledge, vol. 21 (2017)

    Google Scholar 

  17. Shanks, D.R., Lopez, F.J., Darby, R.J., Dickinson, A.: Distinguishing associative and probabilistic contrast theories of human contingency judgment. In: Psychology of learning and motivation, vol. 34, pp. 265–311. Elsevier (1996)

    Google Scholar 

  18. De Houwer, J., Beckers, T.: A review of recent developments in research and theories on human contingency learning. Q. J. Exp. Psychol. Sect. B 55(4), 289–310 (2002)

    Article  Google Scholar 

  19. Pineño, O., Miller, R.R.: Comparing associative, statistical, and inferential reasoning accounts of human contingency learning. Q. J. Exp. Psychol. 60(3), 310–329 (2007)

    Article  Google Scholar 

  20. Shanks, D.R.: Associationism and cognition: human contingency learning at 25. Q. J. Exp. Psychol. 60(3), 291–309 (2007)

    Article  MathSciNet  Google Scholar 

  21. Mackintosh, N.J.: A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 82(4), 276 (1975)

    Article  Google Scholar 

  22. Miller, R.R., Matzel, L.D.: The comparator hypothesis: a response rule for the expression of associations. In: Psychology of learning and motivation, vol. 22, pp. 51–92. Elsevier (1988)

    Google Scholar 

  23. Pearce, J.M., Hall, G.: A model for pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87(6), 532 (1980)

    Article  Google Scholar 

  24. Rescorla, R.A.: A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. Class. Conditioning Curr. Res. Theor. 2, 64–69 (1972)

    Google Scholar 

  25. Wagner, A.R., Rescorla, R.A.: Inhibition in Pavlovian conditioning: application of a theory. In: Boakes, R.A., Halliday, M.S. (eds.) Inhibition and Learning. Academic Press, New York (1972)

    Google Scholar 

  26. Chapman, G.B.: Trial order affects cue interaction in contingency judgment. J. Exp. Psychol. Learn. Mem. Cogn. 17(5), 837 (1991)

    Article  Google Scholar 

  27. De Houwer, J., Beckers, T.: Higher-order retrospective revaluation in human causal learning. Q. J. Exp. Psychol. Sect. B 55(2b), 137–151 (2002)

    Article  Google Scholar 

  28. Dickinson, A.: Within compound associations mediate the retrospective revaluation of causality judgements. Q. J. Exp. Psychol. Sect. B 49(1), 60–80 (1996)

    Article  Google Scholar 

  29. Cheng, P.W., Novick, L.R.: Covariation in natural causal induction. Psychol. Rev. 99(2), 365 (1992)

    Article  Google Scholar 

  30. López, F.J., Almaraz, J., Fernández, P., Shanks, D.: Adquisición progresiva del conocimiento sobre relaciones predictivas: Curvas de aprendizaje en juicios de contingencia. Psicothema, pp. 337–349 (1999)

    Google Scholar 

  31. Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., Pezzulo, G.: Active inference: a process theory. Neural Comput. 29(1), 1–49 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Blanco, F., Matute, H., Vadillo, M.A.: Mediating role of activity level in the depressive realism effect (2012)

    Google Scholar 

  33. Blanco, F., Matute, H., Vadillo, M.A.: Interactive effects of the probability of the cue and the probability of the outcome on the overestimation of null contingency. Learn. Behav. 41(4), 333–340 (2013). https://doi.org/10.3758/s13420-013-0108-8

    Article  Google Scholar 

  34. Byrom, N., Msetfi, R., Murphy, R.: Two pathways to causal control: use and availability of information in the environment in people with and without signs of depression. Acta Physiol. (Oxf) 157, 1–12 (2015)

    Google Scholar 

  35. Griffiths, T.L., Tenenbaum, J.B.: Structure and strength in causal induction. Cogn. Psychol. 51(4), 334–384 (2005)

    Article  Google Scholar 

  36. Waldmann, M.R.: Competition among causes but not effects in predictive and diagnostic learning. J. Exp. Psychol. Learn. Mem. Cogn. 26(1), 53 (2000)

    Article  Google Scholar 

  37. Kruschke, J.K.: Bayesian approaches to associative learning: from passive to active learning. Learn. Behav. 36(3), 210–226 (2008). https://doi.org/10.3758/LB.36.3.210

    Article  Google Scholar 

  38. Tenenbaum, J.B., Griffiths, T.L., Kemp, C.: Theory-based bayesian models of inductive learning and reasoning. Trends Cogn. Sci. 10(7), 309–318 (2006)

    Article  Google Scholar 

  39. Chater, N., Oaksford, M., Hahn, U., Heit, E.: Bayesian models of cognition. Wiley Interdisc. Rev. Cogn. Sci. 1(6), 811–823 (2010)

    Article  Google Scholar 

  40. Doya, K., Ishii, S., Pouget, A., Rao, R.P.: Bayesian brain: Probabilistic Approaches to Neural Coding. MIT Press, Cambridge (2007)

    Google Scholar 

  41. Von Helmholtz, H.: Handbuch der physiologischen Optik. Voss, vol. 9 (1867)

    Google Scholar 

  42. Nelson, J.D.: Finding useful questions: on bayesian diagnosticity, probability, impact, and information gain. Psychol. Rev. 112(4), 979 (2005)

    Article  Google Scholar 

  43. De Ath, G., Everson, R.M., Rahat, A.A., Fieldsend, J.E.: Greed is good: exploration and exploitation trade-offs in bayesian optimisation. ACM Trans. Evol. Learn. Optim. 1(1), 1–22 (2021)

    Article  Google Scholar 

  44. Friston, K.: The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11(2), 127–138 (2010)

    Article  Google Scholar 

  45. Parr, T., Pezzulo, G., Friston, K.J.: Active Inference: the Free Energy Principle in Mind, Brain, and Behavior. MIT Press, Cambridge (2022)

    Google Scholar 

  46. Friston, K.J., Daunizeau, J., Kiebel, S.J.: Reinforcement learning or active inference? PLoS ONE 4(7), e6421 (2009)

    Article  Google Scholar 

  47. Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., Pezzulo, G.: Active inference and epistemic value. Cogn. Neurosci. 6(4), 187–214 (2015)

    Article  Google Scholar 

  48. Smith, R., Friston, K.J., Whyte, C.J.: A step-by-step tutorial on active inference and its application to empirical data. J. Math. Psychol. 107, 102–632 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  49. Heins, C., Millidge, B., Demekas, D., et al.: Pymdp: a python library for active inference in discrete state spaces, arXiv preprint arXiv:2201.03904 (2022)

  50. Friston, K.J., Rosch, R., Parr, T., Price, C., Bowman, H.: Deep temporal models and active inference. Neurosci. Biobehav. Rev. 90, 486–501 (2018)

    Article  Google Scholar 

  51. Blanco, F., Matute, H., Vadillo, M.A.: Depressive realism: wiser or quieter? Psychol. Record 59(4), 551–562 (2009)

    Article  Google Scholar 

  52. Friston, K.J., Parr, T., de Vries, B.: The graphical brain: belief propagation and active inference. Netw. Neurosci. 1(4), 381–414 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. VERSES Research Lab, Los Angeles, CA, 90016, USA

    Riddhi J. Pitliya

  2. Department of Experimental Psychology, University of Oxford, Oxford, UK

    Riddhi J. Pitliya & Robin A. Murphy

Authors
  1. Riddhi J. Pitliya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robin A. Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riddhi J. Pitliya .

Editor information

Editors and Affiliations

  1. University of Sussex, Brighton, UK

    Christopher L. Buckley

  2. La Sapienza University of Rome, Rome, Italy

    Daniela Cialfi

  3. Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands

    Pablo Lanillos

  4. VERSES Research Lab, Los Angeles, CA, USA

    Maxwell Ramstead

  5. University College London, London, UK

    Noor Sajid

  6. Kyoto University, Kyoto, Japan

    Hideaki Shimazaki

  7. VERSES Research Lab, Los Angeles, CA, USA

    Tim Verbelen

  8. Technische Universiteit Delft, Delft, The Netherlands

    Martijn Wisse

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pitliya, R.J., Murphy, R.A. (2024). A Model of Agential Learning Using Active Inference. In: Buckley, C.L., et al. Active Inference. IWAI 2023. Communications in Computer and Information Science, vol 1915. Springer, Cham. https://doi.org/10.1007/978-3-031-47958-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47958-8_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47957-1

  • Online ISBN: 978-3-031-47958-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation