City Research Online

Active inference and learning

Friston, K. J., FitzGerald, T., Rigoli, F. , Schwartenbeck, P., O'Doherty, J. & Pezzulo, G. (2016). Active inference and learning. Neuroscience & Biobehavioral Reviews, 68, pp. 862-879. doi: 10.1016/j.neubiorev.2016.06.022

Abstract

This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.

Publication Type: Article
Publisher Keywords: Active inference; Habit learning; Bayesian inference; Goal-directed; Free energy; Information gain; Bayesian surprise; Epistemic value; Exploration; Exploitation
Subjects: B Philosophy. Psychology. Religion > BF Psychology
Departments: School of Health & Psychological Sciences > Psychology
SWORD Depositor:
[thumbnail of act_inf_learn.pdf]
Preview
Text - Published Version
Available under License Creative Commons Attribution.

Download (5MB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login