Active inference and learning

Friston, K. J., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O'Doherty, J. & Pezzulo, G. (2016). Active inference and learning. Neuroscience & Biobehavioral Reviews, 68, pp. 862-879. doi: 10.1016/j.neubiorev.2016.06.022

[img]
Preview
Text - Published Version
Available under License Creative Commons Attribution.

Download (5MB) | Preview

Abstract

This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.

Item Type: Article
Uncontrolled Keywords: Active inference; Habit learning; Bayesian inference; Goal-directed; Free energy; Information gain; Bayesian surprise; Epistemic value; Exploration; Exploitation
Subjects: B Philosophy. Psychology. Religion > BF Psychology
Divisions: School of Social Sciences > Department of Psychology
URI: http://openaccess.city.ac.uk/id/eprint/16681

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics