City Research Online

Retrospective model-based inference guides model-free credit assignment

Moran, R., Keramati, M. ORCID: 0000-0002-1120-5867, Dayan, P. and Dolan, R.J. (2019). Retrospective model-based inference guides model-free credit assignment. Nature Communications, 10(1), 750.. doi: 10.1038/s41467-019-08662-8

Abstract

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions.

Publication Type: Article
Subjects: B Philosophy. Psychology. Religion > BF Psychology
R Medicine > RC Internal medicine > RC0321 Neuroscience. Biological psychiatry. Neuropsychiatry
Departments: School of Arts & Social Sciences > Psychology
URI: https://openaccess.city.ac.uk/id/eprint/23482
[img]
Preview
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login