City Research Online

Model based planners reflect on their model-free propensities

Moran, R. ORCID: 0000-0002-7641-2402, Keramati, M. ORCID: 0000-0002-1120-5867 & Dolan, R. J. (2021). Model based planners reflect on their model-free propensities. PLOS Computational Biology, 17(1), article number e1008552. doi: 10.1371/journal.pcbi.1008552


Dual-reinforcement learning theory proposes behaviour is under the tutelage of a retrospective, value-caching, model-free (MF) system and a prospective-planning, model-based (MB), system. This architecture raises a question as to the degree to which, when devising a plan, a MB controller takes account of influences from its MF counterpart. We present evidence that such a sophisticated self-reflective MB planner incorporates an anticipation of the influences its own MF-proclivities exerts on the execution of its planned future actions. Using a novel bandit task, wherein subjects were periodically allowed to design their environment, we show that reward-assignments were constructed in a manner consistent with a MB system taking account of its MF propensities. Thus, in the task participants assigned higher rewards to bandits that were momentarily associated with stronger MF tendencies. Our findings have implications for a range of decision making domains that includes drug abuse, pre-commitment, and the tension between short and long-term decision horizons in economics.

Publication Type: Article
Additional Information: © 2021 Moran et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Subjects: B Philosophy. Psychology. Religion > BF Psychology
H Social Sciences > HM Sociology
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QH Natural history > QH301 Biology
Departments: School of Health & Psychological Sciences
School of Health & Psychological Sciences > Psychology
SWORD Depositor:
[thumbnail of Model based planners reflect on their model-free propensities.pdf]
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login