City Research Online

Abstract

This thesis investigates how Reinforcement Learning agents can systematically leverage the Free Energy Principle to enhance intrinsic motivation, effectively differentiate between uncertainty types, and identify latent causal structures underlying sensory data. The central hypothesis is that aligning intrinsic motivation and exploration with the theoretical framework of active inference, as defined by the Free Energy Principle, can yield significant theoretical clarity and empirical improvements over existing heuristic approaches.

First, existing intrinsic motivation methods, including prediction error, information gain, and compression progress, are reframed within active inference. By treating exploration as a natural consequence of minimizing expected free energy, the thesis demonstrates improved exploratory efficiency and theoretical grounding. Developed methods illustrate how intrinsic rewards can be systematically derived from variational principles, consistently achieving robust exploration in sparse reward scenarios. Second, the thesis explicitly differentiates aleatoric (irreducible environmental noise) from epistemic uncertainty (incomplete knowledge) within Reinforcement Learning contexts. The approaches utilize inverse dynamics models and variational inference to prioritize exploration aimed specifically at epistemic uncertainty reduction, thereby enhancing both exploration quality and agent robustness. Empirical results consistently indicate improved task performance and adaptive behavior through this principled uncertainty management. Finally, the thesis addresses how agents can identify latent causal structures underlying sensory data by systematically guiding exploration toward experiences that resolve uncertainty about hidden environmental variables. The research demonstrates that structured latent representations informed by active inference facilitate effective discovery and encoding of task-specific parameters and environmental dynamics.

Publication Type:	Thesis (Doctoral)
Subjects:	B Philosophy. Psychology. Religion > BF Psychology Q Science Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Departments:	School of Science & Technology > Department of Computer Science School of Science & Technology > School of Science & Technology Doctoral Theses Doctoral Theses

Export

Downloads