City Research Online

S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay

Caselles-Dupre, H., Garcia-Ortiz, M. ORCID: 0000-0003-4729-7457 and Filliat, D. (2021). S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay. Proceedings of the International Joint Conference on Neural Networks, 2021-J, doi: 10.1109/IJCNN52387.2021.9533683 ISSN 2161-4393

Abstract

We consider the problem of building a state representation model for control, in a continual learning setting. As the environment changes, the aim is to efficiently compress the sensory state information without losing past knowledge, and then use Reinforcement Learning on the resulting features for efficient policy learning. To this end, we propose S-TRIGGER, a general method for Continual State Representation Learning applicable to Variational Auto-Encoders and its many variants. The method is based on Generative Replay, i.e. the use of generated samples to maintain past knowledge. It comes along with a statistically sound method for environment change detection, which self-triggers the Generative Replay. Our experiments on VAEs show that S-TRIGGER learns state representations that allows fast and high-performing Reinforcement Learning, while avoiding catastrophic forgetting. The resulting system has a bounded size and is capable of autonomously learning new information without using past data.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Publisher Keywords: Neural networks, Buildings, Reinforcement learning
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Mathematics, Computer Science & Engineering > Computer Science
Date available in CRO: 09 Dec 2021 11:51
Date deposited: 9 December 2021
Date of acceptance: 10 April 2021
Date of first online publication: 27 September 2021
URI: https://openaccess.city.ac.uk/id/eprint/27245
[img]
Preview
Text - Accepted Version
Download (1MB) | Preview

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login