City Research Online

Instilling moral value alignment by means of multi-objective reinforcement learning

Rodriguez-Soto, M., Serramia, M. ORCID: 0000-0003-0993-024X, Lopez-Sanchez, M. & Antonio Rodriguez-Aguilar, J. (2022). Instilling moral value alignment by means of multi-objective reinforcement learning. Ethics and Information Technology, 24(1), article number 9. doi: 10.1007/s10676-022-09635-0


AI research is being challenged with ensuring that autonomous agents learn to behave ethically, namely in alignment with moral values. Here, we propose a novel way of tackling the value alignment problem as a two-step process. The first step consists on formalising moral values and value aligned behaviour based on philosophical foundations. Our formalisation is compatible with the framework of (Multi-Objective) Reinforcement Learning, to ease the handling of an agent’s individual and ethical objectives. The second step consists in designing an environment wherein an agent learns to behave ethically while pursuing its individual objective. We leverage on our theoretical results to introduce an algorithm that automates our two-step approach. In the cases where value-aligned behaviour is possible, our algorithm produces a learning environment for the agent wherein it will learn a value-aligned behaviour.

Publication Type: Article
Additional Information: This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
Publisher Keywords: Value alignment, Reinforcement learning, Multi-objective reinforcement learning, Ethics
Subjects: B Philosophy. Psychology. Religion > B Philosophy (General)
B Philosophy. Psychology. Religion > BJ Ethics
H Social Sciences > HM Sociology
H Social Sciences > HN Social history and conditions. Social problems. Social reform
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
T Technology
Departments: School of Science & Technology > Computer Science
SWORD Depositor:
[thumbnail of s10676-022-09635-0.pdf]
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login