City Research Online

Impact of Machine Learning on Safety Monitors

Terrosi, F., Strigini, L. ORCID: 0000-0002-4246-2866 & Bondavalli, A. (2022). Impact of Machine Learning on Safety Monitors. In: Lecture Notes in Computer Science. 41st International Conference, SAFECOMP 2022, 6-9 Sep 2022, Munich, Germnay. doi: 10.1007/978-3-031-14835-4_9


Machine Learning components in safety-critical applications can perform some complex tasks that would be unfeasible otherwise. However, they are also a weak point concerning safety assurance. An aspect requiring study is how the interactions between machine-learning components and other non-ML components evolve with training of the former. It is theoretically possible that learning by Neural Networks may reduce the effectiveness of error checkers or safety monitors, creating a major complication for safety assurance. We present an initial exploration of this problem focused on automated driving, where machine learning is heavily used. We simulated operational testing of a standard vehicle architecture, where a machine learning-based Controller is responsible for driving the vehicle and a separate Safety Monitor is provided to detect hazardous situations and trigger emergency action to avoid accidents. Among the results, we observed that indeed improving the Controller could make the Safety Monitor less effective; it is even possible for a training increment to make the Controller’s own behaviour safer but the vehicle’s less safe. We discuss implications for practice and for research.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: This version of the contribution has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: Use of this Accepted Version is subject to the publisher’s Accepted Manuscript terms of use
Publisher Keywords: safety, autonomous vehicles automotive, machine-learning
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology > Computer Science > Software Reliability
[thumbnail of SAFECOMP22_paper_10.pdf]
Text - Accepted Version
Download (392kB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login