City Research Online

Why Black-Box Bayesian Safety Assessment of Autonomous Vehicles is Problematic and What Can be Done About it?

Popov, P. ORCID: 0000-0002-3434-5272 (2025). Why Black-Box Bayesian Safety Assessment of Autonomous Vehicles is Problematic and What Can be Done About it?. IEEE Transactions on Intelligent Vehicles,

Abstract

This paper deals with the Bayesian safety assessment of autonomous vehicles (AV) conducted via driving AVs on the public roads, often referred to as “driving to safety.” A key safety measure is the probability of catastrophic failure (i.e., a road accident) per mile of driving (pfm), assumed a random variable.

We argue that a Bayesian prediction based on a univariate (“black-box”) probabilistic model has an intrinsic deficiency: it cannot accommodate the variation of pfm due to changing road conditions, which in turn may affect significantly the predicted pfm and may lead to optimistic predictions.

A multivariate probabilistic model is developed to overcome this limitation of the univariate model. Using a set of contrived examples the predictions of the multivariate model are compared with those derived with univariate models. Our results provide an intriguing insight that even when AV driving does not lead to accidents at all, the pfm predictions with the multivariate model may be more pessimistic than the assumed prior, and those derived with a black-box model, including the predictions using the recently developed “conservative Bayesian inference”.

The multivariate Bayesian safety assessment can be applied to autonomous vehicles and to other complex intelligent systems such as robots, UAVs, etc., where the operating conditions vary.

Publication Type: Article
Additional Information: © 2025 IEEE. For the purpose of open access, the author(s) has applied a Creative Commons Attribution (CC BY) license to any Accepted Manuscript version arising.
Publisher Keywords: Autonomous vehicle, Bayesian inference, “driving to safety,” Safety Assessment
Subjects: H Social Sciences > HD Industries. Land use. Labor > HD61 Risk Management
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
T Technology > TL Motor vehicles. Aeronautics. Astronautics
Departments: School of Science & Technology
School of Science & Technology > Department of Computer Science
School of Science & Technology > Department of Computer Science > Software Reliability
SWORD Depositor:
[thumbnail of IEEE_Format_BlackBoxCriticism_revised2_clean.pdf]
Preview
Text - Accepted Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login