City Research Online

Abstract

This paper presents Bayesian techniques for conservative claims about software reliability, particularly when evidence suggests the software’s executions are not statistically independent. We formalise informal notions of “doubting” that the executions are independent, and incorporate such doubts into reliability assessments. We develop techniques that reveal the extent to which independence assumptions can undermine conservatism in assessments, and identify conditions under which this impact is not significant. These techniques – novel extensions of conservative Bayesian inference (CBI) approaches – give conservative confidence bounds on the software’s failure probability per execution. With illustrations in two application areas – nuclear powerplant safety and autonomous vehicle (AV) safety – our analyses reveals: 1) the confidence an assessor should possess before subjecting a system to operational testing. Otherwise, such testing is futile – favourable operational testing evidence will eventually decrease one’s confidence in the system being sufficiently reliable; 2) the independence assumption supports conservative claims sometimes; 3) in some scenarios, observing a system operate without failure gives less confidence in the system than if some failures had been observed; 4) building confidence in a system is very sensitive to failures – each additional failure means significantly more operational testing is required, in order to support a reliability claim.

Publication Type:	Article
Additional Information:	This is the peer reviewed version of the following article: Salako, K. & Zhao, X. (2023). Demonstrating Software Reliability using Possibly Correlated Tests: Insights from a Conservative Bayesian Approach. Quality and Reliability Engineering International,, which is to be published in final form at http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1099-1638. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited.
Publisher Keywords:	reliability engineering, software reliability, conservative Bayesian inference, software testing, safety-critical systems
Subjects:	Q Science > QA Mathematics Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology > Computer Science > Software Reliability
SWORD Depositor:	Symplectic Administrator

Export

Downloads