City Research Online

Abstract

In many safety-critical applications, binary classifiers detect undesirable operational states. Optimal adjudication schemes may provide an affordable means of constructing Bayes-optimal hybrid classifiers (i.e. a classifier configuration with the lowest expected cost of classification errors) by combining the outputs from multiple classifiers. However, statistical uncertainty poses significant challenges when applying such schemes, so that their use cannot be guaranteed to produce truly optimal configurations. We present statistical methods for estimating the extent to which various adjudication schemes, including optimal adjudication, fail to produce optimal classifier configurations. Applying these methods to two critical contexts—pneumonia diagnosis and concrete defect detection—classical statistical bounds reveal how training-sample uncertainty, validation-sample uncertainty, and the relative cost of classification errors jointly constrain both the efficacy of adjudication schemes and the confidence that can be placed in the configurations they produce. So-called “smoothed” hybrid classifiers consistently give the lowest risk with the tightest optimality confidence bounds, particularly when the relative cost of classification errors is very low/high.

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	This version of the article has been accepted for publication, after peer review and is subject to Springer Cham AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-032-34867-8_19
Subjects:	H Social Sciences > HA Statistics Q Science > QA Mathematics Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology School of Science & Technology > Department of Computer Science
SWORD Depositor:	Symplectic Administrator

Export

Downloads