Bishop, P. G. & Cyra, L. (2012). Overcoming non-determinism in testing smart devices: how to build models of device behaviour. Paper presented at the 11th International Probabilistic Safety Assessment and Management Conference and the Annual European Safety and Reliability Conference 2012 (PSAM11 & ESREL 2012), 25 - 29 June 2012, Helsinki, Finland.
Download (158kB) | Preview
Justification of smart instruments has become an important topic in the nuclear industry. In practice, however, the publicly available artefacts are often the only source of information about the device. Therefore, in many cases independent black-box testing may be the only way to increase the confidence in the device. In this paper we provide a set of recommendations, which we consider to be the best practices for performing black-box assessments. We present our method of testing smart instruments, in which we use the publicly available artefacts only. We present a test harness and describe a method of test automation. We focus on the analysis of test results, which is made particularly complex by the inherent non determinism in the testing of analogue devices. In the paper we analyse the sources of non-determinism, which for instance may arise from inaccuracy in an analogue measurement made by the device when two alternative actions are possible. We propose three alternative ideas on how to build models of device behaviour, which can cope with this kind of non-determinism. We compare and contrast these three solutions, and express our recommendations. Finally, we use a case study, in which a black box assessment of two similar smart instruments is performed to illustrate the differences between the solutions.
|Item Type:||Conference or Workshop Item (Paper)|
|Uncontrolled Keywords:||Smart sensor, testing, non-determinism, test automation|
|Subjects:||Q Science > QA Mathematics > QA76 Computer software|
|Divisions:||School of Informatics > Centre for Software Reliability|
Actions (login required)
Downloads per month over past year