City Research Online

Overcoming non-determinism in testing smart devices: how to build models of device behaviour

Bishop, P. G. & Cyra, L. (2012). Overcoming non-determinism in testing smart devices: how to build models of device behaviour. Paper presented at the 11th International Probabilistic Safety Assessment and Management Conference and the Annual European Safety and Reliability Conference 2012 (PSAM11 & ESREL 2012), 25 - 29 June 2012, Helsinki, Finland.


Justification of smart instruments has become an important topic in the nuclear industry. In practice, however, the publicly available artefacts are often the only source of information about the device. Therefore, in many cases independent black-box testing may be the only way to increase the confidence in the device. In this paper we provide a set of recommendations, which we consider to be the best practices for performing black-box assessments. We present our method of testing smart instruments, in which we use the publicly available artefacts only. We present a test harness and describe a method of test automation. We focus on the analysis of test results, which is made particularly complex by the inherent non determinism in the testing of analogue devices. In the paper we analyse the sources of non-determinism, which for instance may arise from inaccuracy in an analogue measurement made by the device when two alternative actions are possible. We propose three alternative ideas on how to build models of device behaviour, which can cope with this kind of non-determinism. We compare and contrast these three solutions, and express our recommendations. Finally, we use a case study, in which a black box assessment of two similar smart instruments is performed to illustrate the differences between the solutions.

Publication Type: Conference or Workshop Item (Paper)
Publisher Keywords: Smart sensor, testing, non-determinism, test automation
Subjects: Q Science > QA Mathematics > QA76 Computer software
Departments: School of Science & Technology > Computer Science > Software Reliability
[thumbnail of _02-Fr3-2_-_Smart.pdf]
Download (158kB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login