City Research Online

Comparing the effectiveness of testing methods in improving programs: the effect of variations in program quality

Pizza, M. & Strigini, L. (1998). Comparing the effectiveness of testing methods in improving programs: the effect of variations in program quality. In: NINTH INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING, PROCEEDINGS. The Ninth International Symposium on Software Reliability Engineering, 4 - 7 Nov 1998, Paderborn, Germany.


We compare the efficacy of different testing methods for improving the reliability of software. Specifically, we use modelling to compare “operational” testing, in which test cases are chosen according to their probability of occurring in actual use of the software, against “debug” testing methods, in which the testers look for test cases which they consider likely to cause failure, or that satisfy some coverage criterion. We base our comparisons on the reliability reached by the program at the end of testing. Differently from previous studies, we consider the probability distribution of the achieved reliability, and thus the probability of satisfying specific requirements, rather than just the average reliability achieved. We take account of two sources of variation. The variation between the actual test histories that are possible for a given program and a given test method: and the fact that different programs start testing with different faults and initial reliability levels. By necessity, we use very simplified models of reality. Yet, we can show some interesting conclusions with important practical consequences. In general, there are stronger arguments in favor of operational testing than previous studies have shown

Publication Type: Conference or Workshop Item (Paper)
Additional Information: © 1998 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
Subjects: Q Science > QA Mathematics > QA76 Computer software
Departments: School of Science & Technology > Computer Science > Software Reliability
[thumbnail of ISSRE98testing.pdf]
Download (89kB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login