City Research Online

The legal rule that computers are presumed to be operating correctly – unforeseen and unjust consequences

Bohm, N., Christie, J., Ladkin, P. B. , Littlewood, B. ORCID: 0000-0002-6397-8685, Marshall, P., Mason, S., Newby, M., Murdoch, S., Thimbleby, H. & Thomas, M. (2022). The legal rule that computers are presumed to be operating correctly – unforeseen and unjust consequences. London, UK: Information Security Research & Education, University College London (UCL).

Abstract

In England and Wales, courts consider computers, as a matter of law, to have been working correctly unless there is evidence to the contrary. Therefore, evidence produced by computers is treated as reliable unless other evidence suggests otherwise. This way of handling evidence is known as a ‘rebuttable presumption’. A court will treat a computer as if it is working perfectly unless someone can show why that is not the case. This presumption poses a challenge to those who dispute evidence produced by a computer system. Frequently the challenge is insurmountable, particularly where a substantial institution operates the system. The Post Office Horizon scandal clearly exposes the problem and the harm that may result. From 1999, the Post Office prosecuted hundreds of postmasters and Post Office employees for theft and fraud based on evidence produced by the Horizon computer system showing shortfalls in their branch accounts. In those prosecutions, the Post Office relied on the presumption that computers were operating correctly. Hundreds of postmasters and others were convicted, sentenced to terms of imprisonment, fined, or had their property confiscated. This clearly demonstrated that the Law Commission’s assertion that ‘such a regime would work fairly’ was flawed. In the December 2019 judgment in the group litigation Bates v The Post Office Ltd (No 6: Horizon Issues) Rev 1, Mr Justice Fraser concluded that it was possible that software errors in Horizon could have caused apparent shortfalls in branch accounts, rather than these being due to theft or fraud. Following this judgement, the Criminal Cases Review Commission referred an unprecedented number of convictions, based upon the supposed shortfalls in the Horizon accounts, to the Court of Appeal. Appeal courts have quashed more than 70 convictions at the time of writing. There will be many more appeals and many more convictions quashed in what is likely the largest miscarriage of justice in British history. Were it not for the group litigation, the fundamental unreliability of the software in the Post Office’s Horizon computer system would not have been revealed, as previous challenges to Horizon’s correctness were unable to rebut the presumption of reliability for computer evidence. The financial risk of bringing legal action deterred other challenges. Similar issues apply in other situations where the reliability of computer evidence is questioned, such as in payment disputes. The legal presumption, as applied in practice, has exposed widespread misunderstanding about the nature of computer failures – in particular, the fact that these are almost invariably failures of software. The presumption has been the cause of widespread injustice. There is a pressing requirement for the presumption to be re-evaluated to avoid the risk of further or continuing injustice. We propose that the presumption that computer evidence is reliable be replaced with a process where if computer evidence is challenged, a party must justify the correctness of the evidence upon which they rely. The proposed process, summarised below, requires the disclosure of documents that would already exist in any well-managed computer system. The procedural and evidential safeguards of the kind we propose would probably have avoided the disastrous repeated miscarriages of justice over the past 20 years.

Publication Type: Report
Additional Information: This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International Licence
Subjects: H Social Sciences > HV Social pathology. Social and public welfare
K Law > K Law (General)
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology
School of Science & Technology > Computer Science
SWORD Depositor:
[thumbnail of briefing-presumption-that-computers-are-reliable.pdf]
Preview
Text - Published Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (116kB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login