On why we lack confidence in some signal-detection-based analyses of confidence
Arnold, D. H., Johnston, A., Adie, J. & Yarrow, K. ORCID: 0000-0003-0666-2163 (2023).
On why we lack confidence in some signal-detection-based analyses of confidence.
Consciousness and Cognition, 113,
103532.
doi: 10.1016/j.concog.2023.103532
Abstract
Signal-detection theory (SDT) is one of the most popular frameworks for analyzing data from studies of human behavior – including investigations of confidence. SDT-based analyses of confidence deliver both standard estimates of sensitivity (d’), and a second estimate informed by high-confidence decisions – meta d’. The extent to which meta d’ estimates fall short of d’ estimates is regarded as a measure of metacognitive inefficiency. These analyses rely on a key but questionable assumption – that repeated exposures to an input will evoke a normally-shaped distribution of perceptual experiences (the normality assumption). We show that when distributions of experience do not conform with the normality assumption, meta d’ can be systematically underestimated relative to d'. We explain why violations of the normality assumption are especially a problem for some popular SDT-based analyses of confidence, in contrast to other analyses inspired by the SDT framework, which are more robust.
Publication Type: | Article |
---|---|
Additional Information: | © 2023. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/ |
Publisher Keywords: | Perceptual metacognition; Confidence; Visual Adaptation; Signal Detection Theory |
Subjects: | B Philosophy. Psychology. Religion > BF Psychology R Medicine > RC Internal medicine > RC0321 Neuroscience. Biological psychiatry. Neuropsychiatry |
Departments: | School of Health & Psychological Sciences > Psychology |
![[thumbnail of Direction_AE_Confidence_CC_rnd3.pdf]](https://openaccess.city.ac.uk/style/images/fileicons/text.png)
This document is not freely accessible until 7 June 2024 due to copyright restrictions.
Available under License Creative Commons Attribution Non-commercial No Derivatives.
To request a copy, please use the button below.
Request a copyExport
Downloads
Downloads per month over past year