City Research Online

Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles

Merz, M., Richman, R., Tsanakas, A. ORCID: 0000-0003-4552-5532 & Wüthrich, M. V. (2021). Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles. .

Abstract

A vastly growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specifically, we consider variable importance by fixing (global) output levels and, thus, explain how features marginally contribute across different regions of the prediction space. Hence, MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects (ALE), which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effect, and visually illustrate the 3-way relationship between marginal attribution, output level, and feature value.

Publication Type: Monograph (Working Paper)
Additional Information: Copyright the authors, 2021.
Publisher Keywords: explainable AI (XAI), model-agnostic tools, deep learning, attribution, accumulated local effects (ALE), partial dependence plot (PDP), locally interpretable model-agnostic explanation (LIME), variable importance, post-hoc analysis
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: Bayes Business School > Actuarial Science & Insurance
[thumbnail of Manuscript.pdf]
Preview
Text - Pre-print
Download (839kB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login