Cutting the Black Box: Conceptual ...
Document type :
Direction scientifique d'une publication (ouvrage, numéro spécial de revue, proceedings): Proceedings
DOI :
Title :
Cutting the Black Box: Conceptual Interpretation of a Deep Neural Net with Multi-Modal Embeddings and Multi-Criteria Decision Aid
Author(s) :
Atienza, Nicolas [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Thales Research and Technology [Palaiseau]
Bresson, Roman [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Thales Research and Technology [Palaiseau]
Rousselot, Cyriaque [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Integrated Optimization with Complex Structure [INOCS]
Caillou, Philippe [Auteur]
Laboratoire de Recherche en Informatique [LRI]
Machine Learning and Optimisation [TAO]
Cohen, Johanne [Auteur]
Algorithmes, Apprentissage et Calcul [AAC]
Graphes, Algorithmes et Combinatoire - LISN [GALaC]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Labreuche, Christophe [Auteur]
Thales Research and Technology [Palaiseau]
Sebag, Michèle [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Thales Research and Technology [Palaiseau]
Bresson, Roman [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Thales Research and Technology [Palaiseau]
Rousselot, Cyriaque [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Integrated Optimization with Complex Structure [INOCS]
Caillou, Philippe [Auteur]
Laboratoire de Recherche en Informatique [LRI]
Machine Learning and Optimisation [TAO]
Cohen, Johanne [Auteur]
Algorithmes, Apprentissage et Calcul [AAC]
Graphes, Algorithmes et Combinatoire - LISN [GALaC]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Labreuche, Christophe [Auteur]
Thales Research and Technology [Palaiseau]
Sebag, Michèle [Auteur]
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Conference title :
Thirty-Third International Joint Conference on Artificial Intelligence {IJCAI-24}
Publisher :
International Joint Conferences on Artificial Intelligence Organization
Publication date :
2024
HAL domain(s) :
Informatique [cs]
Informatique [cs]/Complexité [cs.CC]
Informatique [cs]/Mathématique discrète [cs.DM]
Sciences cognitives/Informatique
Informatique [cs]/Complexité [cs.CC]
Informatique [cs]/Mathématique discrète [cs.DM]
Sciences cognitives/Informatique
English abstract : [en]
This paper tackles the concept-based explanation of neural models in computer vision, building upon the state of the art in Multi-Criteria Decision Aid (MCDA). The novelty of the approach is to leverage multi-modal embeddings ...
Show more >This paper tackles the concept-based explanation of neural models in computer vision, building upon the state of the art in Multi-Criteria Decision Aid (MCDA). The novelty of the approach is to leverage multi-modal embeddings from CLIP to bridge the gap between pixel-based and concept-based representations. The proposed Cut the Black Box (CB2) approach disentangles the latent representation of a trained pixel-based neural net, referred to as teacher model, along a 3-step process. Firstly, the pixel-based representation of the samples is mapped onto a conceptual representation using multi-modal embeddings. Secondly, an interpretable-by-design MCDA student model is trained by distillation from the teacher model, using the conceptual sample representation. Thirdly, the alignment of the teacher and student latent representations spells out the concepts relevant to explaining the teacher model. The empirical validation of the approach on ResNet, VGG, and VisionTransformer on Cifar-10, Cifar-100, Tiny ImageNet, and Fashion-MNIST showcases the effectiveness of the interpretations provided for the teacher models. The analysis reveals that decision-making predominantly relies on few concepts, thereby exposing potential bias in the teacher's decisions.Show less >
Show more >This paper tackles the concept-based explanation of neural models in computer vision, building upon the state of the art in Multi-Criteria Decision Aid (MCDA). The novelty of the approach is to leverage multi-modal embeddings from CLIP to bridge the gap between pixel-based and concept-based representations. The proposed Cut the Black Box (CB2) approach disentangles the latent representation of a trained pixel-based neural net, referred to as teacher model, along a 3-step process. Firstly, the pixel-based representation of the samples is mapped onto a conceptual representation using multi-modal embeddings. Secondly, an interpretable-by-design MCDA student model is trained by distillation from the teacher model, using the conceptual sample representation. Thirdly, the alignment of the teacher and student latent representations spells out the concepts relevant to explaining the teacher model. The empirical validation of the approach on ResNet, VGG, and VisionTransformer on Cifar-10, Cifar-100, Tiny ImageNet, and Fashion-MNIST showcases the effectiveness of the interpretations provided for the teacher models. The analysis reveals that decision-making predominantly relies on few concepts, thereby exposing potential bias in the teacher's decisions.Show less >
Language :
Anglais
Audience :
Internationale
Collections :
Source :