Titre :

Visual Reasoning with Multi-hop Feature Modulation

Auteur(s) :

Strub, Florian [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Université de Lille
Sequential Learning [SEQUEL]
Seurin, Mathieu [Auteur]
Université de Lille
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Sequential Learning [SEQUEL]
Perez, Ethan [Auteur]
Rice University [Houston]
de Vries, Harm [Auteur]
Department of Computer Science and Operations Research [Montreal]
Montreal Institute for Learning Algorithms [Montréal] [MILA]
Mary, Jérémie [Auteur]
Criteo [Paris]
Preux, Philippe [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Université de Lille
Sequential Learning [SEQUEL]
Courville, Aaron [Auteur]
Montreal Institute for Learning Algorithms [Montréal] [MILA]
CIFAR
Pietquin, Olivier [Auteur]
Google Inc

Éditeur(s) ou directeur(s) scientifique(s) :

Vittorio Ferrari
Martial Hebert
Cristian Sminchisescu
Yair Weiss

Titre de la manifestation scientifique :

ECCV 2018 - 15th European Conference on Computer Vision

Ville :

Munich

Pays :

Allemagne

Date de début de la manifestation scientifique :

2018-09-08

Titre de la revue :

Proc. ECCV

Éditeur :

Springer

Date de publication :

2018-09-14

Mot(s)-clé(s) en anglais :

Deep Learning
Computer Vision
Natural Language Understanding
Multi-modal Learning

Discipline(s) HAL :

Informatique [cs]/Intelligence artificielle [cs.AI]
Informatique [cs]/Vision par ordinateur et reconnaissance de formes [cs.CV]
Informatique [cs]/Apprentissage [cs.LG]
Informatique [cs]/Réseau de neurones [cs.NE]

Résumé en anglais : [en]

Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue. For such tasks, one successful approach ...
Lire la suite >Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue. For such tasks, one successful approach is to condition image-based convolutional network computation on language via Feature-wise Linear Modulation (FiLM) layers, i.e., per-channel scaling and shifting. We propose to generate the parameters of FiLM layers going up the hierarchy of a convolutional network in a multi-hop fashion rather than all at once, as in prior work. By alternating between attending to the language input and generating FiLM layer parameters, this approach is better able to scale to settings with longer input sequences such as dialogue. We demonstrate that multi-hop FiLM generation achieves state-of-the-art for the short input sequence task ReferIt-on-par with single-hop FiLM generation-while also significantly outperforming prior state-of-the-art and single-hop FiLM generation on the GuessWhat?! visual dialogue task.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Projet Européen :

Child-Robot Communication and Collaboration: Edutainment, Behavioural Modelling and Cognitive Development in Typically Developing and Autistic Spectrum Children
CHIST-ERA project "Interactive Grounded Language Understanding"

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL