Lip Reading with Hahn Convolutional Neural Networks moments

Mesbah, Abderrahim; Hammouchi, Hicham; Berrahou, Aissam; Berbia, Hassan; Qjidaa, Hassan; Daoudi, Mohamed

Document type :

Compte-rendu et recension critique d'ouvrage

Title :

Lip Reading with Hahn Convolutional Neural Networks moments

Author(s) :

Mesbah, Abderrahim [Auteur]
Université Sidi Mohamed Ben Abdellah [USMBA]
Hammouchi, Hicham [Auteur]
Université Sidi Mohamed Ben Abdellah [USMBA]
Université Mohammed V de Rabat [Agdal] [UM5]
Berrahou, Aissam [Auteur]
Université Mohammed V de Rabat [Agdal] [UM5]
Berbia, Hassan [Auteur]
Université Mohammed V de Rabat [Agdal] [UM5]
Qjidaa, Hassan [Auteur]
Université Sidi Mohamed Ben Abdellah [USMBA]
Daoudi, Mohamed [Auteur]

Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Ecole nationale supérieure Mines-Télécom Lille Douai [IMT Lille Douai]

Journal title :

Image and Vision Computing

Publisher :

Elsevier

Publication date :

2019-04-22

ISSN :

0262-8856

English keyword(s) :

Visual speech recognition
Lipreading
Laryngectomy
Deep learning

HAL domain(s) :

Informatique [cs]/Vision par ordinateur et reconnaissance de formes [cs.CV]

English abstract : [en]

Lipreading or Visual speech recognition is the process of decoding speech from speakers mouth movements. It is used for people with hearing impairment , to understand patients attained with laryngeal cancer, people with ...
Show more >Lipreading or Visual speech recognition is the process of decoding speech from speakers mouth movements. It is used for people with hearing impairment , to understand patients attained with laryngeal cancer, people with vocal cord paralysis and in noisy environment. In this paper we aim to develop a visual-only speech recognition system based only on video. Our main targeted application is in the medical field for the assistance to la-ryngectomized persons. To that end, we propose Hahn Convolutional Neu-ral Network (HCNN), a novel architecture based on Hahn moments as first layer in the Convolutional neural network (CNN) architecture. We show that HCNN helps in reducing the dimensionality of video images, in gaining training time. HCNN model is trained to classify letters, digits or words given as video images. We evaluated the proposed method on three datasets, AVLetters, OuluVS2 and BBC LRW, and we show that it achieves significant results in comparison with other works in the literature.Show less >

Language :

Anglais

Popular science :

Non

Collections :