ScanTalk: 3D Talking Heads from Unregistered Scans
Document type :
Communication dans un congrès avec actes
Title :
ScanTalk: 3D Talking Heads from Unregistered Scans
Author(s) :
Nocentini, Federico [Auteur]
Università degli Studi di Firenze = University of Florence = Université de Florence [UniFI]
Besnier, Thomas [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Ferrari, Claudio [Auteur]
Arguillère, Sylvain [Auteur]
Laboratoire Paul Painlevé - UMR 8524 [LPP]
Berretti, Stefano [Auteur]
Università degli Studi di Firenze = University of Florence = Université de Florence [UniFI]
Daoudi, Mohamed [Auteur]
Ecole nationale supérieure Mines-Télécom Lille Douai [IMT Nord Europe]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Università degli Studi di Firenze = University of Florence = Université de Florence [UniFI]
Besnier, Thomas [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Ferrari, Claudio [Auteur]
Arguillère, Sylvain [Auteur]

Laboratoire Paul Painlevé - UMR 8524 [LPP]
Berretti, Stefano [Auteur]
Università degli Studi di Firenze = University of Florence = Université de Florence [UniFI]
Daoudi, Mohamed [Auteur]

Ecole nationale supérieure Mines-Télécom Lille Douai [IMT Nord Europe]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Conference title :
European Conference on Computer Vision (ECCV)
City :
Milan
Country :
Italie
Start date of the conference :
2024-09-29
English keyword(s) :
3D Talking Heads 3D Scans Animation DiffusionNet
3D Talking Heads
3D Scans Animation
DiffusionNet
3D Talking Heads
3D Scans Animation
DiffusionNet
HAL domain(s) :
Informatique [cs]/Vision par ordinateur et reconnaissance de formes [cs.CV]
English abstract : [en]
<div><p>Speech-driven 3D talking heads generation has emerged as a significant area of interest among researchers, presenting numerous challenges. Existing methods are constrained by animating faces with fixed topologies, ...
Show more ><div><p>Speech-driven 3D talking heads generation has emerged as a significant area of interest among researchers, presenting numerous challenges. Existing methods are constrained by animating faces with fixed topologies, wherein point-wise correspondence is established, and the number and order of points remains consistent across all identities the model can animate. In this work, we present ScanTalk, a novel framework capable of animating 3D faces in arbitrary topologies including scanned data. Our approach relies on the DiffusionNet architecture to overcome the fixed topology constraint, offering promising avenues for more flexible and realistic 3D animations. By leveraging the power of DiffusionNet, ScanTalk not only adapts to diverse facial structures but also maintains fidelity when dealing with scanned data, thereby enhancing the authenticity and versatility of generated 3D talking heads. Through comprehensive comparisons with state-of-the-art methods, we validate the efficacy of our approach, demonstrating its capacity to generate realistic talking heads comparable to existing techniques. While our primary objective is to develop a generic method free from topological constraints, all state-of-the-art methodologies are bound by such limitations. Code for reproducing our results, and the pre-trained model are available at https://github.com/miccunifi/ScanTalk.</p></div>Show less >
Show more ><div><p>Speech-driven 3D talking heads generation has emerged as a significant area of interest among researchers, presenting numerous challenges. Existing methods are constrained by animating faces with fixed topologies, wherein point-wise correspondence is established, and the number and order of points remains consistent across all identities the model can animate. In this work, we present ScanTalk, a novel framework capable of animating 3D faces in arbitrary topologies including scanned data. Our approach relies on the DiffusionNet architecture to overcome the fixed topology constraint, offering promising avenues for more flexible and realistic 3D animations. By leveraging the power of DiffusionNet, ScanTalk not only adapts to diverse facial structures but also maintains fidelity when dealing with scanned data, thereby enhancing the authenticity and versatility of generated 3D talking heads. Through comprehensive comparisons with state-of-the-art methods, we validate the efficacy of our approach, demonstrating its capacity to generate realistic talking heads comparable to existing techniques. While our primary objective is to develop a generic method free from topological constraints, all state-of-the-art methodologies are bound by such limitations. Code for reproducing our results, and the pre-trained model are available at https://github.com/miccunifi/ScanTalk.</p></div>Show less >
Language :
Anglais
Peer reviewed article :
Oui
Audience :
Internationale
Popular science :
Non
Collections :
Source :
Files
- document
- Open access
- Access the document
- ECCV_2024_ScanTalk___Camera_Ready_Version-2.pdf
- Open access
- Access the document