An In-Cell Weight Update Scheme Using One-Hot Gradients for On-Chip Learning

Chêne, Mathieu; Larras, Benoit; Kaiser, Andreas; Frappe, Antoine

Type de document :

Communication dans un congrès avec actes

DOI :

10.1109/ICECS61496.2024.10848987

URL permanente :

http://hdl.handle.net/20.500.12210/124241

Titre :

An In-Cell Weight Update Scheme Using One-Hot Gradients for On-Chip Learning

Auteur(s) :

Chêne, Mathieu [Auteur]
Microélectronique Silicium - IEMN [MICROELEC SI - IEMN]
Centrale Lille
JUNIA [JUNIA]
Institut d’Électronique, de Microélectronique et de Nanotechnologie - UMR 8520 [IEMN]
Larras, Benoit [Auteur]

JUNIA [JUNIA]
Microélectronique Silicium - IEMN [MICROELEC SI - IEMN]
Institut d’Électronique, de Microélectronique et de Nanotechnologie - UMR 8520 [IEMN]
Kaiser, Andreas [Auteur]

JUNIA [JUNIA]
Institut d’Électronique, de Microélectronique et de Nanotechnologie - UMR 8520 [IEMN]
Microélectronique Silicium - IEMN [MICROELEC SI - IEMN]
Frappe, Antoine [Auteur]

Microélectronique Silicium - IEMN [MICROELEC SI - IEMN]
JUNIA [JUNIA]
Institut d’Électronique, de Microélectronique et de Nanotechnologie - UMR 8520 [IEMN]

Titre de la manifestation scientifique :

2024 31st IEEE International Conference on Electronics, Circuits and Systems (ICECS)

Ville :

Nancy

Pays :

France

Date de début de la manifestation scientifique :

2024-11-18

Éditeur :

IEEE

Discipline(s) HAL :

Physique [physics]
Sciences de l'ingénieur [physics]

Résumé en anglais : [en]

In the context of edge AI, the embedded models need to be retrained on-chip. The more the backward path and the calculation the gradient (two important steps during learning) are optimized by literature, the more it becomes ...
Lire la suite >In the context of edge AI, the embedded models need to be retrained on-chip. The more the backward path and the calculation the gradient (two important steps during learning) are optimized by literature, the more it becomes interesting to optimize weight update operation. This work proposes to encode the weight update gradient as a power of two. It is shown that this encoding method has a low impact on the learning capability for neural network configurations and it reduces the amount of weight update operations during training by a factor of 12.5 in the case of a feedforward neural network and by a factor of 9.5 in the case of a convolutional configuration. It is also compatible with transfer learning approaches. In addition, using in-memory computing, it is possible to get rid of the read operation required for the weight update operation and to reduce the amount of writing operation by 8, thereby reducing further the energy required for a weight update by another factor of 13.7. Finally considering the reduction of energy needed per update and the reduction of update operation done during training process, the energy needed for weight update can be reduced by 137 in the case of a convolutional architecture.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Collections :

Institut d'Électronique, de Microélectronique et de Nanotechnologie (IEMN) - UMR 8520

Source :

Harvested from HAL

Date de dépôt :

2025-02-25T06:36:28Z

An In-Cell Weight Update Scheme Using ... BibTeX CSV Excel RIS

An In-Cell Weight Update Scheme Using ...

BibTeX

CSV

Excel

RIS