Industry-sensitive language modeling for business
Type de document :
Compte-rendu et recension critique d'ouvrage
Titre :
Industry-sensitive language modeling for business
Auteur(s) :
Borchert, Philipp [Auteur]
Lille économie management - UMR 9221 [LEM]
Coussement, Kristof [Auteur]
Lille économie management - UMR 9221 [LEM]
de Weerdt, Jochen [Auteur]
Catholic University of Leuven = Katholieke Universiteit Leuven [KU Leuven]
De Caigny, Arno [Auteur]
Lille économie management - UMR 9221 [LEM]
Lille économie management - UMR 9221 [LEM]
Coussement, Kristof [Auteur]
Lille économie management - UMR 9221 [LEM]
de Weerdt, Jochen [Auteur]
Catholic University of Leuven = Katholieke Universiteit Leuven [KU Leuven]
De Caigny, Arno [Auteur]
Lille économie management - UMR 9221 [LEM]
Titre de la revue :
European Journal of Operational Research
Pagination :
691-702
Éditeur :
Elsevier
Date de publication :
2024-06-01
ISSN :
0377-2217
Discipline(s) HAL :
Sciences de l'Homme et Société/Gestion et management
Résumé en anglais : [en]
We introduce BusinessBERT, a new industry-sensitive language model for business applications. The key novelty of our model lies in incorporating industry information to enhance decision-making in business-related natural ...
Lire la suite >We introduce BusinessBERT, a new industry-sensitive language model for business applications. The key novelty of our model lies in incorporating industry information to enhance decision-making in business-related natural language processing (NLP) tasks. BusinessBERT extends the Bidirectional Encoder Representations from Transformers (BERT) architecture by embedding industry information during pretraining through two innovative approaches that enable BusinessBert to capture industry-specific terminology: (1) BusinessBERT is trained on business communication corpora totaling 2.23 billion tokens consisting of company website content, MD&A statements and scientific papers in the business domain; (2) we employ industry classification as an additional pretraining objective. Our results suggest that BusinessBERT improves data-driven decision-making by providing superior performance on business-related NLP tasks. Our experiments cover 7 benchmark datasets that include text classification, named entity recognition, sentiment analysis, and question-answering tasks. Additionally, this paper reduces the complexity of using BusinessBERT for other NLP applications by making it freely available as a pretrained language model to the business community. The model, its pretraining corpora and corresponding code snippets are accessible via https://github.com/pnborchert/BusinessBERT.Lire moins >
Lire la suite >We introduce BusinessBERT, a new industry-sensitive language model for business applications. The key novelty of our model lies in incorporating industry information to enhance decision-making in business-related natural language processing (NLP) tasks. BusinessBERT extends the Bidirectional Encoder Representations from Transformers (BERT) architecture by embedding industry information during pretraining through two innovative approaches that enable BusinessBert to capture industry-specific terminology: (1) BusinessBERT is trained on business communication corpora totaling 2.23 billion tokens consisting of company website content, MD&A statements and scientific papers in the business domain; (2) we employ industry classification as an additional pretraining objective. Our results suggest that BusinessBERT improves data-driven decision-making by providing superior performance on business-related NLP tasks. Our experiments cover 7 benchmark datasets that include text classification, named entity recognition, sentiment analysis, and question-answering tasks. Additionally, this paper reduces the complexity of using BusinessBERT for other NLP applications by making it freely available as a pretrained language model to the business community. The model, its pretraining corpora and corresponding code snippets are accessible via https://github.com/pnborchert/BusinessBERT.Lire moins >
Langue :
Anglais
Vulgarisation :
Non
Collections :
Source :