Industry-sensitive language modeling for business

Borchert, Philipp; Coussement, Kristof; de Weerdt, Jochen; De Caigny, Arno

Document type :

Compte-rendu et recension critique d'ouvrage

DOI :

10.1016/j.ejor.2024.01.023

Title :

Industry-sensitive language modeling for business

Author(s) :

Borchert, Philipp [Auteur]
Lille économie management - UMR 9221 [LEM]
Coussement, Kristof [Auteur]

Lille économie management - UMR 9221 [LEM]
de Weerdt, Jochen [Auteur]
Catholic University of Leuven = Katholieke Universiteit Leuven [KU Leuven]
De Caigny, Arno [Auteur]
Lille économie management - UMR 9221 [LEM]

Journal title :

European Journal of Operational Research

Pages :

691-702

Publisher :

Elsevier

Publication date :

2024-06-01

ISSN :

0377-2217

HAL domain(s) :

Sciences de l'Homme et Société/Gestion et management

English abstract : [en]

We introduce BusinessBERT, a new industry-sensitive language model for business applications. The key novelty of our model lies in incorporating industry information to enhance decision-making in business-related natural ...
Show more >We introduce BusinessBERT, a new industry-sensitive language model for business applications. The key novelty of our model lies in incorporating industry information to enhance decision-making in business-related natural language processing (NLP) tasks. BusinessBERT extends the Bidirectional Encoder Representations from Transformers (BERT) architecture by embedding industry information during pretraining through two innovative approaches that enable BusinessBert to capture industry-specific terminology: (1) BusinessBERT is trained on business communication corpora totaling 2.23 billion tokens consisting of company website content, MD&A statements and scientific papers in the business domain; (2) we employ industry classification as an additional pretraining objective. Our results suggest that BusinessBERT improves data-driven decision-making by providing superior performance on business-related NLP tasks. Our experiments cover 7 benchmark datasets that include text classification, named entity recognition, sentiment analysis, and question-answering tasks. Additionally, this paper reduces the complexity of using BusinessBERT for other NLP applications by making it freely available as a pretrained language model to the business community. The model, its pretraining corpora and corresponding code snippets are accessible via https://github.com/pnborchert/BusinessBERT.Show less >

Language :

Anglais

Popular science :

Non

Collections :

Lille Économie Management (LEM) - UMR 9221

Source :

Harvested from HAL

Industry-sensitive language modeling for business BibTeX CSV Excel RIS

Industry-sensitive language modeling for business

BibTeX

CSV

Excel

RIS