BERTaú: Itaú BERT for digital customer service

01/28/2021
by   Paulo Finardi, et al.
0

In the last few years, three major topics received increased interest: deep learning, NLP and conversational agents. Bringing these three topics together to create an amazing digital customer experience and indeed deploy in production and solve real-world problems is something innovative and disruptive. We introduce a new Portuguese financial domain language representation model called BERTaú. BERTaú is an uncased BERT-base trained from scratch with data from the Itaú virtual assistant chatbot solution. Our novel contribution is that BERTaú pretrained language model requires less data, reached state-of-the-art performance in three NLP tasks, and generates a smaller and lighter model that makes the deployment feasible. We developed three tasks to validate our model: information retrieval with Frequently Asked Questions (FAQ) from Itaú bank, sentiment analysis from our virtual assistant data, and a NER solution. All proposed tasks are real-world solutions in production on our environment and the usage of a specialist model proved to be effective when compared to Google BERT multilingual and the DPRQuestionEncoder from Facebook, available at Hugging Face. The BERTaú improves the performance in 22 in NER F1 score and can also represent the same sequence in up to 66 tokens when compared to "shelf models".

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/19/2019

BERTje: A Dutch BERT Model

The transformer-based pre-trained language model BERT has helped to impr...
research
12/03/2021

Augmenting Customer Support with an NLP-based Receptionist

In this paper, we show how a Portuguese BERT model can be combined with ...
research
02/17/2020

A Financial Service Chatbot based on Deep Bidirectional Transformers

We develop a chatbot using Deep Bidirectional Transformer models (BERT) ...
research
03/05/2021

Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis

Although previous research on Aspect-based Sentiment Analysis (ABSA) for...
research
04/23/2021

Optimizing small BERTs trained for German NER

Currently, the most widespread neural network architecture for training ...
research
07/03/2020

Playing with Words at the National Library of Sweden – Making a Swedish BERT

This paper introduces the Swedish BERT ("KB-BERT") developed by the KBLa...
research
03/19/2021

Cost-effective Deployment of BERT Models in Serverless Environment

In this study we demonstrate the viability of deploying BERT-style model...

Please sign up or login with your details

Forgot password? Click here to reset