DeepAI AI Chat
Log In Sign Up

An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale

by   Yong Liu, et al.

This paper conducts an empirical investigation to evaluate transfer learning for classifying sales engagement emails arising from digital sales engagement platforms. Given the complexity of content and context of sales engagement, lack of standardized large corpora and benchmarks, limited labeled examples and heterogenous context of intent, this real-world use case poses both a challenge and an opportunity for adopting a transfer learning approach. We propose an evaluation framework to assess a high performance transfer learning (HPTL) approach in three key areas in addition to commonly used accuracy metrics: 1) effective embeddings and pretrained language model usage, 2) minimum labeled samples requirement and 3) transfer learning implementation strategies. We use in-house sales engagement email samples as the experiment dataset, which includes over 3000 emails labeled as positive, objection, unsubscribe, or not-sure. We discuss our findings on evaluating BERT, ELMo, Flair and GloVe embeddings with both feature-based and fine-tuning approaches and their scalability on a GPU cluster with increasingly larger labeled samples. Our results show that fine-tuning of the BERT model outperforms with as few as 300 labeled samples, but underperforms with fewer than 300 labeled samples, relative to all the feature-based approaches using different embeddings.


DeepEmotex: Classifying Emotion in Text Messages using Deep Transfer Learning

Transfer learning has been widely used in natural language processing th...

On transfer learning using a MAC model variant

We introduce a variant of the MAC model (Hudson and Manning, CVPR 2018) ...

Experiments on transfer learning architectures for biomedical relation extraction

Relation extraction (RE) consists in identifying and structuring automat...

ETHNO-DAANN: Ethnographic Engagement Classification by Deep Adversarial Transfer Learning

Student motivation is a key research agenda due to the necessity of both...

Transfer Learning Approach for Arabic Offensive Language Detection System – BERT-Based Model

Developing a system to detect online offensive language is very importan...

Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading

Automatic Short Answer Grading (ASAG) is the process of grading the stud...

Detecting Insincere Questions from Text: A Transfer Learning Approach

The internet today has become an unrivalled source of information where ...