DeepAI AI Chat
Log In Sign Up

NorBERT: NetwOrk Representations through BERT for Network Analysis and Management

06/21/2022
by   Franck Le, et al.
ibm
0

Deep neural network models have been very successfully applied to Natural Language Processing (NLP) and Image based tasks. Their application to network analysis and management tasks is just recently being pursued. Our interest is in producing deep models that can be effectively generalized to perform well on multiple network tasks in different environments. A major challenge is that traditional deep models often rely on categorical features, but cannot handle unseen categorical values. One method for dealing with such problems is to learn contextual embeddings for categorical variables used by deep networks to improve their performance. In this paper, we adapt the NLP pre-training technique and associated deep model BERT to learn semantically meaningful numerical representations (embeddings) for Fully Qualified Domain Names (FQDNs) used in communication networks. We show through a series of experiments that such an approach can be used to generate models that maintain their effectiveness when applied to environments other than the one in which they were trained.

READ FULL TEXT
04/21/2019

NeuronBlocks -- Building Your NLP DNN Models Like Playing Lego

When building deep neural network models for natural language processing...
09/03/2017

Investigating how well contextual features are captured by bi-directional recurrent neural network models

Learning algorithms for natural language processing (NLP) tasks traditio...
09/15/2021

Enhancing Clinical Information Extraction with Transferred Contextual Embeddings

The Bidirectional Encoder Representations from Transformers (BERT) model...
04/15/2019

Human-Guided Learning of Column Networks: Augmenting Deep Learning with Advice

Recently, deep models have been successfully applied in several applicat...
12/17/2020

BERT Goes Shopping: Comparing Distributional Models for Product Representations

Word embeddings (e.g., word2vec) have been applied successfully to eComm...
08/31/2021

Effectiveness of Deep Networks in NLP using BiDAF as an example architecture

Question Answering with NLP has progressed through the evolution of adva...