SocialBERT – Transformers for Online SocialNetwork Language Modelling

11/13/2021
by   Ilia Karpov, et al.
0

The ubiquity of the contemporary language understanding tasks gives relevance to the development of generalized, yet highly efficient models that utilize all knowledge, provided by the data source. In this work, we present SocialBERT - the first model that uses knowledge about the author's position in the network during text analysis. We investigate possible models for learning social network information and successfully inject it into the baseline BERT model. The evaluation shows that embedding this information maintains a good generalization, with an increase in the quality of the probabilistic model for the given author up to 7.5 majority of groups for the chosen social network, and still able to work with previously unknown groups. The obtained model, as well as the code of our experiments, is available for download and use in applied tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/19/2015

Overcoming Language Variation in Sentiment Analysis with Social Attention

Variation in language is ubiquitous, particularly in newer forms of writ...
research
06/24/2022

Text and author-level political inference using heterogeneous knowledge representations

The inference of politically-charged information from text data is a pop...
research
04/07/2022

BERTuit: Understanding Spanish language in Twitter through a native transformer

The appearance of complex attention-based language models such as BERT, ...
research
02/21/2017

Efficient Social Network Multilingual Classification using Character, POS n-grams and Dynamic Normalization

In this paper we describe a dynamic normalization process applied to soc...
research
09/09/2019

BERT-Based Arabic Social Media Author Profiling

We report our models for detecting age, language variety, and gender fro...
research
12/29/2020

Presenting a Dataset for Collaborator Recommending Systems in Academic Social Network: a Case Study on ReseachGate

Collaborator finding systems are a special type of expert finding models...
research
11/09/2019

xSLUE: A Benchmark and Analysis Platform for Cross-Style Language Understanding and Evaluation

Every natural text is written in some style. The style is formed by a co...

Please sign up or login with your details

Forgot password? Click here to reset