MM-Rec: Multimodal News Recommendation

04/15/2021
by   Chuhan Wu, et al.
0

Accurate news representation is critical for news recommendation. Most of existing news representation methods learn news representations only from news texts while ignore the visual information in news like images. In fact, users may click news not only because of the interest in news titles but also due to the attraction of news images. Thus, images are useful for representing news and predicting user behaviors. In this paper, we propose a multimodal news recommendation method, which can incorporate both textual and visual information of news to learn multimodal news representations. We first extract region-of-interests (ROIs) from news images via objective detection. Then we use a pre-trained visiolinguistic model to encode both news texts and news image ROIs and model their inherent relatedness using co-attentional Transformers. In addition, we propose a crossmodal candidate-aware attention network to select relevant historical clicked news for accurate user modeling by measuring the crossmodal relatedness between clicked news and candidate news. Experiments validate that incorporating multimodal news information can effectively improve news recommendation.

READ FULL TEXT

page 1

page 4

research
04/10/2022

News Recommendation with Candidate-aware User Modeling

News recommendation aims to match news with personalized user interest. ...
research
10/06/2022

VLSNR:Vision-Linguistics Coordination Time Sequence-aware News Recommendation

News representation and user-oriented modeling are both essential for ne...
research
08/30/2021

N15News: A New Dataset for Multimodal News Classification

Current news datasets merely focus on text features on the news and rare...
research
09/26/2021

Why Do We Click: Visual Impression-aware News Recommendation

There is a soaring interest in the news recommendation research scenario...
research
04/11/2023

Prompt Learning for News Recommendation

Some recent news recommendation (NR) methods introduce a Pre-trained Lan...
research
11/15/2022

MM-Locate-News: Multimodal Focus Location Estimation in News

The consumption of news has changed significantly as the Web has become ...
research
01/12/2022

GateFormer: Speeding Up News Feed Recommendation with Input Gated Transformers

News feed recommendation is an important web service. In recent years, p...

Please sign up or login with your details

Forgot password? Click here to reset