Explainable Fashion Recommendation with Joint Outfit Matching and Comment Generation

06/23/2018
by   Yujie Lin, et al.
0

Most previous work on fashion recommendation focuses on designing visual features to enhance recommendations. Existing work neglects user comments of fashion items, which have been proved effective in generating explanations along with better recommendation results. We propose a novel neural network framework, neural fashion recommendation (NFR), that simultaneously provides fashion recommendations and generates abstractive comments. NFR consists of two parts: outfit matching and comment generation. For outfit matching, we propose a convolutional neural network with a mutual attention mechanism to extract visual features of outfits. The visual features are then decoded into a rating score for the matching prediction. For abstractive comment generation, we propose a gated recurrent neural network with a cross-modality attention mechanism to transform visual features into a concise sentence. The two parts are jointly trained based on a multi-task learning framework in an end-to-end back-propagation paradigm. Extensive experiments conducted on an existing dataset and a collected real-world dataset show NFR achieves significant improvements over state-of-the-art baselines for fashion recommendation. Meanwhile, our generated comments achieve impressive ROUGE and BLEU scores in comparison to human-written comments. The generated comments can be regarded as explanations for the recommendation results. We release the dataset and code to facilitate future research.

READ FULL TEXT

page 1

page 9

page 11

page 14

research
08/24/2019

Improving Outfit Recommendation with Co-supervision of Fashion Generation

The task of fashion recommendation includes two main challenges: visual ...
research
04/13/2017

Fashion Conversation Data on Instagram

The fashion industry is establishing its presence on a number of visual-...
research
06/19/2017

Exploring Content-based Artwork Recommendation with Metadata and Visual Features

Compared to other areas, artwork recommendation has received little atte...
research
06/22/2017

Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation

Advances in image processing and computer vision in the latest years hav...
research
11/01/2021

Comparative Explanations of Recommendations

As recommendation is essentially a comparative (or ranking) process, a g...
research
08/01/2017

Neural Rating Regression with Abstractive Tips Generation for Recommendation

Recently, some E-commerce sites launch a new interaction box called Tips...
research
02/28/2018

Neural Aesthetic Image Reviewer

Recently, there is a rising interest in perceiving image aesthetics. The...

Please sign up or login with your details

Forgot password? Click here to reset