Learning Type-Aware Embeddings for Fashion Compatibility

by   Mariya I. Vasileva, et al.

Outfits in online fashion data are composed of items of many different types (e.g. top, bottom, shoes, accessories) that share some stylistic relationship with each other. A representation for building outfits requires a method that can learn both notions of similarity (for example, when two tops are interchangeable) and compatibility (items of possibly different type that can go together in an outfit). This paper presents an approach to learning an image embedding that respects item type, and demonstrates that doing so leads to such representation. Jointly learning item similarity and compatibility in an end-to-end fashion enables our model to support a range of novel queries that exploit item type which prior work has been ill-equipped to handle. For example, one can look for a set of tops that (a) substitute for a particular top in an outfit and (b) varies widely (thereby encouraging browsing). To evaluate the learned representation, we collect a new dataset containing some 68,306 outfits created by users on the Polyvore website. Our approach obtains 3-5 prediction and fill-in-the-blank tasks using our dataset as well as an established smaller dataset. Our extensive qualitative and quantitative evaluation provides strong evidence that the proposed type-respecting embedding strategy outperforms existing methods while also allowing for a variety of useful queries.


page 24

page 31

page 33

page 35

page 36

page 38

page 39

page 40


Learning Fashion Compatibility with Bidirectional LSTMs

The ubiquity of online fashion shopping demands effective recommendation...

Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Existing works about fashion outfit compatibility focus on predicting th...

Using Discriminative Methods to Learn Fashion Compatibility Across Datasets

Determining whether a pair of garments are compatible with each other is...

Compatibility Family Learning for Item Recommendation and Generation

Compatibility between items, such as clothes and shoes, is a major facto...

SAT: Self-adaptive training for fashion compatibility prediction

This paper presents a self-adaptive training (SAT) model for fashion com...

A Correlation Maximization Approach for Cross Domain Co-Embeddings

Although modern recommendation systems can exploit the structure in user...

Answering Compositional Queries with Set-Theoretic Embeddings

The need to compactly and robustly represent item-attribute relations ar...

Please sign up or login with your details

Forgot password? Click here to reset