CookGAN: Meal Image Synthesis from Ingredients

02/25/2020
by   Fangda Han, et al.
35

In this work we propose a new computational framework, based on generative deep models, for synthesis of photo-realistic food meal images from textual list of its ingredients. Previous works on synthesis of images from text typically rely on pre-trained text models to extract text features, followed by generative neural networks (GAN) aimed to generate realistic images conditioned on the text features. These works mainly focus on generating spatially compact and well-defined categories of objects, such as birds or flowers, but meal images are significantly more complex, consisting of multiple ingredients whose appearance and spatial qualities are further modified by cooking methods. To generate real-like meal images from ingredients, we propose Cook Generative Adversarial Networks (CookGAN), CookGAN first builds an attention-based ingredients-image association model, which is then used to condition a generative neural network tasked with synthesizing meal images. Furthermore, a cycle-consistent constraint is added to further improve image quality and control appearance. Experiments show our model is able to generate meal images corresponding to the ingredients.

READ FULL TEXT

page 5

page 6

page 7

page 8

research
05/09/2019

The Art of Food: Meal Image Synthesis from Ingredients

In this work we propose a new computational framework, based on generati...
research
10/29/2019

Semantic Object Accuracy for Generative Text-to-Image Synthesis

Generative adversarial networks conditioned on simple textual image desc...
research
03/24/2023

Factor Decomposed Generative Adversarial Networks for Text-to-Image Synthesis

Prior works about text-to-image synthesis typically concatenated the sen...
research
06/06/2019

How to make a pizza: Learning a compositional layer-based GAN model

A food recipe is an ordered set of instructions for preparing a particul...
research
08/20/2022

Vision-Language Matching for Text-to-Image Synthesis via Generative Adversarial Networks

Text-to-image synthesis aims to generate a photo-realistic and semantic ...
research
09/16/2021

Aesthetics and neural network image representations

We analyze the spaces of images encoded by generative networks of the Bi...
research
01/18/2016

Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

This paper studies a combination of generative Markov random field (MRF)...

Please sign up or login with your details

Forgot password? Click here to reset