Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images

by   Jiatong Li, et al.

Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. In this paper, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. A demo of our system and our data is availableat: foodai.cs.rutgers.edu.



There are no comments yet.


page 1

page 2

page 3

page 4


Deep Cooking: Predicting Relative Food Ingredient Amounts from Images

In this paper, we study the novel problem of not only predicting ingredi...

Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Images

Food computing is playing an increasingly important role in human daily ...

Leveraging Context to Support Automated Food Recognition in Restaurants

The pervasiveness of mobile cameras has resulted in a dramatic increase ...

Cross-modal Retrieval and Synthesis (X-MRS): Closing the modality gap in shared subspace

Computational food analysis (CFA), a broad set of methods that attempt t...

Recipe1M: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images

In this paper, we introduce Recipe1M, a new large-scale, structured corp...

Learning Program Representations for Food Images and Cooking Recipes

In this paper, we are interested in modeling a how-to instructional proc...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.