Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings

04/30/2018
by   Micael Carvalho, et al.
0

Designing powerful tools that support cooking activities has rapidly gained popularity due to the massive amounts of available data, as well as recent advances in machine learning that are capable of analyzing them. In this paper, we propose a cross-modal retrieval model aligning visual and textual data (like pictures of dishes and their recipes) in a shared representation space. We describe an effective learning scheme, capable of tackling large-scale problems, and validate it on the Recipe1M dataset containing nearly 1 million picture-recipe pairs. We show the effectiveness of our approach regarding previous state-of-the-art models and present qualitative results over computational cooking use cases.

READ FULL TEXT

page 7

page 8

page 9

research
05/02/2018

Images & Recipes: Retrieval in the cooking context

Recent advances in the machine learning community allowed different use ...
research
01/31/2019

Self-Supervised Visual Representations for Cross-Modal Retrieval

Cross-modal retrieval methods have been significantly improved in last y...
research
03/24/2021

Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning

Cross-modal recipe retrieval has recently gained substantial attention d...
research
09/06/2022

Cross Modal Compression: Towards Human-comprehensible Semantic Compression

Traditional image/video compression aims to reduce the transmission/stor...
research
10/19/2020

DIME: An Online Tool for the Visual Comparison of Cross-Modal Retrieval Models

Cross-modal retrieval relies on accurate models to retrieve relevant res...
research
06/07/2023

Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata

Virtual assistants have been widely used by mobile phone users in recent...
research
09/13/2019

Co-Attentive Cross-Modal Deep Learning for Medical Evidence Synthesis and Decision Making

Modern medicine requires generalised approaches to the synthesis and int...

Please sign up or login with your details

Forgot password? Click here to reset