Visual Aware Hierarchy Based Food Recognition

12/06/2020
by   Runyu Mao, et al.
0

Food recognition is one of the most important components in image-based dietary assessment. However, due to the different complexity level of food images and inter-class similarity of food categories, it is challenging for an image-based food recognition system to achieve high accuracy for a variety of publicly available datasets. In this work, we propose a new two-step food recognition system that includes food localization and hierarchical food classification using Convolutional Neural Networks (CNNs) as the backbone architecture. The food localization step is based on an implementation of the Faster R-CNN method to identify food regions. In the food classification step, visually similar food categories can be clustered together automatically to generate a hierarchical structure that represents the semantic visual relations among food categories, then a multi-task CNN model is proposed to perform the classification task based on the visual aware hierarchical structure. Since the size and quality of dataset is a key component of data driven methods, we introduce a new food image dataset, VIPER-FoodNet (VFN) dataset, consists of 82 food categories with 15k images based on the most commonly consumed foods in the United States. A semi-automatic crowdsourcing tool is used to provide the ground-truth information for this dataset including food object bounding boxes and food object labels. Experimental results demonstrate that our system can significantly improve both classification and recognition performance on 4 publicly available datasets and the new VFN dataset.

READ FULL TEXT

page 9

page 11

page 15

research
09/06/2021

Improving Dietary Assessment Via Integrated Hierarchy Food Classification

Image-based dietary assessment refers to the process of determining what...
research
07/29/2016

Can a CNN Recognize Catalan Diet?

Nowadays, we can find several diseases related to the unhealthy diet hab...
research
04/20/2022

A Mobile Food Recognition System for Dietary Assessment

Food recognition is an important task for a variety of applications, inc...
research
10/11/2019

Semi-Automatic Crowdsourcing Tool for Online Food Image Collection and Annotation

Assessing dietary intake accurately remains an open and challenging rese...
research
09/14/2017

Food Recognition using Fusion of Classifiers based on CNNs

With the arrival of convolutional neural networks, the complex problem o...
research
04/27/2016

Simultaneous Food Localization and Recognition

The development of automatic nutrition diaries, which would allow to kee...
research
05/08/2017

ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition

In this paper, we introduce a new and challenging large-scale food image...

Please sign up or login with your details

Forgot password? Click here to reset