Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation

03/29/2017
by   Ryan Szeto, et al.
0

We motivate and address a human-in-the-loop variant of the monocular viewpoint estimation task in which the location and class of one semantic object keypoint is available at test time. In order to leverage the keypoint information, we devise a Convolutional Neural Network called Click-Here CNN (CH-CNN) that integrates the keypoint information with activations from the layers that process the image. It transforms the keypoint information into a 2D map that can be used to weigh features from certain parts of the image more heavily. The weighted sum of these spatial features is combined with global image features to provide relevant information to the prediction layers. To train our network, we collect a novel dataset of 3D keypoint annotations on thousands of CAD models, and synthetically render millions of images with 2D keypoint information. On test instances from PASCAL 3D+, our model achieves a mean class accuracy of 90.7 obtains 85.7 human-in-the-loop inference.

READ FULL TEXT

page 1

page 5

page 7

page 8

page 12

page 14

page 15

research
02/05/2018

Adviser Networks: Learning What Question to Ask for Human-In-The-Loop Viewpoint Estimation

Humans have an unparalleled visual intelligence and can overcome visual ...
research
02/05/2021

CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

We introduce CharacterGAN, a generative model that can be trained on onl...
research
12/13/2019

Joint Viewpoint and Keypoint Estimation with Real and Synthetic Data

The estimation of viewpoints and keypoints effectively enhance object de...
research
09/28/2021

Weakly Supervised Keypoint Discovery

In this paper, we propose a method for keypoint discovery from a 2D imag...
research
03/09/2023

KGNv2: Separating Scale and Pose Prediction for Keypoint-based 6-DoF Grasp Synthesis on RGB-D input

We propose a new 6-DoF grasp pose synthesis approach from 2D/2.5D input ...
research
10/07/2021

Using Keypoint Matching and Interactive Self Attention Network to verify Retail POSMs

Point of Sale Materials(POSM) are the merchandising and decoration items...
research
02/03/2020

Towards High Performance Human Keypoint Detection

Human keypoint detection from a single image is very challenging due to ...

Please sign up or login with your details

Forgot password? Click here to reset