Hard Example Generation by Texture Synthesis for Cross-domain Shape Similarity Learning

by   Huan Fu, et al.

Image-based 3D shape retrieval (IBSR) aims to find the corresponding 3D shape of a given 2D image from a large 3D shape database. The common routine is to map 2D images and 3D shapes into an embedding space and define (or learn) a shape similarity measure. While metric learning with some adaptation techniques seems to be a natural solution to shape similarity learning, the performance is often unsatisfactory for fine-grained shape retrieval. In the paper, we identify the source of the poor performance and propose a practical solution to this problem. We find that the shape difference between a negative pair is entangled with the texture gap, making metric learning ineffective in pushing away negative pairs. To tackle this issue, we develop a geometry-focused multi-view metric learning framework empowered by texture synthesis. The synthesis of textures for 3D shape models creates hard triplets, which suppress the adverse effects of rich texture in 2D images, thereby push the network to focus more on discovering geometric characteristics. Our approach shows state-of-the-art performance on a recently released large-scale 3D-FUTURE[1] repository, as well as three widely studied benchmarks, including Pix3D[2], Stanford Cars[3], and Comp Cars[4]. Codes will be made publicly available at: https://github.com/3D-FRONT-FUTURE/IBSR-texture


3D-FUTURE: 3D Furniture shape with TextURE

The 3D CAD shapes in current 3D benchmarks are mostly collected from onl...

Introspective Deep Metric Learning

This paper proposes an introspective deep metric learning (IDML) framewo...

Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding

Temporal grounding aims to localize a video moment which is semantically...

Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"

We present the first method to perform automatic 3D pose, shape and text...

Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning

Learning the distance metric between pairs of samples has been studied f...

Supervised Metric Learning for Retrieval via Contextual Similarity Optimization

Existing deep metric learning approaches fall into three general categor...

Large-to-small Image Resolution Asymmetry in Deep Metric Learning

Deep metric learning for vision is trained by optimizing a representatio...

Please sign up or login with your details

Forgot password? Click here to reset