3D Shape Knowledge Graph for Cross-domain and Cross-modal 3D Shape Retrieval

by   Weizhi Nie, et al.

With the development of 3D modeling and fabrication, 3D shape retrieval has become a hot topic. In recent years, several strategies have been put forth to address this retrieval issue. However, it is difficult for them to handle cross-modal 3D shape retrieval because of the natural differences between modalities. In this paper, we propose an innovative concept, namely, geometric words, which is regarded as the basic element to represent any 3D or 2D entity by combination, and assisted by which, we can simultaneously handle cross-domain or cross-modal retrieval problems. First, to construct the knowledge graph, we utilize the geometric word as the node, and then use the category of the 3D shape as well as the attribute of the geometry to bridge the nodes. Second, based on the knowledge graph, we provide a unique way for learning each entity's embedding. Finally, we propose an effective similarity measure to handle the cross-domain and cross-modal 3D shape retrieval. Specifically, every 3D or 2D entity could locate its geometric terms in the 3D knowledge graph, which serve as a link between cross-domain and cross-modal data. Thus, our approach can achieve the cross-domain and cross-modal 3D shape retrieval at the same time. We evaluated our proposed method on the ModelNet40 dataset and ShapeNetCore55 dataset for both the 3D shape retrieval task and cross-domain 3D shape retrieval task. The classic cross-modal dataset (MI3DOR) is utilized to evaluate cross-modal 3D shape retrieval. Experimental results and comparisons with state-of-the-art methods illustrate the superiority of our approach.


page 2

page 5

page 7

page 8

page 11

page 17

page 18


Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences

A recent method employs 3D voxels to represent 3D shapes, but this limit...

Uncertainty-Aware Cross-Modal Transfer Network for Sketch-Based 3D Shape Retrieval

In recent years, sketch-based 3D shape retrieval has attracted growing a...

CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction

Knowledge graph (KG) link prediction is a fundamental task in artificial...

Cross-modal Cross-domain Learning for Unsupervised LiDAR Semantic Segmentation

In recent years, cross-modal domain adaptation has been studied on the p...

Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval

Our goal in this research is to study a more realistic environment in wh...

Geometric Cross-Modal Comparison of Heterogeneous Sensor Data

In this work, we address the problem of cross-modal comparison of aerial...

Knowledge Embedding and Retrieval Strategies in an Informledge System

Informledge System (ILS) is a knowledge network with autonomous nodes an...

Please sign up or login with your details

Forgot password? Click here to reset