Visual Tactile Fusion Object Clustering

by   Tao Zhang, et al.
Indiana University

Object clustering, aiming at grouping similar objects into one cluster with an unsupervised strategy, has been extensivelystudied among various data-driven applications. However, most existing state-of-the-art object clustering methods (e.g., single-view or multi-view clustering methods) only explore visual information, while ignoring one of most important sensing modalities, i.e., tactile information which can help capture different object properties and further boost the performance of object clustering task. To effectively benefit both visual and tactile modalities for object clustering, in this paper, we propose a deep Auto-Encoder-like Non-negative Matrix Factorization framework for visual-tactile fusion clustering. Specifically, deep matrix factorization constrained by an under-complete Auto-Encoder-like architecture is employed to jointly learn hierarchical expression of visual-tactile fusion data, and preserve the local structure of data generating distribution of visual and tactile modalities. Meanwhile, a graph regularizer is introduced to capture the intrinsic relations of data samples within each modality. Furthermore, we propose a modality-level consensus regularizer to effectively align thevisual and tactile data in a common subspace in which the gap between visual and tactile data is mitigated. For the model optimization, we present an efficient alternating minimization strategy to solve our proposed model. Finally, we conduct extensive experiments on public datasets to verify the effectiveness of our framework.


VisTaNet: Attention Guided Deep Fusion for Surface Roughness Classification

Human texture perception is a weighted average of multi-sensory inputs: ...

Visual-Tactile Sensing for In-Hand Object Reconstruction

Tactile sensing is one of the modalities humans rely on heavily to perce...

Visual-Tactile Multimodality for Following Deformable Linear Objects Using Reinforcement Learning

Manipulation of deformable objects is a challenging task for a robot. It...

Multi-view Clustering with Deep Matrix Factorization and Global Graph Refinement

Multi-view clustering is an important yet challenging task in machine le...

Elastic Tactile Simulation Towards Tactile-Visual Perception

Tactile sensing plays an important role in robotic perception and manipu...

A Framework for Multisensory Foresight for Embodied Agents

Predicting future sensory states is crucial for learning agents such as ...

Graph-Collaborated Auto-Encoder Hashing for Multi-view Binary Clustering

Unsupervised hashing methods have attracted widespread attention with th...

Please sign up or login with your details

Forgot password? Click here to reset