The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution

05/30/2022
by   Ronghan Chen, et al.
0

Rotation-invariant (RI) 3D deep learning methods suffer performance degradation as they typically design RI representations as input that lose critical global information comparing to 3D coordinates. Most state-of-the-arts address it by incurring additional blocks or complex global representations in a heavy and ineffective manner. In this paper, we reveal that the global information loss stems from an unexplored pose information loss problem, which can be solved more efficiently and effectively as we only need to restore more lightweight local pose in each layer, and the global information can be hierarchically aggregated in the deep networks without extra efforts. To address this problem, we develop a Pose-aware Rotation Invariant Convolution (i.e., PaRI-Conv), which dynamically adapts its kernels based on the relative poses. To implement it, we propose an Augmented Point Pair Feature (APPF) to fully encode the RI relative pose information, and a factorized dynamic kernel for pose-aware kernel generation, which can further reduce the computational cost and memory burden by decomposing the kernel into a shared basis matrix and a pose-aware diagonal matrix. Extensive experiments on shape classification and part segmentation tasks show that our PaRI-Conv surpasses the state-of-the-art RI methods while being more compact and efficient.

READ FULL TEXT
research
02/06/2023

PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration

Recent interest in point cloud analysis has led rapid progress in design...
research
03/16/2020

A Rotation-Invariant Framework for Deep Point Cloud Analysis

Recently, many deep neural networks were designed to process 3D point cl...
research
03/06/2023

CRIN: Rotation-Invariant Point Cloud Analysis and Rotation Estimation via Centrifugal Reference Frame

Various recent methods attempt to implement rotation-invariant 3D deep l...
research
11/01/2019

Rotation Invariant Point Cloud Classification: Where Local Geometry Meets Global Topology

Point cloud analysis is a basic task in 3D computer vision, which attrac...
research
07/28/2022

Self-supervised learning with rotation-invariant kernels

A major paradigm for learning image representations in a self-supervised...
research
07/13/2020

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

This paper targets on learning-based novel view synthesis from a single ...
research
04/03/2022

Shape-Pose Disentanglement using SE(3)-equivariant Vector Neurons

We introduce an unsupervised technique for encoding point clouds into a ...

Please sign up or login with your details

Forgot password? Click here to reset