MATE: Masked Autoencoders are Online 3D Test-Time Learners

11/21/2022
by   M. Jehanzeb Mirza, et al.
0

We propose MATE, the first Test-Time-Training (TTT) method designed for 3D data. It makes deep networks trained in point cloud classification robust to distribution shifts occurring in test data, which could not be anticipated during training. Like existing TTT methods, which focused on classifying 2D images in the presence of distribution shifts at test-time, MATE also leverages test data for adaptation. Its test-time objective is that of a Masked Autoencoder: Each test point cloud has a large portion of its points removed before it is fed to the network, tasked with reconstructing the full point cloud. Once the network is updated, it is used to classify the point cloud. We test MATE on several 3D object classification datasets and show that it significantly improves robustness of deep networks to several types of corruptions commonly occurring in 3D point clouds. Further, we show that MATE is very efficient in terms of the fraction of points it needs for the adaptation. It can effectively adapt given as few as 5 sample, which reduces its memory footprint and makes it lightweight. We also highlight that MATE achieves competitive performance by adapting sparingly on the test data, which further reduces its computational overhead, making it ideal for real-time applications.

READ FULL TEXT

page 4

page 11

research
08/31/2023

Test-Time Adaptation for Point Cloud Upsampling Using Meta-Learning

Affordable 3D scanners often produce sparse and non-uniform point clouds...
research
11/23/2022

ActMAD: Activation Matching to Align Distributions for Test-Time-Training

Test-Time-Training (TTT) is an approach to cope with out-of-distribution...
research
08/21/2020

Orderly Disorder in Point Cloud Domain

In the real world, out-of-distribution samples, noise and distortions ex...
research
08/31/2023

Point-TTA: Test-Time Adaptation for Point Cloud Registration Using Multitask Meta-Auxiliary Learning

We present Point-TTA, a novel test-time adaptation framework for point c...
research
06/01/2022

CAFA: Class-Aware Feature Alignment for Test-Time Adaptation

Despite recent advancements in deep learning, deep networks still suffer...
research
02/09/2023

Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study

A deployed question answering (QA) model can easily fail when the test d...
research
04/10/2023

Revisiting Test Time Adaptation under Online Evaluation

This paper proposes a novel online evaluation protocol for Test Time Ada...

Please sign up or login with your details

Forgot password? Click here to reset