DeepAI AI Chat
Log In Sign Up

Mixup for Test-Time Training

by   Bochao Zhang, et al.

Test-time training provides a new approach solving the problem of domain shift. In its framework, a test-time training phase is inserted between training phase and test phase. During test-time training phase, usually parts of the model are updated with test sample(s). Then the updated model will be used in the test phase. However, utilizing test samples for test-time training has some limitations. Firstly, it will lead to overfitting to the test-time procedure thus hurt the performance on the main task. Besides, updating part of the model without changing other parts will induce a mismatch problem. Thus it is hard to perform better on the main task. To relieve above problems, we propose to use mixup in test-time training (MixTTT) which controls the change of model's parameters as well as completing the test-time procedure. We theoretically show its contribution in alleviating the mismatch problem of updated part and static part for the main task as a specific regularization effect for test-time training. MixTTT can be used as an add-on module in general test-time training based methods to further improve their performance. Experimental results show the effectiveness of our method.


page 1

page 2

page 3

page 4


MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption

An unresolved problem in Deep Learning is the ability of neural networks...

Changing Model Behavior at Test-Time Using Reinforcement Learning

Machine learning models are often used at test-time subject to constrain...

Utilizing Excess Resources in Training Neural Networks

In this work, we suggest Kernel Filtering Linear Overparameterization (K...

"Prompt-Gamma Neutron Activation Analysis (PGNAA)" Metal Spectral Classification using Deep Learning Method

There is a pressing market demand to minimize the test time of Prompt Ga...

Treebank Embedding Vectors for Out-of-domain Dependency Parsing

A recent advance in monolingual dependency parsing is the idea of a tree...

TTTFlow: Unsupervised Test-Time Training with Normalizing Flow

A major problem of deep neural networks for image classification is thei...

Updating the VESICLE-CNN Synapse Detector

We present an updated version of the VESICLE-CNN algorithm presented by ...