On Evolving Attention Towards Domain Adaptation

03/25/2021
by   Kekai Sheng, et al.
0

Towards better unsupervised domain adaptation (UDA). Recently, researchers propose various domain-conditioned attention modules and make promising progresses. However, considering that the configuration of attention, i.e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario. For the first time, this paper proposes EvoADA: a novel framework to evolve the attention configuration for a given UDA task without human intervention. In particular, we propose a novel search space containing diverse attention configurations. Then, to evaluate the attention configurations and make search procedure UDA-oriented (transferability + discrimination), we apply a simple and effective evaluation strategy: 1) training the network weights on two domains with off-the-shelf domain adaptation methods; 2) evolving the attention configurations under the guide of the discriminative ability on the target domain. Experiments on various kinds of cross-domain benchmarks, i.e., Office-31, Office-Home, CUB-Paintings, and Duke-Market-1510, reveal that the proposed EvoADA consistently boosts multiple state-of-the-art domain adaptation approaches, and the optimal attention configurations help them achieve better performance.

READ FULL TEXT
research
04/14/2020

Multi-source Attention for Unsupervised Domain Adaptation

Domain adaptation considers the problem of generalising a model learnt u...
research
05/21/2018

DiDA: Disentangled Synthesis for Domain Adaptation

Unsupervised domain adaptation aims at learning a shared model for two r...
research
12/20/2022

ADAS: A Simple Active-and-Adaptive Baseline for Cross-Domain 3D Semantic Segmentation

State-of-the-art 3D semantic segmentation models are trained on the off-...
research
12/20/2021

Reciprocal Normalization for Domain Adaptation

Batch normalization (BN) is widely used in modern deep neural networks, ...
research
10/25/2019

Low-Resource Domain Adaptation for Speaker Recognition Using Cycle-GANs

Current speaker recognition technology provides great performance with t...
research
03/18/2023

Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation

For video models to be transferred and applied seamlessly across video t...
research
08/19/2016

Automatic Selection of Context Configurations for Improved Class-Specific Word Representations

This paper is concerned with identifying contexts useful for training wo...

Please sign up or login with your details

Forgot password? Click here to reset