DWMD: Dimensional Weighted Orderwise Moment Discrepancy for Domain-specific Hidden Representation Matching

07/18/2020
by   Rongzhe Wei, et al.
0

Knowledge transfer from a source domain to a different but semantically related target domain has long been an important topic in the context of unsupervised domain adaptation (UDA). A key challenge in this field is establishing a metric that can exactly measure the data distribution discrepancy between two homogeneous domains and adopt it in distribution alignment, especially in the matching of feature representations in the hidden activation space. Existing distribution matching approaches can be interpreted as failing to either explicitly orderwise align higher-order moments or satisfy the prerequisite of certain assumptions in practical uses. We propose a novel moment-based probability distribution metric termed dimensional weighted orderwise moment discrepancy (DWMD) for feature representation matching in the UDA scenario. Our metric function takes advantage of a series for high-order moment alignment, and we theoretically prove that our DWMD metric function is error-free, which means that it can strictly reflect the distribution differences between domains and is valid without any feature distribution assumption. In addition, since the discrepancies between probability distributions in each feature dimension are different, dimensional weighting is considered in our function. We further calculate the error bound of the empirical estimate of the DWMD metric in practical applications. Comprehensive experiments on benchmark datasets illustrate that our method yields state-of-the-art distribution metrics.

READ FULL TEXT
research
02/28/2017

Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

The learning of domain-invariant representations in the context of domai...
research
12/27/2019

HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation

Minimizing the discrepancy of feature distributions between different do...
research
11/16/2017

Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment

A novel approach for unsupervised domain adaptation for neural networks ...
research
09/21/2021

Multi-Source Video Domain Adaptation with Temporal Attentive Moment Alignment

Multi-Source Domain Adaptation (MSDA) is a more practical domain adaptat...
research
06/08/2016

A moment-matching Ferguson and Klass algorithm

Completely random measures (CRM) represent the key building block of a w...
research
03/25/2019

Manifold Criterion Guided Transfer Learning via Intermediate Domain Generation

In many practical transfer learning scenarios, the feature distribution ...

Please sign up or login with your details

Forgot password? Click here to reset