Image-to-Image Translation with Multi-Path Consistency Regularization
share
Image translation across different domains has attracted much attention in both machine learning and computer vision communities. Taking the translation from source domain D_s to target domain D_t as an example, existing algorithms mainly rely on two kinds of loss for training: One is the discrimination loss, which is used to differentiate images generated by the models and natural images; the other is the reconstruction loss, which measures the difference between an original image and the reconstructed version through D_s→D_t→D_s translation. In this work, we introduce a new kind of loss, multi-path consistency loss, which evaluates the differences between direct translation D_s→D_t and indirect translation D_s→D_a→D_t with D_a as an auxiliary domain, to regularize training. For multi-domain translation (at least, three) which focuses on building translation models between any two domains, at each training iteration, we randomly select three domains, set them respectively as the source, auxiliary and target domains, build the multi-path consistency loss and optimize the network. For two-domain translation, we need to introduce an additional auxiliary domain and construct the multi-path consistency loss. We conduct various experiments to demonstrate the effectiveness of our proposed methods, including face-to-face translation, paint-to-photo translation, and de-raining/de-noising translation.
READ FULL TEXT