Neural Style Transfer and Unpaired Image-to-Image Translation to deal with the Domain Shift Problem on Spheroid Segmentation

Background and objectives. Domain shift is a generalisation problem of machine learning models that occurs when the data distribution of the training set is different to the data distribution encountered by the model when it is deployed. This is common in the context of biomedical image segmentation due to the variance of experimental conditions, equipment, and capturing settings. In this work, we address this challenge by studying both neural style transfer algorithms and unpaired image-to-image translation methods in the context of the segmentation of tumour spheroids. Methods. We have illustrated the domain shift problem in the context of spheroid segmentation with 4 deep learning segmentation models that achieved an IoU over 97 whose performance decreased up to an 84% when applied to images captured under different conditions. In order to deal with this problem, we have explored 3 style transfer algorithms (NST, deep image analogy, and STROTSS), and 6 unpaired image-to-image translations algorithms (CycleGAN, DualGAN, ForkGAN, GANILLA, CUT, and FastCUT). These algorithms have been integrated into a high-level API that facilitates their application to other contexts where the domain-shift problem occurs. Results. We have considerably improved the performance of the 4 segmentation models when applied to images captured under different conditions by using both style transfer and image-to-image translation algorithms. In particular, there are 2 style transfer algorithms (NST and deep image analogy) and 1 unpaired image-to-image translations algorithm (CycleGAN) that improve the IoU of the models in a range from 0.24 to 76.07. Therefore, reaching a similar performance to the one obtained with the models are applied to images following the training distribution.


page 5

page 12


Time-of-Day Neural Style Transfer for Architectural Photographs

Architectural photography is a genre of photography that focuses on capt...

Generating Embroidery Patterns Using Image-to-Image Translation

In many scenarios in computer vision, machine learning, and computer gra...

Random Style Transfer based Domain Generalization Networks Integrating Shape and Spatial Information

Deep learning (DL)-based models have demonstrated good performance in me...

AdaWCT: Adaptive Whitening and Coloring Style Injection

Adaptive instance normalization (AdaIN) has become the standard method f...

Review Neural Networks about Image Transformation Based on IGC Learning Framework with Annotated Information

Image transformation, a class of vision and graphics problems whose goal...

Image-to-Image Translation: Methods and Applications

Image-to-image translation (I2I) aims to transfer images from a source d...

Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation

In this work, we present a fully automatic method to segment cardiac str...

Please sign up or login with your details

Forgot password? Click here to reset