Great advancements have been achieved in machine learning, particularly with supervised learning algorithms, reaching human-level performance on applications that a few years ago would be considered extremely challenging. However, a common assumption in machine learning is that training and test data are drawn from the same probability distribution. Methods are trained on data from a source domain , where is a feature space, the data and the marginal distribution that their features follow. In an image segmentation problem, for example, could be samples (voxels or patches) from multi-spectral MR scans, is the feature space defined by the available MR sequences and is the distribution of intensities in the sequences. In the developing stage of a supervised algorithm, given corresponding ground truth labels , such as segmentation masks, where the label space, a predictive function is learnt via training and configuration of hyper-parameters on the data (). tries to approximate the optimal function that generated . At the time of deployment, however, these methods often under-perform or fail if the testing data come from a different target domain , with and/or . This is because the optimal predictive function for may differ from , and so the learnt will not perform well on . The above scenario is common in biomedical applications due to variations in image acquisition, in particular, in multi-center studies. Training and testing data may differ in contrast, resolution, noise levels () or even type of sequences (). Despite the rapid advancements in representation learning, this issue has been shown to affect even the latest models . Generating labelled databases is time consuming and often expensive, and assuming annotations for training are available for each new domain is neither realistic nor scalable. Instead, it is desired to develop methods that can learn from existing databases and generalize well or adapt to the target domain without the need for additional training data.
Transfer learning (TL)  investigates development of predictive models by leveraging knowledge from potentially different but related domains and tasks. Even between tasks where label spaces and differ, TL can take advantage of similarities in the underlying structure of the mappings and . A subclass of TL is multi-task learning, where a model is trained on multiple related tasks simultaneously. Most related to this work, domain adaptation (DA) is the subclass of TL that assumes and only the domains differ. It explores learning a function that performs well on both domains, under the basic assumption that such a function exists .
In this work we investigate unsupervised domain adaptation (UDA) . In this setting we assume the availability of a labeled database from source domain , along with an unlabeled database from a different but related target domain . We wish to model the unknown optimal function for labelling . However since no labels are available for , cannot be learnt. This is in contrast to supervised DA, which requires at least some labelled data for . Instead, we try to learn a representation that maps and to a feature space that is invariant to differences between the two domains, as well as a function learnt using data , such that approximates and is closer to than any function that can be learnt using only the source data .
Contributions: In this work we develop a domain adaptation method based on adversarial neural networks [5, 6]. We propose the adversarial training of a segmenter and a domain-classifier, which aims to make the representation learnt by the segmenter invariant to domain-specific factors. We describe and analyse the development of domain-adversarial networks for the purpose of segmentation, which to the best of our knowledge has not been previously performed. We investigate the adaptation of layers at various depths and propose multi-connected adversarial networks, which we show improve domain adaptation. We employ our system for the segmentation of traumatic brain injuries (TBI), investigating adaptation between databases acquired using two different scanners with difference in the available MR sequences. We show that without utilizing any labels in the target domain, our method closes the performance gap with respect to supervised learning with target labels to a large extent.
Related Work: TL and DA have attracted significant interest over the years. Comprehensive reviews of early works can be found in [16, 1, 9]. Popularity of TL increased with the wide adoption of neural networks when their features were found to be effective when transferred across tasks. For example, features learnt from natural images were used off-the-shelf for detecting peri-fissural nodules . More commonly, TL is performed via pre-training on a source task, followed by fine-tuning for the target task via supervised training . A representative example of TL via multi-task learning was presented in . A network was trained simultaneously for segmentation of brain tissue, pectoral muscle and coronary arteries. These experiments show that much of a network’s capacity can be shared between a variety of tasks. Note, all of the above require labels in .
In contrast, DA explores the case where label spaces are the same and little or no labelled data is available in . In  the authors explored supervised DA with SVM-based adaptive classifiers in the scenario where source and target data are acquired with different protocols. This method, however, requires labelled target data. Unsupervised DA was tackled in  via instance weighting, but this relies on strong assumptions about the data distributions.  performed UDA with boosted decision stumps with a search for visual correspondences between source and target samples. This is not as flexible as our approach nor scales well to large databases. The authors in  question the feasibility of DA with neural networks on 3D data due to memory requirements. Here, we show that using adversarial 3D networks is indeed a viable approach.
2 Unsupervised domain adaptation with adversarial nets
The accuracy of a binary classifier that distinguishes between samples from two domains can serve as a proxy of the divergence of distributions and , which otherwise is not straightforward to compute. This idea was first introduced in . Inspired by this, the authors of 
presented a method for simultaneously learning a domain-invariant representation and a task-related classifier by a single neural network. This is done by minimizing the accuracy of an auxiliary network, a domain-discriminator, that processes a hidden representation of the main network and tries to classify the domain of the input sample. This approach formed the basis of our work. We below describe its extension for segmentation and our proposed multi-connected system.
2.1 Segmentation system with domain discriminator
At the core of our system is a fully convolutional neural network (CNN) for image segmentation. Given an input of arbitrary size, which can be a whole image or a sub-segment, this type of network predicts labels for multiple voxels in
, one for each stride of the network’s receptive field over the input. The parameters of the networkare learnt by iteratively minimizing a segmentation loss
using stochastic gradient descent (SGD). The loss is commonly the cross-entropy of the predictions on a training batchof samples. In our settings, are sampled from the source database , for which labels are available. We borrowed the 3D multi-scale CNN architecture from , depicted in Fig 1 and adopt the same configuration for all meta-parameters.
Domain discriminator: When processing an input , the activations of any feature map (FM) in the segmenter encode a hidden representation . If samples come from different distributions , e.g. due to different domains, and the filters of the segmenter are not invariant to the domain-specific variations, the distributions of the corresponding activations will differ as well, . This is expected when the segmenter is trained only on samples from where learnt features will be specific to the source domain. Similar to , we choose a certain representation from the segmenter and use a second network as a domain-classifier that takes as input and tries to classify whether it comes from or . This is equivalent to classifying the domain of . Classification accuracy serves as an indication of how source-specific the representation is. The architecture we use for a domain classifier is a 3D CNN with five layers. The first four have 100 kernels of size . The last classification layer uses kernels. This architecture has a receptive field of with respect to its input and was chosen for compatibility with the size of feature maps in the 3 last layers of the segmenter.
We train this domain-discriminator simultaneously with the segmenter. For this, we form a second training batch . Equal number of samples are extracted from and , so there is no bias towards either. is a label that encodes the domain of , used as the training target. is processed by the segmenter, at the same time with or interleaved to lower memory requirements, computing activations . These activations are then processed by the discriminator, which classifies the domain of each sample in . The discriminator’s classification loss is minimized through optimization of the parameters .
A complication arises for the joint training. The samples from are shared in an SGD iteration for the two losses in the algorithm of . However, many segmentation methods use weighted sampling in order to mitigate class-imbalance, for example by oversampling rare classes [10, 7, 14]. Such sampling requires segmentation masks that are not available for whose samples are extracted randomly. In this case, the discriminator should not compare those against non-randomly extracted samples from , as it could easily associate activations for the over-weighted classes with domain and fail to learn useful domain-discriminative features. Hence, we resort to forming entirely separate batches. is formed of 20 image segments, randomly extracted from images in and . As done in , weighted sampling is used for extracting 10 segments from to form . This ensures countering of class-imbalance for the segmenter, while being unbiased on the samples used for the discriminator.
Domain adaptation via adversarial training: We aim at adapting the representation to become invariant to variations between and . To this end, we expose the accuracy of the domain-discriminator to the segmenter and let it alter its parameters such that its FMs that comprise do not contain cues about the input domain. This is done by incorporating the domain-discriminator’s loss into the training objective of the segmenter, which now aims to simultaneously maximize the domain classification loss and minimize the segmentation loss , or:
is a positive weight that defines the relative importance of the domain-adaptation task for the segmenter. This optimization is possible with regular SGD, as the adversarial networks are interconnected and gradients of can propagate back through the discriminator and into the segmenter. This process was implemented in  via a custom gradient-reversal layer, which is not needed if the optimization is formulated as in Eq. (1), as also noted by the authors.
2.2 Multi-connected adversarial networks
A natural question to arise concerns which layer(s) of the segmenter should be adapted. In , the authors investigated which of the last three fully connected layers of an AlexNet leads to better accuracy when adapted via MMD , concluding it is the last hidden layer that is optimal in their settings. Earlier layers are commonly not adapted as their features are considered rather generic and transferable across related tasks [5, 13].
We argue that adapting only the last layers might not be ideal, especially for the case of segmentation. The accuracy of classification networks depends mostly on high-level patterns. For precise segmentation, however, fine patterns such as detailed texture and small contrast variations are likely to be important. These fine patterns are extracted in early layers and are more susceptible to image-quality variations between domains. Adapting top layers makes them invariant to such variations, but its still a loss of capacity if such features have been already extracted by early layers, which may not be well adapted by the weakened adversarial gradients that reach them. On the other hand, if only early layers are adapted, assuming that the adaptation is not ideal and the features not entirely free of factors of variation between the two domains, the network could recover source-specific patterns at greater depth. For these reasons we propose an architecture where the domain discriminator is connected at multiple layers of the segmenter. First, this removes source-specific patterns early on but also disallows their recovery at deeper layers. Furthermore, the discriminator is enabled to process a large variety of features for discriminating between the domains, increasing its performance and thus the quality of the gradients for the domain adaptation. Finally, by seeing the whole adversarial network as an auxiliary cost function for the segmenter, this type of connections can be compared with deep-supervision , which allows better flow of the gradients incoming from throughout the segmenter and as such can improve learning of quality features. Our main results are based on feeding input to the discriminator from FMs of layers 4,6 and 8 of both high and low resolution pathways, as well as the 10-th hidden layer of the segmenter (cf. Fig. 1). After the FMs of the low resolution pathway are upsampled, all FMs are cropped to match the size of the deepest layer and concatenated. A detailed analysis of the effect of adapting different layers is presented in Sec. 3.4.
We make use of two databases with multi-spectral MR brain scans of patients with moderate to severe TBI, acquired within the first week of injury. The first database consists of 61 subjects, imaged on a 3-T Siemens Magnetom TIM Trio. The MR sequences are isotropic MPRAGE (1mm), axial FLAIR, T2 and Proton Density (PD) (0.70.75mm), and Gradient-Echo (GE) (0.860.865mm). The second database consists of 41 subjects, imaged on a 3-T Siemens Magnetom Verio. This database includes MPRAGE, FLAIR, T2 and PD sequences, acquired at the same resolution as in the first database. The important difference is that instead of GE, a Susceptibility Weighted Image (SWI) is acquired (0.70.75mm). On both databases, all visible lesions were manually annotated on the FLAIR, GE and SWI by clinical experts. Here, we focus on binary segmentation of abnormalities within the brain tissue, and extra-cerebral pathologies were treated as background. All images are skull-stripped, resampled to isotropic 1mm
and affinely registered to MNI space. Image intensities under the brain masks are normalized to zero-mean and unit-variance, after windowing the lowest and top 2% of the intensity histograms.
3.1.1 Source () and target () databases:
GE and SWI are commonly used in TBI studies due to their great sensitivity to haemorrhages, allowing detection of lesions not visible in other sequences (cf. Fig. 2). SWI is a type of GE that offers greater sensitivity and image quality . For the purpose of this study, the first database, with GE available, is considered the source database used to train the segmenter in a supervised manner. The second database, with SWI available, is considered the target database on which we aim to successfully apply the trained segmenter. This corresponds to a typical scenario where a training database is generated on data coming from one clinical site, and new test data coming from another site with varying protocol. Motivated by the similarity between GE and SWI, we will consider them as an interchangeable input channel to our segmentation system, unless stated otherwise. The difference between GE and SWI is contributing the largest variation between distributions and , although some variation may come from differences between other sequences. Using our unsupervised domain adaptation, we aim to learn features that are invariant to these domain differences without the need for any annotations on the target domain. Treating different sequences as the same input is also considered in , however, using a supervised adaptation approach.
3.2 Configuration of the training schedule
A complication of adversarial training concerns the training schedule of the two connected networks, which influences the way they interact. The strength with which the segmenter is adapting its features in order to counter the domain-discriminator is controlled by the parameter (cf. Eq. (1)). We set for the first epochs and let both networks learn independently. This allows the segmenter to initially learn features for the segmentation of without being influenced by noisy adversarial gradients from an initially poorly performing domain-discriminator. After epochs , when the discriminator’s performance has increased, we start countering it to learn domain invariant features with the segmenter. For this, we increase according to the linear schedule , where and is the maximum weighting, so equals after epoch . Finally, at epoch 43 we start refining the segmenter’s features by gradually lowering its learning rate. The discriminator is optimized with constant learning rate 0.001. In the following, is used. In Sec. 3.4 we present a sensitivity analysis showing robust behavior across a range of values for . , and the total duration of this piecewise linear schedule were determined empirically for satisfactory convergence without prolonging training time. Optimal settings are not fully explored yet and may vary between different tasks and the relative difficulty of each network’s specific task.
We performed multiple experiments to obtain upper and lower bounds of baseline accuracy on the challenging task of TBI lesion segmentation. The experiments are discussed below, quantitative results are summarized in Tab. 1 and examples of segmentations are given in Fig. 2. For a fair comparison, the same 2-fold split of was used in all experiments that utilized annotated samples from .
|Train on S||15.7(13.5)||80.4(12.3)||09.5(09.0)|
|Train on S (No GE/SWI)||59.7(22.1)||55.7(22.6)||69.7(21.5)|
|Train on S UDA to T (ours)||62.7(19.8)||58.9(21.2)||71.6(18.4)|
|Train on T||63.5(20.2)||60.6(21.1)||71.5(19.8)|
|Train on S+T||66.5(17.7)||66.6(19.1)||69.4(19.0)|
|Train on S+T (GE/SWI diff chan.)||64.7(19.2)||65.7(20.2)||67.0(20.8)|
Train on , test on : We perform standard supervised training of the segmenter on without adaptation. To segment , motivated by the similarity between GE and SWI sequences, at test time we use SWI in the channel used for GE during training. Even though these sequences can serve similar purposes in the analysis of TBI by radiologists, this approach totally fails, proving them not directly interchangeable as input to a CNN.
Train on (No GE/SWI), test on : We repeat the previous experiment but only use the common sequences of and in both training and testing, neglecting GE and SWI. The experiment was repeated twice to reduce random variations between training sessions. This corresponds to a practical scenario, where we need to segment by only using annotated training data from , and serves as the lower bound of accuracy for our system.
Train on , test on : We perform a 2-fold validation using supervised training on half of and testing on the other half. We use all sequences of . The obtained performance is similar to what was reported in , although on a different database. This experiment provides another indication for the expected accuracy on this challenging segmentation task.
Train on and , test on : To obtain an upper bound of accuracy, we train the segmenter on all data of and half the data of , using their manual annotations. The same input channel is used for GE of and SWI of . We then test on the other half of data from . The experiment is repeated for the other split of . We balance the samples from the two domains in each batch to avoid biasing the segmenter towards that has more subjects. With supervised training on
, the system learns to interchange GE and SWI successfully. This setting uses all available data from both domains, both images and manual annotations, and serves as an estimate of optimal, supervised transfer learning.
Train on and , test on (GE/SWI in different channels): We perform a sanity check that using GE and SWI in the same input channel is reasonable. We repeat the previous experiment but using a CNN with six channels, with separate ones for GE and SWI. The channel is filled with when the sequence is not available, which corresponds to a very low value after our intensity normalization. From this the CNN learns when the sequence is missing and we found this to behave better than common zero-filling. The segmenter performs better than supervised training on only. This indicates that information from both domains is used. However, knowledge transfer is not as strong as when GE and SWI, which share much information, are used in the same channel.
Proposed unsupervised domain adaptation: We train the segmenter on all data of and adapt the domains using half the subjects of , but no labels. GE and SWI share the same input channel. We test segmentation accuracy on the other half of . The experiment is repeated for the other fold. Our method learns filters invariant to the two imaging protocols and transfers knowledge from to , allowing the system to segment haemorrhages only visible on SWI without ever seeing a manual annotation from (Fig. 2). This improves by 3% DSC over the non-adapted segmenter that uses only information from and the common sequences, covering 44% of the difference between the practical lower bound and the upper bound achieved by supervised domain adaptation using labels from both domains.
3.4 Analysis of system
3.4.1 Effect of adapting layers at different depths:
We investigate how the depth of the adapted layers affects our system. For this, we repeat the experiment with domain adaptation from to , changing the layers from which input to the domain-discriminator is provided. Results are shown on Fig. 3 and Tab. LABEL:table:connections. Note that we connect the discriminator at the same layers of both multi-scale pathways of the segmenter (for example, L4 means connections to the 4th layers of both pathways). Adaptation of early layers tends towards over-segmentation (increased recall but lower precision). It has been noticed that severe over-segmentation occurs without adaptation (Fig. 2). These observations make us believe that the segmenter recovers source-specific features between the adapted and the classification layer. Comparing L2 and L(2,4,6,8,10) shows that this is alleviated by multiple connections that enforce domain invariance throughout the segmenter. Since, however, the behaviour of multi-connected adversarials is strongly defined by the shallowest connection, we avoid adapting the earliest layers which seems less beneficial but would slow down convergence.
3.4.2 Effect of adaptation’s strength via :
Here we investigate the sensitivity of our method with respect to , which defines how strongly the discriminator is countered by the segmenter. Fig. 4 shows that higher values lead to quicker adaptation but the accuracy is rather stable for a significant range of values . We note this range might differ for other applications and that smooth convergence is generally preferred for learning high quality features over steep schedules that alter the loss surface aggressively. Finally, we observe that strongly countering the discriminator does not guarantee better performance on . A theoretical reason is that a more domain-invariant representation likely encodes less information about . This information loss increases the Bayes error rate and the entropy of the predictions by the learnt . After a certain level of invariance, this can outweigh the benefits of domain-adaptation [1, 9].
We present an unsupervised domain adaptation method for image segmentation by using adversarial training of two 3D neural networks. To the best of our knowledge this is the first work of such an approach on a biomedical imaging problem. Additionally, we propose multi-connected adversarial networks, which perform better by enabling flow of higher quality adversarial gradients throughout the adapted network. We investigate aspects of adversarial training such as the depth of the adapted layer and the strength of adaptation, providing valuable insights for development of future approaches. While unsupervised in the target domain, our method performs close to the accuracy of supervised baselines. We believe our work makes an important contribution in the context of multi-center studies where domain differences are a major limitation in current image analysis methods. Future work will investigate the capabilities of such methods on databases with different types of variations. We also intend to explore domain adaptation via minimization of maximum mean descripancy , which has recently shown competitive results outside the biomedical domain [19, 13]. An implementation of the proposed system will be made publicly available on https://biomedia.doc.ic.ac.uk/software/deepmedic/.
This work is supported by the EPSRC (grant No: EP/N023668/1) and partially funded by an European Union Framework Program 7 grant (CENTER-TBI; Agreement No: 60215). Part of this work was carried on when KK was an intern at Microsoft Research Cambridge. KK is also supported by the President’s PhD Scholarship of Imperial College London. VN is supported by an Academy of Medical Sciences/Health Foundation Clinician Scientist Fellowship. DM is supported by the Neuroscience Theme of the NIHR Cambridge Biomedical Research Centre and NIHR Senior Investigator awards. We gratefully acknowledge the support of NVIDIA Corporation with the donation of two Titan X GPUs.
-  Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Mach. learning 79(1-2), 151–175 (2010)
-  Bermúdez-Chacón, R., Becker, C., Salzmann, M., Fua, P.: Scalable unsupervised domain adaptation for electron microscopy. In: MICCAI (2016)
-  Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Schölkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)
-  Ciompi, F., de Hoop, B., van Riel, S.J., Chung, K., Scholten, E.T., Oudkerk, M., de Jong, P.A., Prokop, M., van Ginneken, B.: Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2d views and a convolutional neural network out-of-the-box. MedIA 26(1), 195–202 (2015)
-  Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. Journal of Machine Learning Research 17(59), 1–35 (2016)
-  Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS (2014)
-  Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. MedIA (2016)
-  Heimann, T., Mountney, P., John, M., Ionasec, R.: Learning without labeling: Domain adaptation for ultrasound transducer localization. In: MICCAI (2013)
-  Jiang, J.: A literature survey on domain adaptation of statistical classifiers. URL: http://sifaka. cs. uiuc. edu/jiang4/domainadaptation/survey (2008)
-  Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. MedIA 36, 61–78 (2016)
-  Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. In: AISTATS. vol. 2, p. 6 (2015)
-  Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)
-  Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: ICML (2015)
Moeskops, P., Wolterink, J.M., van der Velden, B.H., Gilhuijs, K.G., Leiner, T., Viergever, M.A., Isgum, I.: Deep learning for multi-task medical image segmentation in multiple modalities. In: MICCAI (2016)
-  van Opbroek, A., Ikram, M.A., Vernooij, M.W., De Bruijne, M.: Transfer learning improves supervised image segmentation across imaging protocols. TMI 34(5), 1018–1030 (2015)
-  Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22(10), 1345–1359 (2010)
Shenton, M., Hamoda, H., Schneiderman, J., Bouix, S., Pasternak, O., Rathi, Y., Vu, M.A., Purohit, M., Helmer, K., Koerte, I., et al.: A review of magnetic resonance imaging and diffusion tensor imaging findings in mild traumatic brain injury. Brain imaging and behavior 6(2), 137–192 (2012)
-  Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. TMI 35(5), 1285–1298 (2016)
-  Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)
Ullman, S., Assif, L., Fetaya, E., Harari, D.: Atoms of recognition in human and computer vision. Proc. of the Nat. Academy of Sciences 113(10), 2744–2749 (2016)
-  Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (Nov 1984), http://doi.acm.org/10.1145/1968.1972