I Introduction
Ia Motivation
Ischemic Heart Disease (IHD) remains a major problem in the United States. It is typically caused by a prolonged coronary artery narrowing, reducing blood oxygen supply to myocardial tissue. Without intervention, this leads to reversible tissue damage, myocardial ischemia and eventually irreversible tissue damage, myocardial infarction. Myocardial infarction leads to scarring, left ventricular (LV) remodeling, and dysfunction. Therefore, a reliable quantitative assessment of regional cardiac function for localization of myocardial injury is valuable for early detection and prompt, targeted interventions. A number of regional quantitative indicators including regional ejection fraction, wall thickness, and wall motion have been proposed utilizing imaging modalities that include Cardiac Magnetic Resonance imaging (CMR), nuclear imaging, and Computed Tomography Angiography (CTA). 4dimensional echocardiography (4DE) has advantages of costeffectiveness and being riskfree to the patient over other modality options. In this work, we focused on developing robust methods for estimation of regional myocardial strain for the left ventricle (LV) from 4DE.
Regional myocardial strain estimation requires accurate and reliable motion tracking of the myocardium. Tracking methods typically follow image appearance or imagederived features over the cardiac cycle to produce a dense Lagrangian displacement field
, where all vectors reference a material point in the enddiastole (ED) frame. Although advantageous in terms of cost and acquisition time, quantitative measurement of displacements from 4DE can be challenging due to image artifacts such as inhomogeneity, bone shadows, and signal dropouts that cause poor motion estimation results. Therefore, intelligent regularization of the dense displacement field is a necessary step for producing more reliable strain analysis, which provides objective evaluation of regional heart health that leads to improved ability for diagnosis and targeted therapy.
IB Related Works
IB1 IntensityBased Tracking
Block Matchingbased assumes a consistent speckle pattern that is propagated from frametoframe across the entire echo sequence. For a particular image patch or ”block”, a search region is defined in the subsequent image frame. Blockmatching is applied to find the block in the subsequent frame that maximizes a similarity metric from the original patch of interest. Tissue motion is estimated by computing the center distance between the block of interest and the block in the subsequent frame with maximum similarity in the search region. These steps are performed for every voxel in the region of interest independently, which renders the resulting displacement field to be noisy both spatially and temporally. Therefore, an optional processing step is often performed, such as smoothing the displacement field [1, 2, 3, 4, 5]. The major drawback of this method is that, even with smoothing, the estimated displacements are often extremely noisy due to independent voxelwise estimations.
Elastic registration or NonRigid registration deforms a particular frame of interest from an image sequence to match a subsequent frame optimally, and as a result, produces a displacement field for the entire image. The resulting displacement field is represented by kernels such as Bsplines, Thinplate splines, or Radial Basis Functions (RBF), which implicitly constrained the displacement field to be spatially and/or temporally smooth. In contrast to block matchingbased approaches estimating motion from each voxel independently, registrationbased approaches simultaneously estimate the motion of each voxel via global optimization. However, this optimization problem is nonconvex; therefore, it is usually solved in multiple steps. First, an affine registration step is taken to approximately align the two images. Then, multiple nonrigid registration steps are taken at coarsetofine scales to avoid local minima
[6]. LedesmaCarbayo first used nonrigid registration registering neighboring frames and parameterized using cubic splines for cardiac motion analysis [7]. Elen et al. used a BSpline transformation model regularized with 3D bending energy and volume conservation penalties on 4DE using mutual information as the similarity metric [8]. However, registrationbased methods tend to be computationally intensive due to solving for all voxel displacements jointly and the necessity for coarsetofine optimization. In addition, registrationbased methods require explicit placement of grid points in the image, and misplaced grid points may bias myocardial motion. Heyde et al. proposed a LVshaped coordinate system parameterized with Bsplines [9], but any anatomicalbased coordinate systems require an accurate segmentation of the myocardium, which is also a difficult task. To regularize spatiotemporal cardiac motion, LedesmaCarbayo et al. registered the entire 2D+time echo image sequence (i.e. ED frame to frame for all in the image sequence) globally by using a spatiotemporal Bspline mode [10]. De Craene et al. extended aforementioned work to the application of 3D+time echo image sequence using a 4dimensional spatiotemporal Bspline model regularizing velocities instead of displacements [11].IB2 FeatureBased Tracking
In contrast to tracking intensity changes, another general approach is to extract relevant image features and track these features over the cardiac cycle. These features include image curvature [12], texture [13] or shapes and surfaces [12, 14, 15, 16, 17]
. Examples of feature extraction include segmentation
[18] for myocardial surfaces or extracting image curvatures [16]. Then, point correspondences are estimated using various matching methods. Iterative Closest Point (ICP) iteratively finds correspondences and transforms points using a least squares transformation [19]. Robust Point Matching (RPM) performs a similar procedure as ICP but includes fuzzy assignment and simulated annealing [20]. Generalized Robust Point Matching (GRPM) extends RPM to include feature distances in addition to Euclidean distance [17]. Feature distances such as Euclidean distance [14], surface features differences [14, 16, 12] or image appearance dissimilarity [15] may be used. Shi et al. proposed matching surface points using shape curvature as features to track and computed confidence measures based on uniqueness and correctness of the match [16]. Papademetris et al. used a similar curvaturebased shape tracking approach and then regularized the displacements by modeling myocardial motion with a linear elastic model [12]. The previously mentioned methods focus on spatial regularization and all lack temporal coherence. Parajuli et al. [14] imposed temporal regularization by modeling cardiac motion as the shortest path through a graph. The points of the graph are surface vertices, and the edge weights are defined based on spatial and feature distance among temporal neighborhood points. This work was extended to impose spatiotemporal constraints [15]. However, feature extraction (e.g. myocardial segmentation) is a challenging problem, especially for echocardiography. Therefore, performance of all surface tracking methods is limited by accuracy of segmentation method. In addition, myocardial surfaces lack shape features for capturing the torsional motion of the heart, and surfacebased features, such as Euclidean distance, curvature[16, 12] or shape context [21] are not rotationally invariant. Thus, torsional motion is often underestimated by surface trackingbased approaches.IB3 Regularization Models
Embedded in intensity and featurebased tracking methods are regularization models that enforce physiologicallyplausible motion behavior, such as spatiotemporal smoothness, tissue incompressibility, and temporal periodicity. Free Form Deformation (FFD) models lay a lattice of control points on the image domain [22, 23]. These control points are displaced from their original locations, and the resulting deformation is represented by a set of polynomial basis functions such as Bsplines. As a result, the local displacements enclosed within the control points are implicitly regularized. The choice of basis function determines local smoothness. For example, inclusion of higher order Bsplines would allow more deformation with the raised possibility of fitting to noise. Therefore, there is an inherent tradeoff between smoothness and accuracy. FFD explicitly defines a rectangular set of grid points, but myocardial deformation requires a more complex geometry. Improperly placed control points may bias displacement estimation. Extended Free Form Deformation (EFFD) models are designed to overcome issues caused by the rectangular grid from FFD [24]. EFFD models defines control point lattices that are adapted to the heart, such as cylindrical or anatomical [7, 10, 9, 11, 25]. These EFFD models that are especially adapted to cardiac deformation typically outperforms standard FFD models, but they are complicated to construct and requires accurate segmentation of the myocardium.
Finite Element Method(FEM) models start by dividing myocardium using meshing techniques that facilitate incorporation of biomechanical modeling parameters. For example, Papademetris et al. [12] imposed a transversely isotropic linear elastic model that incorporated a fiber model that enforced motion in myocardial directions. However, cardiac deformation in ischemic regions is not linearly elastic. Instead, the relationship between stress and segment length was determined as exponential [26]. Furthermore, the finite element mesh was complex and difficult to construct. In contrast to FEM, Radial Basis Function (RBF)based displacement representation do not require explicit mesh construction and are hence referred to as meshfree. Choices for RBF kernels include thinplate splines [27] and Gaussian kernels. Compas et al. specifically used the computationally advantageous Compactly Supported RBF (CSRBF) for both displacement field representation and integration of shape and speckle tracking for strain analysis [28]. Parajuli et al. extended this work to incorporate a sparsity penalty for datadriven selection of RBF centers [29].
The aforementioned regularization models use manually crafted features that may not adhere to typical cardiac motion patterns. Furthermore, they impose spatial regularization only, and extension to spatiotemporal regularization is nontrivial. Utilizing the assumption that a wellregularized displacement field has a sparse representation, we proposed learning a sparse dictionary representation. The learned dictionary was then used for sparse coding of noisy or corrupted displacement estimates to recover the true displacements. However, sparse coding of low error trajectories resulted in quantization errors. To address this, trajectories were classified as either high or low error, and dictionary regularization was imposed on higherror trajectories only. This reduced the bias to low error trajectories when imposed with the learned dictionary but limited regularization effectiveness
[30].In order to address this limitation, we extended the dictionary learningbased regularization to a supervised learning framework with a feed forward neural network (FFNN) for spatiotemporal regularization of noisy displacements. This regularization function was learned by supplying 4D Lagrangian displacement patches to a MultiLayered Perceptron (MLP) network. We then showed the ability of this framework to generalize to various tracking methods. We also proposed combining complementary tracking methods using a multiview learning model and showed further improved tracking and strain estimation performance. Finally, we applied the multiview network to in vivo data and showed plausibility for domain adaptation
[31].IC Key Contributions
(1) 
In this paper, we now fully address the problem of domain adaptation related to our previously proposed neural networkbased approach in [31]. Training a supervised machine learning model requires groundtruth, which is especially difficult to obtain for dense Lagrangian displacements. Crossdomain prediction, where a model was trained in one domain and tested in another, is possible as shown in [31]
, but its performance is typically poor. This is a welldocumented problem in the deep learning community, and domain adaptation, defined as training one network that generalizes to multiple data domains, is needed. Previous domain adaptation methods focused on forcing the two data domains to have indistinguishable latent representations typically trained via an adversarial process
[32, 33]. However, these methods are 1.) dedicated to classification problems, 2.) difficult to train due to adversarial training, and 3.) difficult to interpret. Thus, this work is a substantial expansion of [31], where we make the following contributions:
Develops a complete approach for supervised regularization base on an autoencoder design

Presents a novel semisupervised neural network framework with biomechanical constraints for displacement regularization and domain adaptation.

Validation of proposed methods on in vivo data with implanted sonomicrometers.

Illustrates the promise of proposed method for identifying injury zones using estimated regional strain maps.
Ii FeedForward Neural Network Learning
Iia Tracking Methods for Initial Displacement Estimation
In this work, we utilize the following methods for producing initial noisy estimates of the displacement field for regularization. We chose three different representative methods: Radiofrequencybased Block Matching (RFBM), Flow Network Tracking (FNT), and Elastic Registration with Free Form Deformation model (FFD). These algorithms are described in the following sections.
IiA1 RFbased Block Matching
We utilized the RFbased block matching (RFBM) algorithm from Chen et al [4] as an input to our proposed framework. This method is performed on phasesensitive radiofrequency (RF) images, which precede the logcompression and envelop detection steps and are complex valued. As a result, additional intensitylevel information was retained for tracking in contrast to BMode images, which are filtered for enhanced visualization. RFBM performs tracking in the natural spherical ultrasound coordinate system that spans axial (in the direction of ultrasound beam), lateral, and elevational directions. For each voxel, a block around the voxel was defined, and RFBM searched for its reappearance in the next 3D image frame. The similarity between blocks was measured by a complex normalized crosscorrelation function (NCC) defined in Equation 1, where is the normalized 3D correlation coefficient at voxel , , as a function shifts in 3 directions , , . and are the successive blocks at time and for comparison. Search for the block with highest NCC was confined within a local search region in the successive frame. To ensure spatial smoothness, a spatial smoothness filter was applied to the computed NCC map (i.e. NCC value for each voxel). Finally, the displacement at the voxel of interest ,, was computed as distance between the voxel and maxima of the smoothed NCC map. The subvoxel precision displacement in the axial direction was estimated by finding the zerocrossing of the phase of complex NCC, and a second order polynomial was fitted to the voxellevel displacement field in the elevational and lateral directions. Further details of this method can be found in [5].
IiA2 Flow Network Tracking
We also utilized Parajuli et al. called Flow Network Tracking (FNT) as an input to our proposed framework. First, 3D surfaces of endocardium and epicardium were extracted for each image frame using the segmentation method called Dynamical Appearance Model (DAM) developed by Huang et al. [18]. DAM discriminated class appearance differences at multiple scales by finding sparse representations of image patches for each class (i.e. blood vs. myocardium). The trained dictionaries were updated online through the cardiac cycle leveraging spatiotemporal coherence. ChanVese level set functions were fitted to the discriminated classes to produce smooth myocardial surfaces [34]. FNT sampled points from the extracted myocardial surfaces and assigned these points as nodes on a graph, and edges were the potential paths through the graph. FNT then solved for the optimal flow across the graph given following constraints:

Sum of outgoing flow is less than equal to one

Sum of outgoing flow and incoming flow should be equal
The above problem was solved with Linear Program (LP). The edge weights were precomputed as a function of Euclidean and feature distances among neighborhood points. The feature distances were learned by training a Siamese network that finds an optimal feature distance between two image patches. Feature distance was minimized when two patches were most similar and maximized when most dissimilar
[35]. Details of this algorithm can be found in [15].IiA3 NonRigid Registration
We implemented the elastic registrationbased method developed by Rueckert et al. [6]. This method found an optimal transformation T that mapped every voxel in image at time to a corresponding voxel in reference image frame . This transformation can be split into two components:
The global transformation model is an affine transformation that accounts for any potential global temporal deviations that may be caused by inconsistent placement of the ultrasound probe or breathing from the patient. Local deformation was modeled by , which was described by a 3D Free Form Deformation (FFD) model based on Bsplines. Due to the nonconvex nature of the optimization, we utilized a coarse to fine optimization scheme. Finally, the overall cost function to be minimized was:
where is a similarity function between the transformed image and target image. There were a number of similarity function to use. We chose to use Squared Sum Distance (SSD) which is defined as:
for two arbitrary time points, and . was a 3D bending energy penalty. The objective function was solved with Limited MemoryBroydenFletcherGoldfarbShanno (LBFGS) [36].
This registration method produced a displacement field between two image frames, and we had two ways of utilizing this method to produce a 4dimensional Lagrangian displacement field. In the first approach, we first registered adjacent frames to produce an Eulerian displacement field for each image frame. We then converted the Eulerian displacement field to a Lagrangian displacement field by temporally interpolating the displacements over time. We referred to this approach as FrametoFrame Registration using FFD model (
FFD FtoF). In the second approach, we registered every frame in the cardiac sequence to the enddiastole frame. This approach directly produced a 4D Lagrangian displacement without the need for conversion. We referred to this approach as Frame 1toFrame Registration using FFD (FFD 1toF). Because FFD 1toF does not require temporal interpolation, this method does not incur propagation of error. In contrast, FFD FtoF would propagate errors through the cardiac sequence, which often results in “drift”. On the other hand, theoretically, registering adjacent image frames is an easier task due to relatively smaller deformation, but registering all frames of an image sequence to 1 reference frame requires solving for higher deformation and transformations, which is often problematic. Therefore, these two approaches are complementary. We utilized results with both of these registration approaches as part of our overall work.IiB Synthetic Data
Our use of synthetic dataset was pivotal in not only validating the performance of our algorithms but providing groundtruth for development of learningbased approaches. The synthetic dataset contained eight volumetric cardiac sequences developed by Alessandrini et al. [37] that covered a variety of different physiological conditions. The process of generating these synthetic images was as follows:

Two real apical view clinical acquisitions were acquired from one health volunteer and one patient with dyssynchrony and candidate for cardiac resynchronization therapy (CRT).

The first frame of each acquisition was segmented for both the left ventricle (LV) and right ventricle (RV)

The meshes were passed to a motion simulator i.e. the BestelClementSorine (BCS) Electromechanical simulator

As the parameters of the simulators are varied, different physiological cases were created, and the volume meshes obtained from the simulations were used as groundtruth

Independently, sparse demons was applied to the two real recordings to produce 4dimensional contours for the entire image sequences. These 4D contours were further edited by an experienced cardiologists.

The set of 4D landmarks were used to spatiotemporally align the real clinical image sequences with the simulated image sequences.

To simulate local echogenicity, scattering amplitude from the real recordings were sampled to create a scatter map and input into an ultrasound simulation environment (COLE) [38] to produce the final simulated ultrasound image sequences.
These synthetic image sequences incorporated realistic ultrasounds features that simulated the difficulty in tracking ubiquitous in real ultrasound image sequences. The 8 individual sequences from the dataset simulated different physiological conditions, including one normal, 4 sequences with occlusions in the proximal (ladprox) and distal (laddist) parts of the LAD artery, left circumflex (lcx), right coronary artery (rca), and 3 sequences with dilated geometry with 1 synchronous (sync) and 2 dyssynchronous (lbbb, lbbbsmall). The nondilated geometry image had image sizes of voxels with a voxel size of with frame rate 23 Hz. The dilated geometry had the same image dimensions as the nondilated geometry but acquired with a frame rate of 21 Hz. Each image sequence contained 2250 sparse groundtruth trajectories for interpolation to dense field.
IiC Data Preprocessing
Lagrangian dense displacement field is a 4dimensional vector field, where the displacements at each voxel represent the motion in relation to a material point in the reference point, usually enddiastole in cardiac cycle. 4D patches were extracted from the dense field for learning spatiotemporal patterns. The sparse groundtruth trajectories in our synthetic dataset [37] were spatially interpolated to produce dense groundtruth displacement field. Given sparse trajectories for image frame , we solved for frametoframe groundtruth dense displacement field for frame with the following objective function:
where defined the radial basis function kernels(RBF), are the optimal weights of RBF, and and are hyperparameters for and divergencefree regularization terms. The resulting frametoframe or Eulerian displacement field for all were temporally interpolated with respect to material coordinates of the enddiastole frame and accumulated to produce Lagrangian displacement field for patch extraction.
In order to learn spatiotemporal patterns, overlapping 4dimensional patches were extracted from Lagrangian displacement fields as illustrated in Figure 1a. The 4D patches were flattened and concatenated to form the training data. This was applied to both groundtruth Lagrangian displacement field and initial noisy displacement field estimates to form and , respectively. Corresponding pairs of and patches were fed to the feedforward neural network for learning the regularization function.
IiD Spatiotemporal Regularization Learning
Our goal was to learn the condition distribution that maps the noisy corrupted displacements to by minimizing the negative loglikelihood, which is equivalent to the crossentropy between the data distribution and model distribution [39], defined as:
where are the parameters of the model. The specific form of
determines the loss function. Assuming that
has a Gaussian distribution, then the mean squared error(MSE) loss would was derived as:
where
is a function of the variance. While MSE loss may be used, we chose to use a LogCosh function, which is a smooth Huber loss function that has
behavior for high loss, and behavior for small loss [40]. Thus, our objective function for supervised regularization loss was:where is the total number of data samples. In addition to feeding pairs of noisy and displacement patches to the network model, pairs of and displacement patches were also fed to the model for data augmentation. In this way, the model learned to both regularize higherror displacement patches and avoid biasing lowerror displacement patches via learning the identity function. The process workflow for training the MLP network and prediction were illustrated in Figure 1b.
IiE Combining Complementary Methods
Our goal was to integrate RFbased Block Matching (RFBM) and Flow Network Tracking (FNT), two complementary tracking methods applied to intermodal ultrasound images, for overall improved estimations. RFBM has better performance inside the myocardium but has poor performance near the boundaries due to speckle decorrelation. On the other hand, FNT tracks extracted myocardial surfaces and provides more reliable displacement estimation performance near the boundaries. Therefore, our goal was to capture the complementary nature of the two methods.
Our approach was to learn the relationship of between RFBM and FNT. Multiview learning [41] is a class of machine learning models that combine multiple independent sources of features and has classically been used in the medical imaging community for integrating multiple instances or views of the same object. Inspired by this, we combined the extracted displacement patches from RFBM and FNT at the input layer of our feedforward neural network, and the network produced one set of regularized displacements from both of these sources. As a result, the network implicitly learned to weigh the inputs to produce one set of displacement estimates that captured the complementary nature of the inputs. Figure 2 illustrated the network architecture.
Iii Domain Adaptation for Regularization of In Vivo Data
Our previously proposed method required the availability of true displacement for learning. This groundtruth is difficult to acquire in practice. Furthermore, training a network with data from one domain (i.e. synthetic domain) and applied on another domain (i.e. in vivo domain) was challenging and usually produced poor results.
Iiia Autoencoder with Biomechanical Constraints
We proposed using a biomechanicallyconstrained autoencoder network for learning the latent representation of noisy displacements. Autoencoders must be constrained in order to learn a useful representation, such as undercomplete hidden layers, penalty on the parameters of the hidden layers, or sparsity constraint on the outputs of hidden layers [39]. Without these constraints, the autoencoder would simply learn the identity function. In this work, we utilized prior knowledge that wellregularized displacement patches should be biomechanically plausible. Specifically, cardiac tissue deformation is near incompressible, where the volume of myocardial tissue is constant when deformed. In addition, tissue motion is approximately periodic over the cardiac cycle. Leveraging these assumptions, we introduced biomechanicallyinspired constraints to the autoencoder with the following objective function:
where the first term is data adherence between and predicted displacements . The second term penalized incompressibility at each frame , which was measured with
norm of divergence computed as trace of displacement gradient tensor:
where , , and were the displacement field in , , and directions. The third term penalized nonperiodicity of cardiac motion. Summation of temporal derivatives over the temporal dimension of perfectly periodic Lagrangian displacements is zero. Thus, we penalized the norm of temporal derivative of Lagrangian displacements. and were hyperparameters that controlled the influence of divergence and periodicity constraints, respectively. Utilizing these constraints, the autoencoder was forced to learn a biomechanicallyplausible representation of noisy Lagrangian displacement patches.
IiiB SemiSupervised Learning with Biomechanical Constraints
Finally, we proposed a method for incorporating a supervised learning term into the aforementioned biomechanicallyconstrained autoencoder network. We utilized the key assumption that the neural network model is able to regularize noisy displacements from both synthetic and in vivo data domains. Using this assumption, we introduced a supervised loss into the previous autoencoder objective function that minimized the loss between pairs of synthetic noisy displacement patches and their corresponding groundtruth displacement patches. Thus, our objective function that incorporated the supervised term(boxed) is:
This framework was illustrated in Figure 3. We concatenated both supervised displacement patches (with true displacements) and unsupervised displacement patches (without true displacements) within each minibatch for training the network. Influence of supervised term was controlled by . If , then this network is simply an unsupervised autoencoder model with biomechanical constraints. We applied both unsupervised and semisupervised neural network models on our in vivo dataset and validated with sonometric crystals.
Iv Experimental Results
Iva Synthetic Data Experiments


We quantitatively evaluated the performance of our algorithm on dense trajectories (i.e. trajectory for each voxel in the myocardium). For computational efficiency, we resampled each voxel to 0.5 mm with image size of
voxels. We imposed a leaveoneout scheme, where we trained on 7 images and tested on the 8th image. Training patches were sampled with a stride of 2 in each direction. For normal geometry datasets (
normal, laddist, ladprox, lcx, rca), our patch sizes were five dimensional: for 3 spatial directions, temporal direction, and xyz displacement directions. For dilated geometry datasets (sync, lbbb, lbbbsmall), our patch sizes were. In total, we collected around 100,000 patches for training and 22,000 patches for testing. Our network architecture consists of 3 hidden layers with 1000 nodes per layer along with Dropout with probability of 0.2.
Table Ia shows the median tracking error in mm for various different tracking methods. We applied Dictionary Learningbased Regularization (DLR)[30] and Neural Networkbased Regularization (NNR) to RFBM, FNT, and FFD FtoF estimates. We observed that NNR yielded significant improvements in tracking accuracy for all three methods over both initial tracked and dictionary learningregularized trajectories (DL).
We further compared the approach by Compas et al. [28] with our proposed multiview network architecture for integration of surface tracking (FNT) and speckle tracking (RFBM) methods, with this method denoted as RBFComb. in Table I. We used RBF kernels to interpolate the sparse FNT displacements and RFBM displacements, with each sample weighted by a confidence measure. We assumed that FNT was optimal on the myocardial surfaces; thus, we assigned maximum confidence value for all FNTderived displacements on the surfaces. For RFBM, we used NCC as a confidence measure. We compared the RBFbased combination method with our proposed learningbased integration method, where we input displacement estimates from RFBM and FNT into the multiview learning framework denoted as NNRComb.. Furthermore, we noticed that RBFComb.’s performance was in between the tracking accuracies of FNT and RFBM. Thus, the resulting displacement field estimate was simply an averaging between FNT and RFBM, which resulted in tracking performance that was improved from RFBM but worse than FNT. In comparison, our proposed method NNRComb. produced better tracking performance than both FNT and RFBM, suggesting that it was effective in leveraging the complementary nature of the two methods. NNRComb. produced the highest overall tracking accuracy.
We also analyzed our performance via regional strain analysis, where we computed regional strain from dense displacement fields. The computed strain tensors were projected in clinically relevant radial (Rad.), circumferential (Cir.), and longitudinal (Long.) directions. Strain estimation performances were shown in Table Ib. RFBM produced incredibly high radial strain errors relative to other directions. This was because deformation was highest in the radial direction relative to other directions. Thus, RFBM needed a larger search region to capture high deformation, which meant that it was more likely for RFBM to overfit to noise. Overall, strain performance trends mirrored that of tracking performance. NNR consistently produced improved performances over DLRproduced displacements and all three initial tracking methods. Finally, additional qualitative evaluations of strain curves and maps are shown in [31].
IvB In Vivo Data Experiments
IvB1 Image Acquisition Parameters
We also acquired in vivo 4DE from anesthetized openchest canine. These canine images were acquired using a Philips iE33 scanner (Philips Medical Systems, Andover, MA) and X72 probe and conducted in compliance with Institutional Animal Care and Use Committee policies. We used imaging frequency that ranges from 5060 frames per second and typically produced around 1530 3D volumes for each 4dimensional sequence.
For each study, we acquired images from the animal under 3 physiological conditions. First, we acquired a baseline image (BL) of the animal. We then introduced a total occlusion in the left anterior descending (LAD) artery for simulation of high stenosis (HO). Finally, we pharmacologically induced a stress condition by injecting the animal with relatively lowdose dobutamine ().
IvB2 Effect of Regularization Terms
In this section, we analyzed the effect of the regularization parameters , , and on the performance of our method. Typically, machine learning models trained in one domain has worse performance when applied in another domain. Domain adaptation is the task of mitigating this issue and improving crossdomain prediction performance. We visualized the effect of these regularization parameters on domain adaptation using tSNE, which is a nonlinear dimensionality reduction algorithm commonly used for examining the relationship of latent data representations of from different domains. In our experiment, both synthetic displacement patches and in vivo displacement patches were inputted into our semisupervised learning model, and tSNE was applied to the output of the last hidden layer, which reduced the number of dimensions from 1000 to 2. We plotted the outputs from the hidden layer for three levels of regularization in Figure 4: part (a) no regularization (), part (b) low regularization (), and part (c) high regularization (). In part (a), the hidden layer outputs from the synthetic and in vivo data were completely separated. This indicated that the network implicitly classified the synthetic data (where outputs were true displacements) and in vivo data (where outputs were noisy displacements). As a result, the network predicted in vivo noisy displacements when the input was in vivo noisy displacements, which were not spatially smooth or periodic. This motivated the use of biomechanical regularization to force the predicted invivo noisy displacements to be spatiotemporally regularized, and the resulting displacements would better resemble synthetic displacements, achieving domain adaptation. Thus, we experimented with low regularization and observed a slight “mixing” effect in part (b). We further increased regularization and observed a more significant mixing of the outputs of the two domains in part (c). This suggested that biomechanical regularization positively influenced the domain adaptation ability of the network model.
IvB3 Sonometric Crystals
We used sonomicrometric transducer crystals with recording instrument and software Sonosoft and Sonoview (Sonometrics Corporation, London, Ontario, Canada) for recording intercrystal distances over the cardiac cycle. Cubic arrays with 3 cubes and 16 total crystals were implanted across the myocardium, where 8 crystals were placed near the endocardial surface, and 8 additional crystals were placed near the epicardial surface. One cube is approximately in the infarct zone (Infarct) near the occluded artery. One cube was away from the infarct zone (Remote). The last cube is in the middle of the two previous described cubes (Border). These cubes were shown overlaid on example LV surfaces in Figure 5a. We computed strain from the 3D cubic array of crystals based on the work in [42]. For each cube, we defined approximately 50 tetrahedral units, or elementary units that consists of 4 out of the possible 8 vertices. Strain tensor was computed for each tetrahedral unit of the cube. Then, we computed the median strain tensors computed from all of the tetrahedral units to yield one final strain tensor. Finally, principal strain was computed via eigendecomposition of the strain tensor.
In order to utilize the crystalsderived principal strains for validation, we used reference crystals implanted on the ultrasound transducer. We made assumptions regarding the locations of those reference crystals and solved a system of equations for the locations of 16 myocardial crystals. On the X72 probe, we implanted the crystals in the configuration as seen in Figure 5b. The two crystals facing each other were placed approximately 28 mm apart from each other (as measured by a ruler). The third crystal was approximately 13 mm from the center of the probe surface. We assumed that these crystals were located approximately 5mm from top of the “ultrasound fan”. Based on these assumptions, we estimated locations of all 3 crystals. Then, for each myocardial crystal, the distances to the reference crystal, computed from crystal positions, should equal to the recorded distances from the crystals. We formulated this relationship as a 3 variable 3 equation problem and solved for the crystal locations.
IvB4 Experimental Parameters
For each dataset, we collected image sequences for three different physiological condition. The first condition was Baseline (BL), where the animal is healthy. The second condition was High Occlusion (HO), where we occluded the LAD artery. The third condition was High Occlusion with Dobutamine (HODOB), where we both occluded the LAD artery and stressed the animal. This allowed us to quantify the performance of our algorithm from a clinical perspective, ensuring that we captured the regional variations (i.e. strain variations across the 3 cubes) and physiological variations. Table II describes the invivo data and configurations for training, validation, and testing.
Studies  BL  HO  HODOB  Usage 

DSEA08*  Available  Available  Available  Training, Test 
DSEA09  Available  N/A  N/A  Validation 
DSEA10  Available  N/A  N/A  Training 
DSEA14  Available  Available  Available  Training, Test 
DSEA15  Available  Available  Available  Training, Test 
DSEA16  Available  Available  Available  Training Test 
We computed peak principal strain from each crystal cube and compared with imagederived peak principal strains. We have 4 studies (DSEA08, 14, 15, 16) with 3 physiological conditions (BL, HO, HODOB) with 3 cubes (Infarct, Border, Remote) for each image sequence. In our leaveoneout testing scheme, we have N=36 samples for comparison. Our testing metric was Pearson correlation computed for the 36 samples. We tested our semisupervised learning framework on Flow Network Tracking (FNT), RFBlock Matching (RFBM), Frametoframe nonrigid registration (FFD FtoF) and registering frame 1 to each frame (FFD 1toF). For each experiment, we extracted approximately 100,000 displacement patches from the synthetic datasets and 100,000 displacement patches from the invivo datasets, totaling approximately 200,000 patches. To accommodate for the increase in data, we increased the number of hidden layers from 3 (in our synthetic data experiments) to 7 for these invivo experiments. For each dataset, we computed correlations from peak strains estimated using the initial tracking method (FFD FtoF, FFD 1toF, RFBM, FNT). We also showed the computed correlations from peak strains regularized with a neural network model trained only on the synthetic dataset (Synthetic). Further, we showed computed correlations with an Autoencoder with biomechanical regularization, which was equivalent to setting (Autoencoder). Finally, we showed peak strain correlations with our semisupervised framework (Supervised term and Autoencoder) with biomechanical regularization (Semisupervised), where we set .
IvB5 Comparison of Methods
Studies  RFBM  FNT  FFD FtoF  FFD 1toF 

Without Regularization  0.01  0.17  0.33  0.60 
Synthetic Model  0.15  0.04  0.37  0.49 
Autoencoder  0.27  0.26  0.52  0.60 
SemiSupervised  0.26  0.25  0.52  0.63 
Pearson correlations for the various tracking methods were presented in Table III. RFBMproduced displacements were regularized using and . As expected, RFBM without any regularization produced poor results with correlation of 0.01. The severely low correlation stemmed from the fact that principal strain captured the highest deformation and was similar to radial strain, which increased the possibility of overfitting to noise. A model trained on synthetic data only improved the global correlation to 0.15. However, with using our unsupervised and semisupervised learningbased regularization, we were able to capture higher correlations of 0.27 and 0.26, respectively.
We applied our algorithm to noisy displacement estimates from FNT using and . We observed an overall increase in global correlations from 0.17 to 0.25. The relatively low correlation for FNT was due to poor segmentation results used for surface tracking. Performance of FNT or any feature trackingbased methods was heavily dependent on the accuracy of the feature extraction process. In the case, FNT relied on segmentation accuracy. In our experiments, we used segmentation method described in [18]. In this method, the enddiastole (ED) frame was manually segmented, and the resulting contours were propagated bidirectionally towards the endsystole (ES) frame. As a result, segmentation errors propagated temporally and were highest at ES frame. This was problematic for computing peak strain, which was typically from computed from ED to ES.
We applied our algorithm to noisy displacement estimates from both nonrigid registration approaches. Both of these methods were implemented using and . For FFD FtoF, correlation improved from 0.33 to 0.52 for both unsupervised and semisupervised learning approaches. We observed that FFD 1toF produced the highest correlation of 0.6 compared to FFD FtoF, RFBM, and FNT and improved to 0.63 with semisupervised regularization. FFD 1toF produced higher correlation than FFD FtoF likely due to error of propagation significantly affecting performance of FFD FtoF.
In summary, we observed that supervised loss term did not significantly affect peak strain correlations. This was likely due to synthetic dataset being significantly different from invivo dataset. First, synthetic datasets were significantly less noisy compared to the invivo dataset. Second, the synthetic datasets were acquired from humans, but our invivo datasets were acquired from canines. Third, our synthetic datasets were always oriented vertically in the image domain, but the invivo datasets were acquired in a variety of probe angles, which resulted in the LV being oriented at different angles. Data normalization between the two domains with augmentation to the synthetic domain should significantly improve the performance of our semisupervised regularization approach.
Nonetheless, we observed evidence of improvements in peak strain trends for DSEA16 in Figure 6a. We saw that peak strain in the infarct cubes were in a more clear “V” shape in our proposed approach. Furthermore, in the high occlusion condition dataset (HO), we expected to see a positive gradient, where the ischemic cube had the lowest strain, and the remote cube had the highest strain. RFBMderived peak strains had the opposite trend, which was implausible. After regularization, we observed closer to the “upward” trend or positive gradient that we expected and observed from the sonomicrometer data. Figure 6b illustrated an example of correcting peak strain trend from FFD 1toF to a positive gradient in the HODOB condition after regularization.
IvB6 Combining Multiple Methods
Studies  Correlation 

FNT + FFD FtoF Autoencoder  0.57 
FNT + FFD FtoF SemiSupervised  0.60 
FFD 1toF + FFD FtoF Autoencoder  0.65 
FFD 1toF + FFD FtoF SemiSupervised  0.67 
We tested integration of multiple tracking methods, where we inputted two sets of noisy displacements concatenated at the input layer and produce one set of regularized displacement output. We experimented two combinations that were thought to be promising. We first combined FNT and FFD FtoF. FNT relied on tracking surface points, but nonrigid registration tracked intensity information. Therefore, these two methods provided independent features and produced overall better performance than the individual methods. Specifically, FNT produced 0.17 correlation, and FFD FtoF produced 0.33 correlation. This combined method produced 0.57 and 0.60 for the autoencoder and semisupervised models, respectively.
We also experimented with utilizing both FFD 1toF and FFD FtoF. FFD 1toF registered each frame in the image sequence to 1 reference frame, and in contrast, FFD FtoF registered adjacent frames in the image sequence and converted the resulting Eulerian displacements to Lagrangian displacements. Theoretically, registering adjacent image frames is easier due to smaller deformation between adjacent frames. In contrast, registering between a reference image frame (e.g. ED) to another frame in the image cycle (e.g. ES) would be harder due to high deformation between the two image frames. On the other hand, the process of converting Eulerian to Lagrangian displacements incurs a propagation of error, but FFD 1toF directly produced Lagrangian displacements that did not require this conversion. Therefore, these two methods were complementary in the temporal domain. Our network combining these two methods produced a correlation of 0.67, which was higher than correlations of 0.33 and 0.60 from the individual methods.
Finally, we showed an example of the FNT and FFD FtoF combination in Figure 7. The first column showed the crystalderived strains. The middle column showed peak strains from FFD FtoF (top) and FNT (bottom) for DSEA15 and DSEA08, respectively. FFD produced higher peak strain for HO condition in the infarct zone than that in BL, which contradicted crystalderived peak strains. Also, FNT produced peak strains in the infarct zone that completely contradicted the crystalderived peak strains and was due to poor segmentation results. Last column showed the peak strains estimated by the combined method. We noticed that peak strain trends had better agreement to corresponding crystalderived strains in both DSEA15 and DSEA08. Overall, our combined approach had a boosting effect, where two sets of independent displacement estimates were jointly utilized to produce better overall peak strain estimations.
IvB7 Comparison to Infarct Zones
Stress echocardiography is useful for detecting hidden ischemia in patients. However, currently the rest and stress images are analyzed empirically for identifying regional dysfunction, which introduces interobserver variability. We developed a quantitative indicator that captures relative strain difference between rest and stress states that we call differential strain.
We collected N=4 animal studies with chronic occlusions conducted in compliance with Institutional Animal Care and Use Committee policies. Specifically, a balloon occlusion was introduced in either the Left Anterior Descending (LAD) or the Left Circumflex (LCX) arteries. This procedure was done on day 0, and the animals was imaged on day 9. For each study, images from two animal states were acquired: rest and stressed with dobutamine. For both rest and stress image sequences, peak strains were computed using FFD 1toF with semisupervised learning regularization.
To compute differential strain, we registered ED frame of rest image sequence to that of stress image sequence, and the resulting transformation was applied to the strain map estimated from rest image sequence. Then, differential strain map was computed as the difference between registered stress strain map and transformed rest strain map. For validation, we extracted postmortem excised LV from the animal, manually traced the infarct regions for each LV slice, and reconstructed the 2D slices into a 3D surface. Using this surface, we manually traced the infarct region onto the ultrasound image. This process was illustrated in Figure 8.



Table V presented the infarct vs. noninfarct peak strains for four studies at baseline rest (a), stress (b), and differential strains (c). In part (a) of Table V, we observed that the change in strain between Infarct and Noninfarct zones was relatively small in the rest study: from in the infarct zone to in the noninfarct zone. However, in the stress study, we noticed that the difference in strain between infarct and noninfarct zones increased dramatically: from to . This suggested that stress imaging was critical for revealing the ischemic zones in the myocardium. The computed differential strain should approximately equal to the difference between the stress imageproduced strains and rest imageproduced strains, but this was not always the case for the four studies. This was likely due to the misalignment between the rest and stress images during the registration process. Nonetheless, we were able to observe enhanced visualizations in Figure 9.
V Conclusion
In this work, we greatly expanded our previous work and added the ability for domain adaptation. First, we illustrated the effectiveness of our supervised neural network regularization model on synthetic data, showing improvements in both tracking and strain estimation performance. We further proposed a novel unsupervised autoencoder network with biomechanical constraints for learning a latent representation that produced more physiologically plausible displacements. We extended this framework to include a supervised loss term on synthetic data and showed the effects of biomechanical constraints on the network’s ability for domain adaptation. To our knowledge, this was the first domain adaptation method for regression in the context of modern machine learning. We validated both the autoencoder and semisupervised regularization method on invivo data with implanted sonomicrometers. Finally, we showed ability of our semisupervised learning regularization approach for identifying infarcted regions using estimated regional strain maps with good agreement to manuallytraced infarct regions from postmortem excised hearts.
As for future directions, we are interested developing generative models for augmenting training of supervised displacement regularization. We believe this would greatly improve generalizability of supervised regularization learning. Furthermore, integration of learningbased regularization loss terms into the objective function of traditional tracking methods such as nonrigid registration would be interesting. We are also interested in experimenting with direct prediction of displacement field, strain map, or level of abnormality from image sequences.
Acknowledgment
The authors would like to thank the past and current members of Dr. Albert Sinusas’ lab group who were involved in the image acquisition process. This work was supported in part by the National Institute of Health (NIH) grant numbers R01HL121226 and T32HL098069.
References
 [1] B. K. Horn and B. G. Schunck, “Determining optical flow,” Artificial intelligence, vol. 17, no. 13, pp. 185–203, 1981.
 [2] B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” 1981.
 [3] M. A. Lubinski, S. Y. Emelianov, and M. O’Donnell, “Speckle tracking methods for ultrasonic elasticity imaging using shorttime correlation,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, vol. 46, no. 1, pp. 82–96, 1999.
 [4] X. Chen, H. Xie, R. Erkamp, K. Kim, C. Jia, J. Rubin, and M. O’Donnell, “3d correlationbased speckle tracking,” Ultrasonic Imaging, vol. 27, no. 1, pp. 21–36, 2005.
 [5] C. Jia, P. Yan, A. J. Sinusas, D. P. Dione, B. A. Lin, Q. Wei, K. Thiele, T. J. Kolias, J. M. Rubin, L. Huang et al., “3d elasticity imaging using principal stretches on an openchest dog heart,” in Ultrasonics Symposium (IUS), 2010 IEEE. IEEE, 2010, pp. 583–586.
 [6] D. Rueckert, L. I. Sonoda, C. Hayes, D. L. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid registration using freeform deformations: application to breast mr images,” IEEE transactions on medical imaging, vol. 18, no. 8, pp. 712–721, 1999.
 [7] M. J. LedesmaCarbayo, J. Kybic, M. Desco, A. Santos, and M. Unser, “Cardiac motion analysis from ultrasound sequences using nonrigid registration,” in International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer, 2001, pp. 889–896.
 [8] A. Elen, H. F. Choi, D. Loeckx, H. Gao, P. Claus, P. Suetens, F. Maes, and J. D’hooge, “Threedimensional cardiac strain estimation using spatio–temporal elastic registration of ultrasound images: A feasibility study,” IEEE transactions on medical imaging, vol. 27, no. 11, pp. 1580–1591, 2008.
 [9] B. Heyde, R. Jasaityte, D. Barbosa, V. Robesyn, S. Bouchez, P. Wouters, F. Maes, P. Claus, and J. D’hooge, “Elastic image registration versus speckle tracking for 2d myocardial motion estimation: A direct comparison in vivo,” IEEE transactions on medical imaging, vol. 32, no. 2, pp. 449–459, 2013.
 [10] M. J. LedesmaCarbayo, J. Kybic, M. Desco, A. Santos, M. Suhling, P. Hunziker, and M. Unser, “Spatiotemporal nonrigid registration for ultrasound cardiac motion estimation,” IEEE transactions on medical imaging, vol. 24, no. 9, pp. 1113–1126, 2005.
 [11] M. De Craene, G. Piella, O. Camara, N. Duchateau, E. Silva, A. Doltra, J. D’hooge, J. Brugada, M. Sitges, and A. F. Frangi, “Temporal diffeomorphic freeform deformation: Application to motion and strain estimation from 3d echocardiography,” Medical image analysis, vol. 16, no. 2, pp. 427–450, 2012.
 [12] X. Papademetris, A. J. Sinusas, D. P. Dione, R. T. Constable, and J. S. Duncan, “Estimation of 3d left ventricular deformation from medical images using biomechanical models,” IEEE transactions on medical imaging, vol. 21, no. 7, pp. 786–800, 2002.
 [13] K. Kaluzynski, X. Chen, S. Y. Emelianov, A. R. Skovoroda, and M. O’Donnell, “Strain rate imaging using twodimensional speckle tracking,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, vol. 48, no. 4, pp. 1111–1123, 2001.
 [14] N. Parajuli, A. Lu, J. C. Stendahl, M. Zontak, N. Boutagy, M. Eberle, I. Alkhalil, M. O’Donnell, A. J. Sinusas, and J. S. Duncan, “Integrated dynamic shape tracking and rf speckle tracking for cardiac motion analysis,” in International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer, 2016, pp. 431–438.
 [15] N. Parajuli, A. Lu, J. C. Stendahl, M. Zontak, N. Boutagy, I. Alkhalil, M. Eberle, B. A. Lin, M. O’Donnell, A. J. Sinusas et al., “Flow network based cardiac motion tracking leveraging learned feature matching,” in International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer, 2017, pp. 279–286.
 [16] P. Shi, A. J. Sinusas, R. T. Constable, E. Ritman, and J. S. Duncan, “Pointtracked quantitative analysis of left ventricular surface motion from 3d image sequences,” IEEE transactions on medical imaging, vol. 19, no. 1, pp. 36–50, 2000.
 [17] N. Lin and J. S. Duncan, “Generalized robust point matching using an extended freeform deformation model: application to cardiac images,” in Biomedical Imaging: Nano to Macro, 2004. IEEE International Symposium on. IEEE, 2004, pp. 320–323.
 [18] X. Huang, D. P. Dione, C. B. Compas, X. Papademetris, B. A. Lin, A. Bregasi, A. J. Sinusas, L. H. Staib, and J. S. Duncan, “Contour tracking in echocardiographic sequences via sparse representation and dictionary learning,” Medical image analysis, vol. 18, no. 2, pp. 253–271, 2014.
 [19] S. Rusinkiewicz and M. Levoy, “Efficient variants of the icp algorithm,” in 3D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on. IEEE, 2001, pp. 145–152.
 [20] A. Rangarajan, H. Chui, E. Mjolsness, S. Pappu, L. Davachi, P. GoldmanRakic, and J. Duncan, “A robust pointmatching algorithm for autoradiograph alignment,” Medical image analysis, vol. 1, no. 4, pp. 379–398, 1997.
 [21] S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE transactions on pattern analysis and machine intelligence, vol. 24, no. 4, pp. 509–522, 2002.
 [22] T. W. Sederberg and S. R. Parry, “Freeform deformation of solid geometric models,” ACM SIGGRAPH computer graphics, vol. 20, no. 4, pp. 151–160, 1986.
 [23] S. Lee, G. Wolberg, K.Y. Chwa, and S. Y. Shin, “Image metamorphosis with scattered feature constraints,” IEEE Transactions on Visualization and Computer Graphics, vol. 2, no. 4, pp. 337–354, 1996.
 [24] S. Coquillart, Extended freeform deformation: a sculpturing tool for 3D geometric modeling. ACM, 1990, vol. 24, no. 4.
 [25] B. Heyde, M. Alessandrini, J. Hermans, D. Barbosa, P. Claus, and J. D’hooge, “Anatomical image registration using volume conservation to assess cardiac deformation from 3d ultrasound recordings,” IEEE transactions on medical imaging, vol. 35, no. 2, pp. 501–511, 2016.
 [26] M. AKAISHI, R. M. SCHNEIDER, P. A. SEELAUS, L. W. KLEIN, J. B. AGARWAL, R. H. HELFANT, and W. S. WEINTRAUB, “A nonlinear elastic model of contraction of ischaemic segments,” Cardiovascular research, vol. 22, no. 12, pp. 889–899, 1988.
 [27] F. L. Bookstein, “Principal warps: Thinplate splines and the decomposition of deformations,” IEEE Transactions on pattern analysis and machine intelligence, vol. 11, no. 6, pp. 567–585, 1989.
 [28] C. B. Compas, E. Y. Wong, X. Huang, S. Sampath, B. A. Lin, P. Pal, X. Papademetris, K. Thiele, D. P. Dione, M. Stacy et al., “Radial basis functions for combining shape and speckle tracking in 4d echocardiography,” IEEE transactions on medical imaging, vol. 33, no. 6, pp. 1275–1289, 2014.
 [29] N. Parajuli, C. B. Compas, B. A. Lin, S. Sampath, M. O’Donnell, A. J. Sinusas, and J. S. Duncan, “Sparsity and biomechanics inspired integration of shape and speckle tracking for cardiac deformation analysis,” in International Conference on Functional Imaging and Modeling of the Heart. Springer, 2015, pp. 57–64.
 [30] A. Lu, M. Zontak, N. Parajuli, J. C. Stendahl, N. Boutagy, M. Eberle, M. O’Donnell, A. J. Sinusas, and J. S. Duncan, “Dictionary learningbased spatiotemporal regularization for 3d dense speckle tracking,” in Medical Imaging 2017: Ultrasonic Imaging and Tomography, vol. 10139. International Society for Optics and Photonics, 2017, p. 1013904.
 [31] A. Lu, M. Zontak, N. Parajuli, J. C. Stendahl, N. Boutagy, M. Eberle, I. Alkhalil, M. O’Donnell, A. J. Sinusas, and J. S. Duncan, “Learningbased spatiotemporal regularization and integration of tracking methods for regional 4d cardiac deformation analysis,” in International Conference on Medical Image Computing and ComputerAssisted Intervention. Springer, 2017, pp. 323–331.
 [32] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” arXiv preprint arXiv:1409.7495, 2014.
 [33] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in Computer Vision and Pattern Recognition (CVPR), vol. 1, no. 2, 2017, p. 4.
 [34] L. A. Vese and T. F. Chan, “A multiphase level set framework for image segmentation using the mumford and shah model,” International journal of computer vision, vol. 50, no. 3, pp. 271–293, 2002.
 [35] S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1. IEEE, 2005, pp. 539–546.
 [36] C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal, “Algorithm 778: Lbfgsb: Fortran subroutines for largescale boundconstrained optimization,” ACM Transactions on Mathematical Software (TOMS), vol. 23, no. 4, pp. 550–560, 1997.
 [37] M. Alessandrini, M. De Craene, O. Bernard, S. GiffardRoisin, P. Allain, I. WaechterStehle, J. Weese, E. Saloux, H. Delingette, M. Sermesant et al., “A pipeline for the generation of realistic 3d synthetic echocardiographic sequences: Methodology and openaccess database,” IEEE transactions on medical imaging, vol. 34, no. 7, pp. 1436–1451, 2015.
 [38] H. Gao, H. F. Choi, P. Claus, S. Boonen, S. Jaecques, G. H. Van Lenthe, G. Van der Perre, W. Lauriks, and J. D’hooge, “A fast convolutionbased methodology to simulate 2dd/3d cardiac ultrasound images,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, vol. 56, no. 2, pp. 404–409, 2009.
 [39] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT press Cambridge, 2016, vol. 1.
 [40] R. He, B. Hu, X. Yuan, L. Wang et al., Robust recognition via information theoretic learning. Springer, 2014.
 [41] S. Sun, “A survey of multiview machine learning,” Neural Computing and Applications, vol. 23, no. 78, pp. 2031–2038, 2013.
 [42] L. K. Waldman, Y. Fung, and J. W. Covell, “Transmural myocardial deformation in the canine left ventricle. normal in vivo threedimensional finite strains.” Circulation research, vol. 57, no. 1, pp. 152–163, 1985.
Comments
There are no comments yet.