Obstructive coronary artery disease (CAD) is the most common type of cardiovascular disease . Obstructive CAD develops when atherosclerotic plaque builds up in the wall of the coronary arteries, narrowing the coronary artery lumen . This is defined as coronary stenosis, which can potentially limit blood supply to the myocardium, and could lead to ischemia and irreversible damage . Only functionally significant stenoses, i.e. those stenoses which significantly limit blood flow, need to be invasively treated in order to reduce CAD morbidity [3, 4, 5, 6]. Contrarily, invasively treating a functionally non-significant stenosis may lead to harmful output [5, 7]. Therefore, it is crucial to assess the functional significance of a coronary stenosis to guide treatment.
Cardiac CT angiography (CCTA) is typically used to noninvasively identify patients with suspected CAD and visually detect coronary artery stenosis . Although CCTA has high sensitivity in determining the functional significance of the stenosis, its specificity for this task is low [9, 10, 11]. Therefore, to determine whether a coronary artery stenosis is functionally significant, patients with obstructive CAD typically undergo invasive coronary angiography (ICA) to measure the fractional flow reserve (FFR) in the coronary arteries. FFR is currently the reference standard for establishing the functional significance of a coronary stenosis and it is used to guide treatment [3, 4]. However, because of the low specificity of CCTA, up to 50% of patients undergo invasive FFR measurement unnecessarily . To reduce the number of unnecessary invasive procedures, noninvasive determination of the functional significance of stenoses based on CCTA images has been intensively investigated. Several automatic methods for determination the functional significance of coronary artery stenosis in CCTA have been proposed . These methods can be divided into those that simulate and analyze blood flow in the coronary arteries [13, 14, 15, 16], and those that analyze and characterize the left ventricle (LV) myocardium [12, 17].
Methods that simulate and analyze the blood flow in the coronary arteries in CCTA images estimate FFR values along the coronary artery, which can be used to determine the functional significance of coronary artery stenosis. Taylor et al.[13, 18] were the first to propose noninvasive flow-based FFR estimation from CCTA images. To determine FFR values along the coronary artery, computational fluid dynamics, coupled with assumptions of physiological boundary conditions, were used. Itu et al.  also presented a method to estimate FFR in the coronary artery tree in CCTA images by simulating blood flow. This method uses a parametric lumped heart model, while modeling the patient-specific hemodynamics in both healthy and diseased coronary arteries. Nickisch et al.  determined FFR values along the coronary artery by simulating blood flow and pressure along the coronary artery arteries using an electrical patient-specific parametric lumped model. Moreover, Itu et al. 
presented a machine-learning-based model for estimating FFR along the coronary artery. The model is trained on a large number of synthetically generated coronary anatomies, where the target values are computed using a blood flow-based model. While these techniques [13, 14, 15, 16] achieved high accuracy, they are remarkably dependent on the accuracy of coronary artery lumen segmentation . Manual annotation of the coronary artery lumen is a time consuming and a complex task, where commercially available automatic software tools typically require substantial manual interaction and correction, especially in CCTA scans with excessive atherosclerotic calcifications or imaging artefacts due to stents and cardiac motion .
Recently, methods that do not model the blood flow in the coronary arteries but employ characteristics extracted from the myocardium in CCTA scans, have shown to be feasible. Our recent work [12, 21] presented a deep learning approach to automatically identify patients with a functionally significant coronary artery stenosis using analysis of the LV myocardium in CCTA. The method first characterizes the LV myocardium using a convolutional autoencoder (CAE). Thereafter, using the extracted characteristics, patients are classified according to the presence of functionally significant stenosis using an SVM classifier. Previously, Xiong et al.  presented a machine learning based approach to detect patients with anatomically significant stenosis using characteristics of the LV myocardium derived from a CCTA scan. In this method, the LV myocardium is aligned with the standard 17-segments model  to relate each myocardial segment to its perfusing coronary artery. Then, hand-crafted features, describing each myocardial segment, are extracted and used for supervised classification of patients according to the presence of anatomical significant stenosis. Thereafter, Han et al.  employed the technique described in  to detect patients with functionally significant stenosis, as defined by the invasively measured FFR. Although these new methods [12, 17] have presented promising results without the need for accurate coronary artery lumen segmentation, they still need to be validated in large and diverse patients cohorts.
Here, we present a method to automatically and non-invasively classify complete coronary arteries according to the presence of functionally significant stenosis as determined by the invasively measured FFR. Blood flow in the coronary artery may be affected by multiple coronary artery stenoses and arterial plaques [3, 24]. Therefore, to detect the presence of a functionally significant stenosis in an artery, local analysis of a single stenosis may be insufficient. Hence analysis of the complete artery should be performed. Moreover, in clinical practice, usually a single, i.e. lowest, FFR value per coronary artery is reported. Consequently, employing supervised machine learning methods to directly analyze a whole volume of an artery (e.g. with 3D-CNN or ) to detect the functional significance of each stenosis or estimating the invasively measured FFR values at every point along the coronary artery is unfeasible. Therefore, in the proposed work, a complete artery is analyzed in an unsupervised manner to extract lower-dimensional encoding, and thereafter to determine the presence of abnormal FFR. First, using the extracted coronary artery centerline , the straightened 3D multi-planar reformatted (MPR) volume is reconstructed. Then, an MPR volume of a complete artery is characterized with a fixed number of encodings using convolutional autoencoders (CAEs) [27, 28, 29], which serve as unsupervised feature extractors. As MPR volumes of complete coronary arteries have large volumetric sizes and variable lengths and shapes, a single traditional CAE cannot be successfully and directly applied to efficiently encode a complete artery. Therefore, in the here proposed work, two disjoint CAEs are employed. The first CAE performs spatial encoding of local sub-volumes along the artery. Then, a second CAE encodes the output of the first CAE - which depends on the artery length - into a fixed-length encoding. Finally, a support vector machine (SVM)  classifies arteries based on these encodings according to the presence of functionally significant stenosis, as defined by the invasively measured FFR. The proposed approach is illustrated in Fig. 1. Our contributions are twofold. Firstly, we propose to jointly employ two disjoint CAEs that perform spatial and sequential encoding of large volumes with varying lengths. Secondly, in contrast to previous methods that detect the presence of functionally significant stenosis or determine FFR values non-invasively, our method does not require accurate and difficult to obtain segmentation of the coronary artery lumen or LV myocardium. Instead, it only requires the coronary artery centerline, which can be obtained automatically or semi-automatically .
Ii-a Patient and Image Data
This study includes retrospectively collected CCTA scans of 187 patients (age: years, 145 males) acquired between 2012 and 2016. The Institutional Ethical Review Board waived the need for informed consent.
All CCTA scans were acquired using an ECG-triggered step-and-shoot protocol on a 256-detector row scanner (Philips Brilliance iCT, Philips Medical, Best, The Netherlands). A tube voltage of 120 kVp and tube current between 210 and 300 mAs were used. For patients kg contrast medium was injected using a flow rate of 6 mL/s for a total of 70 mL iopromide (Ultravist 300 mg I/mL, Bayer Healthcare, Berlin, Germany), followed by a 50 mL mixed contrast medium and saline (50:50) flush, and next a 30 mL saline flush. For patients kg the flow rate was 6.7 mL/s and the volumes of the boluses were 80, 67 and 40 mL, respectively. Images were reconstructed to an in-plane resolution ranging from 0.38 to 0.56 mm, and 0.9 mm thick slices with 0.45 mm spacing.
In each CCTA scan, coronary arteries were tracked and their centerlines were extracted using the method previously described by Wolterink et al. . The method tracks the visible coronary arteries, where the arterial centerlines are extracted between the ostia and the most distal visible locations. Using the extracted centerlines, a 3D straightened MPR volumes with 0.3 isotropic resolution were reconstructed for all coronary arteries and used for further analysis.
Ii-B FFR Measurements
Out of the 187 patients, 137 patients suspected of obstructive CAD underwent invasive FFR measurements (, interquartile range: 0.74-0.89), up to one year after the acquisition of the CCTA scan. In these patients, FFR was measured in 192 different arteries. FFR was recorded with a coronary pressure guidewire (Certus Pressure Wire, St. Jude Medical, St. Paul, Minnesota) at maximal hyperemia conditions. Maximal hyperemia was induced by administration of intravenous adenosine (at a rate of 140 g/kg per minute) through a central vein. The FFR wire was placed at the most distal part possible in the target artery. Using manual pullback, a single minimal FFR value was assessed and recorded for each artery.
Blood flow in the coronary artery may be affected by a single or multiple coronary artery stenoses [3, 24]. Therefore, to detect the presence of a functionally significant stenosis in an artery, local analysis of a single stenosis may be insufficient. Moreover, in clinical practice, determining invasive FFR values for each voxel within the artery lumen, or recording an FFR value for each point on the coronary artery centerline, is impractical and typically not performed. Instead, the minimal single FFR value per coronary artery is recorded, resulting in a single reference label per artery. Hence, employing supervised end-to-end machine learning methods to directly analyze a whole MPR volume of an artery (e.g. with 3D-CNN or ) to detect the functional significance of each stenosis or estimating the invasively measured FFR values at every point along the coronary artery is infeasible. Therefore, in the proposed work, an MPR of a complete artery is analyzed to determine the presence of abnormal FFR. First, to extract robust features of complete arteries, MPR volumes are characterized by a fixed number of encodings using convolutional autoencoders (CAEs) [27, 28, 29], regardless of the artery length. Then, the extracted encodings are used as input to an SVM classifier that determines whether the artery contains a functionally significant stenosis.
Iii-a Encoding the artery
The main purpose of a convolutional autoencoder (CAE) is to extract robust compact features from unlabeled data, while removing input redundancies and preserving essential aspects of the data [27, 28, 32]. A CAE consists of two main parts, an encoder and a decoder [27, 28]. The encoder compresses the data to a lower dimensional latent space by convolutions and down-sampling. The decoder expands the compressed form to reconstruct the input data by deconvolutions and upsampling. A CAE is trained to minimize a difference loss between the encoder input and decoder output. This ensures that the encodings contain sufficient information to reconstruct inputs with low error . Once the CAE is trained, the decoder is removed and the encoder is used to generate encodings for unseen data.
Coronary arteries are complex anatomical 3D structures, with varying lengths and anomalies across patients . The resolution of modern CT scanners is high and a large number of voxels (millions) is contained in an MPR volume of a single artery. Therefore, following the straightforward approach of training a single CAE, applied directly to the complete artery volume without a large reconstruction error, is infeasible. Therefore, in this work, we propose a two-stage encoding approach to encode a complete MPR volume of the coronary artery, regardless of its length. Fig. 2 illustrates the proposed encoding flow. First, a 3D variational convolutional autoencoder (3D-VCAE) is applied to local sub-volumes extracted from the MPR along the artery centerline. As the 3D-VCAE is only applied to small input volumes, the number of its trainable parameters is relatively low. The 3D-VCAE encodes each sub-volume into a set of small number of encodings. When applied to all sequential sub-volumes along the artery, the result is a features map of the same height as the number of encodings and the same length as the artery length. This features map is then represented as a set of individual 1D sequences of encodings. Each sequence contains an individual encoding out of the set of encodings, running along the artery (colored signals in Fig. 2). This allows the analysis of complete arteries with varying length by a 1D convolutional autoencoder (1D-CAE), with low number of trainable parameters which decreases the chance of overfitting. Hence, the 1D-CAE encodes the varying length sequences of encodings further into a fixed number of encodings, that represent the complete artery, regardless of its length.
Iii-A1 Spatial encoding with 3D variational convolutional autoencoder
VAEs are generative models, which approximate data generating distributions . Through approximation and compression, the resulting models have been shown to capture the underlying data manifold; a constrained, smooth, continuous, lower dimensional latent (feature) space where data is distributed [34, 35]. Inspired by these advantageous properties of the latent space, a VCAE is employed to compress and encode local volumes along the artery. To capture local volumetric characteristics of the artery, the input to the 3D-VCAE is set to a volume of 40x40x5 voxels, centered around a coronary artery centerline point. The size of the input is chosen so that it contains the whole arterial lumen and the vicinity of the artery . The output of the encoder in the 3D-VCAE is set to 16; i.e. an encoding in a
latent space. The dimension of the encoding is determined in preliminary experiments to balance between the compactness and the expressiveness of the encodings. To encode the complete artery, overlapping volumes with stride of 1 are extracted and encoded with 3D-VCAE (Fig.2). This results in 16xL encodings, where L is the length of the artery. The 3D-VCAE architecture used in this work is shown in Fig. 3
(a). In the 3D-VCAE , batch normalization
layers and rectified linear units (ReLUs) are used after all convolution layers except the encoder and decoder output layers.
Iii-A2 Sequential encoding with 1D convolutional autoencoder
When representing the coronary artery to determine the functionally significant stenosis according to FFR, characteristics along the artery, starting from the ostium to the most distal part of the artery, need to be taken into account [13, 14, 15]. Therefore, to analyze the complete artery at once, the local encodings extracted previously by the 3D-VCAE along the length of the artery need to be merged. To accomplish this, the features map, consisting of L sets of 16 values of encoding generated by 3D-VAE at each coronary artery center point, is represented as L 1D sequences. Each sequence consists of 1xL values, where L represents the length of the artery, i.e. number of coronary artery centerline points. To encode arteries with different lengths, sequences of encodings of short arteries were padded into a maximum length 800, which corresponds to the longest artery in the dataset. This representation leads each sequence to represent a specific member of the encoding in the latent space along the artery (colored 1D signals in Fig. 2). This, consequently, allows us to apply a 1D-CAE to each of the 16 sequences separately. The weights of the 16 1D-CAEs are shared, where each 1D-CAE encodes one of the 16 sequences into an encoding of a second latent space of 64 dimensions (). This results in 1024 (16x64) features that represent the complete artery. The 1D-CAE architecture used in this work is shown in Fig. 3(b). In the 1D-CAE, the exponential linear units (ELUs) are used after all convolutions layers except the encoder and decoder output layers.
Iii-B Classification according to the presence of functionally significant stenosis
Based on the extracted encodings from the encoding stage, arteries are classified according to the presence of functionally significant stenosis. This was defined by the invasively measured FFR. In our experiments, the threshold on FFR value was set to 0.9 resulting in a positive class with representing patients with likely present stenosis requiring treatment and a negative class with representing absence of functionally significant stenosis. This threshold was set to identify and exclude arteries and patients unnecessarily undergoing invasive FFR. The classification is performed using an SVM classifier with a linear kernel and an regularization. For each classified artery, the continuous output of the trained SVM is used to assign a predicted class.
As patients with suspected obstructive CAD undergo ICA to measure the FFR in all diseased coronary arteries, in this work, classification of patients is also performed. To classify patients, the highest output value of all classified arteries in a patient is used to assign a predicted class to the patient. The minimal FFR across the arteries of a patient is taken as a reference.
Classification performance is evaluated using a receiver operating characteristic (ROC) curve and the corresponding area under the ROC curve (AUC).
Iv Experiments and Results
Iv-a Encoding the artery
To train the 3D-VCAE and the 1D-CAE, a set of CCTA images of 50 patients, who did not undergo ICA and hence had no FFR measurements, were used. From these, 38 CCTA images were randomly selected for training, and the remaining 12 images were used for validation. In both sets, MPR volumes of the extracted arteries were used to train and validate the autoencoders. Both autoencoders’ hyperparameters were determined in preliminary experiments using the validation set.
To train and validate the 3D-VCAE, 40x40x5 voxels volumes were randomly extracted along centerlines of arteries in the training and validation sets, respectively. Mini-batches of 32 volumes were used to minimize the loss function with Adam optimizer with a learning rate . The mean squared error between the input and the reconstructed volumes, and the Kullback-Leibler (KL) divergence with the reparameterization trick  were employed as a loss function for the variational autoencoder. L2 regularization was used with for all layers. Training was performed until convergence. Fig. 4(a)-(b) show an example of a complete artery which was encoded and reconstructed with the trained 3D-VCAE. This was performed by extracting, encoding and reconstructing input volumes around each point along the MPR centerline. Fig. 4(c) shows the absolute reconstruction error, i.e. the absolute difference between the input and the reconstructed artery. Fig. 4(d) shows all 16 sets of encodings, presented as continuous sequences running along the artery. These sequences are to be encoded further in a later stage using the 1D-CAE into a fixed number of encodings.
To train and validate the 1D-CAE, arteries with the corresponding sets of 16 encodings sequences, obtained by the 3D-VCAE, were randomly chosen from the training and validation sets, respectively. Sequences of encodings of short arteries were padded into a maximum length 800, which corresponds to the longest artery in the dataset. Mini-batches of 32 sets of encodings sequences were used to minimize the loss function with Adam optimizer with a learning rate . The masked mean squared error was employed as a loss function, where padded values in the input sequences did not contribute to the loss value or its gradients and were therefore ignored. L2 regularization was used with for all layers. Training was performed until convergence. Fig. 4(e) shows an example of 3 randomly chosen encoding sequences of a complete artery which were encoded and reconstructed with the trained 1D-CAE.
To demonstrate the effectiveness of the proposed combined two-stage encoding approach in preserving the original shape and appearance of the artery, both disjoint trained autoencoders were combined and tested on complete arteries. To accomplish this, an inference with four steps was performed. First, the encoder of the 3D-VCAE was applied to local volumes along the MPR volume of a complete artery, resulting in 16 sequences of encodings with L values each. Second, the encoder of the 1D-CAE encoded the sequences into a single encoding vector of 1024 values. Third, the decoder of the 1D-CAE decoded the encoding vector back to 16 encodings sequences. Last, the decoder of the 3D-VCAE reconstructed those reconstructed encodings sequences to the original MPR volume size. Fig. 4(a),(f),(g) show an example of a complete artery that was encoded with the combined strategy, reconstructed back to the original volume dimensions, and the corresponding reconstruction error. Fig. 5 compares the average mean absolute reconstruction percentage errors (MAPE) between the local reconstructions made by only the 3D-VCAE and the combined approach reconstruction, across a range of CT Hounsfield units (HU). Fig. 4 and Fig. 5 demonstrate the high resemblance and the low reconstruction error between the results of the local and the combined approaches compared to the original volume.
Iv-B Evaluation of alternative encoding strategies
To demonstrate that the proposed combined encoding strategy is advantageous, two additional encoding strategies were evaluated and compared to the proposed sequential disjoint autoencoders.
First, the most straightforward approach was evaluated, where a single autoencoder analyzes the complete MPR volume, encodes it to a fixed number of encodings (1024), and reconstructs it back to the input size. To handle arteries with different lengths, shorter arteries were padded to the maximal artery length in our dataset (800). Hence, the input of the autoencoder was defined as 40x40x800. The architecture of the evaluated VCAE is shown in Fig. 6.
Second, the 1D-CAE used in the combined encoding strategy (Fig. 2) was replaced by a 2D-CAE to jointly process the encoding sequences. While the proposed 1D-CAE convolved and encoded each encoding sequence separately, the here evaluated 2D-CAE convolved and encoded all sequences of encodings combined. This was performed by representing the encodings map as a 2D image and applying two-dimensional convolutional kernels. The architecture of the evaluated 2D-CAE is identical to the 1D-CAE (Fig. 3(b)), but the convolutions were performed by applying two-dimensional kernels of the same size (3x3). Additionally, dropout of 0.1 was applied between fully connected layers to avoid overfitting.
The two additional autoencoders were trained and validated in a similar manner as in the combined encoding strategy. In the case of the 2D-CAE, the trained decoder of the 3D-VCAE was used to reconstruct the MPR volume. Fig. 7 shows an example of an artery that was encoded and reconstructed using the two evaluated auto-encoding strategies, and compared with the reconstruction of the proposed combined encoding strategy. Fig. 5 also shows the average MAPE across a range of different HUs. Both figures demonstrate a clear advantage for the reconstruction of the combined strategy over the two additionally evaluated approaches.
Iv-C Classification according to the presence of functionally significant stenosis
Classification of arteries was performed using the arteries’ encodings extracted by the two disjoint autoencoders (Section IV-A). All 50 CCTA images of patients used in training and validation of the autoencoders were excluded. Thus, CCTA images of 137 patients and the corresponding reference FFR measurements in 192 different arteries were used for this analysis. To assess the performance and the robustness of the classification, 1000 stratified Monte-Carlo cross-validation experiments were performed. In each experiment 10 arteries were randomly chosen as test set, where all arteries from the same patient were included either in the training or the test set. Optimal SVM parameters were selected in every experiment using a grid search on the training set only.
The obtained results are shown in Fig. 10. On the artery-level, an average AUC of was achieved, while on the patient-level, an average AUC of was achieved.
Iv-D Comparison with other FFR classification methods
We compare the classification results of the here presented method with the results of previous methods. These methods either analyzed of the blood flow in the coronary arteries or analyzed the LV myocardium. Table I lists the results as originally reported. The table demonstrates that the methods achieving highest accuracy and AUC relied on blood flow analysis in the coronary arteries. Moreover, the here presented method seems to outperform the methods that rely on the LV myocardium analysis. However, all of the compared methods were evaluated with different datasets, using different patients cohorts and more importantly employing different FFR thresholds to define the functional significance of a stenosis. Therefore, these results only indicate the differences in performance but should not be directly compared.
), or, as proposed in this work; classifying coronary arteries with features extracted by CAEs.
A method for automatic and non-invasive detection of functionally significant stenosis in the coronary arteries has been presented. The method uses two convolutional autoencoders that characterize the MPR volume of a complete artery with general robust features, and encode the complete artery into a fixed number of encodings to reduce the dimensions of the input. Then, these encodings are used with an SVM classifier to identify arteries with functionally significant stenosis in an supervised manner. Unlike previous methods that detect functionally significant stenosis by relying either on the coronary artery lumen segmentation [13, 14, 15, 16] or the left ventricle myocardium segmentation [12, 17], the proposed method requires only the coronary artery centerline as an input along with the CCTA scan. Artery centerline extraction is a simplified task compared to myocardium segmentation and to the arterial lumen segmentation, where the latter occasionally requires substantial manual interaction, especially in diseased population with heavily calcified arteries. In this work, to extract the coronary artery centerlines, we have employed our previously designed method for artery centerline extraction . However, any other manual, semi-automatic or automatic method could be employed instead.
As the dimensions of MPR volumes of complete arteries are large, and the reference labels are only provided on the artery level, employing a straight-forward supervised 3D-CNN to detect the functional significance of a stenosis is far from feasible. Therefore, here, we have used unsupervised learning to characterize and encode each artery before employing a supervised classifier to detected abnormal FFR. To do so, two disjoint CAEs, that were applied sequentially, were employed. This is contrary to the more common approach of using a single CAE that encodes the complete artery volume at once. The results show that the learned encodings were able to represent the artery shape and appearance accurately, as was demonstrated qualitatively (Fig.4) and quantitatively by the relatively small mean absolute error between the input and the reconstructed volumes (Fig. 5).
The output of the combined encoding strategy was examined and compared to the output of the 3D-VCAE on local volumes. Fig. 4 and Fig. 5 show that, in the range of CT values of the artery lumen (250-450 Hounsfield units), both the local and the combined encoders (with 1D-CAE) achieved satisfactory results, where the local approach was slightly advantageous. However, the lower error achieved with the local approach could be explained by the larger number of encodings used per artery compared to the combined approach. It can be noticed (Fig. 4(b) and (f)) that the reconstructions of both approaches preserved the shape and the morphology of the artery, while not being able to preserve the texture of neither the lumen nor the background. Moreover, due to the low numbers of encodings used in the 3D-VCAE (16), some calcified spots in the artery could not be preserved either (Fig. 4(b)(c) and (f)(g)).
In the proposed combined encoding strategy, both autoencoders were disjoint during training and were combined only during inference, which could lead to error propagation. To overcome this, one alternative would be training both autoencoders simultaneously and end-to-end. However, in preliminary experiments, this was proven difficult, mainly due to hardware limitations. Another alternative would be training the 3D-VCAE separately, and then, using its trained decoder during training the 1D-CAE. This could be done by directly minimizing the mean squared error between the original and the reconstructed MPR volumes of the complete artery instead of between its original and reconstructed encodings sequences. Such training might potentially compensate for errors or prevent error propagation between the two disjoint training processes. Future work might address this.
Additional encoding strategies were performed and compared with the proposed one. Training a single 3D-VCAE (Fig. 6), applied directly to the complete artery volume without a large reconstruction error, was proven infeasible (Fig. 5 and Fig. 7). This could be due to the very large number of trainable parameters () of the CAE and the high variability among the shapes and lengths of the arteries. Moreover, treating all encoding sequences as a single 2D image and encoding this image with a 2D-CAE proved inferior when compared with the proposed 1D-CAE (Fig. 5 and Fig. 7). This might be explained by the lack of local spatial relations between the different encodings along the coronary artery. These spatial relations or local patterns motivate the use of such two dimensional kernels in a typical CAE, when analyzing natural or medical images.
The artery was represented by multiple sequences of encodings, obtained after applying the 3D-VCAE to local sub-volumes along the artery. To further encode the artery to a fixed number of encodings, a 1D-CAE was applied to each sequence of encodings separately. As the fully connected layer in 1D-CAE (layer in Fig. 3(b)) expects a fixed number of inputs, the input sequences were padded to the maximal length of an artery in the dataset. To enable the autoencoder to handle variable length sequences, without the need of padding it, a recurrent autoencoder could be employed 
. In such recurrent autoencoder, known as sequence-to-sequence autoencoders, a recurrent layer, with Gated Recurrent Units (GRUs)
or long short-term memory (LSTM) units, replaces the fully connected layer in the proposed 1D-CAE, to recursively process and encode a sequential varying length input. Future work might investigate this direction. as well.
Our experiments show that moderate classification performance for functionally significant stenosis on both the artery- and patient-levels was achieved, while using only features derived in an unsupervised manner from a CCTA of coronary arteries (Fig. 10). These results show that the proposed approach could potentially lead to a reduction in the number of patients that unnecessarily undergo invasive coronary angiography. For example, as seen in Fig. 10(b), at the sensitivity of 80% or 90% in detecting patients requiring ICA, i.e. those having , unnecessary ICA could have been prevented in 76% or 53% of the negative patients, i.e. those having , respectively.
Unlike this study, most previous methods that analyze blood flow for detection of functionally significant stenosis as determined by invasive FFR differentiate between the functionally significant and non-significant stenoses using a threshold of 0.8 on FFR measurements . However, in preliminary experiments with such a threshold, the proposed method achieved unsatisfactory classification results. The reason may be twofold. First, unlike this study, previous methods don’t analyze a single artery, but they take the blood flow in the complete coronary tree into account. Second, the small number of encodings used in this work only preserved the coarse shape and morphology of the analyzed artery (see Fig. 4(f)). Analysis of the entire coronary tree and/or employing finer morphology characterization might be crucial to differentiate between arteries with functionally significant or non-significant stenoses. Future work might investigate these directions, by analyzing all arteries at once while accumulating their encodings, or by representing the finer morphology of arteries using a larger number of encodings, or the combination of both.
To conclude, this study presented an automatic and non-invasive method for detection of functionally significant stenosis in the coronary arteries in CCTA images. The method is based on two disjoint convolutional autoencoders that characterize and encode volumes of complete coronary arteries into a set of encodings. Thereafter, a support vector machine classifier classifies arteries based on these encodings according to the presence of functionally significant stenosis, as defined by the invasively measured FFR. The achieved moderate classification performance shows the feasibility of reducing the number of patients that unnecessarily undergo invasive FFR measurements.
-  E. J. Benjamin, S. S. Virani, C. W. Callaway, A. M. Chamberlain, A. R. Chang, S. Cheng et al., “Heart disease and stroke statistics-2018 update: a report from the american heart association.” Circulation, vol. 137, no. 12, p. e67, 2018.
-  R. C. Cury, S. Abbara, S. Achenbach, A. Agatston, D. S. Berman, M. J. Budoff et al., “CAD-RADSTM coronary artery disease–reporting and data system. an expert consensus document of the society of cardiovascular computed tomography (SCCT), the american college of radiology (ACR) and the north american society for cardiovascular imaging (NASCI). endorsed by the american college of cardiology,” Journal of Cardiovascular Computed Tomography, vol. 10, no. 4, pp. 269–281, 2016.
-  N. H. Pijls, B. de Bruyne, K. Peels, P. H. van der Voort, H. J. Bonnier, J. Bartunek et al., “Measurement of fractional flow reserve to assess the functional severity of coronary-artery stenoses,” New England Journal of Medicine, vol. 334, no. 26, pp. 1703–1708, 1996.
-  P. A. Tonino, B. De Bruyne, N. H. Pijls, U. Siebert, F. Ikeno, M. vant Veer et al., “Fractional flow reserve versus angiography for guiding percutaneous coronary intervention,” New England Journal of Medicine, vol. 360, no. 3, pp. 213–224, 2009.
-  N. H. Pijls, W. F. Fearon, P. A. Tonino, U. Siebert, F. Ikeno, B. Bornschein et al., “Fractional flow reserve versus angiography for guiding percutaneous coronary intervention in patients with multivessel coronary artery disease: 2-year follow-up of the FAME (fractional flow reserve versus angiography for multivessel evaluation) study,” Journal of the American College of Cardiology, vol. 56, no. 3, pp. 177–184, 2010.
-  L. X. van Nunen, F. M. Zimmermann, P. A. Tonino, E. Barbato, A. Baumbach, T. Engstrøm et al., “Fractional flow reserve versus angiography for guidance of PCI in patients with multivessel coronary artery disease (FAME): 5-year follow-up of a randomised controlled trial,” The Lancet, vol. 386, no. 10006, pp. 1853–1860, 2015.
-  N. H. Pijls, N. Tanaka, and W. F. Fearon, “Functional assessment of coronary stenoses: can we live without it?” European Heart Journal, vol. 34, no. 18, pp. 1335–1344, 2013.
-  M. J. Budoff, D. Dowe, J. G. Jollis, M. Gitter, J. Sutherland, E. Halamert et al., “Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY trial,” Journal of the American College of Cardiology, vol. 52, no. 21, pp. 1724–1732, 2008.
-  W. B. Meijboom, C. A. Van Mieghem, N. van Pelt, A. Weustink, F. Pugliese, N. R. Mollet et al., “Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina,” Journal of the American College of Cardiology, vol. 52, no. 8, pp. 636–643, 2008.
-  F. Bamberg, A. Becker, F. Schwarz, R. P. Marcus, M. Greif, F. von Ziegler et al., “Detection of hemodynamically significant coronary artery stenosis: incremental diagnostic value of dynamic CT-based myocardial perfusion imaging,” Radiology, vol. 260, no. 3, pp. 689–698, 2011.
-  B. S. Ko, J. D. Cameron, M. Leung, I. T. Meredith, D. P. Leong, P. R. Antonis et al., “Combined CT coronary angiography and stress myocardial perfusion imaging for hemodynamically significant stenoses in patients with suspected coronary artery disease: a comparison with fractional flow reserve,” JACC: Cardiovascular Imaging, vol. 5, no. 11, pp. 1097–1111, 2012.
-  M. Zreik, N. Lessmann, R. W. van Hamersvelt, J. M. Wolterink, M. Voskuil, M. A. Viergever et al., “Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis,” Medical image analysis, vol. 44, pp. 72–85, 2018.
-  C. A. Taylor, T. A. Fonte, and J. K. Min, “Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: Scientific basis,” Journal of the American College of Cardiology, vol. 61, no. 22, pp. 2233 – 2241, 2013.
-  L. Itu, P. Sharma, V. Mihalef, A. Kamen, C. Suciu, and D. Lomaniciu, “A patient-specific reduced-order model for coronary circulation,” in Biomedical Imaging (ISBI), 2012 9th IEEE International Symposium on. IEEE, 2012, pp. 832–835.
-  H. Nickisch, Y. Lamash, S. Prevrhal, M. Freiman, M. Vembar, L. Goshen et al., “Learning patient-specific lumped models for interactive coronary blood flow simulations,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp. 433–441.
-  L. Itu, S. Rapaka, T. Passerini, B. Georgescu, C. Schwemmer, M. Schoebinger et al., “A machine-learning approach for computation of fractional flow reserve from coronary computed tomography,” Journal of Applied Physiology, vol. 121, no. 1, pp. 42–52, 2016.
-  G. Xiong, D. Kola, R. Heo, K. Elmore, I. Cho, and J. K. Min, “Myocardial perfusion analysis in cardiac computed tomography angiographic images at rest,” Medical Image Analysis, vol. 24, no. 1, pp. 77–89, 2015.
-  B. L. Nørgaard, J. Leipsic, S. Gaur, S. Seneviratne, B. S. Ko, H. Ito et al., “Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (analysis of coronary blood flow using CT angiography: Next steps),” Journal of the American College of Cardiology, vol. 63, no. 12, pp. 1145–1155, 2014.
-  C. Tesche, C. N. De Cecco, M. H. Albrecht, T. M. Duguay, R. R. Bayer, S. E. Litwin et al., “Coronary ct angiography–derived fractional flow reserve,” Radiology, vol. 285, no. 1, pp. 17–33, 2017.
-  H. Kirişli, M. Schaap, C. Metz, A. Dharampal, W. B. Meijboom, S.-L. Papadopoulou et al., “Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography,” Medical Image Analysis, vol. 17, no. 8, pp. 859–876, 2013.
-  R. W. van Hamersvelt, M. Zreik, M. Voskuil, M. A. Viergever, I. Išgum, and T. Leiner, “Deep learning analysis of left ventricular myocardium in CT angiographic intermediate-degree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis,” European radiology, pp. 1–10, 2018.
-  M. D. Cerqueira, N. J. Weissman, V. Dilsizian, A. K. Jacobs, S. Kaul, W. K. Laskey et al., “Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association,” Circulation, vol. 105, no. 4, pp. 539–542, 2002.
-  D. Han, J. H. Lee, A. Rizvi, H. Gransar, L. Baskaran, J. Schulman-Marcus et al., “Incremental role of resting myocardial computed tomography perfusion for predicting physiologically significant coronary artery disease: A machine learning approach,” Journal of Nuclear Cardiology, pp. 1–11, 2017.
-  B.-K. Koo, H.-M. Yang, J.-H. Doh, H. Choe, S.-Y. Lee, C.-H. Yoon et al., “Optimal intravascular ultrasound criteria and their accuracy for defining the functional significance of intermediate coronary stenoses of different locations,” JACC: Cardiovascular Interventions, vol. 4, no. 7, pp. 803–811, 2011.
-  M. Zreik, R. W. van Hamersvelt, J. M. Wolterink, T. Leiner, M. A. Viergever, and I. Išgum, “A recurrent CNN for automatic detection and classification of coronary artery plaque and stenosis in coronary CT angiography,” IEEE transactions on medical imaging, 2018.
-  J. M. Wolterink, R. W. van Hamersvelt, M. A. Viergever, T. Leiner, and I. Išgum, “Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier,” Medical image analysis, vol. 51, pp. 46–60, 2019.
-  J. Masci, U. Meier, D. Cireşan, and J. Schmidhuber, “Stacked convolutional auto-encoders for hierarchical feature extraction,” in Artificial Neural Networks and Machine Learning–ICANN 2011. Springer, 2011, pp. 52–59.
-  Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, 2013.
-  D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
-  C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.
-  M. Schaap, C. T. Metz, T. van Walsum, A. G. van der Giessen, A. C. Weustink, N. R. Mollet et al., “Standardized evaluation methodology and reference database for evaluating coronary artery centerline extraction algorithms,” Medical Image Analysis, vol. 13, no. 5, pp. 701–714, 2009.
-  Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
-  H. K. Pannu, T. G. Flohr, F. M. Corl, and E. K. Fishman, “Current concepts in multi–detector row CT evaluation of the coronary arteries: principles, techniques, and anatomy,” Radiographics, vol. 23, no. suppl_1, pp. S111–S125, 2003.
D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling, “Semi-supervised learning with deep generative models,” inAdvances in neural information processing systems, 2014, pp. 3581–3589.
-  I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT press Cambridge, 2016, vol. 1.
-  S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of The 32nd International Conference on Machine Learning, 2015, pp. 448–456.
-  D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
-  A. Coenen, M. M. Lubbers, A. Kurata, A. Kono, A. Dedic, R. G. Chelu et al., “Fractional flow reserve computed from noninvasive CT angiography data: diagnostic performance of an on-site clinician-operated computational fluid dynamics algorithm,” Radiology, vol. 274, no. 3, pp. 674–683, 2015.
-  I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104–3112.
-  K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014.
-  S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
-  R. Petraco, S. Sen, S. Nijjer, M. Echavarria-Pinto, J. Escaned, D. P. Francis et al., “Fractional flow reserve–guided revascularization: practical implications of a diagnostic gray zone and measurement variability on clinical decisions,” JACC: Cardiovascular Interventions, vol. 6, no. 3, pp. 222–225, 2013.