The electrocardiogram (ECG) records the electrical signals generated by the heart by using external electrodes, which is a painless and non-invasive way to help diagnose many heart conditions [15, 24]. The standard ECG consists of 12 leads, looking at the heart from various angles. There are six limb leads and six precordial leads. The limb leads (I, II, III, aVR, aVL, and aVF) record signals of the heart at the arms and/or legs, while the precordial leads (V1-V6) are placed on the chest, and record anteriorly directed electrical activity). The limb and precordial leads are mutually orthogonal.
Recently, there have been several studies on deep learning models using ECGs to support the diagnosis of cardiac diseases such as atrial fibrillation[3, 11], heart failure with preserved ejection fraction , myocardial infarction [1, 5], and hypertrophic cardiomyopathy .
With the advancement of smart devices, various mobile and wearable devices have appeared and enable access to the ECG immediately anywhere [2, 6, 10, 23]. These devices obtain a couple of leads or multiple leads  that do not meet the diagnostic capabilities of the standard 12-lead ECG taken using medical equipment. Insufficient leads result in a missing of particular signals representing the heart condition. This degrades the performance of deep learning models in diagnosing diseases [8, 17] .
However, some leads enable the conjecture of other leads through the combination of their vectors because a lead has the signal projected for the vector representing the magnitude and direction of electrical signals at any particular axis. Though subject records concurrently multiple leads with wearable devices, they could measure two leads (e.g., I & II) asynchronously. With assumption of this circumstance, we propose a deep learning model for ECG synthesis from two asynchronous leads to ten leads (ECGT2T).
For ECGT2T, we adopt the concept of image-to-image translation models based on a generative adversarial network (GAN)[14, 29, 9], which translates source images to reference images by learning styles of reference images. ECGT2T learns the heart condition by referring to two given leads and translates other leads based on any given single lead.
The ECGT2T architecture consists of four networks: The reference encoder represents the heart condition, referring to two given leads. The heart generator produces the heart condition from latent vectors. The lead generator generates ten leads using any single-base lead with the heart condition derived from the reference encoder. The discriminator distinguishes between real and fake lead samples.
ECGT2T provides ten good quality leads, removing artifacts such as noise and baseline wandering, which result in inaccurate and misleading clinical interpretation . In addition, the synthesized ten leads retain the rhythm and amplitudes of an input lead.
To evaluate the availability of ECGT2T in practice, we used it as a data augmentation method. For comparison, we additionally developed a model for ECG synthesis from single leads to the other 11 leads (ECGS2E). We obtained two up-to-date ECG datasets collected at the Physikalisch Technische Bundesanstalt  and Chapman University and Shaoxing People’s Hospital , respectively.
We constructed six lead-combination sets: single-lead ECG set (i.e., I), two-lead ECG set (i.e., I & II), asynchronous two-lead ECG set (i.e., I & II), 12-lead ECG set by ECGS2E (i.e. I & generated 11 leads), 12-lead ECG set by ECGT2T (i.e., I & II, and ten generated leads), and original 12-lead ECG set. We then applied them to train ResNet 
to classify heart diseases. As a result, the ResNet using ECGs augmented by ECGT2T outperforms other models in all metrics, except for the model using the original 12-lead ECG set. It is obvious that the accuracy and performance of the model that uses a single-lead ECG is the least reliable. This result implies that the portable devices could full use of diagnosis for cardiac diseases, though they provide insufficient leads.
Our contribution is as follows.
We proposed ECGT2T to generate ten leads from two asynchronous leads representing the heart condition to construct the standard 12-lead ECG.
We showed that both the rhythm and amplitude of leads generated by ECGT2T closely resemble those of the original ones. We also identified that ECGT2T removes noise and baseline fluctuations appearing in the original leads.
We validated the practicality of ECGT2T by using it as a data augmentation method. The trained model by augmented dataset outperforms those by datasets composed with insufficient leads.
Ii Proposed method
Ii-a Problem Definition
ECG represents atrial and ventricular depolarization and repolarization as a series of waves: P wave, QRS complex, and T wave. The standard ECG consists of six limb leads and six precordial leads. The six limb leads record the electrophysiologic changes of the heart in a horizontal plane, while the other six (precordial) leads record changes in the heart at the axial plane. A lead has a signal projected for the vector representing the magnitude and direction of electrical activity generated by the cardiomyocytes at any particular axis. Therefore, a complete 12-lead ECG, which takes account of various directions of this electrical activity, provides a global view of the physiological status of the heart. The use of only one or two leads in an ECG results in a limitation in the interpretation of the heart condition due to spatial information loss for some leads.
For example, when the right coronary artery, which supplies blood to the inferior aspect of the heart, is occluded by a blood clot, the inferior leads of the ECG (namely, II, III, and aVF) capture the ST segment deviation. Without the inferior leads, it is difficult to reach the precise diagnosis. Previous studies also reported that the lack of availability of leads using wearable devices might result is misdiagnosis of acute myocardial infarction .
A study reported that subjects record multiple leads using a recently introduced wearable device, providing the facility for only single-lead ECG recording, depending on different positions of the device 
. The hypothesis of using at least two asynchronous leads with the aim of extrapolating to the real-life situation of 12 leads appears to be impossible at first sight. However, their vector operation enables the estimation of other leads. We propose a method for the ECG synthesis from two asynchronous leads to ten leads; we have called this ECGT2T .
The mechanism of ECGT2T is similar to that of image-to-image translation models, which translate a source image to an image with the style of a reference image [19, 29, 9]. ECGT2T first encodes a heart condition (style) from two leads (reference), and then generates ten leads based on the heart condition. Figure 1 shows the architecture of the ECGT2T. It is based on GAN consisting of four networks: a reference encoder, a heart generator, a lead generator, and a discriminator. Let be any lead of the 12-lead ECG, where ranges from I to V6. The roles of the networks are as follows:
Reference encoder : This encodes a representation for the heart condition for a corresponding lead using two leads (namely, ). It consists of three modules where two reference, an encoding, and multiple lead-specific modules. Each reference network extracts features from I & II, respectively. The encoding network captures the heart condition representation for all leads from an input concatenated with two features. Each lead-specific network encodes a representation of the heart condition corresponding to lead .
Heart generator : This generates a representation for heart condition for each lead, given latent codes (namely, ). This consists of encoding and multiple lead-specific modules. The encoding network produces the common code for the heart condition, which is used to generate via a lead-specific network for the corresponding .
Lead generator : Ten leads are generated using a base lead with representations for the heart condition(namely, ). There are both encoding and decoding modules. Given as any single-base lead, the encoding network extracts the features. Then, the decoding network generates from features injecting on the adaptive instance normalization .
Discriminator : It distinguishes whether the input criteria is real or not (namely, ). It is composed of only a single discriminative module as the role of the discriminator of a traditional conditional GAN .
Our framework includes four objective functions as follows.
Adversarial objective: As an adversarial loss in a typical GAN, we calculate the average of the log probability for ten real leads and the log of the inverted probabilities of the generated ones. The adversarial loss is expressed as follows:
where is set to I, and can be alternated to by .
Lead reconstruction: To construct ten realistic leads using , we set an objective for lead reconstruction by comparing the original and generated leads using the following equation:
where the mean squared error is used since it is sensitive to outliers (e.g., R peaks) in the ECGs.can be alternated to
Lead consistency: To maintain the consistency of generated leads regardless of any base lead, we define the objective for the lead consistency as follows:
where or are I and II, respectively, and can be alternated to . The mean squared error was used as .
Heart condition reconstruction: To represent heart conditions well, we compared the outputs between and . This objective affects the two networks complementarily.
where the mean absolute error is used.
This section evaluates the quality of ECGT2T ECGs compared to the original ECGs. For the availability of ECGT2T ECGs, ECGT2T is used as the augmentation method to complement insufficient leads to construct the standard 12-lead ECG.
Iii-a Experimental setup
Table LABEL:tab:data lists the datasets used in our experiments, which denotes the number of ECG samples and the proportion of samples for cardiac diseases. We obtained both (1) a large publicly available ECG dataset collected at the Physikalisch-Technische Bundesanstalt (PTB-XL)  and (2) an ECG dataset collected at Chapman University and Shaoxing People’s Hospital (CUSPH) .
The PTB-XL dataset provides ECGs with multiple diagnostic statements. This is composed of 21,837 standard 12-lead ECGs, each of which has 10 seconds length with a sampling rate of 500Hz. We selected 15,012 ECGs with 9,527 normal ECG samples and 5,485 ECG samples from patients with myocardial infarction, respectively. CUSPH provides 10,646 ECGs with diverse types of arrhythmia, each of which consists of 12 leads taken over a period of 10 seconds with a 500 Hz sampling rate. For cardiac rhythms, we selected ECG samples for 7,314 normal and 1,780 atrial fibrillation ECG samples. Both datasets were split into training, validation, and test with a ratio of 7:1:2. Each split dataset has the same distribution for normal and cardiac disease ECG samples.
|Dataset||#total||#train||#validation||#test||Prop. samples for diseases|
We implemented models for two different tasks: ECG generation and ECG classification. To demonstrate the superiority of ECGT2T, we developed another generative model for ECG synthesis from single to 11 leads (ECGS2E). For hyperparameters, we set the size of the style vector to 512, the size of the latent vector to 512, and the learning rate to . is set to 2, while the others are set to 1. Under the above settings, we took seven days to train two generative models (ECGT2T and ECGS2E) on the PTB-XL training set.
To evaluate how realistic 12-lead ECGs constructed by generative models are, we applied them as data augmentation methods. When a classification model is given some leads, it first constructs a 12-lead ECG using any generative model, and then learns using them. As a classification model, we adopt ResNet18  which uses a focal loss with and set to 0.5 and 2, respectively. Adam as an optimization method is used with the learning rate and weight decay set to and respectively.
All our models are implemented on the PyTorch framework and executed on a server equipped with Intel Xeon Silver 4210, 256 GB memory, and four GEFORCE RTX 3090 with 24 GB memory.
Iii-B ECG generation
We compared the 12-lead ECGs generated using ECGS2E and ECGT2T over the original ECGs. Figure 4 shows the ECG samples for both PTB-XL and CUSPH datasets. The black, red, and blue lines indicate the signals for the original, ECGS2E, and ECGT2T, respectively. The x-axis indicates the time in the second unit, and the y-axis indicates the amplitude of the signal. There are two subfigures (3(a) and 3(b)) for the PTB-XL dataset and a subfigure (3(c)) for the CUSPH dataset as an unseen sample. ECGS2E synthesizes 11 leads because it is trained using only Lead I, while ECGT2T using asynchronous I & II generates 10 leads. For convenience, all leads are illustrated synchronously, although ECGT2T uses two asynchronous leads.
The ECG in Figure 3(a) is an example of a normal subject. In all ECGs, the amplitudes of the signals over seconds are similar. As expected, the 10 leads generated by ECGT2T are nearly equivalent to the original ones. Interestingly, the leads of ECGS2E, including even Lead II, resemble the original ones. In addition, we note that there are effects for the removal of noise (e.g., aVR, aVL) and baseline wandering (e.g., V6) by generative models.
The ECG in Figure 3(b) is an example of a subject with myocardial infarction, where blood flow to a particular part of the heart decreases or ceases completely. This typically leads to ST-segment elevation, T-wave inversion, and abnormal Q waves on the ECG . As for Lead II in the ECG for ECGS2E, an incorrect signal is generated because of the absence of another vector to the heart. This causes ECGS2E to not draw well parts of the signals in the middle of Lead II and V1. On the other hand, ECGT2T is less seriously affected and generates an ECG similar to the original 12-lead ECG, recognizing the electrical pathophysiology of the heart more accurately using two vector signals.
Figure 3(c) shows an ECG for a subject with atrial fibrillation, which is typically characterized by the absence of P waves, irregularly irregular ventricular rate, and slightly aberrant QRS complexes . Multiple signals in the ECGS2E stray from those in the original ECG. The signals of ECGT2T are very similar to the original signals, even though the CUSPH dataset is not used for training.
The results show that generative models produce tracings almost identical to the original ECGs, especially in the normal ECG sample. In addition, they have the potential to correct the measurement errors of the original ones.
Iii-C ECG classification
To show the effectiveness of ECGT2T as the data augmentation method for the ECG classification, we configured six lead-combination sets and train the models using them. We first introduce a model using the ECGs comprising with only Lead I used as inputs for ECGS2E, called Single. There are two models using the two-lead ECG sets composed of both synchronous I & II and the asynchronous I & II used as inputs for ECGT2T, called to Two and AsyncTwo, respectively. We also implement two other models using ECGS2E and ECGT2T as augmentation methods. They are named to S2E and T2T, respectively. Finally, the model as a baseline uses the original standard 12-lead ECGs, named to Original.
As evaluation metrics, we used the area under the receiver operating characteristic curve (AUROC) with 95% confidence interval(CI) and the area under the precision-recall curve (AUPRC) with 95% CI as metrics.
Figure 2 shows the performance of the classification of myocardial infarction on the PTB-XL dataset. Except for the model using Original, the model using T2T outperforms the other models. However, the performance of the model using S2E is worse than that of the models using two leads (AsyncTwo and SyncTwo). This result also shows only single-lead ECGs performance poorly in the diagnosis of myocardial infarction.
Because myocardial infarction is caused by ischemic necrosis of myocardium supplied by the occluded coronary artery, a 12-lead ECG is required to diagnose the particular type of myocardial infarction, effectively to narrow down to the distribution of the exact blocked vessel. In other words, myocardial infarction cannot not be diagnosed precisely without all the leads affected by each sub-type of infarction: right coronary artery leads (i.e., II, III, and aVF), left anterior descending artery leads (i.e., V1-V4), and left circumflex artery (i.e, V5-V6, I, and aVL), named to inferior infarction, anteromedial infarction, and anterolateral infarction, respectively.
Figure 3 depicts the classification performance for the atrial fibrillation case on the CUSPH dataset. In an AUROC metric, there are fewer differences in the performance of the models on the CUSPH dataset compared to that on the PTB-XL dataset. In fact, the model using Single is slightly better than that using AsyncTwo. However, in the AUPRC metric, the model using T2T is still better than the other models, except for the base line model.
Unlike myocardial infarction, arrhythmia like atrial fibrillation can be diagnosed if there are the irregularly irregular rhythmic patterns in any lead. In other words, it is not required for use of whole 12 leads to observe the rhythm. Therefore, we found the narrow gap of model performances for the arrhythmia classification.
ECG is a painless and non-invasive tool for diagnosing cardiac diseases. It records the electrical signals from different directions through 12 leads for the heart. Recent wearable devices such as Galaxy Watch and Apple Watch provide a facility to access ECG with a couple of leads. However, an ECG with insufficient leads can result in an inaccurate diagnosis of cardiac disease.
To address this issue, we propose a deep generative model based on GAN for ECG synthesis from two asynchronous leads to ten leads called ECGT2T. This model produces ten leads referring to a single base lead with the representation of the heart condition. It consists of four networks. The reference encoder encodes a representation of the heart condition using two leads. The heart generator also generates a representation of the heart condition using a latent code. The lead generator produces ten leads. The discriminator distinguishes whether the given leads are real or not.
ECGT2T generates the standard 12-lead ECG almost identical to the real ECG and provides the effect of removing noise and base line wandering, causing errors in diagnosis. For its capacity as a data augmentation method, it constructs the realistic 12-lead ECG using only a couple of leads. Such ECGs improve the performance of the classification model for diagnosing cardiac diseases.
With the rapid dissemination of many smart watches, we expect that ECGT2T provides facilities to correct measurement errors in ECGs and helps more accurate diagnosis for heart conditions.
Application of deep convolutional neural network for automated detection of myocardial infarction using ecg signals. Information Sciences 415, pp. 190–198. Cited by: §I.
-  AliveCor kardiamobile. External Links: Cited by: §I.
An artificial intelligence-enabled ecg algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. The Lancet 394 (10201), pp. 861–867. Cited by: §I.
-  (2019) Novel use of apple watch 4 to obtain 3-lead electrocardiogram and detect cardiac ischemia. The Permanente Journal 23. Cited by: §I.
-  (2019) Classification of myocardial infarction with multi-lead ecg signals and deep cnn. Pattern Recognition Letters 122, pp. 23–30. Cited by: §I.
-  Beurer mobile ecg device. External Links: Cited by: §I.
-  (2008) ECG signal denoising and baseline wander correction based on the empirical mode decomposition. Computers in biology and medicine 38 (1), pp. 1–13. Cited by: §I.
-  (2020) Artificial intelligence algorithm for detecting myocardial infarction using six-lead electrocardiography. Scientific reports 10 (1), pp. 1–10. Cited by: §I.
Stargan v2: diverse image synthesis for multiple domains.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8188–8197. Cited by: §I, §II-B.
-  Galaxy watch. External Links: Cited by: §I.
Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature medicine 25 (1), pp. 65–69. Cited by: §I.
-  (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778. Cited by: §I, §III-A2.
-  (2017) Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510. Cited by: §II-B.
Learning to discover cross-domain relations with generative adversarial networks.
International Conference on Machine Learning, pp. 1857–1865. Cited by: §I.
-  (2006) Correlation of symptoms to ecg diagnosis following atrial fibrillation ablation. Journal of cardiovascular electrophysiology 17 (2), pp. 146–150. Cited by: §I.
-  (2020) Detection of hypertrophic cardiomyopathy using a convolutional neural network-enabled electrocardiogram. Journal of the American College of Cardiology 75 (7), pp. 722–733. Cited by: §I.
-  (2017) Biometric verification by cross-correlation analysis of 12-lead ecg patterns: ranking of the most reliable peripheral and chest leads. Journal of electrocardiology 50 (6), pp. 847–854. Cited by: §I.
-  (2020) Artificial intelligence assessment for early detection of heart failure with preserved ejection fraction based on electrocardiographic features. European Heart Journal-Digital Health. Cited by: §I.
-  (2017) Unsupervised image-to-image translation networks. arXiv preprint arXiv:1703.00848. Cited by: §II-B.
-  (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. Cited by: §II-B.
-  (2016) Arrhythmia essentials e-book. Elsevier Health Sciences. Cited by: §III-B.
-  (1998) Is this patient having a myocardial infarction?. Jama 280 (14), pp. 1256–1263. Cited by: §III-B.
-  PocketECG. External Links: Cited by: §I.
-  (1968) A point-score system for the ecg diagnosis of left ventricular hypertrophy. American heart journal 75 (6), pp. 752–758. Cited by: §I.
-  (2019) Single-lead ecg recordings including einthoven and wilson leads by a smartwatch: a new era of patient directed early ecg differential diagnosis of cardiac diseases?. Sensors 19 (20), pp. 4377. Cited by: §II-A.
-  (2020) Multichannel electrocardiograms obtained by a smartwatch for the diagnosis of st-segment changes. JAMA cardiology 5 (10), pp. 1176–1180. Cited by: §I, §II-A.
-  (2020) PTB-xl, a large publicly available electrocardiography dataset. Scientific data 7 (1), pp. 1–15. Cited by: §I, §III-A1.
-  (2020) A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients. Scientific data 7 (1), pp. 1–8. Cited by: §I, §III-A1.
-  (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223–2232. Cited by: §I, §II-B.