The application of neural networks in clinical diagnosis processes has gained popularity in recent years. For instance, there have been numerous studies applying pattern recognition on digital medical images such as X-rays and CT scans of patients to spot irregularities such as indicators of strokes or tumors[dnn-imaging-applications]. Conventionally, such a diagnosis requires trained physicians to spend an extended period of time analyzing measurements, where as neural networks are able to quickly and accurately spot suspicious patterns. Machine-learning-powered services specializing in clinical applications [Shah2019, company:enlitic, company:arterys] have been on the rise lately, and the United States Food & Drug Administration has given neural networks clearance for use in medical services [fda-clearance]. Neural networks are not limited to image classification tasks as Pranav et al. [Pranav2017] have demonstrated that neural networks can be leveraged to discover signs of arrhythmia in electrocardiograms (hereinafter referred to as ECGs).
While neural networks show great promise in automated clinical diagnosis, it could be exposed to the threats of adversarial examples [Szegedy2014]
. Past work shows that a trained classifier to diagnose medical images can be fooled by medical images perturbed by adversarial noise, causing classifiers to make mistakes in diagnosing illnesses[Finlayson2019]. Recent studies [Kurakin2016, Goodfellow2015, Athalye2017, Chen2019, Yakura2019, Tu2020] have also explored the robustness of the adversarial examples in the physical world. There are a few existing studies on generating adversarial perturbations applied on ECGs [Han2019, Chen2019], where generated perturbations are injected into ECG measurements to cause misclassifications in segment-based discovery of atrial fibrillation. We note, however, that these previous studies have been limited to software simulations, with limited discussion on the feasibility of adversarial examples of ECG in the real world. In addition, difficultly in realizing adversarial examples via noise injection into sensors have been reported as well, due to noise effects [Yakura2019, Carlini2018].
Given this background, this work aims to bridge the gap between the threats assessed by the software simulation and those that may arise in the real world. We introduce “adversarial beats”, which are adversarial perturbations tailored specifically against ECG, taking into account the physical constraints in implementing the attack.
Our work explores the following research questions:
RQ1: In the real world, what types of attackers would be motivated to leverage adversarial beats against an ECG diagnosis system, under what constraints?
RQ2: Can we generate adversarial beats against machine-learning-powered ECG diagnosis systems to alter classification results to lead to a meaningful manipulation of ECG diagnosis results?
RQ3: Can we apply the generated adversarial beats for ECGs in a hardware attack, taking into account physical constraints?
Key contributions of our work are summarized as follows: We first provide a plausible implementation scheme of neural networks in automated clinical diagnostics of medical data acquired from patients via monitoring, which without proper precautions could be vulnerable to manipulation. We also clarify the types of attackers that may be motivated to attack such implementations, along with their methods and constraints. We then introduce adversarial beats, which aims to spoof classification results of ECG diagnosis. A success rate of up to 65.4% is achieved during training. Finally, we perform real-world evaluation of the adversarial beats through a PoC hardware attack. 3-5 successful attack cases are achieved throughout 40 attempts. Our analysis gives in-depth insight into the feasibility of the attack as well as the realistic scenarios in which adversaries find clear incentives to perform the attack with adversarial beats. We hope future system designers refer to our work to review the threat of real-world attacks leveraging adversarial examples on ECG diagnosis systems.
2 Background: ECG Monitoring and Classification
In this section, we present the basics of ECG monitoring and describe the ECG heartbeat classification model we adopt in this work.
2.1 ECG monitoring
ECGs are waveforms commonly used to visualize and monitor the electrical activity of a patient’s heart over time, allowing physicians to spot certain patterns entailing potential health risks. One of such patterns are arrhythmias, which is a broad class of irregular heartbeats [ecg-workout]. While most arrhythmias are considered to be harmless, some indicate signs of dangerous heart activities which could lead to fatal conditions [ecg-workout]. While Holter monitors [Galli2016] have been a conventional means to monitor ECGs over a extended period of time, recent development of wearable medical devices [Turakhia2013] offers convenient, non-intrusive methods of monitoring patient ECGs, allowing physicians to analyze heart activities of patients throughout their daily lives. As more convenient means of ECG monitoring develop, however, measured patient data will increase, along with the demand for an efficient and accurate analysis of measured ECGs.
2.2 ECG Classification task
Heartbeat Classification Task
|F||Fusion of V and N|
Convolutional neural networks are capable of categorizing individual heartbeats extracted from an ECG [Kachuee2018], or detecting signs of atrial fibrillation from an arbitrary segment of an ECG [Pranav2017]. To evaluate the performance of arrhythmia classification algorithms, it is essential to have a standard protocol/dataset for researchers/engineers to compare the results. Association for Advancement of Medical Instrumentation (AAMI) recommends specific protocols for evaluating performance of automated ECG classifiers, and ANSI/AAMI EC57 [AAMI] specifies the five types of heartbeats in Table 1: non-ectopic beats (N), supra-ventricular ectopic beats (S), ventricular ectopic beats (V), fusion beats (F) and unclassifiable beats (Q), where N is considered normal heartbeats, S, V, and F are considered arrhythmic beats.
3 Threat Model
In this section, we lay out the target system of our proposed attack. To this end, we consider how machine learning-based ECG classifiers would be generally implemented in clinical settings in the future. Furthermore, we identify potential adversaries in such clinical settings.
3.1 A Model of Target Clinical Diagnosis System
There is a significant incentive in building fully-automated clinical diagnosis system, as it offers a potential means to cutting down healthcare expenses [Finlayson2019]. Analyzing how exactly a potential attacker can manipulate a neural network-based diagnosis system is not a straightforward process, because there is no single diagnosis system infrastructure to consider. We thus propose a reference model, considering what is to be expected from future clinical diagnosis systems. Our reference model shown in Figure 1 revolves around daily monitoring of patient ECGs, aimed towards catching signs of suspicious heart activity entailing potential health risks. These tasks are expected to become increasingly convenient if classifiers are implemented in conjunction with newer IoT medical wearables, allowing physicians to analyze large amounts of monitored data and spotting any irregularities quickly. ECGs measured from IoT wearables on patients are sent to a central classifier trained to diagnose the collected data, which the results are sent to healthcare providers to base their decisions on.
3.2 Potential Adversaries
We now identify potential adversaries that could take advantage of the proposed system with malicious intent. We highlight their motives, as well as their abilities to go about an attack, allowing us to identify potential constraints for each adversary.
All but one of the suggested attackers have the potential of conducting our proposed attack. We note, however, that multiple types of attackers could conspire together and share profits earned, also potentially lifting constraints that otherwise would be imposed if attacking on their own.
Patient: Malicious patients could be incentivised to avoid expenses from procedures or medication, by faking diagnosis results to their favor, such as by feigning normal heartbeats to mask arrhythmias (false-negative). Although in the long run, it is the patient’s health at risk, it is possible that those with monetary incentives attempt such an attack, leading to negative impacts on the patient themselves, and other actors involved in the process. We note that, while we conduct experiments to launch a false-positive attack of arrhythmia discovery, false-negative attacks would be possible by setting different target classes. Patients will spend some arbitrary time with the target IoT device, if not for a prolonged period depending on its application. Though limited in the extent of tampering the device, they do have physical access to the target device itself.
Hospital Personnel: Hospital personnel could be physicians, nurses, or any other personnel in charge of patient care. Malicious personnel could be incentivised to conspire against a patient, manipulating diagnosis results to incur additional fees for unnecessary procedures and checkups. Hospital personnel would have arbitrary access to maintain devices, including IoT devices, giving them just as much access to target devices as patients, if not more.
Medical IoT Manufacturers:
IoT manufacturers would be considered a part of the supply chain, in which devices that they manufacture are then utilized by hospitals and other client facilities. Although indirectly, such manufacturers may be motivated to conspire with hospitals for monetary incentives. Manufacturers would have the most extensive access to the device itself, enabling advanced tampering on the device. Attacks via hardware trojans installed within the supply chain is a known attack vector explored in previous work[Kan2016].
Third-Party Healthcare Organizations: Third-party healthcare organizations refer to organizations such as pharmacies and insurance companies. Their profits and services depend heavily on diagnosis results made by hospitals, and malicious actors within such organizations could be incentivised to conspire with other service providers for their own gain. It should be noted, however, that these organizations would have very limited physical access to the target device itself, making it very difficult for them to act on their own. This makes such malicious parties a co-conspirator, rather than the actual attacker, but with potentially the most monetary incentive out of all other attacker profiles.
The proposed types of potential attackers will be discussed in section 7, when they will be considered whether or not it will be feasible for them to go about with the proposed real-world attack method. In addition to the abilities of the attacker, we consider what they know about the target system. While it is unclear how neural network parameters are disclosed to the public in a future clinical diagnosis system, we assume that critical parameters are undisclosed to the public, given recent advancements in machine-learning security [Papernot2018]. Studies regarding ECG classification tasks, however, often utilize datasets available online, which adversaries could also obtain easily. This leads us to believe that all types of adversaries, including patients, are capable of conducting a black-box attack on the classifier. While it is unclear how clinical institutions will regulate how ECGs are diagnosed by neural networks, we assume that healthcare providers are capable of white-box attacks on such a system, provided that they have insider access to such information, such as the exact means of which ECG is recoreded, preprocessed, and fed into classifiers.
4 Adversarial Beats
This section covers the principles and methods we implement to generate adversarial beats against an ECG diagnosis system. We present the physical constraints for achieving the real-world attack, and describe the adversarial beats generation algorithm against the underlying heartbeat classifier.
4.1 Overcoming Physical Constraints
Adversarial beats must overcome physical constraints posed by the ECG pre-processing pipeline.
To ensure that adversarial beats functions in the physical realm, the following challenges are to be considered:
Real-Time Attack: Existing studies [Han2019, Chen2019] attempted to generate specific adversarial perturbations for each sample, similar to the studies of conventional adversarial examples. From a physical standpoint, however, such an approach is not feasible due to the fact that the signals are to be generated in real-time, as ECGs are measured from a patient.
Generating Short Noises: Adversarial perturbations generated in previous work [Chen2019] are around 30 seconds long at its longest, to approximately 5 seconds long, with diminishing effectiveness the shorter it gets, for certain classes at the point of direct digital injection. Longer noise makes managing where to inject it in the ECG difficult, and may suffer from arbitrary changes in the target ECG (such as heart rate/patient movement) during the noise injection, which is assumed to have an influence on its effectiveness. This leads us to believe that shorter perturbations increase the chances of a successful attack via physical injection. Thus, adversarial beats are optimized to be shorter and robust to shifting within the range of a single heartbeat.
Existence of Physical Band-pass Filters: To make the adversarial perturbations work in the real-world, we need to consider the existence of band-pass filters implemented in the preprocessing stage before classification, which filters out any frequency components in the measured signal outside of a certain range. Conventionally, raw ECG segments are processed through certain bandpass filters to remove unwanted noise artifacts after measurement, commonly caused by improper electrode placements, external devices, or patient movement [Kher2019]. The specifications of noise artifacts considered throughout this work are summarized in Table 2. Although Chen et al. [Chen2019] addressed the issue by applying a frequency limit on the generated perturbations, they did not evaluate its effectiveness in a physical environment. Conventional ECGs are sampled at around 300–360 Hz to ensure no information is lost [Kher2019]. For adversarial beats to maintain their effectiveness after filtering, they must be constrained to the range of frequencies that the filter allows. As done in the work of Chen et al.[Chen2019], we limit the frequency components of the adversarial beats within the range target ECGs are filtered with. Adversarial beats are recorded to the extent of its sampling rate, and thus should have granularity no greater than 300–360 Hz.
|Noise Type||Frequency [Hz]||Causes||Effects|
|Baseline Wandering||0.5||Electrode Placement||Vertical Displacement|
|Powerline Interference||50–60||External Devices||Sinusoidal interference|
|Motion Noise||1–10||Patient movement||Artifacts mistaken as QRS complex|
Existence of Heartbeat Segmentation:
Another component in the ECG pre-processing pipeline is the heartbeat segmentation operation. While there are various methods to diagnose an ECG, we implement beat-by-beat classification–a classification scheme to diagnose individual heartbeats in segments of ECGs. We believe it best represents diagnosis of ECGs that are monitored throughout daily life, as signs of arrhythmia throughout daily activities are indicators of health risks. Due to this fact, we propose that considering beat-by-beat classification a distinction from previous work, as it introduces a different impact compared to segmentation-based classification, which classify entire segments of ECGs to a limited range of classification types. Without any regard for its waveform, adversarial beats can substantially alter the general waveform of the ECG regardless of the implemented filter, which is assumed to be the result of strict constraints. Injected intrusive adversarial beats drastically alter the waveform of the ECG from its original state, causing the beat segmentation algorithm to detect additional nonexistent heartbeats. Discrepancies in numbers of detected heartbeats can be the source of arbitrary errors outputted by the classifier. Thus, in addition to detection from human perception, adversarial beats must avoid detection from such heartbeat segmentation algorithms. Preliminary experiments lead to the conclusion that limiting the amplitude of the adversarial beats is the simplest method of preventing interference with segmentation.
Universal perturbations: Physical adversarial perturbations need to be universal in the following two aspects: (1) Beat Invariance: They must be valid for any heartbeat. There is uncertainty in what the class of the original measured heartbeat pertains to. To maximize the success of spoofing certain classes of heartbeat under any occasion, adversarial beats are trained to be effective on any types of heartbeats, and not just normal ones. (2) Positional Invariance: They need to be valid anywhere in the ECG signal. Similar to past work, the concept of Expectation Over Transformation [Athalye2017] is applied to adversarial beats to be effective regardless of where in the target ECG it is applied, as aiming the injection at an exact relative location in an ECG is difficult for an attacker. Specifically, this technique involves implementing random degrees of horizontal shift when applying adversarial beats to an ECG signal [Chen2019]. This is because it is infeasible for hardware to inject adversarial beats in an exact position on the ECG signal in a physical implementation. This contributes to the positional-invariance, or the universal characteristic regarding injection position.
4.2 Adversarial Beats Generation Algorithm
With the physical constraints shown above, we generate adversarial beats by training them on the preprocessed dataset until the misclassification proficiency ceases to increase. They are optimized solely on a digital environment, utilizing datasets available to the public, with digital transformations to simulate physical constraints. To optimize the amplitude of the adversarial beats without compromising its effectiveness, we implement an iterative algorithm shown in Algorithm 1, located in the appendix. To adversarial beats, we adopt the “Expectation Over Transformation,” proposed by Athalye et al. [Athalye2017]. The algorithm aims to generate adversarial examples that remain adversarial over a chosen transformation in the physical world. It has been adopted in several studies such as Chen et al. [Chen2019] and Brown et al. [Brown2017] for generating robust adversarial examples.
The optimization problem for generating adversarial beats is formulated with Equation 1, which aims to minimize the categorical cross-entropy loss for the targeted class while minimizing the frequency components of the adversarial beats that will be filtered out during the physical ECG signal processing stage. In summary, this optimization equation optimizes adversarial beats to adapt to all target heartbeats and horizontal translation, while minimizing the amplitude of frequency components outside of the designated frequency range.
is the set of all heartbeat ECG segments in the training set, is the set of all possible horizontal shift transformations applicable on the adversarial beat within the heartbeat, and is a targeted heartbeat class. Note that. is an operator that represents the horizontal shift transformation, i.e., placing the adversarial beat, , within the target heartbeat ECG segment. A similar approach is taken in previous work [Chen2019]. represents a frequency component of a given signal
; it is computed by applying the Fourier transform to. represents a function to represent the effect of the bandpath filters; i.e., it applies a mask such that frequency components that are to be filtered out during the ECG processing pipeline remain; i.e., if the frequency is to be filtered out, otherwise . During optimization, a capacity on the amplitude of the adversarial beat waveform is set to be contained within 0–, where is specified by Algorithm 1 in the appendix.
The resulting adversarial beat causes an ECG segment to be missclassified as a target class when inserted. Figure 2 illustrates an example case of an adversarial beat being inserted into a target heartbeat ECG segment. Since the set of target heartbeat ECG segments used to optimize the adversarial beats contain heartbeats of any class, they are applicable to heartbeats of any class. This contributes to the universal characteristic of adversarial beat regarding heartbeat classes. We note that the adversarial beat is shorter in length compared to the target ECG heartbeat segment. The main difference from the optimization model used by Chen et al. is that the regularization by -norm disregarded, as we are focused on avoiding interference with the segmentation algorithm. Furthermore, the adversarial beats are intended to be inserted on every heartbeat that the adversary intends to spoof classification results for. Adversarial beats tend to have greater amplitude and less smoothness compared to those reported in previous work [Chen2019, Han2019], which have been demonstrated only in the digital environment. Similar tendency can be observed in physical adversarial perturbations in other domains, such as image [Eykholt2018, Brown2017] and audio [Yakura2019]. The resulting adversarial beats from this optimization algorithm have optimized amplitudes to minimize heartbeat segmentation interference, and are robust to ECG frequency filtering.
5 Simulation-based Evaluation
In this section, we begin by evaluating the proficiency of the trained target heartbeat classifier. We then generate adversarial beats using our proposed algorithm, and evaluate its effectiveness in spoofing certain classes of heartbeats by digitally inserting them in ECG segments.
5.1 Target Heartbeat Classifier
At its core, the ECG diagnosis system is expected to take in a segment of an ECG of a patient and automatically return diagnosis results on them. Several algorithms are applied on the ECG sequentially as discussed in section 4.1, outputting a series of data that the target neural network classifier can properly analyze. In our implementation, we filter out relevant frequencies for baseline wandering and powerline interference using forward-backward filtering with cut-off frequencies allowing frequencies of 0.5–50 Hz. Frequencies to attenuate artifacts caused by patient movement are considered and implemented during heartbeat segmentation. Because the target neural network analyzes the waveform of ECGs of individual heartbeats, the measured ECG data must be segmented into individual heartbeats. Several heartbeat segmentation algorithms have been proposed [Hamilton2002, Christov2004, Engelse1979]. The detected heartbeats are sliced from the original ECG segment and adjusted to fit the specified input of the classifier with their values normalized with 0–1. We implement heartbeat segmentation by adopting algorithms proposed by Hamilton [Hamilton2002].
As the baseline neural network model, we adopt the ECG heartbeat classification model developed by Kachuee et al.[Kachuee2018]. Details of the architecture and training datasets are specified in the appendix.
The confusion matrix for the target classifier on the test dataset consisting of 100 heartbeats for each class sampled from the test dataset is shown in Figure 3. Each axis represents classes of heartbeat, and the X-axis represents the classification result, while the Y-axis represents the ground truth. Each tile represents the number of resulting classifications. The overall accuracy of the trained classifier was 93.4%. The classifier is mostly able to consistently distinguish between different classes of heartbeats. Overall, the classifier made consistent classifications on ECGs measured from a test subject, suggesting translation of the classification proficiency on data measured during the experiments.
5.2 Generating Adversarial Beats
Two variations of adversarial beats each serving unique purposes are trained. Training data and evaluation data prepared in the previous section is used to generate the adversarial beats, similar to the target classifier. The first adversarial beat is trained to cause the target classifier to misclassify any heartbeat injected with it as S class heartbeats, regardless of their original class and spoofing an S class heartbeat (hereinafter referred to as AB-S). The second adversarial beat is trained similarly so that any injected heartbeats are misclassified by the target classifier as V class heartbeats, regardless of their original class, spoofing a V class heartbeat (hereinafter referred to as AB-V). Specifications of the two adversarial beats are shown in Table 3
. AB-S achieved acceptable accuracy of up to 65.4% chance of successfully causing a targeted misclassification, with a relatively low amplitude of 0.1875. On the contrary, AB-V required a higher amplitude of 0.4 and adjustments to its length to achieve acceptable accuracy of 56.2% chance of causing a targeted misclassification. These results were achieved through heuristic adjustments of theparameter from Algorithm 1. While a relatively acceptable degree of effectiveness and amplitude was achieved by threshold value for AB-S, there was difficulty in generating AB-V with similar effectiveness without having to maintain large amplitude. for AB-V was then decreased, which resulted in acceptable amplitude. This suggests S class heartbeats are easier to spoof compared to V class heartbeats, thus requiring less perturbations to cause a misclassification.
|Target Class||Success Rate [%]|
5.3 Testing Adversarial Beats
The trained adversarial beats are tested by introducing them into the test set and counting the cases of misclassification by the target classifier. This is done in a digital environment, isolated from the hardware experimentation to follow. Our intent is to first ensure that the generated adversarial beats possess a certain degree of effectiveness at this point. To simulate the uncertainty in where the adversarial beat may be injected during hardware injection and test whether they generalize to various translation, adversarial beats are digitally injected with random shifts throughout the sampling axis in the original heartbeat ECG. Figure 4(a) and Figure 4(b) presents cases of misclassifications. Each axis represents classes of heartbeat, and the X-axis represents the ground truth, while the Y-axis represents the classification result. Unlike Figure 3, tiles in Figure 4(a) and Figure 4(b) represent cases of misclassifications. AB-S and AB-V both show acceptable performance, as they cause expected instances of misclassifications according to the accuracy denoted throughout training.
6 Real-World Experiments
In this section, we apply the generated adversarial beats into a physical attack, and perform a physical hardware-based attack against our representation of a ECG diagnosis system. We perform a wired-signal injection to the device used for ECG measurement. Hardware setup is explained, and proficiency of the attack is reported with the introduced metrics.
6.1 Experimental Setup and Procedure
The diagram of our hardware setup is shown in Figure 5(a). In our setup, a conventional surface electrode is attached to the chest of a research participant, to emulate an arbitrary ECG measurement device. An ECG controller module reads the measured signals and performs amplification and filtering to the raw signal, outputting an unperturbed ECG as an analog signal. This raw ECG signal is then fed into a signal processing device, consisting of an analog-to-digital converter (ADC), a digital-to-analog converter (DAC), and a PC111We discuss the feasibility of hardware implementation in Section 7.. The signal processing device reads the unperturbed ECG signal for a given period to monitor the timing to inject an adversarial beat. The signal processing device has an adversarial beat pre-computed from the methods covered in section 4 and 5, and transmits a segment of given length, containing an arbitrary number of the prepared adversarial beats separated by the computed interval. The injection waveform generated by the signal processing device (considered as the malicious device in our attack) and the unperturbed ECG signals are then combined via signal addition. Finally, the resulting combined signals are digitized at the target device’s ADC, and sent to our ECG classifier. The resulting PoC hardware setup is shown in Figure 5(b). It’s specifics are explained in the appendix. We note also, that the hardware used for the setup is easily obtainable in terms of accessibility and expenses.
The signal processing device executes an algorithm to output adversarial beats in pulses that synchronize with the patient’s heartbeat, ensuring accurate injection. The adversarial beat is also scaled so that it is injected in the intended amplitude relative to the ECG it is injected in. This algorithm ensures that adversarial beats are aligned to heartbeats, so that they retain their attack success rate regardless of which heartbeat it is injected in. The algorithm executed is as follows:
Read the ECG from the target patient for 5 seconds.
Filter the ECG and perform heartbeat detection.
Compute the distance between individual heartbeats that are detected.
Compute the amplitude of the measured ECG.
Scale the amplitude/length of the pre-computed adversarial beat to match the amplitude of the measured ECG/length of measured heartbeat distance.
Construct a 5 second signal, comprised of duplicates of the scaled adversarial beatspadded by the calculated heartbeat distances so that they occur in the same rhythm as the ECG.
The proposed hardware-injection attack is executed while measuring the ECG of a human participant, who has their heart activity measured for 120 seconds each trial. Before the experiment, we obtained informed consent and confirmed that the participant had no heart defects or disorders, i.e., all the ECG segments measured should be classified as N. Adversarial beats are injected for a total of 40 times every trial, and the injected ECG is filtered before undergoing beat segmentation. Here, we use the trained classifier from section 5.1. Finally, the segmented ECGs with individual heartbeats are fed into the classifier by batch, outputting the predicted class of every heartbeat in the measured ECG. The instances of each detected beat classes are recorded to show how often the proposed hardware attack was able to spoof the targeted heartbeat class. In our experiments, the following ECGs were measured, with a total of 10 trials performed for each: ECGs without any injections as control, ECGs injected with AB-S, and ECGs injected with AB-V.
6.2 Injection Attack Result
Table 4 summarizes the measurement results averaged over the 10 separate trials. A normal ECG waveform, and an ECG waveform measured with AB-S injected during measurement is displayed in Figure 6. When counting the additional arrhythmic beats discovered from injected ECGs compared to ECGs with no injection, we see that we were able to spoof 3-5 cases of heartbeats (approxmiately 5 additional S-class arrhythmia for ECGs injected with AB-S, and 3 additional V-class arrhythmia for ECGs injected with AB-V). This suggests that the adversarial beats are capable of spoofing certain heartbeats, generalize to unseen samples, and retain their effectiveness in a physical environment. We found that in the control sample, several heartbeats were misclassified as S class, resulting in a classification error rate of roughly 7.8%. While the original performance of the target classifier was suspected to be the prime cause of additional S class heartbeat detection, S class heartbeats are considered to possess a unique attribute, as the classifier also seldom misclassified average N class heartbeats as F class beats, which did not occur during the hardware-based attack. Additional observations are included in the appendix due to page limit.
7.1 Feasibility of the Attack
We discuss the feasibility of the attack, considering the cost, integration, and installation of the hardware required for performing the attack with the adversarial beats. We also take into consideration our suggested attacker models, and consider the feasibility of our proposed real-world attack in their perspective.
As mentioned in section 3, attackers are expected to be capable of conducting a black-box attack on the target ECG system. Our experimental results based on a white-box attack shows that the success rate is not necessarily very high, as we are only capable of spoofing 3-5 additional arrhythmic heartbeats in a physical setting. As white-box attacks generally have a higher success rate compared to black-box attacks, we conclude that a successful black-box attack in a physical setting has a limited threat model, leading us to believe that malicious hospital personnel are best fit to utilize adversarial beats, as they are the only actors who are presumed to have remotely any access to confidential information to the target ECG classification model.
Through our experiments we see that creating an adversarial beat is cheap in terms of resources, requiring little resource commitment from the adversary. As for the hardware used in the injection attacks, the adversary only needs to prepare the injection components of the hardware used in this experiment, i.e., a computer, ADC/DAC, and an audio mixer, which all are available at reasonable expenses. This is significant from the perspective of malicious patients or malicious hospital personnel, where advanced hardware tampering consisting of expensive hardware is infeasible. Our PoC hardware setup is possible for any adversary with reasonable knowledge and resourcefulness. This is, however, not to say that an advanced adaptation of our PoC is pointless.
With the growing trend in medical IoT and their expected convenience, we expect daily biomedical monitoring devices to be small in size. We believe that with the simplicity of our injection algorithm, the PoC hardware setup we propose can be embedded into a much smaller circuit component by a resourceful adversary, making potential tampering with the monitoring hardware unnoticeable. We note that the components used in our experimental setup are prototypes of the proposed attack, and integrating all these components into a small board, using a microcontroller with ADC/DAC and op-amps, is possible for an adversary with moderate expertise in embedded systems. Malicious medical IoT manufacturers would be capable of such tampering. With additional functionality, such as generating adversarial beats when prompted via a network signal, adversarial beats can be leveraged to poison a supply chain, enabling a type of backdoor that malicious manufacturers and their co-conspirators can utilize to manipulate diagnosis results.
The installation of malicious hardware could occur in different phases depending on the attacker. As malicious hospital personnel are expected to be the ones maintaining measurement devices before providing them to the patient, they could install the adversarial contraption anytime before provision. On the contrary, patients will have arbitrary access to the measurement device during the extent of the measurement being taken. Malicious manufacturers may install the adversarial hardware contraption in the manufacturing phase. For malicious patients and hospital personnel, it is straightforward to attach an adversarial hardware circuit to an ECG monitoring device as we demonstrate in this work. If such tampering is undetected by other stake holders, the device can be used as an adversarial ECG measuring device, which outputs perturbed heartbeats by command, but otherwise returns the original, unperturbed ECG of the patient.
Taking the aforementioned details into consideration, it can be concluded that our proposed attack requires hospital personnel with access to the physical target device and insider knowledge of the target ECG neural network classifier to succeed. While the exact degree of knowledge hospital personnel would have on the classification model is unclear, our proposed attack could be possible in the case of lack of access restriction to model parameters. We hope that future system designers can refer to our research as an example of a possible threat to look out for.
We discuss further points of optimization and context which could improve our proposed hardware attack.
There is much room for optimization of the adversarial beats. Their lengths are manually adjusted as a static value at the beginning of training, resulting with a heuristically optimized parameter at best. Improvement to the adversarial beat generation algorithm to find the optimal length of the adversarial beat is prioritized in future work. We note, also, that the waveform of adversarial beats are optimized to minimize the disruptiveness of the generated adversarial beats towards beat segmentation algorithms. While experimental results have shown heuristically that minimizing the prominence of the peaks in adversarial beats sufficiently prevents beat segmentation disruption, the specific conditions in which beat segmentation algorithm mistake adversarial beats as QRS complexes are unclear. Thus, specific beat segmentation algorithms should be considered in detail for future work to analyze the specific constraints they put on adversarial beats. An increase in the overall effectiveness of adversarial beats would also lead to a larger variety of potential attackers to consider. Improving on the generation algorithm of adversarial beats to enable a black-box attack will assist in expanding the discussion of a more diverse threat model.
While the hardware setup showcased in this work demonstrates the feasibility of physical adversarial examples for ECGs, implementation could be improved. Besides hardware noise, the injection algorithm implemented in the myDAQ device may also be responsible for the inaccuracy of recreating adversarial beats as the injection signal. As the algorithm only accommodates for difference in lengths of particular heartbeats by scaling the length of adversarial beats by average beat-to-beat distance within a 5-second window, the resulting length adjustment may be inaccurate, which can be taxing on their effectiveness. Furthermore, while adversarial beats are trained to be effective in any horizontal position on a target heartbeat, ending up on a prominent peak in the waveform significantly alters the maximum amplitude of the resulting injected ECG, which introduces a degree of uncertainty that could be detrimental to the success of the attack. Improvements in generating injection signals may allow adversaries for more consistent and versatile manipulation of the diagnosis results.
Our proposed attack is naive in the sense that it disregards the presence of human operators in the automated diagnosis system. Adversarial beats introduce prominent noise into the measured ECG signals. While we assume an automated diagnosis, a specialist in ECGs simply observing the measured ECG is enough to raise suspicion and potentially compromise the hardware attack. In this case, our human observer would be the hospital personnel, making them even more of a likely candidate to carry out the proposed attack. Future work focuses on minimizing the amplitude of adversarial beats, or otherwise overcoming the risk of the attack getting compromised by a human observer.
7.3 Ethical Consideration
The hardware implementation used in this work was not used for any purposes other than for experimentation. We obtained informed consent from the participant who had their ECGs measured; i.e., before the experiments, we explained the objectives of the experiments and that we use the measured ECGs purely for the experiments, and not for other purposes such as medical diagnosis.
Automated clinical diagnosis systems powered by neural networks are exposed to the threats of adversarial examples. To analyze the feasibility of adversarial examples for ECGs in the real world, we attempt a PoC attack against a naively implemented neural network-based automatic ECG diagnosis system. Using off-the-shelf hardware to implement our attack, we demonstrate that our proposed hardware attack is capable of causing a classifier trained to detect arrhythmia to make a misclassification. We also identify attackers and scenarios in which the proposed attack could be executed to commit a healthcare fraud. As this work showed, the specific domain knowledge and the specific schemes in which measurement data is collected play a substantial part in how an adversary may leverage adversarial examples in the real world. We hope our work can be referenced by system designers to implement preventive measures against potential adversarial attacks in future clinical diagnostic systems.
Appendix A: Concrete Attack Scenarios
With the threat model laid out in section 3, we give some concrete examples as to how certain adversaries may go about an attack. Following the heartbeat types shown in Table 1, we present two heartbeat spoofing attack models and explain ways in which the resulting ECG readings can be used for medical fraud to benefit the adversary.
S-Target Spoofing Attack
One of the specific conditions the S class heartbeats entails is Atrial Premature heartbeats [AAMI]. While individual Atrial Premature heartbeats are considered harmless, 3-4 consecutive occurrences entail potential signs of severe stress, side affects of medicine, excessive substance consumption, and potential heart failure [ecg-workout]. Patients with frequent occurrences of this arrhythmia are at times subject to anti-anxiety drugs, or heart medication to prevent further development of serious arrhythmia. The exact hardware attack executed in our experiments can be utilized by a malicious hospital personnel. A physician may be able to embed an adversarial hardware contraption between the input of the readings from the electrodes to feign consecutive S class heartbeats in patient ECGs. This will allow the physician to appoint excessive medication, leading to higher costs billed. Perfect performance of the hardware attack is not required to succeed, as only few of the attempts are required to succeed in order to raise suspicion of dangerous heart activity with occasional consecutive S class heartbeat occurrences.
V-Target Spoofing Attack
V class heartbeats point to Premature Ventricular Contraction [AAMI]. Premature Ventricular Contraction heartbeats indicate heart instability when occurring in succession, or in a pattern and is looked out for during periods following a heart surgery [ecg-workout]. Patients seen with frequent occurences of this arrhythmia are at times subject to antiarrhythmic medication. Malicious hospital personnel may conduct a similar attack to the S-Target Spoofing attack to display such arrhythmia occurring in patterns to warrant unnecessary application of antiarrhythmic medication during post-surgery care. Constant monitoring is required after heart surgeries, and devices may be assigned to patients after heart surgeries to ensure stability in heart activity, making this scenario especially vulnerable to our suggested attack method.
Appendix B: Adversarial Beat Generation Algorithm
Algorithm 1 describes the algorithm used to generate adversarial beats. In each iteration, adversarial beats are optimized to maximize attack success rate, and a stricter amplitude is assigned once the perturbation reaches the threshold attack success rate. After unsuccessful iterations, the algorithm deems further constriction of amplitude difficult and aborts the training. For this experiment, we empirically set the threshold as .
Appendix C: Target Model Details
The model complies with the five-heartbeats classification in Table 1
. The architecture is a series of consecutive residual blocks consisting of convolutional layers and max-pooling layers, followed by two fully-connected layers with 32 neurons each. Convolutional layers perform 1-D convolution, each with 32 kernels of size 5, as specified by Kachuee et al.[Kachuee2018]
. For the Max-pooling layer, size is 5 with a stride of 2. A classification accuracy of up to 93.4% was achieved in Kachuee’s original work, successfully determining heartbeat patterns.
To train the classifier, we adopt the PhysioNet MIT-BIH Arrhythmia Database [Goldberger2000, Moody2001]. All the heartbeats are labeled with their respective heartbeat annotations, which are adapted to the beat annotation standards specified in AAMI EC57 as shown in Table 1. We first segment ECGs in the dataset into individual heartbeats. The resulting dataset contains segments of ECGs of individual heartbeats, each labeled as a class, resized to universal length, so that they can be inserted into the target classifier. We normalize the amplitude of waveforms values to 0–1.0. We subsampled all but one class to counteract disproportionate class distributions. Finally, the resulting dataset is then split with 80% of the dataset used for training the classifier and the remaining 20% used for testing the classifier and the effectiveness of our created adversarial beats.
Appendix D: Hardware Setup
The ECG controller is realized with the Analog Devices AD8232 module which amplifies and filters the raw electrical signal. We use an off-the-shelf data acquisition device, National Instruments myDAQ, capable of acquiring/generating electrical to realize the ADC/DAC in the signal processing device shown in Figure 5(a). A laptop PC is attached to the NI myDAQ device to inject pre-computed adversarial beats with proper timing. Signal addition was executed by using a commercial audio mixer [Mixer]. The final ADC component was implemented with an Arduino via their analog input port.