1 Introduction
Traffic analysis is the art of inferring sensitive information from the patterns of network traffic (as opposed to packet contents), in particular, packet timings and sizes. Traffic analysis is useful in scenarios where network traffic is encrypted, since encryption does not significantly modify traffic patterns. In particular, previous work has studied traffic analysis algorithms that either compromise the privacy of encrypted traffic (e.g., by linking anonymous communications [46, 34]) or enhance its security by fingerprinting malicious, obfuscated connections (e.g., stepping stone attacks [57, 22]).
Recent advances in traffic analysis leverage deep neural networks (DNNs) to design classifiers that are significantly (in some cases, orders of magnitude) more efficient and more reliable than traditional traffic analysis techniques. In particular, the recent website fingerprinting work of Deep Fingerprinting [46] outperforms all prior fingerprinting techniques in classifying webpages, and the DeepCorr [34] flow correlation technique is able to link anonymized traffic flows with accuracies two orders magnitude superior to prior flow correlation techniques. Given the increasing use of DNNs in traffic analysis applications, we ask ourselves the following question: can DNNbased traffic analysis techniques get defeated through adversarially perturbing —live—traffic patterns?
Note that adversarial perturbations is an active area of research in various image processing applications [49, 17, 28, 12, 33, 9, 32, 21] (referred to as adversarial examples). However, applying adversarial perturbations on network traffic is not trivial, as it faces two major challenges. First, the perturbing entity, i.e., the adversary,^{1}^{1}1In our context, the adversary is not necessarily a malicious party; it is the entity who aims to defeat the underlying DNN traffic classifiers. should be able to apply his adversarial perturbations on live network traffic, without buffering the target traffic or knowing the patterns of upcoming network packets. This is because in most traffic analysis applications, as will be introduced, the adversary can not influence the generation of target traffic, but he can only intercept the packets of the target traffic and perturb them on the fly. In this paper, we are the first to design techniques that adversarially perturb live network traffic to defeat DNNbased traffic classifiers; we call our approach blind adversarial perturbations. Our technique applies adversarial perturbations on live packets as they appear on the wire. The key idea of our adversarial perturbations algorithm is that it generates “blind” perturbations that are independent of the target inputs by solving specific optimization problems that will be explained. We design adversarial perturbation mechanisms for the key features commonly used in traffic analysis applications: our adversarial perturbations include changing the timings and sizes of packets, as well as inserting dummy network packets.
The second challenge to applying adversarial perturbations on traffic analysis applications is that, any perturbation mechanism on network traffic should preserve various constraints of traffic patterns, e.g., the dependencies between different traffic features, the statistical distribution of timings/sizes expected from the underlying protocol, etc. This is unlike traditional adversarial example studies (in the context of image processing) that modify image pixel values individually. Therefore, one can not simply borrow techniques from traditional adversarial examples. We consequently design various remapping functions and regularizers, that we incorporate into our optimization problem to enforce such network constraints. As will be shown, in most scenarios the constraints are not differentiable, and therefore we carefully craft custom gradient functions to approximate their gradients.
Evaluations: Our blind adversarial perturbations algorithm is generic and can be applied to various types of traffic classifiers. We demonstrate this by applying our techniques on stateoftheart website fingerprinting [46, 3] and flow correlation[34] techniques, the two moststudied types of traffic analysis. Our evaluations show that our adversarial perturbations can effectively defeat DNNbased traffic analysis techniques through small, live adversarial perturbations. For instance, our perturbations can reduce the accuracy of stateoftheart website fingerprinting [3, 46] works by 90 by only adding 10 bandwidth overhead. Also, our adversarial perturbations can reduce the true positive rate of stateoftheart flow correlation techniques [34]
from 0.9 to 0.2 by applying tiny delays with a 50ms jitter standard deviation.
We also show that our blind adversarial perturbations are transferable between different models and architectures, which signifies their practical importance as they can be implemented by blackbox adversaries.
Countermeasures: We conclude by studying various countermeasures against our adversarial perturbations. We start by leveraging existing defenses against adversarial examples from the image classification literature and adapting them to the traffic analysis scenario. We show that such adapted defenses are not effective against our network adversarial perturbations as they do not take into account the specific constraints of traffic features. Motivated by this, we design a tailored countermeasure for our network adversarial perturbations, which we demonstrate to be more effective than the adapted defenses. The key idea of our countermeasure is performing adversarial training, and using our attack as a regularizer to train robust traffic analysis models.
2 Preliminaries
2.1 Problem Statement
Traffic analysis is to infer sensitive information from the patterns of network traffic, i.e., packet timings and sizes. Therefore, many works have investigated the use of traffic analysis in various scenarios where traffic contents are encrypted. In particular, traffic analysis has been used to compromise anonymity in anonymous communications systems through various types of attacks, specifically, website fingerprinting [46, 3, 43, 53, 38, 5, 54, 37, 18, 55], and flow correlation [34, 35, 22, 11, 60, 30, 45, 23, 36, 48]. Traffic analysis has also been used to trace back cybercriminals who obfuscate their identifies through stepping stone relays [22, 23, 34, 57, 59].
Our problem: Defeating DNNbased traffic analysis algorithms The stateoftheart traffic analysis techniques use deep neural networks (DNN) to offer much higher performances than prior techniques. For instance, DeepCorr [34] provides a flow correlation accuracy of compared to of statisticalbased systems like RAPTOR [48] (in a given setting). Also, VarCNN [3]
leverages deep learning techniques to perform a website fingerprinting attack which achieves
accuracy in a closedworld setting. However, deep learning models are infamous for being susceptible to various adversarial attacks where the adversary adds small perturbations to the inputs to mislead the deep learning model. Such techniques are known as adversarial examples in the context of image processing, but have not been investigated in the traffic analysis domain. In this work, we study the possibility of defeating DNNbased traffic analysis techniques through adversarial perturbations.In our setting, some traffic analysis parties use DNNbased traffic analysis techniques for various purposes, such as breaking Tor’s anonymity or detecting cybercriminals. On the other hand, the traffic analysis adversary(ies) aim at interfering with the traffic analysis process through adversarially perturbing traffic patterns of the connections they intercept. To do so, the traffic analysis adversary(ies) perturb the traffic patterns of the intercepted flows to reduce the accuracy of the DNNbased classifiers used by the traffic analysis parties. To further clarify the distinction between the players, in the flow correlation setting, the traffic analysis “party” can be a malicious ISP who aims at deanonymizing Tor users by analyzing their Tor connections; however, the traffic analysis “adversary” can be some (benign) Tor relays who perturb traffic patterns of their connections to defeat potential traffic analysis attacks.
Challenges: Note that our problem resembles the setting of adversarial examples for image classification. However, applying adversarial perturbations on network traffic presents two major challenges. First, the adversaries should be able to apply adversarial perturbations on live network connections where the patterns of upcoming network packets are unknown to the adversaries. This is because in traffic analysis applications, the adversary is not in charge of generating traffic patterns. For instance, in the flow correlation scenario, the traffic analysis adversary is a benign Tor relay who intercepts and (slightly) perturbs the traffic generated by Tor users. The second challenge to applying network adversarial perturbations is that they should preserve the various constraints of network traffic, e.g., the dependencies of different traffic features.
Sketch of our approach: In this work, we design blind adversarial perturbations, a set of techniques to perform adversarial network perturbations that overcome the two mentioned challenges. To address the first challenge (applying on live traffic), we design blind
perturbation vectors that are independent of their target traffic, therefore, they can be applied on any (unknown) network flows. Figure
1 shows what is needed by our blind perturbations adversary compared to traditional (nonblind) perturbation techniques. We generate such blind adversarial perturbations by solving a specific optimization problem. We address the second challenge (enforcing network constraints) by using various remapping functions and regularizers that adjust perturbed traffic features to follow the required constraints.2.2 Threat Model
Adversary’s knowledge of the inputs. We assume the adversary has no prior knowledge about the patterns of upcoming network packets of the target connections to be perturbed. Therefore, the goal of the adversary is to craft inputagnostic perturbation vectors that once applied to arbitrary network flows, such flows are misclassified by the target DNNbased traffic analysis algorithms.
Adversary’s Knowledge of the model. We start with an adversary who has a whitebox access to the target model to be defeated, i.e., he knows the target DNN model’s architecture and parameters (Section 4). Then, in Section 9 we extend our attack to a blackbox setting where the adversary has no knowledge of the target model’s architecture or parameters, by leveraging the transferability of our technique.
Adversary’s Knowledge of the training data. We assume the adversary knows a set of samples from the same distribution as the training dataset of the target model. For example, in the website fingerprinting application the adversary can browse the target websites to be misclassified to obtain such training samples.
Attack’s target. We consider four types of attacks, i.e., STDT, STDU, SUDT, and SUDU, based on the adversary’s source and destination targets as defined below:
a) Destinationtargeted/untargeted (DT/DU): We call the attack destinationtargeted (DT
) if the goal of the adversary is to make the model misclassify arbitrary inputs into a specific, target output class. On the other hand, we call the attack
destinationuntargeted (DU) if the goal is to misclassify inputs into arbitrary (incorrect) output classes.b) Sourcetargeted/untargeted (ST/SU): A sourcetargeted (ST) adversary is one whose goal is to have inputs from a specific input class misclassified by the traffic analysis model. By contrast, a sourceuntargeted (SU) adversary is one who aims at causing arbitrary inputs classes to get misclassified.
3 Background
3.1 Deep learning
A deep neural network (DNN) consists of a series of linear and nonlinear functions, known as layers. Each layer has a weight matrix
and an activation function. For a given input
, we denote the output of a DNN model by:where is the th layer of the deep neural network (note that we use bold letters to represent vectors as in
). We focus on supervised learning, where we have a set of labeled training data. Let
be a set of data points in the target dimensional space, where each dimension represents one attribute of the input data points. We assume there is an oracle which maps the data points to their labels. For the sake of simplicity, we only focus on classification tasks.The goal of training is to find a classification model that maps each point in to its correct class in the set of classes, . To obtain
, one needs to define a lowerbounded, realvalued loss function
that for each data point measures the difference between and the model’s prediction .Therefore, the loss function for can be defined as:
(1) 
and the objective of training is to find an that minimizes this loss function. Since is not entirely available to the training entities, in practice, a set of samples from it, called the training set , is used to train the model [52]. Therefore, instead of minimizing (1
), machine learning algorithms minimize the expected
empirical loss of the model over its training set :(2) 
Therefore, a deep neural network is trained by tuning its weight parameters to minimize its empirical loss function over a (large) set of known inputoutput pairs . This is commonly performed using a variation of the gradient descent algorithm, e.g., back propagation [14].
3.2 Adversarial Examples
An adversarial example is an adversarially crafted input that fools a target classifier or regression model into making incorrect classifications or predictions. The adversary’s goal is to generate adversarial examples by adding minimal perturbations to the input data attributes. Therefore, an adversarial example can be crafted by solving the following optimization problem:
(3) 
where is a nonadversarial input sample,
is the adversarial perturbation added to it, and represents the true label of its input, as defined in the previous section. The adversary’s objective is to adds a minimal perturbation to force the target model misclassify the input . Adversarial examples are commonly studied in image classification applications, where a constraint in finding adversarial examples is that the added noise should be imperceptible to the human eyes.
In this paper, we will investigate the application of adversarial examples on network connections with different imperceptibility constraints.
Previous works [17, 16, 29, 12, 32] have suggested several ways to generate adversarial examples. The Fast Gradient Sign Method (FGSM) [17] algorithm generates an adversarial sample by calculating the following perturbation for a given model and a loss function :
(4) 
where is the model’s loss gradient w.r.t. the input , and the is the input’s label. Therefore, the adversarial perturbation is the sign of the model’s loss gradient w.r.t. the input and label . Also, is a coefficient controlling the amplitude of the perturbation. Therefore, the adversarial perturbation in FGSM is the sign of model’s gradient. The adversary adds the perturbation to to craft an adversarial example. Kurakin et al. [29] proposed a targeted version of FGSM, where the adversary’s goal is to fool the model to classify inputs as a desired target class (as opposed to any class in FGSM). Kurakin et al. also introduced an iterative method to improve the success rate of the generated examples. Dong et al. [12] showed that using the momentum approach can improve Kurkain et al.’s iterative method. Also, Carlini and Wagner [8] designed a set of attacks that can craft adversarial examples when the adversary has various norm constraints (e.g., , , ). Other variations of adversarial examples [47, 13] have been introduced to craft adversarial examples that consider different sets of constraints or improve the adversary’s success rate. MoosaviDezfooli et al. [32] introduced universal adversarial perturbations where the adversary generates adversarial examples that are independent of the inputs.
3.3 Traffic Analysis Techniques
We overview the two major classes of traffic analysis techniques, which we will use to demonstrate our network adversarial perturbations.
Flow correlation: Flow correlation aims at linking obfuscated network flows by correlating their traffic characteristics, i.e., packet timings and sizes [35, 22, 34, 2]. In particular, the Tor anonymity system has been the target of flow correlation attacks, where an adversary aims at linking ingress and egress segments of a Tor connection by correlating traffic characteristics. Traditional flow correlation techniques mainly use standard statistical correlation metrics to correlate the vectors of flow timings and sizes across flows, in particular mutual information [11, 60], Pearson correlation [30, 45]
[23, 36], and Spearman correlation [48]. More recently, Nasr et al. [34] design a DNNbased approach for flow correlation, called DeepCorr. They show that DeepCorr outperforms statistical flow correlation techniques by large margins.Website Fingerprinting: Website fingerprinting (WF) aims at detecting the websites visited over encrypted channels like VPNs, Tor, and other proxies [46, 3, 43, 53, 38, 5, 54, 37, 18, 55]. The attack is performed by a passive adversary who monitors the victim’s encrypted network traffic, e.g., a malicious ISP or a surveillance agency. The adversary compares the victim’s observed traffic flow against a set of prerecorded webpage traces, to identify the webpage being browsed. Website fingerprinting differs from flow correlation in that the adversary only observes one end of the connection, e.g., the connection between a client and a Tor relay. Website fingerprinting has been widely studied in the context of Tor traffic analysis [43, 46, 3, 38, 5, 54, 37, 18].
Various machine learning classifiers have been used for WF, e.g., using KNN
[53], SVM [37], and random forest
[18]. However, the stateoftheart WF algorithms use Convolutional Neural Networks to perform website fingerprinting, i.e., Sirinam et al.
[46], Rimmer et al. [43], and Bhat et al. [3].Defenses: Note that our blind adversarial perturbations technique serves as a defense mechanism against traffic analysis classifiers (as it aims at fooling the underlying classifiers). The literature has proposed other defenses against website fingerprinting and flow correlation attacks [25, 56, 10, 4]. Similar to our work, such defenses work by manipulating traffic features, i.e., packet timings, sizes, and directions.
In Section 7.5, we compare the performance of our blind adversarial perturbations with stateoftheart defenses, showing that our technique outperforms all of these techniques in defeating traffic analysis.
Also, note that some recent works have considered using adversarial perturbations as a defense against traffic analysis. In particular, Mockingbird [24] generates adversarial perturbations to defeat website fingerprinting, and Zhang et al. [58] apply adversarial examples to defeat video classification using traffic analysis. However, both of these works are nonblind, i.e., the adversary needs to know the patterns of the target flows in advance; therefore, we consider them to be unusable in typical traffic analysis scenarios. By contrast, our blind perturbation technique modifies live network connections.
4 Formalulating Blind Adversarial Perturbations
In this section, we present the key formulation and algorithms for generating blind adversarial perturbations.
4.1 The General Formulation
We formulate the blind adversarial perturbations problem as the following optimization problem:
(5) 
where the objective is to find a (blind) perturbation vector, , such that when added to an arbitrary input from a target input domain , it will cause the underlying DNN model to misclassify. In a sourcetargeted (ST) attack (see definitions in Section 2.2), contains inputs from a target class to be misclassifies, whereas in a sourceuntargeted (SU) attack will be a large set of inputs from different classes.
Note that one cannot find a closedform solution for this optimization problem since the target model is a nonconvex ML model, i.e., a deep neural network. Therefore, (5) can be formulated as follows to numerically solve the problem using empirical approximation techniques:
(6) 
where is the target model’s loss function and is the adversary’s network training dataset.
Note that prior work by MoosaviDezfooli et al. [32] has studied the generation of universal adversarial perturbations for image recognition applications. We, however, take a different direction in generating blind perturbations: in contrast to finding a perturbation vector that maximizes the loss function in [32], we aim to find a perturbation generator model . This generator model will generate adversarial perturbation vectors when provided with a random trigger parameter (we denote the corresponding adversarial perturbation as ), i.e., we are able to generate different perturbations on different random ’s. Therefore, the goal of our optimization problem is to optimize the parameters of the perturbation generator model (as opposed to optimizing a perturbation vector in [32]). Using a generator model increases the attack performance, as shown previously [19, 1] and validated through our experiments. Hence, we formulate our optimization problem as:
(7) 
We can use existing optimization techniques (e.g., Adam [26]) to solve this problem. In each iteration of training, our algorithm selects a batch from the training dataset and a random trigger , then computes the objective function.
4.2 Incorporating Traffic Constraints
Studies of adversarial examples for image recognition applications [16, 29, 12, 17, 32] simply modify image pixel values individually. However, applying adversarial perturbations on network traffic is much more challenging due to the various constraints of network traffic that should be preserved while applying the perturbations. In particular, interpacket delays should have nonnegative values; the target network protocol may need to follow specific packet size/timing distributions; packets should not be removed from a connection; and, packet numbers should get adjusted after injecting new packets.
One can add other network constraints depending on the underlying network protocol. We use remapping and regularization functions to enforce these domain constraints while creating blind adversarial perturbations. A remapping function adjusts the perturbed traffic patterns so they comply with some domain constraints. For example, when an adversary adds a packet to a traffic flow at position , the remapping function should shift the indices of all consecutive packets.
We therefore reformulate our optimization problem by including the remapping function :
(8) 
Moreover, we add a regularization term to the loss function so that the adversary can enforce additional constraints, as will be discussed. Therefore, the following is our complete optimization problem:
(9) 
We adjust (9) for a destinationtargeted (DT) attack by replacing with , where is the target output class. Also, remind that for sourcetargeted attacks, contains samples only from the target classes.
4.3 Overview of the Algorithm
Algorithm 1 summarizes our approach to generate blind adversarial perturbations (Figure 8 in Appendix C illustrates the main components of our algorithm). In each iteration, Algorithm 1 computes the gradient of the objective function w.r.t. the blind perturbation for given inputs, and optimizes it by moving in the direction of the gradient. The algorithm enforces domain constraints using various remapping and regularization functions. We use the iterative minibatch stochastic gradient ascent [14] technique.
5 Perturbation Techniques
The traffic analysis literature uses three main features for building traffic analysis classifiers: 1) packet timings [34, 3], 2) packet sizes [34], and 3) packet directions [3, 46, 43, 53]. Our blind adversarial perturbation technique leverages these features to adversarially perturb traffic. These features can be modified either by delaying packets, resizing packets, or injecting new (dummy) packets (dropping packets is not an option as it will break the underlying applications). We describe how we perform such perturbations.
5.1 Manipulating Existing Packets
The adversary can modify the timings and sizes (but not the directions) of existing packets of a target network connection. We present a network connection as a vector of features: , where can represent the size, timing, direction, or a combination of these features for the th packet. The adversary designs a blind adversarial perturbation model , as introduced in Section 4, such that it outputs a perturbation vector with the same size as . The adversary adds to the original traffic patterns as packets arrive, so is the patterns of the perturbed connection. The main challenge is that the perturbed traffic features, , should not violate the domain constraints of the target network application.
Perturbing timings: We first introduce how the timing features can be perturbed. We use interpacket delays (IPDs) to represent the timing information of packets. An important constraint on the timing features is that the adversary should not introduce excessive delays on the packets as excessive delays will either interfere with the underlying application (e.g., Tor relays are not willing to introduce large latencies) or give away the adversary. We control the amount of delay added by the adversary by using a remapping function as follows:
A second constraint on timing features is that the perturbed timings should follow the statistical distributions expected from the target protocol. Towards this, we leverage a regularizer to enforce the desired statistical behavior on the blind perturbations. Our regularizer enforces a Laplacian distribution for network jitters, as suggested by prior work [35], but it can enforce arbitrary distributions. To do this, we use a generative adversarial network (GAN) [15]: we design a discriminator model which tries to distinguish the generated perturbations from a Laplace distribution. Then, we use this discriminator as our regularizer function to make the distribution of the crafted perturbations similar to a Laplace distribution. We simultaneously train the blind perturbation model and the discriminator model. This is summarized in Algorithm 2.
Perturbing sizes: An adversary can perturb packet sizes by increasing packets sizes (through appending dummy bits). However, modified packet sizes should not violate the expected maximum packet size of the underlying protocol as well as the expected statistical distribution of the sizes. For instance, Tor packets are expected to have certain packet sizes.
We use the remapping function , as shown in Algorithm 3, to adjust the amplitude of size modifications as well as to enforce the desired statistical distributions. The input to Algorithm 3 is the blind adversarial perturbation (), the desired maximum bytes of added traffic (), the desired maximum added bytes to a single packet (), and the expected packet size distribution of the underlying network protocol () (if the network protocol does not have any specific size constraints, then ). Algorithm 3 starts by selecting the highest values from the output of the adversarial perturbations and adds them to the traffic flows up to bytes. Since Algorithm 3 is not differentiable, we cannot simply use Algorithm 1. Instead, we define a custom gradient function for Algorithm 3 which allows us to train the blind adversarial perturbation model. Given the gradient of the target model’s loss w.r.t. the output of Algorithm 3 (i.e., ), we modify the perturbation model’s gradient as:
(11) 
where is the selected training batch. We do not need regularization for packet sizes.
5.2 Injecting Adversarial Packets
In addition to perturbing the features of existing packets, the adversary can also inject packets with specific sizes and at specific times into the target connection to be perturbed. The goal of our adversary is to identify the most adversarial timing and sizes for injected packets. We design a remapping function (Algorithm 4) that obtains the ordering of injected packets as well as their feature values. Similar to the previous attack, Algorithm 4 is not differentiable and we cannot simply use it for Algorithm 1. Instead, we use a custom gradient function for Algorithm 4 which allows us to train our blind adversarial perturbation model. We define the gradient function for different types of feature as described in the following.
Injecting adversarial directions: While an adversary cannot change the directions of existing packets, she can inject adversarial directions by adding packets. A connection’s packet directions can be represented as a series of 1 (downstream) and +1 (upstream) values. However, generating adversarial perturbations with binary values is not straightforward.
We generate a perturbation vector with the same size as the target connection. Each element of this vector shows the effect of inserting a packet at that specific position (i.e., in Algorithm 4). We select positions with largest absolute values for packet injection; the sign of the selected position determines the direction of the injected packet. Finally, we modify the perturbation model’s gradient as:
(12) 
Injecting adversarial timings/sizes: Unlike packet directions, for the timing and size features, we need to learn both the positions and the values of the added packets simultaneously. We design the perturbation generation model to output two vectors for the locations and the values of the added packets, where the value vector represents the selected feature (timing or sizes). We use the gradient function defined in (12) for the position of the inserted packets. We use Algorithm 5 to compute the gradients for the values of the inserted packets.
Injecting multiple adversarial features: To inject packets that simultaneously perturb several features, we modify the perturbation generation model to output one vector for the position of the injected packets and one for each feature set to be perturbed. We use Algorithm 5 to compute the gradient of each vector. Moreover, we cannot use (12) to compute the gradient for the position vector, therefore, we take the average between the gradient of all different input feature vectors.
6 Experimental Setup
6.1 Metrics
For a given blind adversarial perturbation generator and test dataset , we define the attack success metric as:
(13) 
where DU and DT represent destinationuntargeted and targeted attack scenarios, respectively (as defined in Section 2.2). For sourcetargeted (ST) cases, contains instances only from the target source class. Also, in our evaluations of the targeted attacks (ST and DT), we only report the results for target classes with minimum and maximum attack accuracies. For example, “Max STDT” indicates the best results for the source and destination targeted attacks, and we present the target classes using the notation, which means class is the targeted destination class and is the targeted source class. The maximum accuracy shows the worst case scenario for the target model and the minimum accuracy shows the lower bound on the adversary’s success rate. If there are multiple classes that lead to a max/min accuracy, we only mention one of them.
Note that while we can use to evaluate attack success in various settings, for the flow correlation experiments we use a more specific metric (as there are only two output classes for a flow correlation classifier). Specifically, we use the reduction in true positive and false positive rates of the target flow correlation algorithm to evaluate the success of our attack.
6.2 Target Systems
We demonstrate our attack on three stateoftheart DNNbased traffic analysis systems.
DeepCorr: DeepCorr [34] is the stateoftheart flow correlation system, which uses deep learning to learn flow correlation functions for specific network settings like that of Tor. DeepCorr uses interpacket delays (IPDs) and sizes of the packets as the features. DeepCorr uses Convolutional neural networks to extract complex features from the raw timing and size information, and it outperforms the conventional statistical flow correlation techniques by significant margins. Since DeepCorr uses both timings and sizes of packets as the features, we apply the timebased and sizebased attacks on DeepCorr.
As mentioned earlier, nonblind adversarial perturbations [24, 58] are useless in the flow correlation setting, as the adversary does not know the features of the upcoming packets in a target connection. Hence, our blind perturbations are applicable in this setting.
VarCNN: VarCNN [3]
is a deep learningbased website fingerprinting (WF) system that uses both manual and automated feature extraction techniques to be able to work with even small amounts of training data. VarCNN uses ResNets
[20] with dilated casual convolutions, the stateoftheart convolutional neural network, as its base structure. Furthermore, VarCNN shows that in contrast to previous WF attacks, combining packet timing information (IPDs) and direction information can improve the performance of the WF adversary. In addition to packet IPDs and directions, VarCNN uses cumulative statistical information for features of network flows. Therefore, VarCNN combines three different models, two ResNet models for timing and direction information, and one fully connected model for metadata statistical information as the final structure. VarCNN considers both closedworld and openworld scenarios.Similar to the setting of flow correlation, a WF adversary will not be able to use traditional (nonblind) adversarial perturbations [24, 58], as she will not have knowledge on the patterns of upcoming packets in a targeted connection. Therefore, WF is a trivial application for blind perturbations. Since VarCNN uses both IPD and packet direction features for fingerprinting, we use both timingbased and directionbased techniques to generate our adversarial perturbations.
Deep Fingerprinting (DF): Deep Fingerprinting (DF) [46] is a deep learning based WF attack which uses CNNs to perform WF attacks on Tor. DF deploys automated feature extraction, and uses the direction information for training. In contrast to VarCNN, DF does not require handcrafted features of packet sequences. Similar to VarCNN, DF considers both closedworld and openworld scenarios. Sirinam et al. [46]
show that DF outperforms prior WF systems in defeating WF defenses of WTFPAD
[25] and WT [56].Codes.
As we perform our attack in PyTorch, we use the original codes of DeepCorr, DF, and VarCNN models and convert them from TensorFlow to PyTorch. We then train these models using the datasets of those papers.
6.3 Adversary Setup and Models
While our technique can be applied to any traffic analysis setting, we present our setup for the popular Tor application.
Adversary’s Interception Points Our adversary has the same placement as traditional Tor traffic analysis works [46, 56, 43, 54, 55]. For the WF scenario, we assume the adversary is manipulating the traffic between a Tor client and the first Tor hop, i.e., a Tor bridge [50] or a Tor guard relay. Therefore, our blind adversarial perturbation can be implemented as a Tor pluggable transport [42], in which case the blind perturbations are applied by both the Tor client software and the Tor bridge. In the flow correlation setting, similar to the literature, traffic manipulations are performed by Tor entry and exit relays (since flow correlation attackers intercept both egress and ingress Tor connections). In our evaluations, we show that even applying our blind adversarial perturbations on only ingress flows is enough to defeat flow correlation attacks, i.e., the same adversary placement as the WF setting.
Adversarial Perturbation Models As mentioned in Section 4
, we design a deep learning model to generate blind adversarial noises. For each type of perturbation, the adversarial model is a fully connected model with one hidden layer of size 500 and a ReLu activation function. The parameters of the adversarial model are presented in Appendix
B. The input and output sizes of the adversarial model are equal to the length of features in the target flow. In the forward function, the adversarial model takes in a given input, manipulates it based on the attack method, and output a crafted version of the input. In each iteration of training, we update the parameters of the adversarial model based on the loss functions introduced in Section 4. We use Adam optimizer to learn the blind noise with a learning rate of .Discriminator Model As mentioned in Section 4, we use a GAN model to enforce the time constraints of our modified network flows. To do so, we design a fullyconnected discriminator model containing two hidden FC layers of size 1000. The parameters of the discriminator model are presented in Appendix B. The input and output sizes of this model are equal to the sizes of the blind adversarial noise. In the training process, we use Adam optimizer with a learning rate of to learn the discriminator model.
6.4 Datasets
Tor Flow Correlation Dataset For flow correlation experiments, we use the publicly available dataset of DeepCorr [34], which contains flows for training and flows for testing. These flows are captured Tor flows of top Alexa’s websites and contain timings and sizes of each of them. These flows are then used to create a large set of flow pairs including associated flow pairs (flows belonging to the same Tor connection) and nonassociated flow pairs (flows belonging to arbitrary Tor connections). Each associated flow pair is labeled with , and each nonassociated flow pair with .
Tor Website Fingerprinting Datasets VarCNN uses a dataset of 900 monitored sites each with 2,500 traces. These sites were compiled from the Alexa list of most popular websites. VarCNN is fed in with different sets of features representing a given trace; the directionbased ResNet model takes a set of 1’s and 1’s as the direction of each packet such that 1 shows an outgoing packet and 1 represents an incoming packet. The timebased ResNet uses the IPDs of the traces as features. The metadata model takes in seven float numbers as the statistical information of the traces. To be consistent with previous WF attacks [46, 43, 54, 55], we use the first 5000 values of a given trace for both direction and time features.
DF uses a different dataset than VarCNN. For the closedworld setting, they collected the traces of 95 top Alexa websites with 1000 visits for each. DF uses the same representation as VarCNN for direction information of the packets. Since CNNs only take in a fixed length input, DF considers the first 5000 values of each flow.
7 Experiment Results
We evaluate blind adversarial perturbations against the target systems of Section 6.2 using each of the three key traffic features and their combinations. We also compare our attack with traditional attacks.
7.1 Adversarially Perturbing Directions
Bandwith Overhead ()  SUDU (%)  Max STDU (#, )  Min STDU (#, )  Max SUDT (#, )  Min SUDT (#, )  Max STDT (##, )  Min STDT (##, )  

20  0.04  
100  2.04  
500  11.11  
1000  25.0  
2000  66.66 
As explained in Section 4, an adversary cannot change the directions of existing packets, but he can insert packets with adversarial directions. We evaluated our attack for different adversary settings and strengths against VarCNN [3] and DF [46] (which use direction features). We used 10 epochs and Adam optimizer to train the blind adversarial perturbations model with a learning rate of . Tables 1 and 2 show the success of our attack (using in (13)) on DF and VarCNN, respectively, when they only use packet directions as their features. As can be seen, both DF and VarCNN are highly vulnerable to adversarial perturbation attacks when the adversary only injects a small number of packets. Specifically, we were able to generate targeted perturbations that misclassify every input into a target class with only 25 bandwidth overhead.
7.2 Adversarially Perturbing Timings
We consider two scenarios for generating adversarial timing perturbations: with and without an invisibility constraint. In both scenarios, we limited the adversaries’ power such that the added noise to the timings of the packets has a maximum mean and standard deviation as explained in Section 4. For the invisibility constraint, we force the added noise to have the same distribution as natural network jitter, which follows a Laplace distribution [34]. The detailed parameters of our model are presented in Appendix B.
Bandwith Overhead ()  SUDU (%)  Max STDU (#, )  Min STDU (#, )  Max SUDT (#, )  Min SUDT (#, )  Max STDT (##, )  Min STDT (##, )  

20  0.04  
100  2.04  
500  11.11  
1000  25.0  
2000  66.66 
Figure 2 shows the performance of our attack against DeepCorr when the adversary only manipulates the timings of packets. As expected, Figures 1(a) and 1(b) show that increasing the strength (mean or standard deviation) of our blind noise results in better performance of the attack, but even a perturbation with average 0 and a tiny standard deviation of significantly reduces the true positive of DeepCorr from to .
Also, we can create effective adversarial perturbations with high invisibility: Figure 5 shows the histogram of the generated timing perturbations, with parameters , learned under an invisibility constraint forcing it to follow a Laplace distribution. For this invisible noise, Figure 1(c) compares the performance of timing perturbations on DeepCorr with different attack strengths; it also shows the impact of arbitrary Laplace distributed perturbations on DeepCorr.
We also apply our timing perturbations on VarCNN. Table 3 shows our attack success () with and without an invisibility constraint. We realize that timing perturbations have much larger impacts on VarCNN than direction perturbations. Moreover, as expected, in the untargeted scenario (SUDU) and for different bandwidth overheads, our attack has better performance without the invisibility constraint. However, even with an invisibility constraint, our attack reduces the accuracy of VarCNN drastically, i.e., a blind timing perturbation with an average 0 and a tiny standard deviation of reduces the accuracy of VarCNN by .
Limited Noise  Stealthy Noise  

SUDU ()  Max STDU (#, )  Max SUDT (#, )  Max STDT (##, )  SUDU ()  Max STDU (#, )  Max SUDT (#, )  Max STDT (##, )  
0,5  
0,10  
0,20  
0,30  
0,50 
7.3 Adversarially Perturbing Sizes
We evaluate our size perturbation attack on DeepCorr, which is the only system (among the three we studied) that uses packet sizes for traffic analysis. As DeepCorr is mainly studied in the context of Tor, our perturbation algorithm enforces the size distribution of Tor on the generated size perturbations. Figure 5 shows the results when the adversary only manipulates packet sizes. As can be seen, size perturbations are less impactful on DeepCorr than timing perturbations, suggesting that DeepCorr is more sensitive to the timings of packets.
7.4 Perturbing Multiple Features
In this section, we evaluate the performance of our adversarial perturbations when we perturb multiple features simultaneously. VarCNN uses both packet timing and directions to fingerprint websites. Table 4 shows the impact of adversarially perturbing both of these features on VarCNN; we see that combining perturbation attacks increases the impact of the attack, e.g., in the untargeted scenario (SUDU), the combination of both attacks with parameters results in an attack success of while the timebased and directionbased perturbations alone result in and , respectively. Similarly, in Figure 5, we see that by combining time and size perturbations, the accuracy of DeepCorr drops from to (with ) by injecting only 20 packets, while using only time perturbations the accuracy drops to .
,  BW Overhead ()  :  SUDU (%)  Max STDU (#, )  Min STDU (#, )  Max SUDT (#, )  Min SUDT (#, )  Max STDT (##, )  Min STDT (##, ) 

20, 0, 5  0.04  
100, 0, 10  2.04  
500, 0, 20  11.11  
1000, 0, 30  25.0  
2000, 0, 50  66.66 
7.5 Comparison With Traditional Attacks
There exist traditional attacks on DNNbased traffic analysis systems that use techniques other than adversarial perturbations. In this section, we compare our adversarial perturbation attacks with such traditional approaches.
Packet insertion techniques: Several WF countermeasures work by adding new packets. We show that our adversarial perturbations are significantly more effective with similar overheads. WTFPAD [25] is a stateoftheart technique which adaptively adds dummy packets to Tor traffic to evade website fingerprinting systems. Using WTFPAD on the DF dataset reduces the WF accuracy to at the cost of a bandwidth overhead. Similarly, the stateoftheart WalkieTalkie [56] reduces DF’s accuracy to with a bandwidth overhead and a latency overhead [46]. On the other hand, our injectionbased targeted blind adversarial attack reduces the detection accuracy to (close to random guess) with only a bandwidth overhead and no added latency (using the exact same datasets). To compare existing WF countermeasures with our results while using VarCNN model, we refer to their paper [3] where WTFPAD can decrease the accuracy of VarCNN by (from to ) with bandwidth overhead. However, according to Table 2, with a similar bandwidth overhead (1000 inserted packets and overhead), our attack reduces the accuracy by which significantly outperforms WTFPAD. Our results suggest that, our blind adversarial perturbation technique drastically outperforms traditional defenses against deep learning based website fingerprinting systems.
Time perturbation techniques: Figure 1(c) compares our technique with a naive countermeasure of adding random Laplacian noise to the timings of the packets. We see that by adding a Laplace noise with zero mean and 20ms standard deviation, the accuracy of DeepCorr drops from TP (for FP) to TP, but using our adversarial perturbation technique with the same mean and standard deviation, the accuracy drops to and without and with invisibility, respectively.
Nonblind adversarial perturbations: Two recent works [24, 58] use “nonblind” adversarial perturbations to defeat traffic analysis classifiers. As discussed earlier, we consider these techniques unusable in regular traffic analysis applications, as they can not be applied on live connections. Nevertheless, we show that our technique even outperforms these nonblind techniques; for instance, when DF is the target system, Mockingbird [24] reduces the accuracy of DF by with a bandwidth overhead (in fullduplex mode), while our directionbased blind perturbation technique reduces the accuracy of DF by much higher and with a much lower bandwidth overhead of .
8 Countermeasures
In this section, we evaluate defenses against our blind adversarial perturbations. We start by showing why our perturbations are hard to counter. We will then borrow three countermeasure techniques from the image classification domain, and show that they perform poorly against blind adversarial perturbations. Finally, we will design a tailored, more efficient defense on blind adversarial perturbations.
Uniqueness of Our Adversarial Perturbations: A key property that impacts countering adversarial perturbations is the uniqueness of adversarial perturbations: if there is only one (few) possible adversarial perturbations, the defender can identify them and train her model to be robust against the known perturbations. As explained before, our adversarial perturbations are not unique: our algorithm derives a perturbation generator () that for random s can create different perturbation vectors. To demonstrate the nonuniqueness of our perturbations, we created 5,000 adversarial perturbations for the applications studied in this paper (we stopped at 5,000 only due to limited GPU memory). Figure 7 shows the histogram of the distance between the different adversarial perturbations that we generated for DeepCorr. We can say that the generated perturbations are not unique, and, the adversary cannot easily detect them. These different perturbations however cause similar adversarial impacts on their target model, as shown in Figure 7.
Adapting Existing Defenses: Many defenses have been designed for adversarial examples in image classification applications, particularly, adversarial training [29, 51, 31], gradient masking [40, 44], and regionbased classification [6]. In Appendix A, we discuss how we adapt these defenses to our networking application.
Adversary Strength  Transferability 

Adversary Strength  Transferability 

Adversary Strength  Transferability 

Our Tailored Defense: We use the adversarial training approach in which the defender uses adversarial perturbations crafted by our attack to make the target model robust against the attacks. We assume the defender knows the objective function and its parameters. We evaluate our defense when the defender does not know if the attack is targeted or untargeted (for both source and destination). The defender trains the model for one epoch, and then generates blind adversarial perturbations from all possible settings using algorithm 1. Then, he extends the training dataset by including all of the adversarial samples generated by the adversary and trains the target model on the augmented train dataset. Algorithm 6 in Appendix D sketches our defense algorithm.
Comparing our defense vs. prior defenses: We compare our defense with previous defenses borrowed from the image classification literature. Tables 5 and 6 compare the performances of different defenses on DF and DeepCorr scenarios, respectively. As we see, none of the prior defenses for adversarial examples are robust against our blind adversarial attacks, and in some cases, utilizing them even improves the accuracy of the attack. However, the results show that our tailored defense is more robust than prior defenses. Since the attacker knows the exact attack mechanism, all defense methods cannot perform well when the adversary uses higher strengths in crafting adversarial perturbations. While our defense is more robust against blind adversarial attacks, it increases the training time of the target model by orders of magnitude which makes it not scalable for larger models. Therefore, designing efficient defenses against blind adversarial perturbations is an important future work.
9 Transferability
An adversarial perturbation scheme is called to be transferable if the perturbations it creates for a target model can misclassify other models as well. A transferable perturbation algorithm is much more practical, as the adversary will not need to have a whitebox access to its target model; instead, the adversary will be able to use a surrogate (whitebox) model to craft its adversarial perturbations, and then apply them to the original blackbox target model.
In this section, we evaluate the transferability of our blind adversarial perturbation technique. First, we train a surrogate model for our traffic analysis application. Note that, the original and surrogate models do not need to have the same architecture, but they are trained for the same task (likely with difference classification accuracies). Next, we create a perturbation generation function for our surrogate model (as described before). We use this to generate perturbations, and apply these perturbations on some sample flows. Finally, we feed the resulted perturbed flows as inputs to the original model (i.e., the target blackbox model) of the traffic analysis application. We measure transferability using a common metric from [39]: we identify the input flows that are correctly classified by both original and surrogate models before applying the blind adversarial perturbation; then, among these samples, we return the ratio of samples misclassified by the original model over the samples misclassified by the surrogate model as our transferability metric.
Directionbased technique To evaluate the transferability of our directionbased perturbations, we use the DF system [46] as the surrogate model and the WF system of Rimmer et al. [43] as the original model. Note that the model proposed by Rimmer et al. uses CNNs, however, it has a completely different structure than DF. We train both models on DF’s dataset [46], and generate blind adversarial perturbations for the surrogate DF model. Then we test the original model using these perturbations. Table 9 shows the transferability of our directionbased attack with different noise strengths. As can be seen, our directionbased attack is highly transferable.
Timebased technique For the transferability of the timebased attack, we use DeepCorr [34] as the original model. We use AlexNet [27] as the surrogate model, which has a completely different architecture. We train AlexNet on the same dataset used by DeepCorr. Since the main task of AlexNet is image classification, we modify its hyperparameters slightly to make it compatible with the DeepCorr dataset. To calculate transferability, we fix the false positive rates of both surrogate and original models to the same values (by choosing the right flow correlation thresholds). Table 9 shows high degrees of transferability for the timebased attack with different blind noise strengths (for a constant false positive rate of ).
Sizebased technique To evaluate the transferability of the sizebased perturbations, we use DeepCorr as the original model and AlexNet as the surrogate model, and calculate transferability as before. Table 9 shows the transferability of the sizebased technique with different blind noise strengths for a false positive rate of .
To summarize, we show that blind adversarial perturbations are highly transferable between different model architectures, allowing them to be used by blackbox adversaries.
10 Conclusions
In this paper, we introduced blind adversarial perturbations, a mechanism to defeat DNNbased traffic analysis classifiers which works by perturbing the features of live network connections. We presented a systematic approach to generate blind adversarial perturbations through solving specific optimization problems tailored to traffic analysis applications. Our blind adversarial perturbations algorithm is generic and can be applied on various types of traffic classifiers with different network constraints.
We evaluated our attack against stateoftheart traffic analysis systems, showing that our attack outperforms traditional techniques in defeating traffic analysis. We also showed that our blind adversarial perturbations are even transferable between different models and architectures, so they can be applied by blackbox adversaries. Finally, we showed that existing defenses against adversarial examples perform poorly against blind adversarial perturbations, therefore we designed a tailored countermeasure against blind perturbations.
Acknowledgements
The project is generously supported by the NSF CAREER grant CNS1553301 and the NSF grant CNS1564067. Milad Nasr is supported by a Google PhD Fellowship in Security and Privacy.
References
 [1] (2019) Universal Adversarial Audio Perturbations. arXiv preprint arXiv:1908.03173. Cited by: §4.1.
 [2] (2020) Practical Traffic Analysis Attacks on Secure Messaging Applications. In NDSS, Cited by: §3.3.
 [3] (2018) VarCNN and DynaFlow: Improved Attacks and Defenses for Website Fingerprinting. CoRR abs/1802.10215. External Links: Link, 1802.10215 Cited by: §1, §2.1, §2.1, §3.3, §3.3, §5, §6.2, §7.1, §7.5, Table 2, Table 3, Table 4.
 [4] (2014) Csbuflo: A congestion sensitive website fingerprinting defense. In Proceedings of the 13th Workshop on Privacy in the Electronic Society, Cited by: §3.3.
 [5] (2012) Touching from a distance: Website fingerprinting attacks and defenses. In Proceedings of the ACM CCS, Cited by: §2.1, §3.3.
 [6] (2017) Mitigating evasion attacks to deep neural networks via regionbased classification. In Proceedings of the 33rd Annual Computer Security Applications Conference, Cited by: Appendix A, Table 5, Table 6, §8.

[7]
(2017)
Adversarial examples are not easily detected: Bypassing ten detection methods.
In
Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security
, Cited by: Appendix A.  [8] (2017) Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP), Cited by: §3.2.
 [9] (2017) EAD: ElasticNet Attacks to Deep Neural Networks via Adversarial Examples. Cited by: §1.
 [10] (2017) Website fingerprinting defenses at the application layer. Proceedings on Privacy Enhancing Technologies. Cited by: §3.3.
 [11] (2011) A statistical test for information leaks using continuous mutual information. In Computer Security Foundations Symposium (CSF), Cited by: §2.1, §3.3.

[12]
(2018)
Boosting adversarial attacks with momentum.
In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
, Cited by: §1, §3.2, §4.2.  [13] (2018) Robust physicalworld attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §3.2.
 [14] (2016) Deep learning. Cited by: §3.1, §4.3.
 [15] (2014) Generative Adversarial Nets. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §5.1.
 [16] (2014) Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. Cited by: §3.2, §4.2.
 [17] (2015) Explaining and Harnessing Adversarial Examples. In 3rd International Conference on Learning Representations, External Links: Link Cited by: §1, §3.2, §4.2.
 [18] (2016) kfingerprinting: A robust scalable website fingerprinting technique. In USENIX Security Symposium, Cited by: §2.1, §3.3, §3.3.
 [19] (2018) Learning Universal Adversarial Perturbations with Generative Models. In 2018 IEEE Security and Privacy Workshops (SPW), Cited by: §4.1.
 [20] (2015) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Cited by: §6.2.
 [21] (2018) Decision Boundary Analysis of Adversarial Examples. In International Conference on Learning Representations, External Links: Link Cited by: §1.
 [22] (2009) RAINBOW: A Robust And Invisible NonBlind Watermark for Network Flows. In NDSS, Cited by: §1, §2.1, §3.3.
 [23] (2014) Nonblind watermarking of network flows. IEEE/ACM Transactions on Networking (TON) 22 (4). Cited by: §2.1, §3.3.
 [24] (2019) Mockingbird: Defending Against DeepLearningBased Website Fingerprinting Attacks with Adversarial Traces. CoRR. Cited by: §3.3, §6.2, §6.2, §7.5.
 [25] (2016) Toward an efficient website fingerprinting defense. In European Symposium on Research in Computer Security, Cited by: §3.3, §6.2, §7.5.
 [26] (2014) Adam: A Method for Stochastic Optimization. International Conference on Learning Representations. Cited by: §4.1.
 [27] (2012) Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, Cited by: §9.
 [28] (2016) Adversarial examples in the physical world. ArXiv abs/1607.02533. Cited by: §1.
 [29] (2016) Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236. Cited by: §3.2, §4.2, §8.
 [30] (2004) Timing attacks in lowlatency mix systems. In International Conference on Financial Cryptography, Cited by: §2.1, §3.3.
 [31] (2017) Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. Cited by: Appendix A, Table 5, Table 6, §8.
 [32] (2017) Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §1, §3.2, §4.1, §4.2.
 [33] (2016) DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, External Links: Link Cited by: §1.
 [34] (2018) Deepcorr: strong flow correlation attacks on tor using deep learning. In Proceedings of the ACM CCS, Cited by: §1, §1, §1, §2.1, §2.1, §3.3, §5, §6.2, §6.4, §7.2, Table 9, §9.
 [35] (2017) Compressive Traffic Analysis: A New Paradigm for Scalable Traffic Analysis. In Proceedings of the ACM CCS, Cited by: §2.1, §3.3, §5.1.
 [36] (2017) Compressive Traffic Analysis: A New Paradigm for Scalable Traffic Analysis. In Proceedings of the ACM CCS, Cited by: §2.1, §3.3.
 [37] (2016) Website Fingerprinting at Internet Scale.. In NDSS, Cited by: §2.1, §3.3, §3.3.
 [38] (2011) Website fingerprinting in onion routing based anonymization networks. In Proceedings of the 10th annual ACM workshop on Privacy in the electronic society, Cited by: §2.1, §3.3.
 [39] (2016) Transferability in Machine Learning: from Phenomena to BlackBox Attacks using Adversarial Samples. ArXiv. Cited by: §9.
 [40] (2016) Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy (SP), Cited by: Appendix A, §8.
 [41] (2017) Automatic Differentiation in PyTorch. In NIPS Autodiff Workshop, Cited by: §6.
 [42] Tor: Pluggable transports. The Tor Project. Note: https://www.torproject.org/docs/pluggabletransports.html.en Cited by: §6.3.
 [43] (2017) Automated website fingerprinting through deep learning. arXiv preprint arXiv:1708.06376. Cited by: §2.1, §3.3, §3.3, §5, §6.3, §6.4, Table 9, §9.
 [44] (2018) Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Thirtysecond AAAI conference on artificial intelligence, Cited by: Appendix A, Table 5, Table 6, §8.
 [45] (2006) Timing analysis in lowlatency mix networks: Attacks and defenses. In European Symposium on Research in Computer Security (ESORICS), Cited by: §2.1, §3.3.
 [46] (2018) Deep fingerprinting: Undermining website fingerprinting defenses with deep learning. In Proceedings of the ACM CCS, Cited by: §1, §1, §1, §2.1, §3.3, §3.3, §5, §6.2, §6.3, §6.4, §7.1, §7.5, Table 1, Table 9, §9.

[47]
(201710)
One Pixel Attack for Fooling Deep Neural Networks.
IEEE Transactions on Evolutionary Computation
PP. External Links: Document Cited by: §3.2.  [48] (2015) RAPTOR: routing attacks on privacy in tor. In USENIX Security Symposium, Cited by: §2.1, §2.1, §3.3.
 [49] (2013) Intriguing properties of neural networks. Cited by: §1.
 [50] Design of a BlockingResistant Anonymity System. Note: https://svn.torproject.org/svn/projects/designpaper/blocking.html Cited by: §6.3.
 [51] (2017) Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204. Cited by: §8.

[52]
(2013)
The nature of statistical learning theory
. Springer science & business media. Cited by: §3.1.  [53] (201408) Effective Attacks and Provable Defenses for Website Fingerprinting. In USENIX Security Symposium, San Diego, CA. External Links: ISBN 9781931971157, Link Cited by: §2.1, §3.3, §3.3, §5.
 [54] (2013) Improved website fingerprinting on tor. In Proceedings of the 12th ACM workshop on Workshop on privacy in the electronic society, Cited by: §2.1, §3.3, §6.3, §6.4.
 [55] (2016) On realistically attacking tor with website fingerprinting. Proceedings on Privacy Enhancing Technologies. Cited by: §2.1, §3.3, §6.3, §6.4.
 [56] (2017) Walkietalkie: An efficient defense against passive website fingerprinting attacks. In USENIX Security Symposium, Cited by: §3.3, §6.2, §6.3, §7.5.
 [57] (2000) Finding a Connection Chain for Tracing Intruders. In ESORICS, Cited by: §1, §2.1.
 [58] (2019) Statistical privacy for streaming traffic.. In NDSS, Cited by: §3.3, §6.2, §6.2, §7.5.
 [59] (2000) Detecting Stepping Stones. In USENIX Security Symposium, Cited by: §2.1.
 [60] (2004) On flow correlation attacks and countermeasures in mix networks. In International Workshop on Privacy Enhancing Technologies, Cited by: §2.1, §3.3.
Appendix A Adapting Traditional Defenses to Adversarial Examples
Madry et al. [31] presented a scalable adversarial training approach to increase the robustness of deep learning models to adversarial examples. In each iteration of training, this method generates a set of adversarial examples and uses them in the training phase. Madry et al.’s defense is the most robust defense among the adversarial training based defenses [7]. We cannot use this method as is, since in the image recognition applications, pixels can take real values, while in directionbased traffic analysis methods, features take only two values (1, +1). Therefore, we modify this defense to our setting. To generate a set of adversarial examples in the training process, we randomly choose a number of packets and flip their directions from 1 to +1 and vice versa. Similarly, for the packet timings and sizes we enforced all of application constraints for generating the adversarial examples.
From the gradient mask approach, we used the input gradient regularization (IGR) technique of Ross and DoshiVelez [44]. IGR is more robust against adversarial attacks compared to its previous work [40]. Their defense trains a model to have smooth input gradients with fewer extreme values which becomes more resistant to adversarial examples. We utilize this approach to train a robust model using DF structure. We evaluated the directionbased attack against this defense with parameter .
While the previous defenses train a robust model against adversarial attacks, Cao and Gong [6] designed a defense method which does not change the training process. They proposed a regionbased classification (RC) method which creates a hypercube centered at the input to predict its label. Then, the method samples a set of data points from the crafted hypercube and uses an existing trained model to produce predicted label for each sampled data point; Finally, it uses majority voting to generate the final class label for the given input. We need to make changes to the regionbased classification defense. In contrast to images, we cannot create a hypercube centered at the input by just adding random real values to the packet direction sequences which have values 1, 1. Instead, for each input, we create the hypercube by randomly choosing a number of packets in the sequence and flipping their directions. To apply regionbased classification in the test phase of the directionbased method while adding blind perturbations, we randomly choose 125 packets and change their directions to form the hypercube. Similar to Cao and Gong, we call this number as the radius of the hypercube. We choose this value for the radius because 125 is the maximum number of packets we can use to form the hypercube while the accuracy of the regionbased method does not go below the accuracy of the original DF model. Using radius of 125 for hypercubes, we apply the regionbased classification against our attack. For time and size based methods, we use the strength of the adversary to generate the hypercubes.
Appendix B Model Parameters
Table 10 shows the structure of the adversarial model for each type of perturbation and also the discriminator model.
Model  Number of hidden layers  Hidden layer units  Optimizer  Learning rate  Activation 

Directionbased  Adam  ReLu  
Timebased  Adam  ReLu  
Sizebased (ordering)  Adam  ReLu  
Sizebased (amplitude)  Adam  ReLu  
Discriminator  Adam  ReLu 
Appendix C The Scheme of Our Blind Adversarial Perturbation Technique
Figure 8 illustrates the main components of our blind adversarial perturbations algorithm.
Appendix D Defense Algorithm
Algorithm 6 summarizes the adversarial approach to train a robust model against blind adversarial attacks in traffic analysis applications.