SecEL: Privacy-Preserving, Verifiable and Fault-Tolerant Edge Learning for Autonomous Vehicles

01/27/2020
by   Jiasi Weng, et al.
0

Mobile edge computing (MEC) is an emerging technology to transform the cloud-based computing services into the edge-based ones. Autonomous vehicular network (AVNET), as one of the most promising applications of MEC, can feature edge learning and communication techniques, improving the safety for autonomous vehicles (AVs). This paper focuses on the edge learning in AVNET, where AVs at the edge of the network share model parameters instead of data in a distributed manner, and an aggregator aggregates parameters from AVs and at the end obtains a trained model. Existing edge learning cases care more about the accuracy performance but neglect discussing its security issues, such as data leakage, computing integrity invasion, and connection failure. To the best of our knowledge, there lacks an effective scheme simultaneously covering the foregoing security issues. Therefore, we propose SecEL, a privacy-preserving, verifiable and fault-tolerant scheme for edge learning in AVNET. First, we leverage the primitive of bivariate polynomial-based secret sharing to encrypt model parameters by one-time padding. Second, we use homomorphic authenticator based on message authentication code to support verifiable computation. Third, we mitigate the computation failure problem caused by connection failure. The employed cryptographic primitives are lightweight, thus the presented scheme is practical. Last, we simulate and evaluate SecEL in terms of time cost, throughput and classification accuracy. The experiment results demonstrate the effectiveness of SecEL.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

09/13/2019

F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds

Autonomous vehicles are heavily reliant upon their sensors to perfect th...
03/01/2021

Blockchain-Based Federated Learning in Mobile Edge Networks with Application in Internet of Vehicles

The rapid increase of the data scale in Internet of Vehicles (IoV) syste...
12/17/2019

Computation Offloading and Content Caching Delivery in Vehicular Edge Computing: A Survey

Autonomous Vehicles (AVs) generated a plethora of data prior to support ...
02/18/2021

AggFT: Low-Cost Fault-Tolerant Smart Meter Aggregation with Proven Termination and Privacy

Smart meter data aggregation protocols have been developed to address ri...
07/21/2019

A Lightweight and Privacy-Preserving Authentication Protocol for Mobile Edge Computing

With the advent of the Internet-of-Things (IoT), vehicular networks and ...
01/06/2022

SPDL: Blockchain-secured and Privacy-preserving Decentralized Learning

Decentralized learning involves training machine learning models over re...
07/17/2020

Privacy-Preserving Distributed Learning in the Analog Domain

We consider the critical problem of distributed learning over data while...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

In recent years, mobile edge computing (MEC) is regarded as a promising way, enabling both local processing and global coordination, to perform computation-consuming tasks [1, 2, 3]. Autonomous vehicular network (AVNET) becomes one of the popular applications of MEC, facilitating smart city development [4, 5]. In particular, MEC nodes, such as smartphones or autonomous vehicles (AVs)’ central controllers at the edge of the network, are allowed to provide sufficient computation resources and locally tackle computing tasks, and meanwhile, a server at an edge node (e.g., base station) is responsible for global coordination [3]. These MEC nodes have powerful central processing units (CPUs) or graphics processing units (GPUs) and usually are close to vehicles, resulting in short response time and low transmission latency. Therefore, more and more advanced technologies have been integrated into MEC.

Edge learning [3, 6]

in AVNET is one of the advanced technologies of MEC, where AVs are able to collaboratively learn a high-accuracy machine learning model used to predict the road environment to improve vehicle safety. A practical case of edge learning in AVNET is the Tesla’s machine learning model for auto-driving feeded with data from millions of Tesla vehicles 

[6]. Following the popular network architecture for MEC [7], in edge learning, there are multiple AVs being collaborative learners and a remote server being a parameter server. Autonomous vehicles share model parameters which are trained on the local sensing data. The server keeps a common machine learning model to train, by aggregating shared model parameters from multiple AVs and updating model parameters. Individual AVs continually train on the same local sensing data using the updated parameters until the pre-defined loss threshold is reached.

Problem Statement: Efforts have been taken to improve the performance of edge learning [3, 6, 8, 9]. However, the privacy and security issues of edge learning may be not well-addressed: (i) sharing model parameters (instead of data) cannot protect data privacy due to the popular membership inference attacks [10] and reconstruct attacks [11] on the sharing model parameters. These attacks can infer the specific vehicle-behaviour information implied in the local sensing data which natively causes privacy concern. Recently, data privacy issue has attracted people’s great attention, and international organization has announced data protection standard, such as GDPR [12]. Existing solutions employ traditional public key-based encryption to ensure privacy-preserving federated learning [13, 14], but they do not consider the features of MEC setting, e.g., resource-limited, bandwidth-hungry and dynamic AVNET. Thus, a lightweight privacy-preserving scheme applied into edge learning is in need. (ii) The aggregator at the remote server can be subject to various attacks, such as computing integrity invasion [15, 16] leading to a manipulated model misbehaving (refer it to Section 3.2). It would disrupt the model accuracy and even poison AVs’ sensing capability. Existing privacy-preserving federated learning methods [13, 17] protect data privacy but neglect data integrity. Therefore, providing privacy-preserving scheme with computing verification while enforcing efficiency is inevitable for edge learning. (iii) Communication failure between MEC nodes and the remote server may happen and cause computation failure, since AVs usually have high mobility. Here, computation failure can decrease the learning performance of the AVs, which may cause the unexpected safety threats. From this view, a proposed security scheme should particularly tolerate the problem of connection failure.

In addition, we concretely focus on the Software-Defined Networking (SDN)-enabled network architecture for MEC, since it is recently standardized by 3GPP [18] and promotes edge learning. SDN is widely adopted and enables the capability of resources management. An SDN-enabled cloud server assembling SDN control modules can select the most suitable technology to ensure maximum-degree reliability of connection. Most importantly, the SDN-enabled server can have a network-wide view of the connection state of communicating AVs [19, 20], meaning that it can be aware of the event of connection failure, which is crucial for the proposed scheme.

Our Contribution: To simultaneously overcome the aforementioned challenges, we propose a privacy-preserving, verifiable, and fault-tolerant scheme for edge learning in AVNET, where AVs asynchronously share local model parameters to an SDN-enabled server in charge of aggregation. The contributions are summarized as follows:

  • We leverage the primitive of bivariate polynomial-based secret sharing and utilize the method of one-time padding technique instead of asymmetric encryption to guarantee the secrecy of shared model parameters, which leads to a lightweight privacy-preserving scheme.

  • We seamlessly combine homomorphic authenticator with the proposed privacy-preserving scheme, which allows participating AVs to verify the computation results from the SDN-enabled server, avoiding getting manipulated results.

  • We enable SecEL to tolerate disconnected AVs, thereby improving the reliability of learning. Particularly, an honest but disconnected autonomous vehicle’s secret can be recovered by a group of alive participants so that he can fluently obtain the computation results from the SDN-enabled server. Considering the rare bandwidth resources, there is no need to introduce the additional interaction to share secrets among the group of alive participants.

  • We lastly simulate SecEL in Python-based setting and evaluate its performance when it is applied into a popular model-learning case. The evaluation results demonstrate the effectiveness of SecEL in terms of time cost, throughput and learning accuracy.

Roadmap: The rest of our paper is organized as follows. In Section 2, we briefly introduce the primitives involved in our paper, including federate learning, bivariate polynomial-based secret sharing and homomorphic authenticator. In addition, we also gives a typical scenario of edge learning in AVNET. In Section 3, we present the security model of SecEL. Subsequently, we show the detailed design of SecEL over five phases in Section 4. Afterwards, in Section 5, we give a security analysis of proposed SecEL with respect to the security model. We lastly demonstrate the performance results by evaluating SecEL in Section 6, survey the related work in Section 7 and make a conclusion in Section 8.

2 Background

In this section, we review the essential algorithm of federated learning. Afterwards, we introduce two utilized cryptographic tools.

2.1 Federated Learning

Federated learning is a kind of distributed machine learning algorithm, where multiple machines and a central sever jointly solve the learning problem using distributed gradient descent techniques [7]

. The learning problem is to obtain the optimized parameters by minimizing the loss function on training dataset.

Formally, let vector

and represent training dataset and training parameters of the model, respectively. Then, defines the loss function on . Therefore, when federated learning is adapted, denotes the -th machine’s loss function, where is maintained locally by the machine, and presents the loss function at a central server. Suppose machines exist and then the following formula will be hold:

To minimize

, stochastic gradient descent method (SGD) is often used to find

Distributed SGD can be naturally derived from SGD. It includes two major learning steps to achieve the learning problem, such as local update and global aggregation. In local update, each machine preforms the machine learning task locally and trains small-scale model. In global aggregation, a central server collects the small-scale models together and updates parameters.

Concretely, it works as follows : (i) Each machine initializes the local model parameters with the same randomized value, where . (ii) The step of local update is executed to update by iteration via the following rule,

where is the learning rate. (iii) After one or multiple local updates, the step of global aggregation is performed by aggregating all machines’ parameters (a.k.a. intermediate gradients) uploaded by the participating machines. Herein, we let the times of local update be , as it is recommended to be in range [21]. That is,

After finishing the global aggregation step, is broadcasted to the joining machines and they continue the local update step. (iv) This learning process is repeated until the loss function at the cental server is minimized to the predefined threshold. In this paper, should be encrypted by the individual participant before being aggregated.

On the other hand, distributed SGD generally is classified into two categories: synchronous SGD and asynchronous SGD, depending on whether the process of uploading local parameters is synchronized. Consider the features of AVNET environment including bandwidth-limitation and latency-sensitivity, this paper employs weakly asynchronous SGD

[22] which neutralizes the effect of synchronous and asynchronous SGD. Particularly, it lets the central server wait for () number of participating machines finishing uploading their parameters instead of all machines as synchronous SGD does. At the same time, it avoids the problem of stale gradients caused by asynchronous SGD.

2.2 Bivariate Polynomial-based Secret Sharing

Derived from Shamir’s secret sharing scheme, bivariate polynomial-based secret sharing uses a bivariate polynomial to share secret instead of using a univariate one [23]. Suppose that a secret is hidden in the constant term of a bivariate polynomial. The secret owner can distribute two polynomials related the secret to a participant, rather than a point. Thus, it distributes more information to participants than the univariate polynomial-based secret sharing does. Due to the additional information, a participant’s lost secret is allowed to be rebuilt by other participants. In addition, each pair of AVs naturally share a common private key, which enables them to privately transmit messages.

Formally, a bivariate polynomial can be defined as mod , where is a prime lager than the secret which is hided in . Herein, the degree in both variate and is . Our scheme uses a symmetric bivariate polynomial, in which coefficients , which allows to have a same threshold in variate and to recover the secret . In the contrary, in an ansymmetric bivariate polynomial, is not equal to , so does not exactly have threshold .

2.3 Homomorphic Authenticator

Basically, homomorphic authenticator is used to indicate which input data is authenticated and how the input data should be correctly computed [24, 25]. A general homomorphic authenticator scheme includes a sequence of the following probabilistic polynomial time algorithms.

. With a security parameter , it outputs authenticated key and public parameter .

. Taking the authenticated key , a message and a label as input, it generates an authenticated tag for under .

. Given , a vector of authenticated tags and an arithmetic computation (it could be an arithmetic circuit), it outputs a new authenticated tag . If is the authenticated tag for () as the output of some label program (to be introduced), then authenticates , where , as the output of the composed program .

. It is a determinatic algorithm to verify whether authenticates by the labeled program .

Evaluation correctness of the above scheme is indicated as follows. If , , and , then .

Remark. Label and labeled program are the essential components for homomorphic authenticator. They define which data is authenticated and how data is evaluated, respectively. Hence, SecEL uses the round index to label the shared parameters and the labeled program is the sum operation on the shared parameters from participants. A labeled program is represented as . The composed program is generated by , where the inputs of correspondingly are the different labeled inputs of .

3 Scenario Model and Security Model

In this section, we first introduce the edge learning scenario in AVNET. Then, we discuss the basic assumptions around the security threats in the network scenario. Further, we present the security goals with respect to the threats.

Fig. 1: Scenario model

3.1 Scenario Model

Fig. 1 shows an overview of the edge learning scenario in AVNET [7], where AVs collaborate to learn the vehicle environment for autonomous navigation with the help of edge nodes. Through edge learning, AVs can indirectly share sensing data to improve the prediction accuracy, i.e., sharing model parameters trained on local sensing data. Specifically, edge nodes can be various base stations (BSs), including eNBs and RSUs/Wi-Fi APs. AVs can connect with the closest BS via different communication technologies, such as dedicated short range communications (DSRC) and White-Fi. V2I (vehicle to infrastructure) link presents the connection between AVs and BSs. V2V (vehicle to vehicle) link means the connection among vehicles.

Suppose that each AV has the small-scale sensing data due to the constrained sensing ability and fails to gain a high-accuracy prediction model. Therefore, each AV can locally train a small-scale model on the local sensing data. After training, AVs send intermediate gradients to an edge node (e.g., an eNB) that keeps and updates a common training model. Then, AVs obtain the latest updated parameters from the edge node to update their local models. The foregoing process repeats until the training loss is small enough. For ease of presentation, it is defined that those AVs and the edge node are in a collaborative group, and they are participants. Respectively, the entities outside the group are outsiders.

Recall that we focus on the SDN-enabled MEC which is capable of resources management. In this scenario, SDN control modules in edge nodes can help to flexibly and efficiently allocate network resources for V2I link according to different requirements of communication qualities. Note that an edge node’s SDN control module maintains the global view of V2I and V2V links in its communication range. Hence, the edge node can be aware of the link deterioration of AVs, if this AV moves out of the range of the connected AVs, like vehicle . Further, it is not difficult for BSs to identify that ciphertexts are from which AVs as shown in Section 4.

3.2 Basic Assumptions

First, we assume that AVs are honest but curious, which means each AV can honestly participate in collaborative learning but want to know more about other AVs’ data, such as intermediate gradients. That is because that there exists membership inference attacks [10] and reconstruct attacks [11]. If AVs’ intermediate gradients are exposed, they can be exploited to infer secret information of the corresponding local data.

Second, we assume that there exist fault AVs in the scenario. Fault AVs are those whose connections are invalid and they cannot response to other AVs inside a group. This is reasonable since AVs may offline due to various reasons.

Third, we assume that the edge node can be malicious, where he may manipulate or substitute the intermediate gradients collected from AVs and return incorrect updated parameters. In this case, malicious attacks can be launched, such as trojaning attacks [15]. Particularly, the adversarial edge node can inject trojaning parameters to the original model, through which the manipulated model would misbehave in a specific case. To notice, prior work on the security of machine learning [17] makes an assumption that the parameter server in their scheme (like the role of the edge node in this paper) is honest. In contract, we consider the worst case of our design, which is more realistic.

Last, we assume that there exist authenticated communication channels between entities (e.g. AVs, BSs). Specifically, a symmetric authenticated encryption algorithm [26] can be used to securely transmit data between any two parties who share a private key, in advance.

3.3 Security Goals

In this section, we claim our security goals of SecEL. We will discuss the security issues, taking the features of AVNET into consideration, such as resource limitation, asynchronous communication and AVs’ high mobility.

First of all, SecEL protects each AV’s data privacy against other participants and outsiders. Specifically, individual AVs’ model parameters cannot be leaked to anyone but itself. Only a threshold number of AVs can obtain the aggregated parameters.

Second, SecEL ensures the authenticity and correctness of computation at the edge node need. It is noteworthy that in this resource-limited setting, SecEL needs to provide an efficient verification method to simultaneously ensure the above security goal.

Third, SecEL can tolerate the link-failed AVs and enforce the computation reliability. Concretely, due to the high mobility of AVs and asynchronous communication, some AV can leave the communication range of other AVs and lose the secret shares (rather than the secret) which are given by other AVs in Setup phase (refer it to Section 4). This makes the AV fail to decrypt parameters returned by BS, which may block his training process. Thus, SecEL guarantees that the AV’s secret shares can be recovered by other participants. For the sake of efficiency, this recovery process should not introduce the additional secret sharing round.

4 SecEL Design

Notation Description
the version number of parameters on the BS
integer, the number of AVs participating in a round ( varies in different rounds)
the -length set that is {}
(the similar meanings for )
the unique identifier of each AV
, is a secure prime larger than the secret
, the threshold to reconstruct the secret
the AV with identifier
a masking secret of
(or )
an authenticated key
a partially authenticated key of AV and is a part of
the size of (the similar meanings for )
TABLE I: Summary of notations

As Fig. 2 shown, an overview of SecEL in one round which contains five phases, including Setup, Masking, Aggregation, Verification, Decryption. A round here refers to a period when AVs share their intermediate gradients to the edge node and obtain aggregated parameters computed by the edge node. Below we elaborate all the phases. To notice, we summarize the notations in Table I for ease of description.

Fig. 2: Overview of SecEL. Herein, the number of active AVs in different phases (i.e., ) and their relationships are discussed in Fig. 3.

Setup Phase. In this step, each AV selects a masking secret as well as a partially authenticated key, and shares them to other counterparts. In particular, each AV privately chooses a symmetric bivariate polynomial with degree in variate and , and then hides the masking secret in . Also, randomly selects another key which is the partially authenticated key. Afterwards, distributes shares {, } of and key to other ( & ). For each AV, no less than AVs are assumed receiving his secret shares.

According to the properties of symmetric bivariate polynomial, we have the following insights: (i) ; (ii) number of (or ) are sufficient to reconstruct ; (iii) If any owned by is lost, it can be recovered by number of (); the similar case for holds.

Remark. The above secret distribution can be bandwidth consuming (up to communication complexity in total), which may be incompatible to the bandwidth-hungry setting. To counter such a drawback, we deploy Algorithm 1 to distribute the secret with communication complexity . Algorithm 1 is borrowed from [27], but we allow each AV to share his secret to counterparts rather than a portion of them in the first step. Basically, Algorithm 1 contains two steps: (i) AVs individually choose a degree- univariate polynomial , where . Then, each of them allocates shares of to other AVs. At the end of this step, each one obtains the share of . is defined as . (ii) Each generates another degree- univariate polynomial and . Then, redistributes secret shares of to all AVs, that is, is distributed to (). In this way, the foregoing number of with degree- encode the bivariate polynomial with degree- and . Besides, . can be reconstructed after collecting at least shares (). Similarly, can be recovered by at least shares () because of .

Input: AVs with identifier
Output: Each () possesses and ()
1 Step 1.
2 for each () do
3       choose a -degree ;
4       // keep secretly;
5       for each () do
6             compute (mod );
7             // distribute () to ;
8             // keep locally;
9            
10      
11for  do
12       compute ;
13       ;
14      
15Step 2.
16 for each () do
17       generate a -degree ;
18       //where ;
19       for each () do
20             compute (mod );
21             // allocate () to ;
22            
23      
Algorithm 1 Each AV gets secret shares with communication complexity

Masking Phase. AVs mask intermediate gradients and generate the corresponding authenticated MAC with the masking secret and authenticated key. first builds . Here, is the pseudorandom generator agreed among AVs. After that, he computes (mod ) to mask parameters, where is the element of . For each element, uses the distinct public nonce to generate random number.

To compute the second component, generates a common authenticated key composed of , i.e., (mod ). Besides, prepares since he possesses . Based on and , can compute (mod ). Then, AVs send their masking results {, ()} () to BS.

Remark. It is worth noting that ’s can be recovered by at least AVs by providing ( & ).

Aggregation Phase. BS aggregates together the received ciphertexts in this phase. BS receives AVs’ ciphertext and records their identifier. If some AV’s ciphertext fails to submit, BS will note his identifier in the returned result because they have no contribution in this round. Suppose that BS successfully receives AVs’ ciphertext, . BS separately aggregates two components of ciphertext: and . Next, it returns the aggregated result enclosing the failed AVs’ identifiers, that is, {}.

Verification Phase. Each AV verifies the correctness of aggregated results returned by BS. Suppose that () number of AVs receive results for keeping liveness [27]. They execute a distributed, unpredictable and unbiased randomness algorithm [28] to select an AV leader to collect shares from other AVs, and then recover secrets. In this way, they collaboratively recover which is used to generate in Masking phase. Recall that at least AVs can rebuild one of and then . can be computed with . Next, each of them verifies the correctness of and by identifying whether is equal to . If yes, they go into the next phase; otherwise, they reject the result.

Remark. Note that active AVs in this phase can privately recover the secrets against outsiders and the edge node. The fact is that each pair of AVs naturally share a common private key after Setup phase, i.e., or , between and . With this common key, each of them can securely transmit data by employing a symmetric authenticated encryption algorithm. In doing so, there is no need to additionally run a key-agreement protocol to share a common key.

Decryption Phase. AVs collaboratively unmask the correct ciphertext. Specifically, AVs further jointly reconstruct (also ) which is used to mask the intermediate messages. Note that at least AVs can collaboratively obtain and then . can be computed with . After that, the aggregated message can be unmasked by . Then, each of AVs use to update the local parameters and a new round begins.

Remark. In the case of link deterioration, we assume that ( but ) loses her share. Her secret still can be recovered only if there are at least AVs in the group. That is because AVs are sufficient to rebuild ’s share of . We emphatically explain it in the following text.

Fig. 3: Case of AVs’ participation during a round. We use eight squares to represent eight AVs and suppose . The dotted lines with different colors dedicate the participated AVs in respective phases, e.g., the green dotted lines appoint the three AVs (in green squares) participating in Setup phase, which means they have received shares. Note that AVs’ identifiers belong to the corresponding sets mentioned in Fig. 2.

Once an honest AV loses secret, saying (i.e. ), no less than AVs who can provide shares () can help rebuild . Here, we demonstrate the possible occurrence of the case of AVs’ shares missing with the help of Fig. 3. Obviously, there are five AVs (five squares) contributing in Aggregation phase and they have to verify and decrypt the returned ciphertext in Verification phase and Decrypt phase, respectively. However, two AVs (in the blue squares) have no shares, since they do not participate in Setup phase due to asynchronous communication. Then, they need help from other three AVs (in the green squares) to reconstruct their shares. In addition, the remain AVs (in the white squares) are not permitted to obtain the returned result since they do not make their contributions in Aggregation phase.

5 Security Analysis

In this section, we give the security analysis with respect to the security goals mentioned in Section 3.3. Recall that SecEL devotes to protect data privacy, guarantee computation integrity and ensure reliability. However, whether these issues can be addressed depends on if the primitive of the symmetric bivariate polynomial is secure enough. Therefore, we first give the analysis that symmetric bivariate polynomial we used is secure (Theorem 1), and then we present the security analysis for the foregoing security goals.

Theorem 1.

Symmetric bivariate polynomial is a secure secret sharing scheme.

Proof. It needs to be satisfied that only participants can reconstruct the secret, thus we prove less than number of participants are unable to gain the secret.

Note that the selected has total coefficients. Suppose that there exist participants colluding. has two ()-degree univariate polynomials {, } (note: ) and can build linear independent equations. Then, colluded participants totally gain linear independent equations. At the same time, those colluded participants, with each other, share points of the bivariate polynomial . Thus, they finally can build linear independent equations at total to solve the bivariate polynomial . Since is larger than , colluded participants cannot reconstruct and gain the secret.

Protecting data privacy. Recall that SecEL uses the secret hidden in the symmetric bivariate polynomial to mask the sharing parameters by one-time padding. In this way, the confidentiality of the sharing parameters is ensured, and then the privacy of corresponding data is protected. Formally, we define the method in SecEL to mask parameters is semantically secure as shown in Lemma 1.

Lemma 1.

If is a secure pseudo-random function (PRF), SecEL is semantically secure according to the Definition 1 as shown in the work [24]

Analysis. Refer it to the work [24]’s Definition 1 which defines the security model of one-time padding by games between challenger and adversary

. We obtain the following conclusion: the probability that an adversary correctly guesses a masking parameter

is less than the probability to successfully distinguish and a truly random function plus , i.e., .

Guaranteeing computation integrity. Recall that SecEL uses homomorphic authenticator to generate homomorphic message authentication codes for each AV’s masking parameters, thereby the computation integrity is guaranteed. Formally, we define the method to generate authentication code is unforgeable as depicted in Lemma 2.

Lemma 2.

If is a secure pseudo-random function (PRF), SecEL is unforgeable according to the work [24]’s Definition 2.

Analysis. Refer it to the work [24]’s Definition 2 which defines the security model of generating authenticated code by games between challenger and adversary. We draw the conclusion: the probability that an adversary successfully forges a correct authenticated code is less than the probability plus the negligible probability for the adversary to make verification queries, i.e., . Herein, is the security parameter.

Ensuring reliability. Recall that reliability is referred to allowing for honest AVs correctly decrypting the aggregated parameters only if they have contributions. Refer to the Decryption phase in Section 4, we demonstrate how an honest but failed AV’s secret shares can be recovered. Particularly, the AV makes his contribution in Aggregation phase but loses his secret shares due to the failed connection in Setup phase. Let alive AVs help the honest but failed AV to obtain his shares. Then the AV fluently decrypts the aggregated result, thereby avoiding hindering his local training process, which ensures reliability.

6 Evaluation

In this section, we simulate the presented SecEL. We apply it into a popular image classification task. Extensive experiments are performed to validate the feasibility and effectiveness of SecEL.

6.1 Simulation Environment

We simulate SecEL as a module using Python programming language (version 3.6.4) and PyCryptodome library (version 3.6.1), about 200-line codes at total. The module covers five phases presented in Section 4. Note that the length of randomly selected secret is 128 bit and

is a randomly chosen 130 bit prime. On the other hand, we build the learning model on the MNIST dataset, which is implemented by Python as well, Numpy (version 1.14.0) and Tensorflow (version 1.7.0). The learning model is collaboratively trained by multiple parties via sharing individual model parameters (also called as gradients). Herein, the party refers to the role of AV. Particularly, MNIST dataset is split equally and distributed to individual parties before learning. Individual parties train a common learning model based on the individual dataset. At the same time, they share model parameters obtained from their local models. Shared parameters are masked by calling the

mask function of the SecEL module. Those masked parameters then are aggregated by calling SecEL’s aggregate function. With verify and decrypt functions, the aggregated result can be verified and decrypted. In addition, all of the experiments are conducted on a desktop computer with 2.70 GHz Intel(R) Xeon(R) CPU and 8 GB memory.

6.2 Performance Analysis

According to the detailed design demonstrated in Section 4, we give the performance analysis for SecEL in each round, in terms of computation and communication overhead.

6.2.1 Computation overhead

First, for each AV, there has computation time cost at total, where is the length of . In detail, this total cost can be broken up into the following parts: (i) in Setup phase, each AV needs to share secret to other AV by using Algorithm 1, which leads to time cost ; (ii) in Masking phase, each of them masks parameters by the encryption method of one-time padding, which results in time cost ; (iii) in Verification phase, each active AV collaborates with others to recover the authenticated key and then verifies the returned result, which has time cost ; (iv) in Decryption phase, each active AV needs to rebuild the masking secret and unmask the returned ciphertext with time cost . Second, for the edge node, it has time cost , since it needs to aggregate parameters sent by participating AVs.

Setup Masking Aggregation Verification Decryption
Each AV - -
Edge node - - - - - - - -
TABLE II: Performance analysis on computation overhead

6.2.2 Communication overhead

First, AVs have two kinds of communication cost depending on whether being the leader or not in Verification and Decryption phase. Recall that we use a randomness algorithm to select a leader among active AVs to help recovering secret, which takes communication cost according to the latest work [28]; if is far smaller that , it would lead to . Next, we analyse the communication cost for the AV being leader and the AV not being leader. On one hand, the AV being leader has cost, reduced to . On the other hand, the AV not being leader has , reduced to . Second, for the edge node, it takes when receiving the masking parameters from AVs.

Setup Masking Aggregation Verification    Decryption
Each AV
or or
Edge node - - - - - - - -
TABLE III: Performance analysis on communication overhead

6.3 Evaluation

We first conduct the experiments to evaluate SecEL’s performance from the aspects of time cost and throughput. Time cost is an essentially crucial metric in the setting of AVNET that indicates processing latency. Throughput also is an important factor to demonstrate the consumption of bandwidth resource. We evaluate the time cost of each phase respectively and throughput in Masking and Aggregation phase.

Fig. 4: Impact of No. of parties on time cost in Setup phase.
(a)
(b)
Fig. 5: Impact of No. of gradients on time cost and throughput in Masking phase.

Fig. 4 shows the time consumption in Setup phase with the increasing number of parties. It is worth noting that the consumed time is independent of the size of learning model, i.e., the number of gradients in the following text. When the number of parties is less than , it will take no more than  s. Especially, the time cost for 100 parties is actually  s.

Fig. (a)a shows the time cost in Masking phase as the number of gradients grows up. It can be observed that the time cost increases linearly with the number of gradients. However, it is irrelative to the number of parties, since each party masks gradients individually. On the other hand, the trend of throughput appears to be slightly downward as depicted in Fig. (b)b.

Fig. (a)a illustrates the time consumption in Aggregation phase, which depends on the quantity of gradients and parties. It can be observed that as the number of gradients increases, it becomes more time consuming when a group with the same number of parties performs in Aggregation phase. On the other hand, Fig. (b)b presents that throughput has a decreased tread with more parties participating in.

It can be observed from Fig. (a)a that the number of parties makes insignificant impact on the time consumed in Verification phase, but the rising number of gradients leads to the linearly increasing time consumption. It is reasonable that the figure of parties is far less than that of gradients and more aggregated gradients take more time to verify. Finally, it can be observed from Fig. (b)b that the Decryption phase is more time consuming when compared with other phases due to the overhead of reconstructing masking keys. As demonstrated, the used time grows up on two hands, i.e., the rising number of gradients and parties.

(a)
(b)
Fig. 6: Impact of No. of gradients and No. of parties on time cost and throughput in Aggregation phase.
(a)
(b)
Fig. 7: Impact of No. of gradients and No. of parties on time cost in Verification phase and Decryption phase.

7 Related Work

Recently, there is a wide and hot discussion on edge learning. Some works study the performance of edge learning, e.g., accuracy and efficiency. For example, Wang et al. [3] focus on the convergence rate of the gradient-descent based distributed learning algorithms in resource-limited MEC systems. The authors present an effective control algorithm after analyzing the trade-off relationship among the number of global aggregation, the classification accuracy and resource cost. Specifically, the control algorithm is to make the best use of given amount of resources to learn by balancing well the number of local update and global aggregation. Also, Zhu et al. [6] research on the communication latency issue of distributed learning in MEC settings and propose the novel concept of learning-driven communication. In a word, the foregoing work can be regarded as the complementary parts to this paper.

On the other hand, to the best of our knowledge, there still have few papers discussing security issues of edge learning. Although there exists a portion of work on privacy-preserving distributed learning, they could not be directly immigrated into the MEC systems, where communication is asynchronous and resources are limited. We compare these works in Table IV, which demonstrates SecEL simultaneously achieves three security goals. First of all, Shokri et al. [29]

implement a privacy-preserving distributed deep learning system, where multiple parties share a small fraction of gradients and learn a deep learning model together. Their system uses the difference privacy technique to add noise into shared parameters, so that data privacy is guaranteed to a certain degree. However, their proposed system has been attacked by Hitaj

et al. [11] by employing the tool of GAN (Generative Adversarial Network). Bonawitz et al. [17] preserve Shokri et al.‘s system model of training and proposes an efficient method to securely aggregate local gradients of participants for a common deep learning model. However, the presented scheme is deployed in the synchronous communication environment and could not help a fraction of link-failed participants to recover their lost secrets. Most unfortunately, the scheme does not support participants to verify the correctness of computation in a server (the server is the edge node in our paper). As for data privacy, Bonawitz et al.’s scheme allows the server to obtain the unmasking aggregated result, while our work does not do that. In SecEL, the masking aggregated result is returned by the edge node and unmasked by participating AVs. Other works on privacy-preserving distributed learning, such as [13, 14], utilize public key-based cryptographic systems to protect data privacy, which takes more computation than the encryption method of one-time padding.

Our [17] [13] [14]
Encryption method one-time padding one-time padding asymmetric encryption asymmetric encryption
Data privacy
Verifiable computation
Lost secret recovery
  • means having the corresponding security property while means not having.

TABLE IV: Comparison of our work and the existing work

8 Conclusion

In this paper, we present a privacy-preserving, verifiable and fault-tolerant scheme, named SecEL for edge learning in AVNET. Specifically, SecEL combines the primitive bivariate polynomial-based secret sharing with homomorphic authenticator. Participating AVs’ sharing parameters are protected by the way of one-time padding and labeled with respective MACs, thus data privacy and verifiable computation are ensured. In addition, SecEL allows an honest but failed participating AV’s secret can be rebuilt by other active participants, which is adapted to the asynchronous AVNET environment. Finally, the evaluation of SecEL demonstrates the acceptable performance results.

References

  • [1] P. Mach and Z. Becvar, “Mobile edge computing: A survey on architecture and computation offloading,” IEEE Communications Surveys & Tutorials, vol. 19, no. 3, pp. 1628–1656, 2017.
  • [2] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.
  • [3] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “When edge meets learning: Adaptive control for resource-constrained distributed machine learning,” in IEEE INFOCOM 2018-IEEE Conference on Computer Communications.   IEEE, 2018, pp. 63–71.
  • [4] N. Kumar, S. Zeadally, and J. J. Rodrigues, “Vehicular delay-tolerant networks for smart grid data management using mobile edge computing,” IEEE Communications Magazine, vol. 54, no. 10, pp. 60–66, 2016.
  • [5] K. Zhang, Y. Mao, S. Leng, Y. He, and Y. Zhang, “Mobile-edge computing for vehicular networks: A promising network paradigm with predictive off-loading,” IEEE Vehicular Technology Magazine, vol. 12, no. 2, pp. 36–44, 2017.
  • [6] G. Zhu, D. Liu, Y. Du, C. You, J. Zhang, and K. Huang, “Towards an intelligent edge: Wireless communication meets machine learning,” arXiv preprint arXiv:1809.00343, 2018.
  • [7] H. B. McMahan, E. Moore, D. Ramage, S. Hampson et al., “Communication-efficient learning of deep networks from decentralized data,” arXiv preprint arXiv:1602.05629, 2016.
  • [8] S. Samarakoon, M. Bennis, W. Saad, and M. Debbah, “Federated learning for ultra-reliable low-latency v2v communications,” in 2018 IEEE Global Communications Conference (GLOBECOM).   IEEE, 2018, pp. 1–7.
  • [9] M. Kamp, L. Adilova, J. Sicking, F. Hüger, P. Schlicht, T. Wirtz, and S. Wrobel, “Efficient decentralized deep learning by dynamic model averaging,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2018, pp. 393–409.
  • [10] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Inference attacks against collaborative learning,” arXiv preprint arXiv:1805.04049, 2018.
  • [11] B. Hitaj, G. Ateniese, and F. Pérez-Cruz, “Deep models under the gan: information leakage from collaborative deep learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.   ACM, 2017, pp. 603–618.
  • [12] L. Marelli and G. Testa, “Scrutinizing the eu general data protection regulation,” Science, vol. 360, no. 6388, pp. 496–498, 2018.
  • [13] Y. Aono, T. Hayashi, L. Wang, S. Moriai et al., “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 5, pp. 1333–1345, 2018.
  • [14] J. Weng, J. Weng, J. Zhang, M. Li, Y. Zhang, and W. Luo, “Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive,” Cryptology ePrint Archive, Report 2018/679, 2018.
  • [15]

    Y. Liu, S. Ma, Y. Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang, “Trojaning attack on neural networks,” 2017.

  • [16] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” arXiv preprint arXiv:1807.00459, 2018.
  • [17] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.   ACM, 2017, pp. 1175–1191.
  • [18] D. Laselva, D. Lopez-Perez, M. Rinne, and T. Henttonen, “3gpp lte-wlan aggregation technologies: functionalities and performance comparison,” IEEE Communications Magazine, vol. 56, no. 3, pp. 195–203, 2018.
  • [19] P. Shantharama, A. S. Thyagaturu, N. Karakoc, L. Ferrari, M. Reisslein, and A. Scaglione, “Layback: Sdn management of multi-access edge computing (mec) for network access services and radio resource sharing,” IEEE Access, vol. 6, pp. 57 545–57 561, 2018.
  • [20] H. Peng, Q. Ye, and X. Shen, “Sdn-based resource management for autonomous vehicular networks: A multi-access edge computing approach,” arXiv preprint arXiv:1809.08966, 2018.
  • [21] H. Su and H. Chen, “Experiments on parallel training of deep neural network using model averaging,” arXiv preprint arXiv:1507.01239, 2015.
  • [22] W. Zhang, S. Gupta, X. Lian, and J. Liu, “Staleness-aware async-sgd for distributed deep learning,” arXiv preprint arXiv:1511.05950, 2015.
  • [23] L. Harn, C.-F. Hsu, Z. Xia, and J. Zhou, “How to share secret efficiently over networks,” Security and Communication Networks, vol. 2017, 2017.
  • [24] N. H. Tran, H. Pang, and R. H. Deng, “Efficient verifiable computation of linear and quadratic functions over encrypted data,” in Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security.   ACM, 2016, pp. 605–616.
  • [25] R. Gennaro and D. Wichs, “Fully homomorphic message authenticators,” in International Conference on the Theory and Application of Cryptology and Information Security.   Springer, 2013, pp. 301–320.
  • [26] D. Engels, M.-J. O. Saarinen, P. Schweitzer, and E. M. Smith, “The hummingbird-2 lightweight authenticated encryption algorithm,” in International Workshop on Radio Frequency Identification: Security and Privacy Issues.   Springer, 2011, pp. 19–31.
  • [27] S. K. D. Maram, F. Zhang, L. Wang, A. Low, Y. Zhang, A. Juels, and D. Song, “Dynamic-committee proactive secret sharing,” 2018.
  • [28] E. Syta, P. Jovanovic, E. K. Kogias, N. Gailly, L. Gasser, I. Khoffi, M. J. Fischer, and B. Ford, “Scalable bias-resistant distributed randomness,” in 2017 IEEE Symposium on Security and Privacy (SP).   Ieee, 2017, pp. 444–460.
  • [29] R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security.   ACM, 2015, pp. 1310–1321.