Joint Sensing, Communication, and Computation Resource Allocation for Cooperative Perception in Fog-Based Vehicular Networks

12/12/2021
by   Xinran Zhang, et al.
0

To enlarge the perception range and reliability of individual autonomous vehicles, cooperative perception has been received much attention. However, considering the high volume of shared messages, limited bandwidth and computation resources in vehicular networks become bottlenecks. In this paper, we investigate how to balance the volume of shared messages and constrained resources in fog-based vehicular networks. To this end, we first characterize sum satisfaction of cooperative perception taking account of its spatial-temporal value and latency performance. Next, the sensing block message, communication resource block, and computation resource are jointly allocated to maximize the sum satisfaction of cooperative perception, while satisfying the maximum latency and sojourn time constraints of vehicles. Owing to its non-convexity, we decouple the original problem into two separate sub-problems and devise corresponding solutions. Simulation results demonstrate that our proposed scheme can effectively boost the sum satisfaction of cooperative perception compared with existing baselines.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

page 5

page 6

12/07/2020

Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning

Cooperative perception plays a vital role in extending a vehicle's sensi...
06/23/2019

Computation Offloading and Resource Allocation for Backhaul Limited Cooperative MEC Systems

In this paper, we jointly optimize computation offloading and resource a...
08/29/2019

Generation of Cooperative Perception Messages for Connected and Automated Vehicles

Cooperative or collective perception (or sensing) enables connected and ...
11/16/2020

Feature Sharing and Integration for Cooperative Cognition and Perception with Volumetric Sensors

The recent advancement in computational and communication systems has le...
05/10/2020

Fogbanks: Future Dynamic Vehicular Fog Banks for Processing, Sensing and Storage in 6G

Fixed edge processing has become a key feature of 5G networks, while pla...
10/09/2020

Hybrid Vehicular and Cloud Distributed Computing: A Case for Cooperative Perception

In this work, we propose the use of hybrid offloading of computing tasks...
01/14/2021

Augmented Informative Cooperative Perception

Connected vehicles, whether equipped with advanced driver-assistance sys...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

With the rapid advancements of embedded systems, sensors, and artificial intelligence in recent decades, autonomous driving has evolved from an impossible dream into a foreseeable reality, and will eventually reshape future transport system and driving experience

[10]. As the key enabling technology, cooperative perception[9] exploits vehicle-to-everything (V2X) communications to exchange sensory messages among neighboring vehicles and road side units, and offers an eagle view of broader road environment. In[8], a remarkable perception performance gain over traditional separate perception is validated through theoretical analysis, regardless of the additional communication and computation overhead. Nevertheless, when the number of involved vehicles is , the required communication capacity reaches Mbps[6, 2], which overwhelms limited communication bandwidth and computation power in vehicular networks (VNETs) and restricts the implementation of cooperative perception.

To balance the exchanging message volume in cooperative perception and the limited resources in VNETs, some efforts have been made for the joint allocation of sensing, communication, and computation resource. Higuchi et al.[4] attempt to reduce the redundancy of cooperative perception messages by selecting more valuable ones to be delivered. Specifically, the value of messages is firstly quantified based on message history and current mobility, and then more valuable messages are transmitted via V2X communications. In[1], a quadtree-based compression mechanism is utilized to partition sensory information into different block messages. Then, the vehicular association, resource allocation, and block message selection are jointly optimized with deep reinforcement learning (DRL) to maximize the satisfaction of cooperative perception. Du et al.[3] go a step further by proposing a platoon-based cooperative perception framework, in which the perception scheduling, computation strategies, and communication resource allocation are jointly optimized to minimize the delay of sensing tasks.

The aforementioned works[4, 1, 3] commonly concentrate on the direct sharing among vehicles, leaving the more general problem of VNET-assisted cooperative perception less investigated. By taking advantages of hierarchical computation resources and flexible resource management, fog-based vehicular networks (FVNETs)[11] provide a promising edge-cloud collaboration framework. In this paper, we examine how to realize cooperative perception with constrained resources in FVNETs, where multiple vehicles offload individual sensory messages to fog access points (F-APs) or the cloud server for cooperative computation. The main contribution are summarized three-folds.

  • Taking spatial-temporal values of sensory messages and latency performance into consideration, we propose a joint sensing block message, communication and computation resource allocation problem to maximize the sum satisfaction of cooperative perception, while satisfying the maximum tolerant latency and sojourn time constraints.

  • The non-convex sum satisfaction maximization problem is decoupled into a joint block message and offloading mode selection problem plus a joint communication and computation resource allocation problem. By solving two problems based on multi-agent DRL and swap matching, the sum satisfaction can be maximized.

  • Through simulation results, we show that our proposed scheme can significantly improve the sum satisfaction of cooperative perception in FVNETs.

Ii System Model and Problem Formulation

Fig. 1: Cooperative Perception in FVNETs.

Considering a cooperative perception scenario in FVNETs, which consists of vehicular user equipments (VUEs), F-APs, and remote radio heads (RRHs) connecting to a centralized cloud server. As shown in Fig.1, each VUE adopts region quadtree technique [1] to compress its sensory data into independent block messages . Here, and denote the block sets of VUE

’s sensory region and the whole region, respectively. With efficient machine learning methods, the cloud server can implicitly estimate VUE

’s trajectory and sojourn time with F-AP for time , where are coordinates.

For cooperative perception, each VUE requests a map constructing task which covers its front region with a length of . To this end, the cloud server chooses different VUEs to constitute independent cooperative clusters, and selects different block messages from cooperative VUEs for uploading. In this article, we assume that the VUEs associating with the same F-AP or the cloud server form a cooperative cluster, and partial messages could be migrated among F-APs and the cloud server via fiber backhaul links.

Once receiving these uploading messages, the cloud server and F-APs carry out customized maps for individual VUEs by sequentially realigning, integrating, inferring, and projecting the collected uploading messages. We define a tuple to refer to VUE ’s task, whose elements denote the data size of a block message, the number of CPU-cycle frequencies required to process 1 bit input, and ’s maximum tolerant latency, respectively.

Ii-a Spatial-Temporal Model in Cooperative Perception

To identify the values of block messages in cooperative perception, the spatial-temporal model is detailed as follows.

From the perspective of timeliness, the value of VUE ’s block message decreases from time of generation at until it hits deadline at . A linear descend function is used to define the temporal value of VUE ’s block message at time as

(1)

where is the message’s initial value. However, considering that the sensing time and the sensing period are unknown at the cloud server, we model the temporal value by Markov process to characterize its randomness. For VUE , the set of potential temporal value is defined as .

is the probability that the temporal value of VUE

’s block message at time is if its value in time is .

As for spatial dimension, each VUE has a rectangular region of interest where the closer sensory blocks are more expected by the VUEs. With VUE ’s and block ’s positions , , the euclidean distance between VUE and block and the angle between VUE ’s moving direction and block can be easily calculated, as illustrated in Fig.1. Thus, we define the spatial value of block for VUE as

(2)

Finally, the spatial-temporal value of VUE ’s block message for VUE can be calculated by .

Ii-B Communication and Computation Models

As illustrated above, includes block message uploading, joint processing, and output downloading process. Like[3], we ignore the download latency. Denote and as the uploading and computation latencies, we have ’s sum latency . Moreover, each task could be handled by following two possible modes: the cloud mode and the F-AP mode. Let denote the offloading mode selection of VUE : if , VUE choose the cloud mode; while , VUE offloads the task to F-AP . Next, we describe the communication and computation models considered in both modes in sequence.

Ii-B1 Communication model

In FVNETs, the total bandwidth is divided into resource blocks (RBs) with bandwidth each and full frequency reuse is considered for all F-APs and RRHs. Define as the RB allocation indicator. if RB is allocated to VUE and , otherwise. Without loss of generality, each VUE is allocated a single RB for uploading. For VUE associating with F-AP , its uploading rate can be expressed as

(3)

where , , and are the transmit power, noise power, and channel gain from VUE to F-AP on RB at time , respectively.

When VUE selects the cloud mode, it associates with its close RRHs, whose set is described as . In the meantime, optimal linear detection, i.e. minimum mean square error (MMSE) detection, is performed at the cloud server to mitigate inter-RRH interference, thus the uplink rate of VUE in the cloud mode at time is

(4)

where is the interference and

is the MMSE detection vector.

is the channel gain from VUE to its associated RRHs on RB at time , while is the channel from VUE to the associated RRHs of VUE .

Note that the offloaded task starts computing only when the whole VUEs finish uploading. Therefore, the communication latency of task relies on the maximum one among cooperative VUEs, i.e.,

(5)

where is fronthaul delay. denotes the block message selection of VUE : if , then block message of VUE is selected for uploading.

Ii-B2 Computation model

For cooperative perception, the offloaded task has to process all uploaded block messages within its region of interest. Assuming that the computation capabilities of the cloud server and each F-AP are characterized by the maximum CPU-cycle frequency and . With CPU-cycle frequency allocated to VUE , the computation latency of task can be obtained by

(6)

Ii-C Problem Formulation

In FVNETs, the cloud server and F-APs are interested in finishing VUEs’ offloaded tasks with more valuable block messages as quickly as possible, therefore we define the satisfaction of VUE at time as

(7)

where and are weight parameters. The first part shows the overall spatial-temporal values of block messages from cooperative VUEs, and the second part denotes the effect of task latency.

The key of this work is to maximize the long-term sum satisfaction for cooperative perception in FVNETs by optimizing the offloading mode selection scheme , the sensory block message selection scheme , the uplink RB allocation scheme , and the frequency resource allocation scheme . Mathematically, it can be formulated as

(8)

where constraints (8a) shows the overall latency of individual VUE should be less than the maximum tolerant latency of and its sojourn time , constraint (8b) regulates that the uploading block messages of different VUEs are non-overlapped, constraints (8c,d) means each VUE can be allocated to a single mode and RB, and constraint (8e) denotes that the allocated CPU-cycle frequencies are not allowed to exceed the frequency budgets .

Iii Joint Sensing, Communication, and Computation Resource Allocation

Note that the formulated problem (8) is a mixed-integer nonlinear programming problem which is in general intractable. In this section, we decouple the original problem into a joint mode selection and block selection sub-problem plus a joint communication and computation resource allocation sub-problem. Afterwards, multi-agent DRL and swap matching-based algorithms are developed.

Iii-a Optimization of Offloading Mode and Block Selection

Under the circumstance that communication and computation resource allocation scheme has been determined, the original problem can be reformulated as

(9)

Since the block message’s value is dynamic and unknown, we cannot solve problem (9) using traditional optimization method. Inspired by[12], we resort to multi-agent DRL to resolve this uncertainty.

Iii-A1 Multi-Agent Markov Game

To be concrete, we treat every VUE as an intelligent agent and convert problem (9) into a Markov game with agents. At time , each agent observes local state and selects its own action . Given the joint actions taken by agents, the FVNET feeds back new state and immediate reward . The formal definitions of three elements are given in the following.

State: Each agent ’s local state includes the following parts: , the maximum tolerant latency of VUE ; , the coordinate of its own; , the temporal value of VUE ’s sensory messages at time ; , the number of serving VUEs of the cloud server and F-APs at time ; and , the latency satisfied ratio.

Action: Consistent with (9), each agent selects the offloading mode and uploading messages .

Reward: We consider this game as a fully cooperative one and define the immediate reward function of each VUE as the sum satisfaction, i.e., .

Iii-A2 Attention Multi-Agent Deep Deterministic Policy Gradient (DDPG)-based Algorithm

Due to non-stationary local states and large action spaces, canonical DRL algorithms, like deep Q-network, are always ineffective. Inspired by[5, 7], multi-agent DDPG algorithm is tuned with an attention mechanism in this article to tackle this problem, namely attention multi-agent DDPG.

In attention multi-agent DDPG, each agent has its own actor network and critic network , acting as the policy and the policy evaluator, respectively. Note that instead of the locals, the global states and actions are considered at the critic network to address the non-stationary and cooperative issues. Furthermore, two target networks ( and ) and experience replay are applied to stabilize training and to remove data correlation. We denote as the replay buffer with capacity and express the stored experience of all agents as the tuple . Here, we omit the time index and denote mark as time for simplicity.

Specifically, the actor network is responsible for finding a deterministic policy to maximize the cumulative discounted reward , where is the discounted factor. Then, can be obtained by

(10)

where is the stochastic noise to encourage exploration. For its updating, the parameter is directly adjusted in the direction of , which is given by

(11)

Here, is the action-value function that is established by the critic network . It takes as the input of the states and actions of all agents and outputs the Q-value for agent . With , the weights of agent ’s critic network

can be updated by minimizing the MSE-based loss function

, i.e.,

(12)

where . Here, and are agent ’s target critic and actor networks with weights and , respectively.

Furthermore, we adjust the critic network with an attention mechanism which facilitates fine-grained and discriminatory treatment of different VUEs. To this end, we firstly customize a multi-layer perception (MLP) network to reduce the input dimension and extract the higher features . Then, agent ’s attention value for the other agents can be calculated by . Here, and denote agent ’s attention weight and value for agent , which is given by

(13)
(14)

Wherein, the parameter matrix and constitute a bilinear mapping for and . In addition,

is a ReLu function and

is a transform matrix. With the derived high feature and the attention value , we could establish a new MLP network to replace the original critic network .

Finally, we conclude the attention multi-agent DDPG procedures to solving problem (9) in Algorithm 1.

Iii-B Optimization of RB and CPU-frequency Allocation

Once the offloading modes and selected blocks have been determined, the original problem is reduced into

(15)

Hereafter, the RBs and frequencies are successively allocated.

1:  Initialize actor, critic, target actor, target critic networks , and the replay memory with capacity .
2:  for

 epoch

 do
3:     Initialize the state .
4:     for step  do
5:        Select action with (10), .
6:        Obtain current reward and next state .
7:        Store tuple into .
8:        for agent  do
9:           Randomly sample a mini-batch of tuples from .
10:           Update the critic network by minimizing (12).
11:           Update the actor network with (11).
12:           Update the target critic and actor networks , .
13:        end for
14:     end for
15:  end for
Algorithm 1 Attention multi-agent DDPG-based algorithm

Iii-B1 Swap Matching-based RB Allocation

With fixed CPU-frequencies, we firstly model the RB allocation problem as a one-to-one matching game and solve it in a distributed and low-complexity manner.

Formally, we define as the matching function for the associated VUE set and the bandwidth set , which has the following properties

(16)

where condition 1) implies that if RB matches with VUE , VUE also matches with RB ; condition 2) gives that each RB can be matched with one VUE, and each VUE can only be matched with one RB in turn.

The utility of VUE is defined as the uploading rate on RB , i.e.,

(17)

which indicates that every VUE selfishly favours the RBs with higher offloading rate. As for RB , it aims to minimize the overall communication latency which equals to the maximum one, thus we define the utility of RB as

(18)

Based on above analysis, a swap matching-based RB allocation algorithm is developed in Algorithm 2. To start the matching process, each VUE and RB establish their own preference lists and in the descending order. A deferred acceptance algorithm is then adopted for initial matching. To overcome the dynamics introduced by inter-cell interference, two matched VUEs could exchange their matched RBs when the maximum communication latency could be reduced.

1:  Initialize the preference lists and the set of unmatched VUEs .
2:  while  and  do
3:     for  do
4:        VUE proposes to its most preferred RB in that has not rejected it before.
5:     end for
6:     for  do
7:        if , RB holds the matching and rejects all proposals; otherwise, RB accepts its favorite VUE and rejects the others.
8:     end for
9:  end while
10:  while there exists a swap-pair which can reduce the maximum latency do
11:     Update .
12:  end while
Algorithm 2 Swap matching-based RB allocation algorithm

Iii-B2 CPU-Cycle Frequency Allocation

Given , the problem (15) is apparently a convex optimization problem with respect to and is easy to be solved by the interior-point method.

Iv Simulation Results and Analysis

In this section, simulation results are provided to verify the sum satisfaction of cooperative perception in FVNETs. We deploy RRHs, F-AP, and VUEs on a -meter-long road. The whole road is equally divided into blocks and the size for each block message is -kbits with -level region quadtree. For cooperative perception, each VUE requests a task to expand its sensory range by m with and ms. In addition, The total bandwidth is MHz, while the cloud server and the F-AP has a maximum computation resource of GHz and GHz, respectively.

Fig. 2: The convergence of algorithm 1.

Fig. 2 shows the convergence performance of the proposed attention multi-agent DDPG-based algorithm, when the number of VUEs is . It is seen that our proposed algorithm could effectively converge to stable sum satisfaction.

Fig. 3: Sum satisfaction versus desired expanding distance.

In Fig. 3, the sum satisfaction performances are provided with different desired expanding distances. For comparison, a distance-based mode selection and full message uploading scheme is adopted as the baseline. It can be observed that our proposed attention multi-agent DDPG algorithm outperforms the baseline, because it could select more valuable block messages and avoid undesired message uploading, meanwhile balancing the load among F-APs and the cloud server. Especially when the network load is large (), our proposed algorithm can effectively balance task load and constrained resource by controlling the selection of block messages.

Fig. 4: The convergence of Algorithm 2.

Fig. 4 verifies the convergence of algorithm 2 with different numbers of VUEs, where the optimal latency which is derived by exhaustive search is adopted for comparison. It can be seen that, our proposed swap matching algorithm can converge to a stationary point within iterations, while its latency performance is close to exhaustive search algorithm.

Fig. 5 evaluates the average latency performances with different communication RB allocation schemes, when the numbers of VUEs are set as . For comparison, the exhaustive search and the matching-based sum rate maximization algorithms are selected as two baselines. Wherein, the former one offers the optimal average latency. It can be observed that our proposed algorithm outperforms the max sum rate algorithm and their latency gap becomes larger when the involved number of VUEs increases. That is because, on the one hand, the uploading latency of cooperative perception in (5) depends on the maximum one among all cooperative VUEs rather than overall latencies of all VUEs; on the other hand, the inter-cell interference has a larger effect on the VUE with lower channel gain. In addition, it is also observed that our proposed algorithm achieves considerable latency performance compared with exhaustive search algorithm.

V Conclusion

In this article, we focus on a vehicular cooperative perception scenario with ultra-low latency requirement, and propose a joint sensing, communication, and computation resource allocation scheme with multi-agent DRL and swap matching for FVENTs, in which multiple VUEs constitute cooperative clusters to offload their computation tasks to either the cloud server or F-APs. Simulation results have verified the effectiveness and superiority of our proposed algorithms on the sum satisfaction and latency performances. In the future, it is interesting to incorporate radio sensing and investigate corresponding resource allocation for cooperative perception.

Fig. 5: Average latency versus the number of VUEs.

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China under No. 61921003 and 62001053, and the Fundamental Research Funds for the Central Universities under No. 24820202020RC11 and 2020RC03.

References

  • [1] M. K. Abdel-Aziz, C. Perfecto, S. Samarakoon, M. Bennis, and W. Saad (2020-12) Vehicular cooperative perception through action branching and federated reinforcement learning. ArXiv201203414 Cs. Cited by: §I, §I, §II.
  • [2] M. Boban, A. Kousaridas, K. Manolakis, J. Eichinger, and W. Xu (2018-09) Connected roads of the future: use cases, requirements, and design considerations for vehicle-to-everything communications. IEEE Veh. Technol. Mag. 13 (3), pp. 110–123. Cited by: §I.
  • [3] H. Du, S. Leng, K. Zhang, and L. Zhou (2020-12) Cooperative sensing and task offloading for autonomous platoons. In Proc. IEEE GLOBECOM, Taipei, Taiwan, pp. 1–6. Cited by: §I, §I, §II-B.
  • [4] T. Higuchi, M. Giordani, A. Zanella, M. Zorzi, and O. Altintas (2019-06) Value-anticipating V2V communications for cooperative perception. In Proc. IEEE IV, Paris, France, pp. 1947–1952. Cited by: §I, §I.
  • [5] S. Iqbal and F. Sha (2019-05) Actor-attention-critic for multi-agent reinforcement learning. In Proc. ICML, California, USA, pp. 2961–2970. Cited by: §III-A2.
  • [6] E. E. Marvasti, A. Raftari, A. E. Marvasti, Y. P. Fallah, R. Guo, and H. Lu (2020-02) Cooperative LIDAR object detection via feature sharing in deep networks. ArXiv200208440 Cs. Cited by: §I.
  • [7] Y. Sun, M. Peng, Y. Zhou, Y. Huang, and S. Mao (2019) Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues. IEEE Commun. Surv. Tutor. 21 (4), pp. 3072–3108. Cited by: §III-A2.
  • [8] Y. Wang, G. de Veciana, T. Shimizu, and H. Lu (2018-06) Deployment and performance of infrastructure to assist vehicular collaborative sensing. In Proc. IEEE VTC Spring, Porto, Portugal, pp. 1–5. Cited by: §I.
  • [9] Q. Yang, S. Fu, H. Wang, and H. Fang (2021-05) Machine-learning-enabled cooperative perception for connected autonomous vehicles: challenges and opportunities. IEEE Netw. 35 (3), pp. 96–101. Cited by: §I.
  • [10] J. Zhang and K. B. Letaief (2020-02) Mobile edge intelligence and computing for the internet of vehicles. Proc. IEEE 108 (2), pp. 246–261. Cited by: §I.
  • [11] K. Zhang, M. Peng, and Y. Sun (2021-02) Delay-optimized resource allocation in fog-based vehicular networks. IEEE Internet Things J. 8 (3), pp. 1347–1357. Cited by: §I.
  • [12] X. Zhang, M. Peng, S. Yan, and Y. Sun (2020-07) Deep-reinforcement-learning-based mode selection and resource allocation for cellular V2X communications. IEEE Internet Things J. 7 (7), pp. 6380–6391. Cited by: §III-A.