Cost-Aware Learning and Optimization for Opportunistic Spectrum Access

04/11/2018
by   Chao Gan, et al.
0

In this paper, we investigate cost-aware joint learning and optimization for multi-channel opportunistic spectrum access in a cognitive radio system. We investigate a discrete time model where the time axis is partitioned into frames. Each frame consists of a sensing phase, followed by a transmission phase. During the sensing phase, the user is able to sense a subset of channels sequentially before it decides to use one of them in the following transmission phase. We assume the channel states alternate between busy and idle according to independent Bernoulli random processes from frame to frame. To capture the inherent uncertainty in channel sensing, we assume the reward of each transmission when the channel is idle is a random variable. We also associate random costs with sensing and transmission actions. Our objective is to understand how the costs and reward of the actions would affect the optimal behavior of the user in both offline and online settings, and design the corresponding opportunistic spectrum access strategies to maximize the expected cumulative net reward (i.e., reward-minus-cost). We start with an offline setting where the statistics of the channel status, costs and reward are known beforehand. We show that the the optimal policy exhibits a recursive double threshold structure, and the user needs to compare the channel statistics with those thresholds sequentially in order to decide its actions. With such insights, we then study the online setting, where the statistical information of the channels, costs and reward are unknown a priori. We judiciously balance exploration and exploitation, and show that the cumulative regret scales in O(log T). We also establish a matched lower bound, which implies that our online algorithm is order-optimal. Simulation results corroborate our theoretical analysis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/22/2018

Cost-aware Cascading Bandits

In this paper, we propose a cost-aware cascading bandits model, a new va...
research
10/16/2017

SpecWatch: A Framework for Adversarial Spectrum Monitoring with Unknown Statistics

In cognitive radio networks (CRNs), dynamic spectrum access has been pro...
research
10/24/2021

Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks

We consider the problem of dynamic spectrum access (DSA) in cognitive wi...
research
06/03/2019

Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

We consider an underlay cognitive radio network where the secondary user...
research
11/27/2018

Optimal Learning for Dynamic Coding in Deadline-Constrained Multi-Channel Networks

We study the problem of serving randomly arriving and delay-sensitive tr...
research
02/13/2022

Reinforcement Learning Based Power Control for Reliable Wireless Transmission

In this paper, we investigate a sequential power allocation problem over...
research
05/10/2021

Sense-Bandits: AI-based Adaptation of Sensing Thresholds for Heterogeneous-technology Coexistence Over Unlicensed Bands

In this paper, we present Sense-Bandits, an AI-based framework for distr...

Please sign up or login with your details

Forgot password? Click here to reset