DeepAI AI Chat
Log In Sign Up

Computing the Feedback Capacity of Finite State Channels using Reinforcement Learning

by   Ziv Aharoni, et al.
Ben-Gurion University of the Negev

In this paper, we propose a novel method to compute the feedback capacity of channels with memory using reinforcement learning (RL). In RL, one seeks to maximize cumulative rewards collected in a sequential decision-making environment. This is done by collecting samples of the underlying environment and using them to learn the optimal decision rule. The main advantage of this approach is its computational efficiency, even in high dimensional problems. Hence, RL can be used to estimate numerically the feedback capacity of unifilar finite state channels (FSCs) with large alphabet size. The outcome of the RL algorithm sheds light on the properties of the optimal decision rule, which in our case, is the optimal input distribution of the channel. These insights can be converted into analytic, single-letter capacity expressions by solving corresponding lower and upper bounds. We demonstrate the efficiency of this method by analytically solving the feedback capacity of the well-known Ising channel with a ternary alphabet. We also provide a simple coding scheme that achieves the feedback capacity.


page 1

page 2

page 3

page 4


Reinforcement Learning Evaluation and Solution for the Feedback Capacity of the Ising Channel with Large Alphabet

We propose a new method to compute the feedback capacity of unifilar fin...

Capacity of Finite-State Channels with Delayed Feedback

In this paper, we investigate the capacity of finite-state channels (FSC...

Youla Coding and Computation of Gaussian Feedback Capacity

In this paper, we propose an approach to numerically compute the feedbac...

Finite-State Channels with Feedback and State Known at the Encoder

We consider finite state channels (FSCs) with feedback and state informa...

How to apply the rubber method for channels with feedback

We give an overview of applications of the rubber method. The rubber met...

Capacity of Continuous Channels with Memory via Directed Information Neural Estimator

Calculating the capacity (with or without feedback) of channels with mem...

QHD: A brain-inspired hyperdimensional reinforcement learning algorithm

Reinforcement Learning (RL) has opened up new opportunities to solve a w...