Gauss-Markov Source Tracking with Side Information: Lower Bounds

04/17/2020 ∙ by Omri Lev, et al. ∙ 0

We consider the problem of causal source coding and causal decoding of a Gauss-Markov source, where the decoder has causal access to a side-information signal. We define the information causal rate-distortion function with causal decoder side information and prove that it bounds from below its operational counterpart.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Motivated by recent advances in tracking and control over networks [SilvaDerpichOstergaard:ECDQ4Control, SahaiMitterPartI, DerpichOstergaard:ECDQ4CausalRDF, TatikondaSahaiMitter, StreamingWithFB:CNS, StavrouOstergaardCharalambousDerpich:ITW2017, BansalBasar:JSCC4Control, LQGoverAWGN:linear:NoBraslavsky, NairEvans:Quantization, MatveevSavkin:LQR-channel:SICON2007, FixedRateLQG:CDC2017, Yuksel:AC2014:LQG:separation, Yuksel:FixedRateControl, YukselBasarBook, KostinaHassibi:RDF4Control:AC, MineroFranceschettiDeyNair, JSCC4Control:AC2019], we consider the setting where a decoder observes the system state corrupted by noise via an internal sensor, while it also receives quantized descriptions of the observations of the state from an external sensor over a rate-limited link.

We focus in this paper on the tracking (estimation) problem of a Gauss–Markov source over a rate-limited channel, i.e., causal encoding and decoding of the source; we view the internal noisy measurements of the state as side information that is available to the decoder but not to the encoder.

The idea of causal rate–distortion function (CRDF) was introduced in [GorbunovPinsker:CausalRDF:Gauss], where [TatikondaPhD, TatikondaSahaiMitter, Charalambous:DirectedInfo-GeneralCost:CDC2011] (see also [KostinaHassibi:RDF4Control:AC, Tanaka:DirectedInfoLQG]) drew the connection between the CRDF and tracking of a Gauss–Markov source over rate-limited links with causal encoding and decoding. Recently, two notable efforts have been made in determining bounds on the performance of these settings in the presence of decoder SI [KostinaHassibi:SideInformationTracking, Starvou:GaussRDF_SI], which provide a comprehensive set of definitions and bounds for this problem, by relying on the seminal work of Wyner and Ziv [Wyner78, WynerZiv76] for rate–distortion with non-causal SI at the decoder. However, since the technique of Wyner and Ziv relies on non-causal knowledge of the SI at the decoder, applying it for scenarios with causal SI imposes an additional slack when used to bound from below the operational CRDF with (causal) SI, on top of the existing gap between the information and operational CRDFs without SI that stems from the causal encoding restriction [LinderZamirCausal, NeuhoffGilbert:CausalSourceCodes:IT1982].

Our goal in this paper is twofold: first, providing short proofs of the lower bounds in [Starvou:GaussRDF_SI] via a simple observation; secondly, deriving a tighter lower bound on the performance of causal source coding with decoder SI that is strictly higher than the bounds in [KostinaHassibi:SideInformationTracking, Starvou:GaussRDF_SI]. To derive the latter, we build on the work of Weissman and El Gamal [WeissmanElGamal_FiniteLookAhead] for rate–-distortion with causal SI and extend their results for CRDFs.

As a by product, we settle a conjecture in the negative by Stavrou and Skoglund [Starvou:GaussRDF_SI] regarding the optimality of Wyner–Ziv-type CRDF bounds for causal tracking over additive white Gaussian noise (AWGN) channels, by proving that an adaptation of our new lower bound is strictly higher for this setting. The rest of the paper is organized as follows. In Sec. II, we formulate the problem of tracking a Gauss–Markov source over a rate-limited link for several different SI scenarios. We review classical results and tools that are used throughout this work in Sec. LABEL:s:Background. We review the CRDF scenario without SI in Sec. LABEL:s:_No_SI_background, and with two-sided SI Sec. LABEL:s:_Two-Sided_SI_background. We provide simple proofs for the existing results along with new tighter bounds on the CRDF with decoder SI in Sec. LABEL:s:_Causal_RDF. We evaluate the expression of the new bound for a Gaussian test channel in Sec. LABEL:s:_Numerical_Evaluation_-_Gaussian_Test_Channel. Finally, we conclude the paper with Sec. LABEL:s:_Conclusions by discussing future research directions.

Ii Problem Statement

Fig. 1: Scalar tracking system with driving WGN. The channel is a bit pipe with instantaneous rate constraint . The presence of the SI in the encoder/decoder is according to the state of switch A/B, respectively. We assume that the SI is the original source after passing through a Gaussian channel.

In this section, we formalize the tracking setting treated in this work, depicted in Fig. 1.


The source is generated by a first order Gauss–Markov model with zero initial condition (

):111The assumption can be easily replaced with a Gaussian that is independent of the system-disturbance sequence .


where is the source sample at time ; is the system disturbance at time

, whose temporal entries are independent and identically distributed (i.i.d.) zero-mean Gaussian of variance

; the eigenvalue

is fixed and known.

Encoder. Observes the state at time and generates a packet of rate .

Channel. At time , a packet is sent over a noiseless channel with rate . The packets are subject to an average-rate constraint:222This is a more lenient constraint than the fixed-rate constraint. Consequently, our lower bounds are valid for both scenarios, although they might be too optimistic for the latter.


Side information. The SI is a noisy version of the current source sample , and is given by


where is zero-mean Gaussian of variance , independent of ,333We denote temporal sequences by . and its temporal entries are i.i.d.

Decoder. At time , receives the packet and constructs an estimate of .

Distortion. The average quadratic distortion at time is defined as


and the average-stage distortion is defined as

Definition 1 (Operational causal rate–distortion function).

The operational causal rate–distortion function (CRDF) is defined as the infimum of all achievable average rates , , subject to an average distortion constraint .