Motivated by recent advances in tracking and control over networks [SilvaDerpichOstergaard:ECDQ4Control, SahaiMitterPartI, DerpichOstergaard:ECDQ4CausalRDF, TatikondaSahaiMitter, StreamingWithFB:CNS, StavrouOstergaardCharalambousDerpich:ITW2017, BansalBasar:JSCC4Control, LQGoverAWGN:linear:NoBraslavsky, NairEvans:Quantization, MatveevSavkin:LQR-channel:SICON2007, FixedRateLQG:CDC2017, Yuksel:AC2014:LQG:separation, Yuksel:FixedRateControl, YukselBasarBook, KostinaHassibi:RDF4Control:AC, MineroFranceschettiDeyNair, JSCC4Control:AC2019], we consider the setting where a decoder observes the system state corrupted by noise via an internal sensor, while it also receives quantized descriptions of the observations of the state from an external sensor over a rate-limited link.
We focus in this paper on the tracking (estimation) problem of a Gauss–Markov source over a rate-limited channel, i.e., causal encoding and decoding of the source; we view the internal noisy measurements of the state as side information that is available to the decoder but not to the encoder.
The idea of causal rate–distortion function (CRDF) was introduced in [GorbunovPinsker:CausalRDF:Gauss], where [TatikondaPhD, TatikondaSahaiMitter, Charalambous:DirectedInfo-GeneralCost:CDC2011] (see also [KostinaHassibi:RDF4Control:AC, Tanaka:DirectedInfoLQG]) drew the connection between the CRDF and tracking of a Gauss–Markov source over rate-limited links with causal encoding and decoding. Recently, two notable efforts have been made in determining bounds on the performance of these settings in the presence of decoder SI [KostinaHassibi:SideInformationTracking, Starvou:GaussRDF_SI], which provide a comprehensive set of definitions and bounds for this problem, by relying on the seminal work of Wyner and Ziv [Wyner78, WynerZiv76] for rate–distortion with non-causal SI at the decoder. However, since the technique of Wyner and Ziv relies on non-causal knowledge of the SI at the decoder, applying it for scenarios with causal SI imposes an additional slack when used to bound from below the operational CRDF with (causal) SI, on top of the existing gap between the information and operational CRDFs without SI that stems from the causal encoding restriction [LinderZamirCausal, NeuhoffGilbert:CausalSourceCodes:IT1982].
Our goal in this paper is twofold: first, providing short proofs of the lower bounds in [Starvou:GaussRDF_SI] via a simple observation; secondly, deriving a tighter lower bound on the performance of causal source coding with decoder SI that is strictly higher than the bounds in [KostinaHassibi:SideInformationTracking, Starvou:GaussRDF_SI]. To derive the latter, we build on the work of Weissman and El Gamal [WeissmanElGamal_FiniteLookAhead] for rate–-distortion with causal SI and extend their results for CRDFs.
As a by product, we settle a conjecture in the negative by Stavrou and Skoglund [Starvou:GaussRDF_SI] regarding the optimality of Wyner–Ziv-type CRDF bounds for causal tracking over additive white Gaussian noise (AWGN) channels, by proving that an adaptation of our new lower bound is strictly higher for this setting. The rest of the paper is organized as follows. In Sec. II, we formulate the problem of tracking a Gauss–Markov source over a rate-limited link for several different SI scenarios. We review classical results and tools that are used throughout this work in Sec. LABEL:s:Background. We review the CRDF scenario without SI in Sec. LABEL:s:_No_SI_background, and with two-sided SI Sec. LABEL:s:_Two-Sided_SI_background. We provide simple proofs for the existing results along with new tighter bounds on the CRDF with decoder SI in Sec. LABEL:s:_Causal_RDF. We evaluate the expression of the new bound for a Gaussian test channel in Sec. LABEL:s:_Numerical_Evaluation_-_Gaussian_Test_Channel. Finally, we conclude the paper with Sec. LABEL:s:_Conclusions by discussing future research directions.
Ii Problem Statement
In this section, we formalize the tracking setting treated in this work, depicted in Fig. 1.
The source is generated by a first order Gauss–Markov model with zero initial condition ():111The assumption can be easily replaced with a Gaussian that is independent of the system-disturbance sequence .
where is the source sample at time ; is the system disturbance at time
, whose temporal entries are independent and identically distributed (i.i.d.) zero-mean Gaussian of variance
; the eigenvalueis fixed and known.
Encoder. Observes the state at time and generates a packet of rate .
Channel. At time , a packet is sent over a noiseless channel with rate . The packets are subject to an average-rate constraint:222This is a more lenient constraint than the fixed-rate constraint. Consequently, our lower bounds are valid for both scenarios, although they might be too optimistic for the latter.
Side information. The SI is a noisy version of the current source sample , and is given by
where is zero-mean Gaussian of variance , independent of ,333We denote temporal sequences by . and its temporal entries are i.i.d.
Decoder. At time , receives the packet and constructs an estimate of .
Distortion. The average quadratic distortion at time is defined as
and the average-stage distortion is defined as
Definition 1 (Operational causal rate–distortion function).
The operational causal rate–distortion function (CRDF) is defined as the infimum of all achievable average rates , , subject to an average distortion constraint .