I Introduction
We revisit the successive refinement problem with causal decoder side information shown in Figure 1, which we refer to as the causal successive refinement problem. There are two encoders and two decoders. The decoders aim to recover the source sequence based on the encoded symbols and causally available private side information sequences. Specifically, given the source sequence , encoders and compress into codewords and respectively. At time , decoder aims to recover the th source symbol using the codeword from encoder and side information up to time , i.e., . Similarly, at time , decoder aims to recover the th source symbol as . Finally, at time , for , decoder
outputs source estimate
that, under a distortion measure , is required to be less than or equal to a specified distortion level . Throughout the paper, it is required that since the decoder at each time has both codewords while decoder has access to only.This problem was first considered by Maor and Merhav in [1] who fully characterized the ratedistortion region for the problem. Maor and Merhav showed that, unlike the case with noncausal side information [2], no special structure e.g., degradedness, is required between the side information and . However, Maor and Merhav only presented a weak converse in [1]. In this paper, we strengthen the result in [1] by providing an exponential strong converse theorem, which states that the joint excessdistortion probability approaches one exponentially fast if the ratedistortion tuple falls outside the ratedistortion region derived by Maor and Merhav.
Ia Related Works
We first briefly summarize existing works on the successive refinement problem. The successive refinement problem was first considered by Equitz and Cover [3] and by Koshelev [4] who considered necessary and sufficient conditions for a sourcedistortion triple to be successively refinable. Rimoldi [5] fully characterized the ratedistortion region of the successive refinement problem under the joint excessdistortion probability criterion while Kanlis and Narayan [6]
derived the excessdistortion exponent in the same setting. The secondorder asymptotic analysis of No and Weissman
[7], which provides approximations to finite blocklength performance and implies strong converse theorems, was derived under the marginal excessdistortion probabilities criteria. This analysis was extended to the joint excessdistortion probability criterion by Zhou, Tan and Motani [8]. Other frameworks for successive refinement decoding include [9, 10, 11, 12].The study of source coding with causal decoder side information was initiated by Weissman and El Gamal in [13] where they derived the ratedistortion function for the lossy source coding problem with causal side information at the decoders (see also [14, Chapter 11.2]). Subsequently, Timo and Vellambi [15] characterized the ratedistortion regions of the GuEffros twohop network [16] and the GrayWyner problem [17] with causal decoder side information; Maor and Merhav [18] derived the ratedistortion region for the successive refinement of the HeegardBerger problem [19] with causal side information available at the decoders; Chia and Weissman [20] considered the cascade and triangular source coding problem with causal decoder side information. However, to the best of our knowledge, no strong converse theorems exist for these problems.
As the information spectrum method will be used in this paper to derive an exponential strong converse theorem for the causal successive refinement problem, we briefly summarize the previous applications of this method to network information theory problems. In [21, 22, 23], Oohama used this method to derive exponential strong converses for the lossless source coding problem with onehelper [24, 25] (i.e., the WynerAhlswedeKörner (WAK) problem), the asymmetric broadcast channel problem [26], and the WynerZiv problem [27] respectively. Furthermore, Oohama’s information spectrum method was also used to derive exponential strong converse theorems for content identification with lossy recovery [28] by Zhou, Tan, Yu and Motani [29] and for Wyner’s common information problem under the total variation distance measure [30] by Yu and Tan [31].
IB Main Contribution and Challenges
We revisit the causal successive refinement problem and present an exponential strong converse theorem. For given rates and blocklength, define the joint excessdistortion probability as the probability that either decoder incurs a distortion level greater than the specified distortion level (see (4)) and define the nonexcessdistortion probability as the probability that both decoders satisfy the specified distortion levels (see (24)). Our proof proceeds as follows. First, we derive a nonasymptotic converse (finite blocklength upper) bound on the nonexcessdistortion probability of any code for the causal successive refinement problem using the information spectrum method. Subsequently, by using Cramér’s inequality and the variational formulation of the ratedistortion region, we show that the nonexcessdistortion probability decays exponentially fast to zero as the blocklength tends to infinity if the ratedistortion tuple falls outside the ratedistortion region of the causal successive refinement problem.
As far as we are aware, this paper is the first to establish a strong converse theorem for any lossy source coding problem with causal decoder side information. Furthermore, our methods can be used to derive exponential strong converse theorems for other lossy source coding problems with causal decoder side information discussed in Section IA. In particular, since the lossy source coding problem with causal decoder side information [13] is a special case of the causal successive refinement problem, the exponential strong converse theorem for the problem in [13] follows as a corollary of our result.
In order to establish the strong converse in this paper, we must overcome several major technical challenges. The main difficulty lies in the fact that for the causal successive refinement problem, the side information is available to the decoder causally instead of noncausally. This causal nature of the side information makes the design of the decoder much more complicated and involved, which complicates the analysis of the excessdistortion probability. We find that classical strong converse techniques like the image size characterization [32] and the perturbation approach [33] cannot lead to a strong converse theorem due to the abovementioned difficulty. However, it is possible that other approaches different from ours can be used to obtain a strong converse theorem for the current problem. For example, it is interesting to explore whether two recently proposed strong converse techniques in [34, 35] can be used for this purpose considering the fact that the methods in [34, 35] have been successfully applied to problems including the WynerZiv problem [27] and the WynerAhlswedeKörner (WAK) problem [24, 25].
Ii Problem Formulation and Existing Results
Notation
Random variables and their realizations are in upper (e.g., ) and lower case (e.g., ) respectively. Sets are denoted in calligraphic font (e.g., ). We use to denote the complement of and use
to denote a random vector of length
. We use and to denote the set of positive real numbers and integers respectively. Given a real number , we often use the shorthand . Given two integers and , we use to denote the set of all integers between and and use to denote. The set of all probability distributions on
is denoted asand the set of all conditional probability distributions from
to is denoted as . For informationtheoretic quantities such as entropy and mutual information, we follow the notation in [32]., we follow [36]. In particular, when the joint distribution of
is , we use and interchangeably.Iia Problem Formulation
Let be a joint probability mass function (pmf) on the finite alphabet with its marginals denoted in the customary way, e.g., , . Throughout the paper, we consider memoryless sources , which are generated i.i.d. according to . Let be the alphabet of the reproduced source symbol at decoder where . Recall the encoderdecoder system model for the causal successive refinement problem illustrated in Figure 1.
A formal definition of a code for the causal successive refinement problem is as follows.
Definition 1.
An code for the causal successive refinement problem consists of

two encoding functions
(1) 
and decoding functions: for any
(2) (3)
For , let be two distortion measures. Given the source sequence and a reproduced version , we measure the distortion between them using the additive distortion measure . To evaluate the performance of any code for the causal successive refinement problem, given distortion specified levels , we consider the following joint excessdistortion probability
(4) 
Given , the ratedistortion region for the causal successive refinement problem is defined as follows.
Definition 2.
Given any , a ratedistortion tuple is said to be achievable if there exists a sequence of codes such that
(5)  
(6)  
(7) 
The closure of the set of all achievable ratedistortion tuples is called the ratedistortion region for the causal successive refinement problem and is denoted as .
IiB Existing Results
In this section, we recall the characterization of the ratedistortion region by Maor and Merhav [1, Theorem 1]. For , let be a random variable taking values in finite alphabet . For simplicity, throughout the paper, we let
(9) 
and let be a particular realization of and its alphabet set, respectively.
Define the following set of joint distributions:
(10) 
Given any joint distribution , define the following set of ratedistortion tuples
(11) 
Maor and Merhav [1] defined the following information theoretical sets of ratedistortion tuples
(12) 
Theorem 1.
The ratedistortion region for the causal successive refinement problem satisfies
(13) 
We remark that in [1], Maor and Merhav considered the average distortion criterion
(14) 
instead of the joint excessdistortion probability criterion (see (7)) in Definition 2. However, with slight modification to the proof of [1], it can be verified (see Appendix A) that the ratedistortion region under the joint excessdistortion probability criterion, is identical to the ratedistortion region derived by Maor and Merhav under the average distortion criterion.
Theorem 1 implies that if a ratedistortion tuple falls outside the ratedistortion region, i.e., , then the excessdistortion probability is bounded away from zero. We strengthen the converse proof of Theorem 1 by showing that if , the excessdistortion probability approaches one exponentially fast as the blocklength tends to infinity.
Iii Main Results
Iiia Preliminaries
In this subsection, we present necessary definitions and a key lemma before stating our main result.
Define the following set of distributions
(15) 
Recall that, for any number , we use and interchangeably. Given any , for any , define the following linear combination of log likelihoods
(16) 
Given any and any , define the cumulant generating function of as
(17) 
Furthermore, define the minimal cumulant generating function over distributions in as
(18) 
Finally, given any ratedistortion tuple , define the exponent functions
(19)  
(20) 
With the above definitions, we have the following lemma establishing the properties of the exponent function .
Lemma 2.
The following claims hold.

For any ratedistortion tuple outside the ratedistortion region, i.e., , we have
(21) 
For any ratedistortion tuple inside the ratedistortion region, i.e., , we have
(22)
The proof of Lemma 2 is inspired by [23, Property 4], [29, Lemma 2] and is given in Section V. As will be shown in Theorem 3, the exponent function is a lower bound on the exponent of the probability of nonexcessdistortion probability for the causal successive refinement problem. Thus, Claim (i) in Lemma 2 is crucial to establish the exponential strong converse theorem which states that the excessdistortion probability (see (4)) approaches one exponentially fast with respect to the blocklength of the source sequences.
IiiB Main Result
Define the probability of nonexcessdistortion as
(23)  
(24) 
Theorem 3.
For any code for the causal successive refinement problem such that
(25) 
we have the following nonasymptotic upper bound on the probability of nonexcessdistortion
(26) 
First, our result is nonasymptotic, i.e., the bound in (26) holds for any . In order to prove Theorem 3, we adapt the recently proposed strong converse technique by Oohama [23] to analyze the probability of nonexcessdistortion probability. We first obtain a nonasymptotic upper bound using the information spectrum of loglikelihoods involved in the definition of (see (16)) and then apply Cramér’s bound on large deviations (see e.g., [29, Lemma 13]) to obtain an exponential type nonasymptotic upper bound. Subsequently, we apply the recursive method [23] and proceed similarly as in [29] to obtain the desired result. Our method can also be used to establish similar results for other source coding problems with causal decoder side information [15, 20, 18].
Second, we believe that classical strong converse techniques including the image size characterization [32] and the perturbation approach [33] cannot lead to the strong converse theorem for the causal successive refinement problem. The main obstacle is that the side information is available causally and thus complicates the decoding analysis significantly.
Invoking Lemma 2 and Theorem 3, we conclude that the exponent on the right hand side of (26) is positive if and only if the ratedistortion tuple is outside the ratedistortion region, which implies the following exponential strong converse theorem.
Theorem 4.
For any sequence of codes satisfying the rate constraints in (25), given any distortion levels , we have that if , then the probability of correct decoding decays exponentially fast to zero as the blocklength of the source sequences tends to infinity.
As a result of Theorem 4, we conclude that for every , the rate distortion region (see Definition 2) satisfies that
(27) 
i.e., strong converse holds for the causal successive refinement problem. Using the strong converse theorem and Marton’s changeofmeasure technique [37], similarly to [29, Theorem 5], we can also derive an upper bound on the excessdistortion probability. Furthermore, applying the oneshot techniques in [38], we can also establish a nonasymptotic achievability bound. Applying the BerryEsseen theorem to the achievability bound and analyzing the nonasymptotic converse bound in Theorem 3, similarly to [23], we conclude that the backoff from the ratedistortion region at finite blocklength scales on the order of . However, nailing down the exact secondorder asymptotics [39, 40] is challenging and is left for future work.
Iv Proof of the NonAsymptotic Converse Bound (Theorem 3)
Iva Preliminaries
Given any code with encoding functions and and decoding functions , we define the following induced conditional distributions:
(28)  
(29)  
(30) 
For simplicity, in the following, we let
(31) 
and let be a particular realization and the alphabet of respectively. With above definitions, we have that the distribution satisfies that for any ,
(32) 
In the remaining part of this section, all distributions denoted by are induced by the joint distribution .
Let auxiliary random variables be and for all . Note that as a function of , the sequence
is a Markov chain under
. Throughout the paper, for each , we let(33) 
and let be a particular realization and the alphabet of , respectively. For , let be arbitrary distributions and let , , , , and be induced distributions of . Given any positive real number , define the following subsets of :
(34)  
(35)  
(36)  
(37)  
(38)  
(39)  
(40)  
(41) 
IvB Proof Steps
Lemma 5.
For any code for the causal successive refinement problem satisfying (25), given any distortion levels , we have
(42) 
The proof of Lemma 5 is given in Appendix B and divided into two steps. First, we derive a letter nonasymptotic upper bound which holds for certain arbitrary letter auxiliary distributions. Subsequently, we singleletterize the derived bound by proper choice of auxiliary distributions and careful decomposition of induced distributions of .
For simplicity, in the following, we will use to denote and use to denote . Given any and any , define
(43) 
Furthermore, given any nonnegative real number , define
(44) 
Finally, given , define the following linear combination of ratedistortion tuples:
(45) 
Using Cramér’s bound [29, Lemma 13], we obtain the following nonasymptotic exponential type upper bound on the probability of nonexcessdistortion, whose proof is given in in Appendix D.
Lemma 6.
For any code satisfying the conditions in Lemma 5, given any distortion levels , we have
(46) 
Furthermore, let
(47) 
and given any such that , let
(48) 
Then we have
(49) 
The following lemma which relates
Comments
There are no comments yet.