    # Adapted Decimation on Finite Frames for Arbitrary Orders of Sigma-Delta Quantization

In Analog-to-digital (A/D) conversion, signal decimation has been proven to greatly improve the efficiency of data storage while maintaining high accuracy. When one couples signal decimation with the ΣΔ quantization scheme, the reconstruction error decays exponentially with respect to the bit-rate. We build on our previous result, which extends signal decimation to finite frames, albeit only up to the second order. In this study, we introduce a new scheme called adapted decimation, which yields polynomial reconstruction error decay rate of arbitrary order with respect to the oversampling rate, and exponential with respect to the bit-rate.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1. Introduction

### 1.1. Background and Motivation

Analog-to-digital (A/D) conversion is a process where bandlimited signals, e.g., audio signals, are digitized for storage and transmission, which is feasible thanks to the classical sampling theorem. In particular, the theorem indicates that discrete sampling is sufficient to capture all features of a given bandlimited signal, provided that the sampling rate is higher than the Nyquist rate.

Given a function

, its Fourier transform

is defined as

 ^f(γ)=∫∞−∞f(t)e−2πıtγdt.

The Fourier transform can also be uniquely extended to as a unitary transformation.

###### Definition 1.1.

Given , if its Fourier transform is supported in .

An important component of A/D conversion is the following theorem:

###### Theorem 1.2 (Classical Sampling Theorem).

Given , for any satisfying

• on

• for ,

with and , , one has

 (1) f(t)=T∑n∈Zf(nT)g(t−nT),

where the convergence is both uniform on compact sets of and in .

In particular, for and , the following identity holds in :

 f(t)=∑f(n)sin(π(t−n))π(t−n).

However, the discrete nature of digital data storage makes it impossible to store exactly the samples . Instead, the quantized samples chosen from a pre-determined finite alphabet are stored. This results in the following reconstructed signal

 ~f(t)=T∑qng(t−nT).

As for the choice of the quantized samples , we shall discuss the following two schemes

• Pulse Code Modulation (PCM):

Quantized samples are taken as the direct-roundoff of the current sample, i.e.,

 (2) qn=Q0(f(nT)):=argminq∈A|q−f(nT)|.
• Quantization:

A sequence of auxiliary variables is introduced for this scheme. is defined recursively as

 qn=Q0(un−1+f(nT)),un=un−1+f(nT)−qn.

quantization was introduced in 1963, , and is still widely used, due to its advantages over PCM. Specifically, quantization is robust against hardware imperfection , a decisive weakness for PCM. For quantization, and the more general noise shaping schemes to be explained below, the boundedness of

turns out to be essential, as most analyses on quantization problems rely on it for error estimation. Schemes with bounded auxiliary variables are said to be

stable.

Despite its merits over PCM, quantization merely produces linear error decay with respect to bits used as opposed to exponential error decay produced by its counterpart PCM. Thus, it is desirable to generalize quantization for higher order error decay.

Given , one can consider an -th order quantization scheme as investigated by Daubechies and DeVore:

###### Theorem 1.3 (Higher Order ΣΔ Quantization, ).

Consider the following stable quantization scheme

 f(nT)−qn=(Δru):=r∑l=0(−1)l(rl)un−l,

where and are the quantized samples and auxiliary variables respectively. Then, for all ,

 |f(t)−T∑n∈Zqng(t−nT)|≤Tr∥u∥∞∥drgdtr∥1.

The existence of such scheme is also proven in the same paper. This has improved the error decay rate from linear to arbitrary polynomial degree while preserving the merits of a first order quantization scheme.

From here, a natural question arises: is it possible to generalize quantization scheme further so that the reconstruction error decay matches the exponential decay of PCM? Two solutions have been proposed for this question. The first one is to create new quantization schemes, known as noise shaping quantization schemes. A brief summary of its development will be provided in Section A.

The other possibility is to drastically enhance data storage efficiency while maintaining the same level of reconstruction accuracy, and signal decimation belongs in this category. The process is as follows: given an r-th order quantization scheme, there exists such that

 f(nT)−qTn=(Δru)n,

where . Then, consider

 ~qT0n:=(SrρqT)(2ρ+1)n,

a down-sampled sequence of , where .

The process where we convert the quantized samples to is called signal decimation.

Decimation has been known in the engineering community , and it was observed that decimation results in exponential error decay with respect to the bit-rate, even though the observation remained a conjecture until 2015 , when Daubechies and Saab proved the following theorem:

###### Theorem 1.4 (Signal Decimation for Bandlimited Functions, ).

Given , , and , there exists a function such that

 f(t)=T0∑[Srρf(T)](2ρ+1)n~g(t−nT0),
 (3) |f(t)−T0∑~qT0n~g(t−nT0)|≤CΣΔCr(TT0)r=:D.

Moreover, the bits needed for each Nyquist interval is

 (4) 1T0log2((2ρ+1)r+1)≤1T0log2(2(T0T)r)=:R.

Consequently,

 D(R)=2CΣΔCr2−T0R

From (3) and (4), we can see that the reconstruction error after decimation still decays polynomially with respect to the sampling frequency. As for the data storage, the bits needed changes from to . Thus, the reconstruction error decays exponentially with respect to the bits used. Figure 1. Illustration of the first order Decimation Scheme: taking averages of samples nearby before down-sampling. The effect on the reconstruction (Replacing qn with yn−qn) is illustrated in parentheses

In , the author made an extension of decimation to signals in finite dimensional spaces. Such signals are sampled by finite frames, and a brief introduction on finite frames is given in Appendix A. Using the alternative decimation operator introduced in the same paper, it is proven that up to the second order sigma-delta quantization, similar results to Theorem 1.4 can be achieved:

###### Definition 1.5 (Alternative Decimation).

Given fixed , the -alternative decimation operator is defined to be , where

• is the integration operator satisfying

 (S+ρ)l,j={1ρifl≥ρ,l−(ρ−1)≤j≤l0otherwise,(S−ρ)l,j={1ρifl≤ρ−1,l+1≤j≤m−ρ+l0otherwise.

Here, the cyclic convention is adopted: For any , .

• is the sub-sampling operator satisfying

 (Dρ)l,j={1ifj=ρ⋅l0otherwise,

where .

###### Definition 1.6 (Unitarily Generated Frames (UGF)).

Given a base vector

and a Hermitian matrix , the unitarily generated frame has its rows as

 (5) Ut:=e2πıΩt,ϕ(m)j=Uj/mϕ0.

The eigenvalues and eigenvectors of

are represented as and .

###### Theorem 1.7 (Alternative Decimation for Finite Frames, ).

Given , , , , and as the generator, base vector, eigenvalues, eigenvectors, and the corresponding UGF, respectively, and . Suppose

• ,

• , and

• ,

then the dual frame combined with the -th order quantization has polynomial reconstruction error decay rate of degree with respect to the oversampling rate :

 Em,ρ,r≤C∥u∥∞1ρr.

Moreover, the total bits used to record the quantized samples are bits, where the constant depends on . Suppose is fixed as , then as a function of bits used at each entry, satisfies

 E(R)≤C∥u∥∞2−12ηR.

The constant is independent of the oversampling rate .

### 1.2. Prior Works

#### 1.2.1. Quantization for Bandlimited Functions

Despite its simple form and robustness, quantization only results in linear error decay with respect to the sampling period as . It was later proven in  that a generalization of quantization, namely the r-th order quantization, exists for any arbitrary , and for such schemes the error decay is of polynomial order . Leveraging the different constants for this family of quantization schemes, sub-exponential decay can also be achieved. A different family of quantization schemes was shown  to have exponential error decay with small exponent (.) In , the exponent was improved to .

#### 1.2.2. Finite Frames

quantization can also be applied to finite frames. It is proven  that for any family of frames with bounded frame variation, the reconstruction error decays linearly with respect to the oversampling rate , where the frame is an matrix. With different choices of dual frames,  proposed that the so-called Sobolev dual achieves minimum induced matrix 2-norm for reconstructions. By carefully matching between the dual frame and the quantization scheme,  proved that using

-dual for random frames will result in exponential decay with near-optimal exponent and high probability.

#### 1.2.3. Decimation

In , using the assumption that the noise in quantization is random along with numerical experiments, it was asserted that decimation greatly reduces the number of bits needed while maintaining the reconstruction accuracy. In , a rigorous proof was given to show that such an assertion is indeed valid, and the reduction of bits used turns the linear decay into exponential decay with respect to the bit-rate.

As for the adaptation to finite dimensional signals, the author proved in  that there exists a similar operator called the alternative decimation operator that behaves similarly to the decimation for bandlimited signals. In particular, for the first and second order of quantization, it is possible to achieve exponential reconstruction error decay with respect to the bit-rate as well. However, similar to the caveat of decimation, it merely improves the storage efficiency while maintaining the same level reconstruction error. Thus, the error rate with respect to the oversampling rate remains the same (quadratic for the second order,) which is still inferior to other noise shaping schemes.

## 2. Main Results

We have seen in Theorem 1.7 that alternative decimation is only useful up to the second order. Thus, we aim to extend our results to arbitrary orders, and the solution we present here is called the adapted decimation.

Given , the -adapted decimation operator is defined to be

 Ar=1ρrDρ¯ΔrρΔ−r,

where is the usual backward difference matrix, satisfies , and has .

It will be shown that, for unitarily generated frames satisfying conditions specified in Theorem 2.2, an -th order quantization coupled with the corresponding adapted decimation has -th order polynomial reconstruction error decay rate with respect to the ratio . As for the data storage, decimation allows for highly efficient storage, making the error decay exponentially with respect to the bits used.

For the rest of the paper, we shall also assume that our quantization scheme is stable, i.e.  remains bounded as the dimension .

We also choose the quantization alphabet to be which is uniformly spaced and symmetric around the origin: Given , we define

 A0={−(2K+1)δ/2,−(2K−1)δ/2,…,(2K−1)δ/2,(2K+1)δ/2}.

For complex Euclidean spaces, we define to be the mid-rise quantizer with length . Throughout this paper we shall always deal with such .

###### Theorem 2.2.

Given , , , , and as the generator, base vector, eigenvalues, eigenvectors, and the corresponding UGF, respectively, and fixed. Suppose

• ,

• ,

• ,

• , and

• there exists from the stable -th order quantization satisfying

 Φ∗m,kx−q=u,

where , then the following statements are true:

• Recursivity: For all , there exists such that .

• Signal reconstruction: is a frame.

• Error estimate: For the dual frame , the reconstruction error satisfies

 (6) Em,ρ≤(4kηmins|cs|2(8ηπ)r)∥u∥∞1ρr.
• Efficient data storage: Suppose the length of the quantization alphabet is , then the total bits used to record the quantized samples are bits. Furthermore, suppose is fixed as , then as a function of bits used at each entry, satisfies

 (7) E(R)≤Ck,η,ϕ0,L∥u∥∞2−12ηR,

where , independent of .

###### Remark 2.3.

Harmonic frames are unitarily generated frames with the generator being the diagonal matrix with entries and the base vector , so Theorem 2.2 is applicable for harmonic frames as well.

We shall prove Theorem 2.2 in several steps. First, we split into one main term and many residual terms in Section 3.1 . Then, we compute the cancellation among residual terms in Section 3.2. We compute the lower frame bound of in Section 3.3 before proving the theorem itself.

## 3. Proof of Theorem 2.2

First, we claim that scales down to the usual backward-difference matrix under the under-sampling matrix :

###### Lemma 3.1.

Given with ,

 Dρ¯Δρ=Δ(η)Dρ,

where is the -dimensional backward difference matrix.

###### Proof.

Note that, for ,

 (Dρ¯Δρ)l,s=(¯Δρ)lρ,s=δ(s−lρ)−δ(s+ρ−lρ)=δ(s−lρ)−δ(s−(l−1)ρ)=(ΔDρ)l,s.

For , . ∎

The most important estimate in Theorem 2.2 is (6), where we find the reconstruction error under adapted decimation . It can be written as follows: given a signal and the matrix , the reconstruction error satisfies

 E=∥x−(ArΦ∗m,k)†Arq∥2=∥(ArΦ∗m,k)†Ar(Φ∗m,kx−q)∥2=∥(ArΦ∗m,k)†1ρrDρ¯ΔrΔ−r(Δru)∥2=1ρr∥((ArΦ∗m,k)∗ArΦ∗m,k)−1(ArΦ∗m,k)∗ΔrDρu∥2≤∥((ArΦ∗m,k)∗ArΦ∗m,k)−1∥2⋅∥(ArΦ∗m,k)∗Δr∥∞→2⋅∥u∥∞1ρr,

where the fourth equality follows from Lemma 3.1, and is the -to-2 norm of . Thus, in order to obtain this estimate, we need to answer two questions:

• Is a frame? What is the lower frame bound of ?

• What is ?

The the first question will be answered positively in Proposition 3.11, and the estimate in the third question is given in Lemma 3.13.

Aside from the reconstruction error estimate, we also need to calculate the bits needed to record the decimated sample , which will be done in Lemma 3.14.

### 3.1. The Effect of Adapted Decimation on the Frame

We start by introducing the following notation:

###### Definition 3.2.

Given , the -by- constant matrix has constant on all entries.

The following two lemmas are needed for us to describe in Proposition 3.5.

###### Lemma 3.3.

Given with generator , we have

 Δ−1Φ∗=(Φ∗−1m,kV)~C,

where and , where

can be diagonalized by a unitary matrix

.

###### Proof.

For any , the -th row of can be written as

 (Δ−1Φ∗m,k)t=(t∑s=1Us/mϕ0)∗=(t∑s=1BTs/mB∗ϕ0)∗=(Bt∑s=1Ts/mB∗ϕ0)∗.

Now,

 t∑s=1(Ts/m)σ=t∑s=1e2πıλσs/m=e2πıλσt/m−1e2πıλσ/m−1=(Tt/m)σ1e2πıλσ/m−1−1e2πıλσ/m−1=(~C0Tt/m−~C0)σ,

where is a diagonal matrix.

Thus,

 (Δ−1Φ∗m,k)t=(Bt∑s=1Ts/mB∗ϕ0)∗=(B~C0B∗Ut/mϕ0)∗−(B~C0B∗ϕ0)∗=ϕ∗t(B~C0B∗)∗−ϕ∗0(B~C0B∗)∗=(Φ∗m,k)t~C−ϕ∗0~C.

Thus, . ∎

###### Lemma 3.4.
 ¯ΔρΦ∗=Φ∗~D+¯Δρ1m,kV,

where .

###### Proof.

For any ,

 (¯ΔρΦ∗m,k)t=(Ut/mϕ0−U(t−ρ)/mϕ0)∗+δ(t−ρ)ϕ∗0=(B(Ik−T−ρ/m)B∗Ut/mϕ0)∗+δ(t−ρ)ϕ∗0=ϕ∗tB(Ik−Tρ/m)B∗+¯Δρ1m,kV=(Φ∗~D)t+(¯Δρ1m,kV)t.

Combining Lemma 3.3 and 3.4, one has the following expansion:

###### Proposition 3.5.

Given ,

 (8) ρrArΦ∗m,k=Dρ¯ΔrρΔ−rΦ∗m,k=Dρ[Φ∗m,k~Dr~Cr+r−1∑j=0¯Δr−jρ1m,kV~Dj~Cr−¯Δrρr−1∑j=0Δ−j1m,kV~Cr−j].
###### Proof.

First, we claim that, for , .

For , by Lemma 3.3. For ,

 Δ−qΦ∗=Δ−1(Φ∗~Cq−1−q−2∑j=0Δ−j1m,kV~Cq−1−j)=Φ∗~Cq−1m,kV~Cq−q−1∑s=1Δ−s1m,kV~Cq−s=Φ∗~Cq−q−1∑j=0Δ−j1m,kV~Cq−j.

As for the effect of , we claim that for .

For , by Lemma 3.4. For ,

 ¯ΔqρΦ∗=¯Δρ(Φ∗~Dq−1+q−2∑j=0¯Δq−1−jρ1m,kV~Dj)=Φ∗~Dq+¯Δρ1m,kV~Dq−1+q−2∑j=0¯Δq−jρ1m,kV~Dj=Φ∗~Dq+q−1∑j=0¯Δq−jρ1m,kV~Dj.

From the two assertions above, we get

 ¯ΔrρΔ−rΦ∗=¯Δrρ(Φ∗~Cr−r−1∑j=0Δ−j1m,kV~Cr−j)=Φ∗~Dr~Cr+r−1∑j=0¯Δr−jρ1m,kV~Dj~Cr−¯Δrρr−1∑j=0Δ−j1m,kV~Cr−j.

### 3.2. Cancellation Between Residual Terms of ArΦ∗m,k

From (8), we can divide into two parts: being the main term, and the rest being residual terms. In this section, we shall investigate the behavior of the residual terms.

Define a double-sequence recursively by

 al,s=⎧⎪⎨⎪⎩1if l=0,s≥10if l=0,s≤0∑j≤sal−1,jifl>0.

Let and . We first examine the form of each before calculating the cancellation between and .

###### Lemma 3.6.

For any and ,

 (Δ−j1m,k)l,s=aj,l.
###### Proof.

First, it can easily be seen that for all by induction on . Then, by definition and induction on ,

 (Δ−j1m,k)l,s=l∑n=1(Δ−j+11m,k)n,s=l∑n=1aj−1,n=∑n≤laj−1,n=aj,l.

###### Lemma 3.7.

For and ,

 (¯ΔκρΔ−q1m,1)lρ=ρ∑s1,…,sκ=1aq−κ,(l−κ)ρ+s1+⋯+sκ.
###### Proof.

We shall prove this by induction on . For and ,

 (¯ΔρΔ−q1m,1)lρ=(Δ−q1m,1)lρ−(Δ−q1m,1)(l−1)ρ=aq,lρ−aq,(l−1)ρ=ρ∑s1=1aq−1,(l−1)ρ+s1.

For ,

 (¯ΔρΔ−q1m,1)ρ=(Δ−q1m,1)ρ=aq,ρ=aq,ρ−aq,0=ρ∑s1=1aq−1,0+s1.

For and ,

 (¯ΔκρΔ−q1m,1)lρ=(¯Δκ−1ρΔ−q1m,1)lρ−(¯Δκ−1ρΔ−q1m,1)(l−1)ρ=ρ∑s1,…,sκ−1=1(aq−κ+1,(l−κ+1)ρ+s1+⋯+sκ−1−aq−κ+1,(l−κ)ρ+s1+⋯+sκ−1)=ρ∑s1,…,sκaq−κ,(l−κ)ρ+s1+⋯+sκ.

As for ,

 (¯ΔκρΔ−q1m,1)ρ=(¯Δκ−1ρΔ−q1m,1)ρ=ρ∑s1,…,sκ−1=1aq−κ+1,(1−κ+1)ρ+s1+⋯+sκ−1=ρ∑s1,…,sκ−1=1aq−κ+1,(1−κ+1)ρ+s1+⋯+sκ−1−aq−κ+1,(0−κ+1)ρ+s1+⋯+sκ−1=ρ∑s1,…,sκaq−κ,(1−κ)rho+s1+⋯+sκ,

where the third equality follows from the fact that . ∎

###### Proposition 3.8.

For ,

 Dρ¯Δlρ1m,kV~Dr−l~Cr−Dρ¯ΔrρΔ−r+l1m,kV~Cl=Δl(1η,kV(