Reproduction Capabilities of Penalized Hyperbolic-polynomial Splines

1 Introduction

This short paper investigates the reproduction capabilities of hyperbolic-polynomial penalized splines. HP-splines, were recently introduced in [1] as a generalization of the better known P-splines (see [2, 3]), and combine a finite difference penalty with HB-splines that piecewise consist of real exponentials and monomials multiplied by these exponentials. Numerical examples show that the exponential nature of HP-splines may turn out to be useful in applications when the data show an exponential trend [4].

The HP-splines we consider have segments in the four-dimensional space

E_{4, α} := span {e^{α x}, x e^{α x}, e^{- α x}, x e^{- α x}}, α \in R,

(1)

with the frequency $α$ being an extra parameter to tune the smoother effects. Even though all details concerning their definition and construction can be already find in [1], the analysis of their reproduction capability is there missing. To fill the gap, here we show that these type of penalized splines reproduce function in the space ${e^{- α x}, x e^{- α x}}$ , that is fit exponential data of the latter type exactly. We also show that they conserve the first and second ‘exponential’ moments, showing that HP-splines are the natural generalization of P-splines even with respect to reproduction and moment preservation.

Given the data points $(x_{i}, y_{i}), i = 1, \dots, m$ , $x_{1} < \dots < x_{m}$ , the uniform knot partition $Ξ := {x_{1} := a = ξ_{1} < ξ_{2} \dots < ξ_{n} = b =: x_{m}}$ with knots distance $h$ , and denoting by ${B_{0}^{α}, \dots, B_{n + 1}^{α}}$ a HB-splines basis of the spline space with segments in $E_{4, α}$ , the HP-spline approximating the given data is obtained by solving the minimization problem

min a_{0}, \dots, a_{n + 1} m \sum i = 1 w_{i} {(y_{i} - n + 1 \sum j = 0 a_{j} B_{j}^{α} (x_{i}))}_{i}^{2} + λ n + 1 \sum j = 2 {((Δ_{2}^{h, α} a)_{j})}_{j}^{2},

(2)

where the minimum is with respect to the HB-splines coefficients $a = (a_{j})_{j = 0}^{n + 1}$ . The values $(w_{1}, \dots, w_{m})$ are non-zero weights, $Δ_{2}^{h, α}$ is the difference operator acting on functions and on sequences respectively as (see [5] for this type of operators),

Δ_{2}^{h, α} u = u (x) - 2 e^{- α h} u (x - h) + e^{- 2 α h} u (x - 2 h), h > 0 the knots distance

(Δ_{2}^{h, α} a)_{j} = a_{j} - 2 e^{- α h} a_{j - 1} + e^{- 2 α h} a_{j - 2}, j \in Z,

and $λ$ is a regularization parameter that can be set in several different ways, e.g. with the discrepancy principle, the generalized cross-validation, or the L-curve method (see [6], for example)
It is not difficult to see that the HP-spline can be written as

s_{h p} (x) = n + 1 \sum j = 0 {^a}_{j} B_{j}^{α} (x),

(3)

where $^a = ({^a}_{0}, \dots, {^a}_{n + 1})^{T} \in R^{(n + 2)}$ is the solution of the linear system

({B^{α}}^{T} W B^{α} + λ {D_{2}^{h, α}}^{T} D_{2}^{h, α}) a = {B^{α}}^{T} W y,

(4)

where $y = (y_{1}, \dots, y_{m})^{T} \in R^{m}$ , $B^{α} \in R^{m \times (n + 2)}$ is the banded collocation matrix $B^{α} := (B_{j}^{α} (x_{i}))_{i = 1, \dots, m}^{j = 0, \dots, n + 1}$ , $W$ is the diagonal matrix

W := (d i a g (w_{i}))_{i = 1, \dots, m} \in R^{m \times m}

and

D_{2}^{h, α} = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ \begin{matrix} 1 & - 2 e^{- α h} & e^{- 2 α h} & 0 & \dots & 0 0 & 1 & - 2 e^{- α h} & e^{- 2 α h} & \dots & 0 ⋮ & ⋱ & ⋱ & ⋱ & \dots & ⋮ ⋮ & ⋮ & ⋮ & 1 & - 2 e^{- α h} & e^{- 2 α h} \end{matrix} ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ \in R^{n \times (n + 2)} .

Note that for $α \to 0$ the space $E_{4, α}$ reduces to ${1, x, x^{2}, x^{3}}$ , HB-splines reduce to cubic B-splines and the difference operator $Δ_{2}^{h, α}$ reduces to the standard forward second order difference operator acting on a sequence $a$ as $(Δ_{2} a)_{j} = a_{j} - 2 a_{j - 1} + a_{j - 2}$ . Therefore, for $α = 0$ HP-splines coincide with P-splines based on classical cubic B-splines proposed by Eilers and Max (see their recent monograph [7]). From now on, without loss of generality, we continue by assuming that $W$

is the identity matrix.

P-splines are known to have a number of useful properties, essentially inherited from B-splines and from the special type of penalty: they can fit polynomial data exactly, they can conserve the first two moments of the data and show no boundary effects. The aim of this paper is to investigate similar reproduction properties of HP-splines. As in the polynomial case, the HP-spline properties are essentially inherited from HB-splines and this is why in Section 2 we first prove that HB-splines reproduce $E_{4, α}$ . Then, in Section 3, we show that, whatever the value of the smoothing parameter $λ$ , HP-splines fit exponential data exactly as they reproduces $E_{2, - α} := {e^{- α x}, x e^{- α x}}$ . Moreover, we show that HP-splines conserve the first and second ‘exponential’ moments. Section 4 draws conclusion and highlights future works.

2 HB-splines and their reproduction properties

As shown in [8], in the ‘cardinal’ situation –corresponding to integer spline knots–, HB-splines can be defined through convolution. For the spline space with segment in $E_{4, α}$ , starting with the first order B-spline

B_{α}^{1} (x) = e^{α x} χ_{[0, 1]} (x),

(5)

supported in $[0, 1]$ , the four order cardinal HB-spline supported on $[0, 4]$ is obtained as

B_{α}^{1} = (B_{α}^{1} * B_{α}^{1} * B_{- α}^{1} * B_{- α}^{1}) with α = (α, α, - α, - α),

hence, for any integer $k$ , the HB-spline supported in $[k, k + 4]$ is obtained by translation as $B_{α}^{1} (\cdot - k)$ . In case the knots are uniform but with a distance $h \neq 1$ , the corresponding HB-splines are defined by dilation,

B_{α}^{h} = B_{α}^{1} (\frac{\cdot}{h}) = (B_{α}^{1} * B_{α}^{1} * B_{- α}^{1} * B_{- α}^{1}) (\frac{\cdot}{h}),

(6)

and then translation. Alternatively, we directly start with the scaled order-one HB-spline $B_{α}^{h} (x) = e^{\frac{α}{h} x} χ_{[0, h]} (x)$ and use repeated convolution. In that case we see that when dealing with grid spacing $h$ , the frequency $α$ is scaled into $\frac{α}{h}$ , a fact that will also enter into the exponential reproduction discussion we are going to make.

Concerning the HB-spline reproduction, we prove the following result.

Proposition 2.1.

Let $α = (α, α, - α, - α)$

be the vector of frequencies, and

B_{α h}^{h}

the HB-spline function with uniform knots defined in (6) having support

[0, 4 h]

. Then,

f = \sum k \in Z c_{k}^{f} B_{α h}^{h} (\cdot - h k), for all f \in E_{4, α} .

Proof.

The starting point is [8, Proposition 2] that yields the particularly simple reproduction formulas for the order-two HB-spline $B_{α, α}^{1} = B_{α}^{1} * B_{α}^{1}$

\sum k \in Z e^{α k} B_{α, α}^{1} (x - k) = e^{α x}, \sum k \in Z (k + 1) e^{α k} B_{α, α}^{1} (x - k) = x e^{α k},

providing, for $t_{k} = k h$ , the reproduction formula for $e^{α x}$ . In fact, from $e^{\frac{α}{h} x} = \sum k \in Z e^{\frac{α}{h} t_{k}} B_{α, α}^{h} (x - t_{k})$ we arrive at

e^{α x} = \sum k \in Z e^{α t_{k}} B_{α h, α h}^{h} (x - t_{k}) .

(7)

Similarly, $\sum_{k \in Z} (k + 1) e^{α k} B_{α, α} (x - k) = x e^{α k}$ gives

\frac{x}{h} e^{\frac{α}{h} x} = \sum k \in Z (k + 1) e^{\frac{α}{h} k h} B_{α, α}^{h} (x - h k)

and hence the reproduction formula for $x e^{α x}$ :

x e^{α x} = \sum k \in Z (t_{k} + h) e^{α t_{k}} B_{α h, α h}^{h} (x - t_{k}) .

(8)

Next, we use the fact that the convolution product of $B_{- h α, - h α}^{h}$ with the functions $f_{1} (x) = e^{α x}$ or $f_{2} (x) = x e^{α x}$ yields another exponential polynomial of the same type, that is

B_{- α h, h α h}^{h} * f_{1} = a_{0} f_{1}, and B_{- α h, α h}^{h} a s t f_{2} = b_{0} f_{1} + b_{1} f_{2}, with a_{0}, b_{0}, b_{1} \in R .

Therefore, since $B_{α}^{1} = B_{α, α}^{1} * B_{- α, - α}^{1}$ and $B_{α h}^{h} = \frac{1}{h} (B_{α h, α h}^{h} * B_{- α h, α h}^{h})$ , if we convolve both side of (7) and (8) with $B_{- α h, - α h}^{h}$ , we arrive at

a_{0} f_{1} = \sum k \in Z e^{α t_{k}} h B_{α h}^{h} (\cdot - t_{k}), and b_{0} f_{1} + b_{1} f_{2} = \sum k \in Z (t_{k} + h) e^{α t_{k}} h B_{α h}^{h} (\cdot - t_{k}),

that are the reproduction formulas for $B_{α h}^{h}$ of a function in $E_{2, α} = {e^{α x}, x e^{α x}}$ . Similarly we prove the reproduction of $E_{2, - α} = {e^{- α x}, x e^{- α x}}$ and therefore the reproduction of $E_{4, α}$ .∎

3 HP-splines reproduction of $E_{2, - α}$ and moment preservation

Based on the HB-splines reproduction properties shown in Proposition 2.1 in this section we show the reproduction capabilities of HP-splines, independently to the value of the smoothing parameter $λ$ .

Proposition 3.1.

Let the data points $(x_{i}, y_{i}), i = 1, \dots, m$ , $x_{1} < \dots < x_{m}$ , be given together with the uniform knots partition $Ξ := {x_{1} := a = ξ_{1} < ξ_{2} \dots < ξ_{n} = b =: x_{m}}$ ( $n < m$ ) extended with the uniform left and right extra knots $ξ_{ℓ} = ξ_{1} + (ℓ - 1) h, ℓ = 0, - 1, - 2, ξ_{n + ℓ} = ξ_{n} + ℓ h, ℓ = 1, 2, 3$ where $h = (b - a) / (n - 1)$ . Let ${B_{0}^{α}, \dots, B_{n + 1}^{α}}$ be the spline basis with segments in $E_{4, α}$ consisting of the uniform HB-splines $B_{0}^{α} := B_{α h}^{h} (\cdot - ξ_{- 2})$ and $B_{j}^{α} = B_{0}^{α} (\cdot - j h), j = 1, \dots, n + 1$ , with $B_{α h}^{h}$ as in (6). Then, if the data are taken form a function $f \in E_{2, - α}$ , i.e., $y_{i} = f (x_{i}), i = 1, \dots, m$ with $f \in E_{2, - α}$ , the HP-splines $s_{h p}$ defined in (3) satisfies

s_{h p} (x_{i}) = y_{i}, i = 1, \dots, m .

Proof.

Form Proposition 2.1 we know that HB-splines reproduce $E_{4, α}$ meaning that there exists a sequence of coefficients $c_{j}^{f}, j = 0, \dots, n + 1$ satisfying

n + 1 \sum j = 0 c_{j}^{f} B_{j}^{α} (x) = f (x), x \in [a, b], f \in E_{4, α} .

(9)

With the notation $^a = (c_{0}^{f}, \dots, c_{n + 1}^{f})$ assuming that the data are taken form a function $f \in E_{2, - α}$ , it is not difficult to see that the solution of the linear system (4) is exactly $^a$ , since from (9) we have

y_{i} = n + 1 \sum j = 0 {^a}_{j} B_{j}^{α} (x_{i}), i = 1, \dots, m,

(10)

and, for $x \in [a, b]$ ,

	$0$	$=$	$Δ_{2}^{h, α} f (x) = n + 1 \sum j = 0 {^a}_{j} Δ_{2}^{h, α} B_{j}^{α} (x) =$		(11)
		$=$	$n + 1 \sum j = 2 (Δ_{2}^{h, α}^a)_{j} B_{j}^{α} (x) + {^a}_{0} B_{0}^{α} (x) + ({^a}_{1} - 2 e^{- α h} {^a}_{0}) B_{1}^{α} (x),$		(11)

that, in combination with the linear independence of HB-splines, shows that $(Δ_{2}^{h, α}^a)_{j} = 0$ for $j = 2, \dots, n + 1$ . The latter means that the model acts like non penalized regression and that the reproduction capabilities of the HB-splines transfer to the HP-splines. ∎

Remark 3.2.

It is important to remark that, even though HB-splines reproduces $E_{4, α}$ , Proposition 3.1 shows that HP-splines reproduces $E_{2, - α}$ only. This limitation is due to the specific definition of the difference operator acting as shown in (11). A model based on $Δ_{2}^{h, - α}$ rather than $Δ_{2}^{h, α}$ would reproduce $E_{2, α}$ .

Next, we show that HP-splines preserve the two ‘exponential’ moments

Proposition 3.3.

In the notation of Proposition (3.1), denoting by $^y = B^{α}^a$ the vector of predicted values with elements ${^y}_{i} = s_{h p} (x_{i}), i = 1, \dots, m$ , we have

m \sum i = 1 e^{- α x_{i}} {^y}_{i} = m \sum i = 1 e^{- α x_{i}} y_{i}, and m \sum i = 1 x_{i} e^{- α x_{i}} {^y}_{i} = m \sum i = 1 x_{i} e^{- α x_{i}} y_{i} .

(12)

Proof.

To see (12), we start from the two equations defining the reproduction of $E_{2, - α}$

n + 1 \sum j = 0 c_{j} B_{j}^{α} (x) = e^{- α x}, and n + 1 \sum j = 0 d_{j} B_{j}^{α} (x) = x e^{- α x}, with c_{j}, d_{j} \in R,

(13)

and evaluate them at $x_{i}, i = 1, \dots, m$ . Hence, for $c = (c_{0}, \dots, c_{n + 1})$ and $e = (e^{- α x_{1}}, \dots, e^{- α x_{m}})$ we have the equivalence

n + 1 \sum j = 0 c_{j} B_{j}^{α} (x_{i}) = e^{- α x_{i}}, i = 1, \dots, m \Leftrightarrow B^{α} c = e,

(14)

and, for $d = (d_{0}, \dots, d_{n + 1})$ and $x e = (x_{1} e^{- α x_{1}}, \dots, x_{m} e^{- α x_{m}})$ , the equivalence

n + 1 \sum j = 0 d_{j} B_{j}^{α} (x_{i}) = x_{i} e^{- α x_{i}}, i = 1, \dots, m \Leftrightarrow B^{α} d = x e .

(15)

Now, in consideration of the linear independence of the HB-splines, with the same reasoning done in Proposition 3.1, from

	$0$	$=$	$Δ_{2}^{h, α} (e^{- α x}) = Δ_{2}^{h, α} (n + 1 \sum j = 0 c_{j} B_{j}^{α} (x)) =$
		$=$	$n + 1 \sum j = 2 (Δ_{2}^{h, α} c)_{j} B_{j}^{α} (x) + c_{0} B_{0}^{α} (x) + (c_{1} - 2 e^{- α h} c_{0}) B_{1}^{α} (x),$

we can write

(Δ_{2}^{h, α} c)_{j} = 0, j = 2, \dots, n + 1,

from which we conclude $D_{2}^{h, α} c = 0 .$ Similarly,

	$0$	$=$	$Δ_{2}^{h, α} (x e^{- α x}) = Δ_{2}^{h, α} (n + 1 \sum j = 0 d_{j} B_{j}^{α} (x)) =$
		$=$	$n + 1 \sum j = 2 (Δ_{2}^{h, α} d)_{j} B_{j}^{α} (x) + d_{0} B_{0}^{α} (x) + (d_{1} - 2 e^{- α h} d_{0})$

implies

(Δ_{2}^{h, α} d)_{j} = 0, j = 2, \dots, n + 1,

(16)

and therefore $D_{2}^{h, α} d = 0 .$ Next, for the predicted values $^y = B^{α}^a$ we have

(17)

Multiplication of both sides of (17) respectively by $c$ and $d$ yields

and

Now, since $D_{2}^{h, α} c = 0$ and $D_{2}^{h, α} d = 0$ using (14) and (15) we arrive at

e^{T} (y -^y) = 0, and {x e}^{T} (y -^y) = 0,

which are the vector versions of (12). ∎

Remark 3.4.

Note that the exponential moments (12), reduce to the classical moments preservation whenever $α = 0$ that is to

m \sum i = 1 {^y}_{i} = m \sum i = 1 y_{i}, and m \sum i = 1 x_{i} {^y}_{i} = m \sum i = 1 x_{i} y_{i} .

We conclude the paper with some figures showing the exponential-reproduction capabilities of HP-splines. Figures 1-2 refers to data taken from the exponential functions $e^{- x}$ while Figure 2 to data from the function $x e^{- x}$ . They display the graph of the HP-spline (black ‘ $-$ ’ ) approximating the data for different selections of $α$

combined with different level of absolute Gaussian noise with zero mean and standard deviation

σ

, both specified in the figure captions. The data sites (red ‘

*

’ ) and the spline knots location (blue ‘

⋄

’ ) are also given in the figures. The smoothing parameter is always

λ = 1

since not relevant to our discussion. The exact exponential fit is evident in absence of noise (left) while it is almost attained in case of a moderate noise (right) as well in case of an uncorrect selection of the frequency. For comparison, the graph of the P-splines approximating the data is also given (magenta ‘

- .

’ ) together with the graph of the function (blue ‘

- -

’ ).

Figure 1: Function $e^{- x}$ . From left to right values of $(α, σ)$ : $(- 1, 0)$ , $(- 1, 0.5 \cdot 10^{- 2})$ , $(- 0.5, 0.5 \cdot 10^{- 2})$ .

Figure 2: Function $x e^{- x}$ . From left to right values of $(α, σ)$ : $(- 1, 0)$ , $(- 1, 0.5 \cdot 10^{- 2})$ , $(- 0.5, 0.5 \cdot 10^{- 2})$ .

4 Conclusions

This short paper enriches the study of HP-splines, penalized hyperbolic - splines with segments in the space $E_{4, α}$ consisting in the exponential polynomials ${e^{α x}, e^{- α x}, x e^{α x}, x e^{- α x}}$ , where $α$ is a real frequency. In particular, it investigates two important analytical properties of HP-splines: that they exactly fit function in $E_{2, - α}$ and that they conserve the first and second ‘exponential’ moments, independently to the value of the smoothing parameter $λ$ . A few numerical examples of reproduction are shown. A dynamic selection strategy of the parameter $α$ , that certainly deserve more attention, is presently under investigation.

Acknowledgement
The authors are members of INdAM-GNCS, partially supporting this work. They are also member of RItA (Rete ITaliana di Approssimazione) and UMI-T.A.A. group.

References

[1] R. Campagna, C. Conti, Penalized hyperbolic-polynomial splines, Applied Mathematics Letters 118 (2021) 107159.
[2] P. H. C. Eilers, B. D. Marx, Flexible smoothing with B-splines and penalties, Statistical Science 11 (2) (1996) 89–121.
[3] P. H. Eilers, B. D. Marx, M. Durbn, Twenty years of P-splines, SORT-Statistics and Operations Research Transactions 39 (2) (2015) 149–186.
[4] R. Campagna, C. Conti, S. Cuomo, Computational error bounds for Laplace transform inversion based on smoothing splines, Applied Mathematics and Computation 383 (2020) 125376.
[5] C. Conti, Sergio López-Ureña and Lucia Romani, Annihilation operators for exponential spaces in subdivision journal, Applied Mathematics and Computation, 418, 126796, 2022
[6] P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, Society for Industrial and Applied Mathematics, 1998.
[7] P. H. Eilers, B. D. Marx, Practical Smoothing: The Joys of P-splines, Cambridge University Press, 2021.
[8] M. Unser, T. Blu, Cardinal exponential splines: part I - theory and filtering algorithms, IEEE Transactions on Signal Processing 53 (4) (2005) 1425–1438 Comp. Math. 11 (1999), 41-54.

Reproduction Capabilities of Penalized Hyperbolic-polynomial Splines

Authors

A Theory of Hyperbolic Prototype Learning

Annihilation Operators for Exponential Spaces in Subdivision

A sufficient condition for penalized polynomial regression to be invariant to translations of the predictor variables

Generalization Error Bound for Hyperbolic Ordinal Embedding

A Quantized Representation of Intertemporal Choice in the Brain

Non-constant bounded holomorphic functions of hyperbolic numbers - Candidates for hyperbolic activation functions

Investigating Flight Envelope Variation Predictability of Impaired Aircraft using Least-Squares Regression Analysis

1 Introduction

2 HB-splines and their reproduction properties

Proposition 2.1.

Proof.

3 HP-splines reproduction of $E_{2, - α}$ and moment preservation

Proposition 3.1.

Proof.

Remark 3.2.

Proposition 3.3.

Proof.

Remark 3.4.

4 Conclusions

References

Reproduction Capabilities of Penalized Hyperbolic-polynomial Splines

Authors

Related Research

A Theory of Hyperbolic Prototype Learning

Annihilation Operators for Exponential Spaces in Subdivision

A sufficient condition for penalized polynomial regression to be invariant to translations of the predictor variables

Generalization Error Bound for Hyperbolic Ordinal Embedding

A Quantized Representation of Intertemporal Choice in the Brain

Non-constant bounded holomorphic functions of hyperbolic numbers - Candidates for hyperbolic activation functions

Investigating Flight Envelope Variation Predictability of Impaired Aircraft using Least-Squares Regression Analysis

This week in AI

1 Introduction

2 HB-splines and their reproduction properties

Proposition 2.1.

Proof.

3 HP-splines reproduction of E2,−α and moment preservation

Proposition 3.1.

Proof.

Remark 3.2.

Proposition 3.3.

Proof.

Remark 3.4.

4 Conclusions

References

3 HP-splines reproduction of $E_{2, - α}$ and moment preservation