Reproduction Capabilities of Penalized Hyperbolic-polynomial Splines

This paper investigates two important analytical properties of hyperbolic-polynomial penalized splines, HP-splines for short. HP-splines, obtained by combining a special type of difference penalty with hyperbolic-polynomial B-splines (HB-splines), were recently introduced by the authors as a generalization of P-splines. HB-splines are bell-shaped basis functions consisting of segments made of real exponentials e^α x, e^-α x and linear functions multiplied by these exponentials, xe^+α x and xe^-α x. Here, we show that these type of penalized splines reproduce function in the space {e^-α x, x e^-α x}, that is they fit exponential data exactly. Moreover, we show that they conserve the first and second 'exponential' moments.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

10/15/2020

A Theory of Hyperbolic Prototype Learning

We introduce Hyperbolic Prototype Learning, a type of supervised learnin...
12/17/2020

Annihilation Operators for Exponential Spaces in Subdivision

We investigate properties of differential and difference operators annih...
02/11/2020

A sufficient condition for penalized polynomial regression to be invariant to translations of the predictor variables

Whereas translating the coding of predictor variables does not change th...
05/21/2021

Generalization Error Bound for Hyperbolic Ordinal Embedding

Hyperbolic ordinal embedding (HOE) represents entities as points in hype...
02/24/2020

A Quantized Representation of Intertemporal Choice in the Brain

Value [4][5] is typically modeled using a continuous representation (i.e...
06/07/2013

Non-constant bounded holomorphic functions of hyperbolic numbers - Candidates for hyperbolic activation functions

The Liouville theorem states that bounded holomorphic complex functions ...
05/20/2019

Investigating Flight Envelope Variation Predictability of Impaired Aircraft using Least-Squares Regression Analysis

Aircraft failures alter the aircraft dynamics and cause maneuvering flig...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

This short paper investigates the reproduction capabilities of hyperbolic-polynomial penalized splines. HP-splines, were recently introduced in [1] as a generalization of the better known P-splines (see [2, 3]), and combine a finite difference penalty with HB-splines that piecewise consist of real exponentials and monomials multiplied by these exponentials. Numerical examples show that the exponential nature of HP-splines may turn out to be useful in applications when the data show an exponential trend [4].

The HP-splines we consider have segments in the four-dimensional space

(1)

with the frequency being an extra parameter to tune the smoother effects. Even though all details concerning their definition and construction can be already find in [1], the analysis of their reproduction capability is there missing. To fill the gap, here we show that these type of penalized splines reproduce function in the space , that is fit exponential data of the latter type exactly. We also show that they conserve the first and second ‘exponential’ moments, showing that HP-splines are the natural generalization of P-splines even with respect to reproduction and moment preservation.

Given the data points , , the uniform knot partition with knots distance , and denoting by a HB-splines basis of the spline space with segments in , the HP-spline approximating the given data is obtained by solving the minimization problem

(2)

where the minimum is with respect to the HB-splines coefficients . The values are non-zero weights, is the difference operator acting on functions and on sequences respectively as (see [5] for this type of operators),

and is a regularization parameter that can be set in several different ways, e.g. with the discrepancy principle, the generalized cross-validation, or the L-curve method (see [6], for example)
It is not difficult to see that the HP-spline can be written as

(3)

where is the solution of the linear system

(4)

where , is the banded collocation matrix , is the diagonal matrix

and

Note that for the space reduces to , HB-splines reduce to cubic B-splines and the difference operator reduces to the standard forward second order difference operator acting on a sequence as . Therefore, for HP-splines coincide with P-splines based on classical cubic B-splines proposed by Eilers and Max (see their recent monograph [7]). From now on, without loss of generality, we continue by assuming that

is the identity matrix.

P-splines are known to have a number of useful properties, essentially inherited from B-splines and from the special type of penalty: they can fit polynomial data exactly, they can conserve the first two moments of the data and show no boundary effects. The aim of this paper is to investigate similar reproduction properties of HP-splines. As in the polynomial case, the HP-spline properties are essentially inherited from HB-splines and this is why in Section 2 we first prove that HB-splines reproduce . Then, in Section 3, we show that, whatever the value of the smoothing parameter , HP-splines fit exponential data exactly as they reproduces . Moreover, we show that HP-splines conserve the first and second ‘exponential’ moments. Section 4 draws conclusion and highlights future works.

2 HB-splines and their reproduction properties

As shown in [8], in the ‘cardinal’  situation –corresponding to integer spline knots–, HB-splines can be defined through convolution. For the spline space with segment in , starting with the first order B-spline

(5)

supported in , the four order cardinal HB-spline supported on is obtained as

hence, for any integer , the HB-spline supported in is obtained by translation as . In case the knots are uniform but with a distance , the corresponding HB-splines are defined by dilation,

(6)

and then translation. Alternatively, we directly start with the scaled order-one HB-spline and use repeated convolution. In that case we see that when dealing with grid spacing , the frequency is scaled into , a fact that will also enter into the exponential reproduction discussion we are going to make.

Concerning the HB-spline reproduction, we prove the following result.

Proposition 2.1.

Let

be the vector of frequencies, and

the HB-spline function with uniform knots defined in (6) having support . Then,

Proof.

The starting point is [8, Proposition 2] that yields the particularly simple reproduction formulas for the order-two HB-spline

providing, for , the reproduction formula for . In fact, from we arrive at

(7)

Similarly, gives

and hence the reproduction formula for :

(8)

Next, we use the fact that the convolution product of with the functions or yields another exponential polynomial of the same type, that is

Therefore, since and , if we convolve both side of (7) and (8) with , we arrive at

that are the reproduction formulas for of a function in . Similarly we prove the reproduction of and therefore the reproduction of .∎

3 HP-splines reproduction of and moment preservation

Based on the HB-splines reproduction properties shown in Proposition 2.1 in this section we show the reproduction capabilities of HP-splines, independently to the value of the smoothing parameter .

Proposition 3.1.

Let the data points , , be given together with the uniform knots partition () extended with the uniform left and right extra knots where . Let be the spline basis with segments in consisting of the uniform HB-splines and , with as in (6). Then, if the data are taken form a function , i.e.,  with , the HP-splines defined in (3) satisfies

Proof.

Form Proposition 2.1 we know that HB-splines reproduce meaning that there exists a sequence of coefficients satisfying

(9)

With the notation assuming that the data are taken form a function , it is not difficult to see that the solution of the linear system (4) is exactly , since from (9) we have

(10)

and, for ,

(11)

that, in combination with the linear independence of HB-splines, shows that for . The latter means that the model acts like non penalized regression and that the reproduction capabilities of the HB-splines transfer to the HP-splines. ∎

Remark 3.2.

It is important to remark that, even though HB-splines reproduces , Proposition 3.1 shows that HP-splines reproduces only. This limitation is due to the specific definition of the difference operator acting as shown in (11). A model based on rather than would reproduce .

Next, we show that HP-splines preserve the two ‘exponential’ moments

Proposition 3.3.

In the notation of Proposition (3.1), denoting by the vector of predicted values with elements , we have

(12)
Proof.

To see (12), we start from the two equations defining the reproduction of

(13)

and evaluate them at . Hence, for and we have the equivalence

(14)

and, for and , the equivalence

(15)

Now, in consideration of the linear independence of the HB-splines, with the same reasoning done in Proposition 3.1, from

we can write

from which we conclude Similarly,

implies

(16)

and therefore Next, for the predicted values we have

(17)

Multiplication of both sides of (17) respectively by and yields

and

Now, since and using (14) and (15) we arrive at

which are the vector versions of (12). ∎

Remark 3.4.

Note that the exponential moments (12), reduce to the classical moments preservation whenever that is to

We conclude the paper with some figures showing the exponential-reproduction capabilities of HP-splines. Figures 1-2 refers to data taken from the exponential functions while Figure 2 to data from the function . They display the graph of the HP-spline (black ‘’ ) approximating the data for different selections of

combined with different level of absolute Gaussian noise with zero mean and standard deviation

, both specified in the figure captions. The data sites (red ‘’ ) and the spline knots location (blue ‘’ ) are also given in the figures. The smoothing parameter is always since not relevant to our discussion. The exact exponential fit is evident in absence of noise (left) while it is almost attained in case of a moderate noise (right) as well in case of an uncorrect selection of the frequency. For comparison, the graph of the P-splines approximating the data is also given (magenta ‘’ ) together with the graph of the function (blue ‘’ ).

Figure 1: Function . From left to right values of : , , .
Figure 2: Function . From left to right values of : , , .

4 Conclusions

This short paper enriches the study of HP-splines, penalized hyperbolic - splines with segments in the space consisting in the exponential polynomials , where is a real frequency. In particular, it investigates two important analytical properties of HP-splines: that they exactly fit function in and that they conserve the first and second ‘exponential’ moments, independently to the value of the smoothing parameter . A few numerical examples of reproduction are shown. A dynamic selection strategy of the parameter , that certainly deserve more attention, is presently under investigation.


Acknowledgement
The authors are members of INdAM-GNCS, partially supporting this work. They are also member of RItA (Rete ITaliana di Approssimazione) and UMI-T.A.A. group.

References

  • [1] R. Campagna, C. Conti, Penalized hyperbolic-polynomial splines, Applied Mathematics Letters 118 (2021) 107159.
  • [2] P. H. C. Eilers, B. D. Marx, Flexible smoothing with B-splines and penalties, Statistical Science 11 (2) (1996) 89–121.
  • [3] P. H. Eilers, B. D. Marx, M. Durbn, Twenty years of P-splines, SORT-Statistics and Operations Research Transactions 39 (2) (2015) 149–186.
  • [4] R. Campagna, C. Conti, S. Cuomo, Computational error bounds for Laplace transform inversion based on smoothing splines, Applied Mathematics and Computation 383 (2020) 125376.
  • [5] C. Conti, Sergio López-Ureña and Lucia Romani, Annihilation operators for exponential spaces in subdivision journal, Applied Mathematics and Computation, 418, 126796, 2022
  • [6] P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, Society for Industrial and Applied Mathematics, 1998.
  • [7] P. H. Eilers, B. D. Marx, Practical Smoothing: The Joys of P-splines, Cambridge University Press, 2021.
  • [8] M. Unser, T. Blu, Cardinal exponential splines: part I - theory and filtering algorithms, IEEE Transactions on Signal Processing 53 (4) (2005) 1425–1438 Comp. Math. 11 (1999), 41-54.