Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lasso

1 Introduction

Since it was introduced by Tibshirani (1996), the lasso regularized method for high-dimensional regression models with sparse coefficients has received a great deal of attention in the literature. Properties of interest for such regression models include the finite sample oracle inequalities. Among the extensive literature of the lasso method, Bunea, Tsybakov, and Wegkamp (2007) and Bickel, Ritov, and Tsybakov (2009)

derived the oracle inequalities for prediction risk and estimation error in a general nonparametric regression model including the high-dimensional linear regression as a special example, and

van de Geer (2008)

provided oracle inequalities for the generalized linear models with Lipschitz loss functions, e.g. logistic regression and classification with hinge loss.

We consider lasso regularized high-dimensional Cox regression. Let $T$ be the survival time and $C$ the censoring time. Suppose we observe a sequence of iid observations $(Y_{i}, Δ_{i}, X_{i})$ , $i = 1, \dots, n$ , where $Y_{i} = T_{i} \land C_{i}$ , $Δ_{i} = I_{{T_{i} \leq C_{i}}}$ , and $X_{i}$ are the covariates in $X$ . Due to largely parallel material, we follow closely the notation in van de Geer (2008). Let

F = {f_{θ} (\cdot) = m \sum k = 1 θ_{k} ψ_{k} (\cdot), θ \in Θ} .

Here $Θ$ is a convex subset of $R^{m}$ , and the functions $ψ_{1}, \dots, ψ_{m}$ are real-valued basis functions on $X$ , which are identity functions of corresponding covariates in a standard Cox model.

Consider the following Cox model (Cox, 1972):

λ (t | X) = λ_{0} (t) e^{f_{θ} (X)},

where $θ$ is the parameter of interest and $λ_{0}$ is the unknown baseline hazard function. The negative log partial likelihood function for $θ$ becomes

(1.1)

The corresponding estimator with lasso penalty is denoted by

^θn:=argminθ∈Θ{ln(θ)+λn^I(θ)},

where $^I (θ) := \sum_{k = 1}^{m} {^σ}_{k} | θ_{k} |$ is the weighted $l_{1}$

norm of the vector

θ \in R^{m}

, with random weights

{^σ}_{k} := {[1 / n \sum_{i = 1}^{n} ψ_{k}^{2} (X_{i})]}_{i}^{1 / 2} .

Clearly the negative log partial likelihood is a sum of non-iid random variables. For ease of theoretical calculation, it is natural to consider the following intermediate function as a “replacement” of the negative log partial likelihood function:

{~ l}_{n} (θ) = - \frac{1}{n} n \sum i = 1 {f_{θ} (X_{i}) - log μ (Y_{i}; f_{θ})} Δ_{i},

(1.2)

which has the desirable iid structure, but with an unknown population expectation

μ (t; f_{θ}) = E_{X, Y} {1 (Y \geq t) e^{f_{θ} (X)}} .

The negative log partial likelihood function (1.1) can then be viewed as a “working” model for the empirical loss function (1.2), and the corresponding loss function becomes

γ_{f_{θ}} = γ (f_{θ} (X), Y, Δ) := - {f_{θ} (X) - log μ (Y; f_{θ})} Δ,

(1.3)

with expected loss

l (θ) = - E_{Y, Δ, X} [{f_{θ} (X) - log μ (Y; f_{θ})} Δ] = P γ_{f_{θ}},

(1.4)

where $P$ denotes the distribution of $(Y, Δ, X)$ . Define the target function $¯ f$ by

¯ f := arg min f \in F P γ_{f},

where $F \supseteq F$ . For simplicity we will assume that there is a unique minimum as in van de Geer (2008). Uniqueness holds for the regular Cox model when $F = F$ , see for example, Andersen and Gill (1982). Define the excess risk of $f$ by

E (f) := P γ_{f} - P γ_{¯ f} .

It is desirable to show similar non-asymptotic oracle inequalities for the Cox regression model as in, for example, van de Geer (2008)

for generalized linear models. That is, with large probability,

Here $V_{θ}$ is called the “estimation error” by van de Geer (2008), which is typically proportional to $λ_{n}^{2}$ times the number of nonzero elements in $θ$ .

Note that the summands in the negative log partial likelihood function (1.1) are not iid, and the intermediate loss function $γ (\cdot, Y, Δ)$ given in (1.3) is not Lipschitz. Hence the conclusion of van de Geer (2008) can not be applied directly. With the Lipschitz condition in van de Geer (2008) replaced by a similar boundedness assumption for regression parameters in Bühlmann (2006), we tackle the problem using pointwise arguments to obtain the oracle bounds of two types of errors: one is between empirical loss (1.2) and expected loss (1.4), and one is between the negative log partial likelihood (1.1) and empirical loss (1.2).

The article is organized as follows. In Section 2, we provide assumptions and additional notation that will be used throughout the paper. In Section 3, following the flow of van de Geer (2008), we first consider the case where the weights $σ_{k} := {[E ψ_{k}^{2} (X)]}^{1 / 2}$ are fixed, then discuss briefly the case with random weights ${^σ}_{k}$ .

2 Assumptions

We impose five basic assumptions in this section. Assumptions A, B, and C are identical to the corresponding assumptions in van de Geer (2008). Assumption D has a similar flavor to the assumption (A2) in Bühlmann (2006) for the persistency property of boosting method in high-dimensional linear regression models. Here it replaces the Lipschitz assumption in van de Geer (2008). Assumption E is commonly used for survival models with censored data, see for example, Andersen and Gill (1982).

Assumption A. $K_{m} := {max}_{1 \leq k \leq m} {∥ ψ_{k} ∥_{\infty} / σ_{k}} < \infty .$

Assumption B. There exists an $η > 0$ and strictly convex increasing G, such that for all $θ \in Θ$ with $∥ f_{θ} - ¯ f ∥_{\infty} \leq η$ , one has $E (f_{θ}) \geq G (∥ f_{θ} - ¯ f ∥) .$

Assumption C. There exists a function $D (\cdot)$ on the subsets of the index set ${1, \dots, m}$ , such that for all $K \subset {1, \dots, m}$ , and for all $θ \in Θ$ and $~ θ \in Θ$ , we have $\sum_{k \in K} σ_{k} | θ_{k} - {~ θ}_{k} | \leq \sqrt{D (K)} ∥ f_{θ} - f_{~ θ} ∥ .$

Assumption D. $L_{m} := {sup}_{θ \in Θ} \sum_{k = 1}^{m} | θ_{k} | < \infty .$

Assumption E. The observation time stops at a finite time $τ > 0$ with $π := P (Y \geq τ) > 0.$

The convex conjugate of function $G$ given in Assumption B is denoted by $H$ such that $u v \leq G (u) + H (v)$ . A typical choice of $G$ is quadratic function with some constant $C_{0}$ , i.e. $G (u) = u^{2} / C_{0}$ , see van de Geer (2008).

From Assumptions A, D and E, we have for any $θ \in Θ$ ,

e^{| f_{θ} (X_{i}) |} \leq e^{K_{m} L_{m} σ_{(m)}} := U_{m} < \infty

(2.1)

for all $i$ , where $σ_{(m)} = {max}_{1 \leq k \leq m} σ_{k}$ .

Let $I (θ) := \sum_{k = 1}^{m} σ_{k} | θ_{k} |$ be the theoretical $l_{1}$ norm of $θ$ , and $^I (θ) := \sum_{k = 1}^{m} {^σ}_{k} | θ_{k} |$ be the empirical $l_{1}$ norm. For any $θ$ and $~ θ$ in $Θ$ , denote

I_{1} (θ | ~ θ) := \sum k : {~ θ}_{k} \neq 0 σ_{k} | θ_{k} |, I_{2} (θ | ~ θ) := I (θ) - I_{1} (θ | ~ θ) .

Similarly we have corresponding empirical versions,

{^I}_{1} (θ | ~ θ) := \sum k : {~ θ}_{k} \neq 0 {^σ}_{k} | θ_{k} |, {^I}_{2} (θ | ~ θ) :=^I (θ) - {^I}_{1} (θ | ~ θ) .

3 Main results

3.1 Non-random normalization weights in the penalty

We show that a similar result to Theorem A.4 of van de Geer (2008) holds for the Cox model. Suppose that $σ_{k} = [E ψ_{k}^{2}]^{1 / 2}$ are known and consider the estimator

^θn:=argminθ∈Θ{ln(θ)+λnI(θ)}.

Denote the empirical probability measure based on the sample ${(X_{i}, Y_{i}, Δ_{i}) : i = 1, \dots n}$ by $P_{n}$ . Let $ε_{1}, \dots, ε_{n}$ be a Rademacher sequence, independent of the training data $(X_{1}, Y_{1}, Δ_{1}), \dots, (X_{n}, Y_{n}, Δ_{n})$ . We fix some $θ^{*} \in Θ$ and denote $F_{M} := {f_{θ} : θ \in Θ, I (θ - θ^{*}) \leq M}$ for some $M > 0$ . For any $θ$ where $I (θ - θ^{*}) \leq M$ , denote

Z_{θ} (M) := ∣ ∣ (P_{n} - P) [γ_{f_{θ}} - γ_{f_{θ^{*}}}] ∣ ∣ = ∣ ∣ [{~ l}_{n} (θ) - l (θ)] - [{~ l}_{n} (θ^{*}) - l (θ^{*})] ∣ ∣ .

Note that van de Geer (2008) has considered the supremum of the above $Z_{θ} (M)$ over $Θ$ . We find that the pointwise argument is adequate for our purpose because only the lasso estimator is of interest, and that the calculation with ${sup}_{f \in F_{M}} Z_{θ} (M)$ in van de Geer (2008) does not apply to the Cox model due to the lack of Lipschitz property.

Lemma 3.1.

Under Assumptions A, D and E, for all $θ$ satisfying $I (θ - θ^{*}) \leq M$ , we have

E Z_{θ} (M) \leq {¯ a}_{n} M,

where

{¯ a}_{n} = 4 a_{n}, a_{n} = \sqrt{\frac{2 K_{m}^{2} log (2 m)}{n}} + \frac{}{K_{m} log (2 m)} n .

Proof.

By the symmetrization theorem, see e.g. van der Vaart and Wellner (1996) or Theorem A.2 in van de Geer (2008), for a class of only one function we have

$E Z_{θ} (M)$	$\leq$	$2 E (∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} {[f_{θ} (X_{i}) - log μ (Y_{i}; f_{θ})] Δ_{i}$
		$- [f_{θ^{}} (X_{i}) - log μ (Y_{i}; f_{θ^{}})] Δ_{i}} ∣ ∣ ∣)$
	$\leq$	$2 E (∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} {f_{θ} (X_{i}) - f_{θ^{*}} (X_{i})} Δ_{i} ∣ ∣ ∣)$
		$+ 2 E (∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} {log μ (Y_{i}; f_{θ}) - log μ (Y_{i}; f_{θ^{*}})} Δ_{i} ∣ ∣ ∣)$
	$=$	$A + B .$

For $A$ we have

A \leq 2 (m \sum k = 1 σ_{k} | θ_{k} - θ_{k}^{*} |) E (max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} ψ_{k} (X_{i}) / σ_{k} ∣ ∣ ∣) .

Applying Lemma A.1 in van de Geer (2008) with $η_{n} = K_{m}$ and $τ_{n}^{2} = K_{m}^{2}$ , we obtain

E (max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} \frac{ψ_{k} (X_{i})}{σ_{k}} ∣ ∣ ∣) \leq a_{n} .

Thus we have

A \leq 2 a_{n} M .

(3.1)

For $B$ , instead of using the contraction theorem that requires Lipschitz, we use the mean value theorem in the following:

	$∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} {log μ (Y_{i}; f_{θ}) - log μ (Y_{i}; f_{θ^{*}})} Δ_{i} ∣ ∣ ∣$
	$= ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} m \sum k = 1 \frac{1}{μ (Y_{i}; f_{θ^{* }})} \int_{Y_{i}}^{\infty} \int_{X} (θ_{k} - θ_{k}^{}) ψ_{k} (x) e^{f_{θ^{* *}} (x)} d P_{X, Y} (x, y) ∣ ∣ ∣$
	$= ∣ ∣ ∣ m \sum k = 1 σ_{k} (θ_{k} - θ_{k}^{}) \frac{1}{n} n \sum i = 1 \frac{ε_{i} Δ_{i}}{μ (Y_{i}; f_{θ^{ }}) σ_{k}} \int_{Y_{i}}^{\infty} \int_{X} ψ_{k} (x) e^{f_{θ^{ *}} (x)} d P_{X, Y} (x, y) ∣ ∣ ∣$
	$\leq ∣ ∣ ∣ m \sum k = 1 σ_{k} (θ_{k} - θ_{k}^{}) ∣ ∣ ∣ max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} F_{θ^{ *}} (k, Y_{i}) ∣ ∣ ∣$
	$\leq M max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} F_{θ^{* *}} (k, Y_{i}) ∣ ∣ ∣,$

where $θ^{* *}$ is between $θ$ and $θ^{*}$ , and

	$F_{θ^{* *}} (k, t)$	$=$	$\frac{E [1 (Y \geq t) ψ_{k} (X) e^{f_{θ^{* }} (X)}]}{μ (t; f_{θ^{ *}}) σ_{k}}$		(3.2)
		$\leq$	$\frac{(∥ ψ_{k} ∥_{\infty} / σ_{k}) E [1 (Y \geq t) e^{f_{θ^{* }} (X)}]}{μ (t; f_{θ^{ *}})} \leq K_{m} .$		(3.2)

Since for all $i$ ,

	$E [ε_{i} Δ_{i} F_{θ^{* }} (k, Y_{i})] = 0, ∥ ε_{i} Δ_{i} F_{θ^{ *}} (k, Y_{i}) ∥_{\infty} \leq K_{m}, and$
	$\frac{1}{n} n \sum i = 1 E [ε_{i} Δ_{i} F_{θ^{* }} (k, Y_{i})]^{2} \leq \frac{1}{n} n \sum i = 1 E [F_{θ^{ *}} (k, Y_{i})]^{2} \leq E K_{m}^{2} = K_{m}^{2},$

following Lemma A.1 in van de Geer (2008), we obtain

B \leq 2 a_{n} M .

(3.3)

Combining (3.1) and (3.3), the upper bound for $E Z_{θ} (M)$ is achieved. ∎

We now can bound $Z_{θ} (M)$ using the Bousquet’s concentration theorem provided in van de Geer (2008) as Theorem A.1.

Corollary 3.1.

Under Assumptions A, D and E, for all $M > 0$ , $r_{1} > 0$ and all $θ$ satisfying $I (θ - θ^{*}) \leq M$ , it holds that

P (Z_{θ} (M) \geq {¯ λ}_{n, 0}^{A} M) \leq exp (- n {¯ a}_{n}^{2} r_{1}^{2}),

where

{¯ λ}_{n, 0}^{A} := {¯ λ}_{n, 0}^{A} (r_{1}) := {¯ a}_{n} (1 + 2 r_{1} \sqrt{2 (K_{m}^{2} + {¯ a}_{n} K_{m})} + \frac{4 r_{1}^{2} {¯ a}_{n} K_{m}}{3})

Proof.

Using the triangular inequality and the mean value theorem, we obtain

$\| γ_{f_{θ}} - γ_{f_{θ^{*}}} \|$	$\leq$	$\| f_{θ} (X) - f_{θ^{}} (X) \| Δ + \| log μ (Y; f_{θ}) - log μ (Y; f_{θ^{}}) \| Δ$
	$\leq$	$∣ ∣ ∣ ∣ m \sum k = 1 σ_{k} \| θ_{k} - θ_{k}^{} \| \frac{ψ_{k} (X)}{σ_{k}} ∣ ∣ ∣ ∣ + \| log μ (Y; f_{θ}) - log μ (Y; f_{θ^{}}) \|$
	$\leq$	$M K_{m} + m \sum k = 1 σ_{k} \| θ_{k} - θ_{k}^{} \| \cdot max 1 \leq k \leq m \| F_{θ^{ *}} (k, Y) \|$
	$\leq$	$2 M K_{m},$

where $θ^{* *}$ is between $θ$ and $θ^{*}$ , $F_{θ^{* *}} (k, Y)$ is defined in (3.2). So we have

∥ γ_{f_{θ}} - γ_{f_{θ^{*}}} ∥_{\infty} \leq 2 M K_{m},

and

P (γ_{f_{θ}} - γ_{f_{θ^{*}}})^{2} \leq 4 M^{2} K_{m}^{2} .

Therefore, in view of Bousquet’s concentration theorem and Lemma 3.1, for all $M > 0$ and $r_{1} > 0$ ,

	$P (Z_{θ} (M) \geq {¯ a}_{n} M (1 + 2 r_{1} \sqrt{2 (K_{m}^{2} + {¯ a}_{n} K_{m})} + \frac{4 r_{1}^{2} {¯ a}_{n} K_{m}}{3}))$
	$\leq exp (- n {¯ a}_{n}^{2} r_{1}^{2}) .$

∎

Now for any $θ$ satisfying $I (θ - θ^{*}) \leq M$ , we bound

R_{θ} (M) := ∣ ∣ [l_{n} (θ) - {~ l}_{n} (θ)] - [l_{n} (θ^{*}) - {~ l}_{n} (θ^{*})] ∣ ∣,

which is equal to

	$\frac{1}{n} n \sum i = 1 ∣ ∣ ∣ [log \frac{1}{n} n \sum j = 1 \frac{1 (Y_{j} \geq Y_{i}) e^{f_{θ} (X_{j})}}{μ (Y_{i}; f_{θ})} - log \frac{1}{n} n \sum j = 1 \frac{1 (Y_{j} \geq Y_{i}) e^{f_{θ^{}} (X_{j})}}{μ (Y_{i}; f_{θ^{}})}] Δ_{i} ∣ ∣ ∣$
	$\leq sup 0 \leq t \leq τ ∣ ∣ ∣ log \frac{1}{n} n \sum j = 1 \frac{1 (Y_{j} \geq t) e^{f_{θ} (X_{j})}}{μ (t; f_{θ})} - log \frac{1}{n} n \sum j = 1 \frac{1 (Y_{j} \geq t) e^{f_{θ^{}} (X_{j})}}{μ (t; f_{θ^{}})} ∣ ∣ ∣ .$

By the mean value theorem, we have

$R_{θ} (M)$	$\leq$	$sup 0 \leq t \leq τ ∣ ∣ ∣ m \sum k = 1 (θ_{k} - θ_{k}^{}) {\frac{\sum_{j = 1}^{n} 1 (Y_{j} \geq t) e^{f_{θ^{ }} (X_{j})}}{μ (t; f_{θ^{ *}})}}^{- 1}$
		${\frac{\sum_{j = 1}^{n} 1 (Y_{j} \geq t) ψ_{k} (X_{j}) e^{f_{θ^{* }} (X_{j})}}{μ (t; f_{θ^{ *}})}$
		$- \frac{\sum_{j = 1}^{n} 1 (Y_{j} \geq t) e^{f_{θ^{* }} (X_{j})} E [1 (Y \geq t) ψ_{k} (X) e^{f_{θ^{ }} (X)}]}{μ (t; f_{θ^{ *}})^{2}}} ∣ ∣ ∣$
	$=$	$sup 0 \leq t \leq τ ∣ ∣ ∣ m \sum k = 1 σ_{k} (θ_{k} - θ_{k}^{}) {\frac{\sum_{j = 1}^{n} 1 (Y_{j} \geq t) {ψ_{k} (X_{j}) / σ_{k}} e^{f_{θ^{ }} (X_{j})}}{\sum_{j = 1}^{n} 1 (Y_{j} \geq t) e^{f_{θ^{ *}} (X_{j})}}$

	$\leq$	$M sup 0 \leq t \leq τ [\frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) e^{f_{θ^{* *}} (X_{i})}]^{- 1}$

		$- E [1 (Y \geq t) {ψ_{k} (X) / σ_{k}} e^{f_{θ^{* *}} (X)}] ∣ ∣ ∣$
		$+ K_{m} ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) e^{f_{θ^{* }} (X_{i})} - E [1 (Y \geq t) e^{f_{θ^{ *}} (X)}] ∣ ∣ ∣},$

where $θ^{* *}$ is between $θ$ and $θ^{*}$ , and by (2.1) we have

sup 0 \leq t \leq τ [\frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) e^{f_{θ^{* *}} (X_{i})}]^{- 1} \leq U_{m} [\frac{1}{n} n \sum i = 1 1 (Y_{i} \geq τ)]^{- 1} .

(3.5)

Lemma 3.2.

Under Assumption E, we have

P (\frac{1}{n} n \sum i = 1 1 (Y_{i} \geq τ) \leq \frac{π}{2}) \leq 2 e^{- n π^{2} / 2} .

Proof.

This is obtained directly from Massart (1990) by taking $r = π \sqrt{n} / 2$ in the following:

	$P (\frac{1}{n} n \sum i = 1 1 (Y_{i} \geq τ) \leq \frac{π}{2})$	$\leq$	$P (sup 0 \leq t \leq τ \sqrt{n} ∣ ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) - π ∣ ∣ ∣ ∣ \geq r)$
		$\leq$	$2 e^{- 2 r^{2}} .$

∎

Lemma 3.3.

Under Assumptions A, D and E, for all $θ$ we have

	$P (sup 0 \leq t \leq τ ∣ ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) e^{f_{θ} (X_{i})} - μ (t; f_{θ}) ∣ ∣ ∣ ∣ \geq U_{m} {¯ a}_{n} r_{1})$		(3.6)
	$\leq \frac{1}{5} W^{2} e^{- n {¯ a}_{n}^{2} r_{1}^{2}},$

where $W$ is a constant that only depends on $K = \sqrt{2}$ .

Proof.

For a class of functions indexed by $t$ , $F = {1 (y \geq t) e^{f_{θ} (x)} / U_{m} : t \in [0, τ], y \in R, e^{f_{θ} (x)} \leq U_{m}}$ , we calculate its bracketing number. For any $ϵ > 0$ , let $t_{i}$ be the $i$ -th $⌈ 1 / ε ⌉$ quantile of $Y$ , i.e.,

P (Y \leq t_{i}) = i ε, i = 1, \dots, ⌈ 1 / ε ⌉ - 1,

where $⌈ x ⌉$ is the smallest integer that is greater than or equal to $x$ . Furthermore, denote $t_{0} = 0$ and $t_{⌈ 1 / ε ⌉} = + \infty$ . For $i = 1, \dots, ⌈ 1 / ε ⌉$ , define brackets $[L_{i}, U_{i}]$ with

L_{i} (x, y) = 1 (y \geq t_{i}) e^{f_{θ} (x)} / U_{m}, U_{i} (x, y) = 1 (y > t_{i - 1}) e^{f_{θ} (x)} / U_{m}

such that $L_{i} (x, y) \leq 1 (y \geq t) e^{f_{θ} (x)} / U_{m} \leq U_{i} (x, y)$ when $t_{i - 1} < t \leq t_{i}$ . Since

	${E [U_{i} - L_{i}]^{2}}_{i}^{1 / 2}$	$\leq$	${⎧ ⎨ ⎩ E {[\frac{e^{f_{θ} (X)}}{U_{m}} {1 (Y \geq t_{i}) - 1 (Y > t_{i - 1})}]}_{i}^{2} ⎫ ⎬ ⎭}^{1 / 2}$
		$\leq$	${P (t_{i - 1} < Y \leq t_{i})}_{i - 1}^{1 / 2} = \sqrt{ε},$

we have $N_{[]} (\sqrt{ε}, F, L_{2}) \leq 2 / ε$ , which yields

N_{[]} (ε, F, L_{2}) \leq \frac{2}{ε^{2}} = {(\frac{K}{ε})}^{2},

where $K = \sqrt{2}$ . Thus, from Theorem 2.14.9 in van der Vaart and Wellner (1996), we have for any $r > 0$ ,

	$P (\sqrt{n} sup 0 \leq t \leq τ ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 \frac{1 (Y_{i} \geq t) e^{f_{θ} (X_{i})}}{U_{m}} - \frac{μ (t; f_{θ})}{U_{m}} ∣ ∣ ∣ \geq r)$	$\leq$	$\frac{1}{2} W^{2} r^{2} e^{- 2 r^{2}}$
		$\leq$	$\frac{1}{5} W^{2} e^{- r^{2}},$

where $W$ is a constant that only depends on $K$ . Note that $r^{2} e^{- r^{2}}$ is bounded by $e^{- 1}$ . Let $r = \sqrt{n} {¯ a}_{n} r_{1}$ , we obtain (3.6).

∎

Lemma 3.4.

Under Assumptions A, D and E, for all $θ$ we have

	$P (sup 0 \leq t \leq τ max 0 \leq k \leq m ∣ ∣ ∣ \frac{}{1} n n \sum i = 1 1 (Y_{i} \geq t) \frac{ψ_{k} (X_{i})}{σ_{k}} e^{f_{θ} (X_{i})}$
	$- E [1 (Y \geq t) \frac{ψ_{k} (X)}{σ_{k}} e^{f_{θ} (X)}] ∣ ∣ ∣ \geq K_{m} U_{m} [{¯ a}_{n} r_{1} + \sqrt{\frac{log (2 m)}{n}}])$
	$\leq \frac{1}{10} W^{2} e^{- n {¯ a}_{n}^{2} r_{1}^{2}} .$		(3.7)

Proof.

Consider the classes of functions indexed by $t$ ,

	$G^{k}$	$=$	${1 (y \geq t) e^{f_{θ} (x)} ψ_{k} (x) / (σ_{k} K_{m} U_{m}) : t \in [0, τ], y \in R,$
			$∣ ∣ e^{f_{θ} (x)} ψ_{k} (x) / σ_{k} ∣ ∣ \leq K_{m} U_{m}}, k = 1, \dots, m .$

Using the same argument in the proof of Lemma 3.3, we have

N_{[]} (ε, G^{k}, L_{2}) \leq {(\frac{K}{ε})}^{2},

where $K = \sqrt{2}$ , and then for any $r > 0$ ,

	$P (\sqrt{n} sup 0 \leq t \leq τ ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 \frac{1 (Y_{i} \geq t) e^{f_{θ} (X_{i})} ψ_{k} (X_{i})}{σ_{k} K_{m} U_{m}}$
	$- E [\frac{1 (Y \geq t) e^{f_{θ} (X)} ψ_{k} (X)}{σ_{k} K_{m} U_{m}}] ∣ ∣ ∣ \geq r) \leq \frac{1}{5} W^{2} e^{- r^{2}} .$

Thus we have


	$- E [1 (Y \geq t) e^{f_{θ} (X)} ψ_{k} (X) / (σ_{k} U_{m} K_{m})] ∣ ∣ ∣ \geq r)$
	$\leq P (m ⋃ k = 1 \sqrt{n} sup 0 \leq t \leq τ ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 1 (Y_{i} \geq t) e^{f_{θ} (X_{i})} ψ_{k} (X_{i}) / (σ_{k} U_{m} K_{m})$
	$- E [1 (Y \geq t) e^{f_{θ} (X)} ψ_{k} (X) / (σ_{k} U_{m} K_{m})] ∣ ∣ ∣ \geq r)$

	$- E [1 (Y \geq t) e^{f_{θ} (X)} ψ_{k} (X) / (σ_{k} U_{m} K_{m})] ∣ ∣ ∣ \geq r)$
	$\leq \frac{m}{5} W^{2} e^{- r^{2}} = \frac{1}{10} W^{2} e^{log (2 m) - r^{2}} .$

Let $log (2 m) - r^{2} = - n {¯ a}_{n}^{2} r_{1}^{2}$ , i.e. $r = \sqrt{n {¯ a}_{n}^{2} r_{1}^{2} + log (2 m)}$ . Since

\sqrt{{¯ a}_{n}^{2} r_{1}^{2} + \frac{log (2 m)}{n}} \leq

	$∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} {log μ (Y_{i}; f_{θ}) - log μ (Y_{i}; f_{θ^{*}})} Δ_{i} ∣ ∣ ∣$
	$= ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} m \sum k = 1 \frac{1}{μ (Y_{i}; f_{θ^{* }})} \int_{Y_{i}}^{\infty} \int_{X} (θ_{k} - θ_{k}^{}) ψ_{k} (x) e^{f_{θ^{* *}} (x)} d P_{X, Y} (x, y) ∣ ∣ ∣$
	$= ∣ ∣ ∣ m \sum k = 1 σ_{k} (θ_{k} - θ_{k}^{}) \frac{1}{n} n \sum i = 1 \frac{ε_{i} Δ_{i}}{μ (Y_{i}; f_{θ^{ }}) σ_{k}} \int_{Y_{i}}^{\infty} \int_{X} ψ_{k} (x) e^{f_{θ^{ *}} (x)} d P_{X, Y} (x, y) ∣ ∣ ∣$
	$\leq ∣ ∣ ∣ m \sum k = 1 σ_{k} (θ_{k} - θ_{k}^{}) ∣ ∣ ∣ max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} F_{θ^{ *}} (k, Y_{i}) ∣ ∣ ∣$
	$\leq M max 1 \leq k \leq m ∣ ∣ ∣ \frac{1}{n} n \sum i = 1 ε_{i} Δ_{i} F_{θ^{* *}} (k, Y_{i}) ∣ ∣ ∣,$

Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lasso

Authors

Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications

Heterogeneous Overdispersed Count Data Regressions via Double Penalized Estimations

Communication-efficient sparse regression: a one-shot approach

Generalized Linear Models with Structured Sparsity Estimators

Deterministic Inequalities for Smooth M-estimators

Kullback-Leibler aggregation and misspecified generalized linear models

On the prediction loss of the lasso in the partially labeled setting

1 Introduction

2 Assumptions

3 Main results

3.1 Non-random normalization weights in the penalty

Lemma 3.1.

Proof.

Corollary 3.1.

Proof.

Lemma 3.2.

Proof.

Lemma 3.3.

Proof.

Lemma 3.4.

Proof.

Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lasso

Authors

Related Research

Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications

Heterogeneous Overdispersed Count Data Regressions via Double Penalized Estimations

Communication-efficient sparse regression: a one-shot approach

Generalized Linear Models with Structured Sparsity Estimators

Deterministic Inequalities for Smooth M-estimators

Kullback-Leibler aggregation and misspecified generalized linear models

On the prediction loss of the lasso in the partially labeled setting

This week in AI

1 Introduction

2 Assumptions

3 Main results

3.1 Non-random normalization weights in the penalty

Lemma 3.1.

Proof.

Corollary 3.1.

Proof.

Lemma 3.2.

Proof.

Lemma 3.3.

Proof.

Lemma 3.4.

Proof.