Adaptive estimation of the excess D.F.

Ion Grama

Université de Bretagne Sud

Laboratoire SABRES Vannes, France

EVA 2005, Gothenburg, August 15-19

Outline

• Introduction and formulation of the problem • Rates of convergence for adaptive estimation • Fitting Pareto d.f. to the tail

Introduction 1

• Explicit computation of the rates of convergence • Simulation results Hall and Welsh (1984). Let F be a d.f. in a neighborhood of Pareto d.f. with index β = 1/α such that

f_{F} (x) = dαx^{−(α+1)} (1 + r (x)) , |r (x)| ≤ Ax^{−αρ}, x ≥ 0,

where |α − α_{0}| ≤ ε, |d − d_{0}| ≤ ε and α_{0}, d_{0}, ε, ρ, A > 0.

⋆ The optimal rate of convergence that can be achieved for estimating α = 1/β is n^{−ρ/(2ρ+1)}. ⋆ This rate is attained for the Hill estimator h_{n,k_{n}} with choice k_{n} ∼ n^{2ρ/(2ρ+1)} depending on ρ.

Dress (1998) gives extensions: Allows arbitrary rates of decay for |r (x)| and constructs sequences k_{n} such that h_{n,k_{n}} attains the optimal rates. Extend the class of d.f. to other types of neighborhoods.

• Remark: There results permits to construct an adaptive estimator by estimating ρ or the rate of |r (x)| and implementing this estimate into the optimal k_{n}. However one needs to know to which class of distributions F belongs. Or generally this information is not available in practice.

Several adaptive procedures have been proposed: Hall and Welsh (1985), Hall (1990), Drees and Kaufman (1998), Matthys and Beirlant (2000), Danielson, de Haan, Pang and de Vries (2001), Guillou and Hall (2001), Gomes and Oliveira (2001), Hisman, Koedijk, Kool and Palm (2001), Beirlant, Goegebeur, Segers and Teugels (2004).

Main drawback: Apriory information on F is needed.

Our goal

1. Give a selection procedure which adapts to the structure of F and attains exactly or nearly optimal rates in the above class of distributions or/and in some larger classes. Otherwise we want to obtain adaptive rates of convergence. Sometimes these types of bounds are called ”oracle” type inequalities.

2. Give some results for estimating the distribution F itself, more precisely for estimating the excess d.f. F_{t} over the threshold t.

Construction of a family of estimators

Observations

X_{1}, ..., X_{n}, i.i.d. r.v.’s with common d.f. F supported on [x_{0}, ∞), x_{0} > 0.

Let F_{t} (x) = 1− \frac{1−F (tx)}{1−F (t)} be the excess d.f. over the threshold t ≥ x_{0}, tied to [1, ∞), and f_{F_{t}} be its density. Consider the family of estimators

θ̂_{n,t} = \frac{1}{n̂_{t}} ∑_{X_{i}>t} log \frac{X_{i}}{t}, t ≥ x_{0},

where n̂_{t} is the number of observations in the interval (t, ∞). It is obtained by maximizing in θ the local quasi log-likelihood corresponding to observations X_{i} > t,

L_{n,t} (F) = ∑_{X_{i}>t} log f_{F_{t}} (X_{i}/t) ,

under the condition that F_{t} is exactly Pareto P_{θ} with θ > 0 and t ≥ x_{0}.

Note that θ̂_{n,X_{n,k}} is the Hill estimator ĥ_{n,k}, where X_{n,1} ≥ ... ≥ X_{n,n} are the order statistics.

Adaptive estimator

Using the family θ̂_{n,t}, t ≥ x_{0} define the adaptive estimator:

θ̂_{n} = θ̂_{n,τ̂_{n}},

where the adaptive location of the tail τ̂_{n} is defined below.

⋆ The idea is to test whether F_{t} belongs to Pareto family against the alternative that F_{t} belongs to the Pareto change point family for all locations t on a grid t_{1} ≥ ... ≥ t_{M_{n}}, with M_{n} ≤ n. The adaptive location τ̂_{n} will be the first detected change point on the grid.

Compare with Drees and Kaufman (1998) [see also Lepski (1991, 1992)]

• another testing procedure

• τ̂_{n} = last location s on the grid when the testing procedure accepts the null hypothesis

Proposed in Grama and Spokoiny (2003) (WIAS preprint) see also Mercurio and Spokoiny (2004) (WIAS preprint) for a time series context.

Adaptive selection of the location of the tail

Define the local quasi log-likelihood ratio test statistic

T_{n,t,τ} = sup_{F ∈P_{t,τ}^{cp}} L_{n,t} (F) − sup_{F ∈P_{t}} L_{n,t} (F) , x_{0} ≤ t < τ,

where:

• P_{t} is the set of d.f. F such that F_{t} is Pareto.

• P_{t,τ}^{CP} is the set of d.f. F such that F_{t} is Pareto change point d.f. with change point at some τ > 1. Let

t̂_{n} ≡ tm̂_{n} = min{t_{m} : T_{n,t_{m}} > λ_{n}},

where

T_{n,t} = max_{t_{k}:ρm≤k≤(1−ρ)m} T_{n,t,t_{k}},

and λ_{n} = μ_{0}+μ_{1} log n is the critical value of the procedure, μ_{i} = const, ρ ∈ (0, \frac{1}{2}). The adaptive location is defined as

τ̂_{n} = arg max_{ρm̂_{n}≤k≤(1−ρ)m̂_{n}} T_{n,t̂_{n},t_{k}}.

A technical assumption on the grid

For some positive constants ρ_{1}, ρ_{2} it holds

ρ_{1} ≤ \frac{n_{t_{m}}}{m} ≤ ρ_{2},

where n_{t_{m}} = n(1 − F (t_{m})) is the expected number of observation in the interval [t_{m}, ∞).

In practice:

• choose F_{0} and set t_{m} as the inverse transformation F_{0}^{−1} of a uniform grid on [0, 1].

• set t_{m} = X_{n,m}, where X_{n,1} ≥ ... ≥ X_{n,n} are the order statistics (in progress).

Main result: adaptive estimator

Compare the performance of the adaptive estimator θ̂_{n} with that of the family θ_{n,t}, t ≥ x_{0} for some values of t.

Theorem 1. Assume that there are sequences τ_{n} and θ_{n} such that \frac{log n}{n(1−F (τ_{n}))} → 0 and

∫_{1}^{∞} log^{2} \frac{f_{F_{τn}} (x)}{p_{θ_{n}} (x)} exp (|log \frac{f_{F_{τn}} (x)}{p_{θ_{n}} (x)}|) F_{τ_{n}} (dx) ≤ ε_{0}\frac{log n}{n (1 − F (τ_{n}))}, (A)

where ε_{0} = const. Then

P_{F} (log^{2} \frac{θ̂_{n}}{θ̂_{n,τ_{n}}} ≤ c\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞,

where c = const depends on ε_{0} and ρ_{i}. ⋆ One requires that the excess d.f. F_{τ_{n}} belongs to a certain neighborhood of Pareto distribution P_{θ_{n}} which is expressed in therms of the log-likelihood ratio [compare with Hall and Welsh (1984) or Drees (1998)]. ⋆ θ̂_{n} and θ̂_{n,τ_{n}} are close with high probability. The rate of convergence of their discrepancy depends on the unknown d.f. F and location τ_{n}.

For applications we need rates of convergence for non-adaptive estimator θ̂_{n,τ_{n}}.

Rates for non-adaptive estimator θ̂_{n,τ_{n}}

Theorem 2. Assume the same condition (A) as in Theorem 1 i.e. there are sequences τ_{n} and θ_{n} such that \frac{log n}{n(1−F (τ_{n}))} → 0 and

∫_{1}^{∞} log^{2} \frac{f_{F_{τn}} (x)}{p_{θ_{n}} (x)} exp (|log \frac{f_{F_{τn}} (x)}{p_{θ_{n}} (x)}|) F_{τ_{n}} (dx) ≤ ε_{0}\frac{log n}{n (1 − F (τ_{n}))}, (A)

where ε_{0} = const. Then a)

P_{F} (log^{2} \frac{θ̂_{n,τ_{n}}}{θ_{n}} ≤ c_{1}\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞,

where c_{1} = const depends on ε_{0} and ρ_{i}.

b) Combining this with the previous result on rate of convergence for the adaptive estimator θ̂_{n} we get

P_{F} (log^{2} \frac{θ̂_{n}}{θ_{n}} ≤ c_{2}\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞,

where c_{2} = const. The latter means that the adaptive estimator has at least the same rate of convergence as θ̂_{n,τ_{n}} for estimating θ_{n}. But we need some insight on θ_{n}.

Illustration 1

Assume that F is regularly varying with index γ : F (x) = cx^{−1/γ}L (x) . In general the condition (A) of Theorem 1 need not be satisfied with θ_{n} = γ. But it can happen that it is satisfied with θ_{n} = γ.

Theorem 3. Assume it is the case, i.e. there is a sequence τ_{n} such that \frac{log n}{n(1−F (τ_{n}))} → 0 and

∫_{1}^{∞} log^{2} \frac{f_{F_{τn}} (x)}{p_{γ} (x)} exp (|log \frac{f_{F_{τn}} (x)}{p_{γ} (x)}|) F_{τ_{n}} (dx) ≤ ε_{0}\frac{log n}{n (1 − F (τ_{n}))}, (A)

where ε_{0} = const. Then

P_{F} (log^{2} \frac{θ̂_{n,τ_{n}}}{γ} ≤ c_{1}\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞.

The adaptive estimator θ̂_{n} attains the same rate for estimating γ.

Assume that F is the same. Let P be the set of Pareto d.f. and K (P, Q) the Kullback-Leibler distance between P and Q, P ≪ Q. Fix a t ≥ x_{0} and ”project” F_{t} on the set P by choosing the closest element in P, say P_{θ_{t}(F)}, where

θ_{t} (F) = arg min_{θ} K (P_{θ}, F_{t}) = ∫_{t}^{∞} log xF_{t} (dx) .

Call θ_{t} (F) fitted Pareto index. It is plotted on the next page.

Theorem 4. Assume the condition (A) of Theorem 1 is satisfied with θ_{n} = θ_{τ_{n}} (F) . Then

P_{F} (log^{2} \frac{θ̂_{n,τ_{n}}}{θ_{τ_{n}} (F)} ≤ c_{1}\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞.

The adaptive estimator θ̂_{n} attains the same rate for estimating θ_{τ_{n}} (F) .

Thus, under given conditions θ̂_{n,τ_{n}} is close to both γ and θ_{τ_{n}} (F) at the same time! What is the difference?

Examples of fitted Pareto indices

1 - Pareto; 2 - log perturbed Pareto; 3 - loggamma; 4 - Cauchy; 5,6,7 - Hall model;

Hill plot and fitted parameter

1 - Hill plot; 2 - index of regular variation; 3 - fitted Pareto index.

Hill plot and fitted parameter

1 - Hill plot; 2 - index of regular variation; 3 - fitted Pareto index.

Fisher-Trippet-Gnedenko theorem

Let F_{t} (x) be the excess d.f. tied to the interval [1, ∞) :

F_{t} (x) = P (X ≤ xt|X > t) = 1 − \frac{1 − F (xt)}{1 − F (t)}, x ≥ 1.

F ∈ MDA(Φ_{α}) iff for any x > 1,

F_{t} (x) → P_{β} (x) ≡ 1 − x^{−1/β}, as t → ∞,

where β = 1/α is called the index of regular variation. Usually the goal is to estimate β and use P_{β} (·) as an approximation of F_{t} (·) .

Our goal

Recover F_{t} (·) directly from the observations using an accompanying Pareto approximation

F_{t} (·) ≈ P_{θ} (·) ,

where θ is generally different from β.

Main result: estimation of the excess d.f.

The estimation of θ_{t} (F) can be related to the estimation of F_{t}. Note that

K (P_{θ}, F_{t}) = K (P_{θ_{t}(F)}, F_{t}) + G (\frac{θ}{θ_{t} (F)} − 1) ,

where G (x) = x − log(1 + x) ∼ x^{2}, x → 0. Using G (x − 1) ∼ log^{2} x, as x → 0, and

K (P_{θ_{t}(F)}, F_{t}) ≤ c ∫_{1}^{∞} log^{2} \frac{f_{F_{t}} (x)}{p_{θ_{t}(F)} (x)} exp (|log \frac{f_{F_{t}} (x)}{p_{θ_{t}(F)} (x)}|) F_{t} (dx)

one can deduce from Theorem 4 Theorem 5. Assume that condition (A) from Theorem 1 holds true with θ_{n} = θ_{τ_{n}} (F) , i.e. there is a sequence τ_{n} such that \frac{log n}{n(1−F (τ_{n}))} → 0 and

∫_{1}^{∞} log^{2} \frac{f_{F_{τn}} (x)}{p_{θ_{τn}(F)} (x)} exp (|log \frac{f_{F_{τn}} (x)}{p_{θ_{τn}(F)} (x)}|) F_{τ_{n}} (dx) ≤ ε_{0}\frac{log n}{n (1 − F (τ_{n}))}, (A)

where ε_{0} = const. Then

P_{F} (K (Pθ̂_{n,τn}, F_{τ_{n}}) ≤ c_{1}\frac{log n}{n (1 − F (τ_{n}))}) → 1, n → ∞.

The same rate can be obtained with adaptive estimator θ̂_{n} replacing θ̂_{n,τ_{n}}.

Explicit computation of the rate: preparation

We need some notations:

F (x) = 1 − exp (− ∫_{x_{0}}^{x} \frac{dt}{tα_{F} (t)}) ,

where α_{F} (x) is the tail index function

α_{F} (x) = \frac{1}{xλ_{F} (x)},

and λ_{F} (x) = \frac{f_{F} (x)}{1−F (x)} is the hazard rate. Condition (A) is satisfied if, there are constants r_{i} > 0 and a sequence τ_{n} such that

∫_{1}^{∞} x^{r_{0}}F_{τ_{n}} (dx) ≤ r_{1}

and

\frac{1}{θ_{τ_{n}}^{2} (F)} sup_{x≥τ_{n}} log^{2} \frac{θ_{τ_{n}} (F)}{α_{F} (x)} ≤ r_{2}\frac{log n}{n (1 − F (τ_{n}))}.

Computing the rate: Example 1

Let F (x) = 1 − c_{β}x^{−1/β} log x, x ≥ x_{0}. By straightforward calculations:

θ_{t} (F) = β (1 + \frac{β}{log t}) and α_{F} (x) = β (1 − \frac{β}{log x})^{−1} .

Then the bias term is

sup_{x≥t} |log \frac{θ_{t} (F)}{α_{F} (x)}| = O (\frac{β}{log t}) , for t ≥ c (β) .

To determine the best location τ_{n} we have the balance equation:

\frac{β}{log τ_{n}} = ε_{0} (\frac{log n}{n (1 − F (τ_{n}))})^{1/2} .

Since n (1 − F (t)) = cnt^{−1/β} log t, the latter gives τ_{n} ∼ \frac{n^{β}}{log^{2} n} and thus

sup_{x≥τ_{n}} |log \frac{θ_{τ_{n}} (F)}{α_{F} (x)}| = O (\frac{1}{log τ_{n}}) = O (\frac{1}{log n}) .

Then the previous results imply that the adaptive estimator attains the rate \frac{1}{log n} :

P_{F} (K (Pθ̂_{n}, F_{τ_{n}}) ≤ c_{1}\frac{1}{log^{2} n}) → 1, n → ∞.

Example 1 (continued)

In the same way one can show that

P_{F} (|log \frac{θ̂_{n}}{β}| ≤ c_{1}\frac{1}{log n}) → 1, n → ∞.

According to the results in Drees (1998) the best achievable rate for estimating β is \frac{1}{log n}, so our adaptive estimator is rate optimal.

Computing the rate: Example 2

Let F (x) = 1 − c_{β}x^{−1/β} − c_{γ}x^{−1/γ}, where β = γ + α, α, β, γ > 0, x ≥ x_{0}. By straightforward calculations:

θ_{t} (F) = \frac{βc_{β}t^{−1/β} + γc_{γ}t^{−1/γ}}{c_{β}t^{−1/β} + c_{γ}t^{−1/γ}} and α_{F} (x) = \frac{c_{β}x^{−1/β} + c_{γ}x^{−1/γ}}{β^{−1}c_{β}x^{−1/β} + γ^{−1}c_{γ}x^{−1/γ}}.

Then

sup_{x≥t} |log \frac{θ_{t} (F)}{α_{F} (x)}| = O (t^{−\frac{1}{γ}+\frac{1}{β}}) , as t → ∞.

Then for the best location τ_{n} we have the balance equation:

τ_{n}^{−\frac{1}{γ}+\frac{1}{β}} = ε_{0} (\frac{log n}{n (1 − F (τ_{n}))})^{1/2}

Since n (1 − F (t)) ∼ nt^{−1/β}, the latter gives t ∼ (\frac{n}{log n})^{\frac{βγ}{2β−γ}} and thus

sup_{x≥t} |log \frac{θ_{t} (F)}{α_{F} (x)}| = O (t^{−\frac{β−γ}{βγ}}) = O ((\frac{log n}{n})^{\frac{β−γ}{2β−γ}} ) = O ((\frac{log n}{n})^{\frac{α}{β+α}}) ,

where β ≥ α.

Example 2 (continued)

Then the previous results imply that the adaptive estimator attains the rate (\frac{log n}{n})^{\frac{α}{β+α}} :

P_{F} (K (Pθ̂_{n}, F_{τ_{n}}) ≤ c_{1} (\frac{log n}{n})^{\frac{2α}{β+α}}) → 1, n → ∞.

For estimating β we have in the same way

P_{F} (|log \frac{θ̂_{n}}{β}| ≤ c_{1} (\frac{log n}{n})^{\frac{α}{β+α}}) → 1, n → ∞.

According to the results in Hall and Welsh (1984) the best rate achievable in the class of functions mentioned in the beginning of the talk is n^{−ρ/(2ρ+1)}. The d.f. F (x) = 1 − c_{β}x^{−1/β} − c_{γ}x^{−1/γ} satisfies that condition with γ = β^{−1} (1 + ρ)^{−1} . Since

\frac{α}{β + α} = \frac{2β − 2γ}{2β − γ} = \frac{ρ}{2ρ + 1}

we see that our adaptive estimator attains the optimal rate up to a log n factor.

Simulations: Quantile estimation

We use Weissman (1975) estimator: for p ∈ (0, 1)

q̂_{k,p} = X_{n,k} (\frac{1 − F̂_{n} (X_{n,k})}{1 − p})ĥ_{n,k} ,

and its adaptive version

q̂_{p} = τ̂_{n} (\frac{1 − F̂_{n} \binom{}{τ_{n}}}{1 − p})θ̂_{n} .

with τ̂_{n} and θ̂_{n} given by our CP-adapive procedure. The quality of estimation is measured by the MSE

σ^{2} (q̂, q) = \frac{1}{n} ∑_{i=1}^{n} (log q̂− log q)^{2} .

Quantile estimation: Cauchy

F = Cauchy d.f.; n = 1000, p = 1 − 1/n = 0.999; # MC simulations = 2000. 1- plot of RMSE σ (q̂_{k,p}, q_{p}) as a function of k; 2 - RMSE σ (q̂_{p}, q_{p}) of adaptive estimator.

Quantile estimation: loggamma

F = loggamma d.f.; n = 1000, p = 1 − 1/n = 0.999; # MC simulations = 2000. 1- plot of RMSE σ (q̂_{k,p}, q_{p}) as a function of k; 2 - RMSE σ (q̂_{p}, q_{p}) of adaptive estimator.

Quantile estimation

We give the values of the ratio r_{p} = σ (q̂_{p}, q_{p}) / min_{k} σ (q̂_{k,p}, q_{p}) as a function of p ∈ [0.9, 0.999999].

Values of r_{p}
p  0.9 0.99 0.999 0.9999 0.99999 0.999999
Cauchy 1.0333 1.0771 1.1073 1.1197 1.1263 1.1305
loggamma 1.0841 1.1772 1.2529 1.2731 1.2782 1.2786

The increase over minimal RMSE introduced by the adaptive procedure for estimating

q_{p}, p ∈ [1 − 100/n, 1 − 0.001/n] ≡ [0.9, 0.999999]

does not exceed 13% for Cauchy observations and 28% for loggamma observations.

Tail index estimation

For the tail index estimation we use the MSE

σ^{2} (β̂, β) = \frac{1}{n} ∑_{i=1}^{n} (β̂ − β)^{2} .

Tail index estimation: Cauchy

F = Cauchy d.f.; n = 1000, p = 1 − 1/n = 0.999; # MC simulations = 2000. 1- plot of RMSE σ (ĥ_{n,k}, β) as a function of k; 2 - RMSE σ (θ̂_{n}, β) of adaptive estimator.

r_{p} = σ (θ̂_{n}, β) / min_{k} σ (ĥ_{n,k}, β) = 1.121888. ≈ 13% increase of RMSE

Tail index estimation: loggamma

F = loggamma d.f.; n = 1000, p = 1 − 1/n = 0.999; # MC simulations = 2000. 1- plot of RMSE σ (ĥ_{n,k}, β) as a function of k; 2 - RMSE σ (θ̂_{n}, β) of adaptive estimator.

r_{p} = σ (θ̂_{n}, β) / min_{k} σ (ĥ_{n,k}, β) = 1.276199; ≈ 28% increase of RMSE.