Variable Kernel Methods

  • 格式:pdf
  • 大小:183.75 KB
  • 文档页数:20

1
Construction of a Kernel Density Estimate
Density
0.0
0.2
0.4
0.6
0.8
2.5
3.0
3.5 x
4.0
4.5
5.0
2
Variable Kernel Methods
n
˘ f (x) = n−1
i=1
Kh(x − Xi)
Local bandwidth: h = b(x) Sample-point adaptive bandwidth: h = α(Xi) Data sharpening: shift location of Xi’s
3
Why Use Variable Bandwidths?
Density
0.1
0.2
0.3
0.4
0.0
h = 4cn−1
5
h =cn−1
5
−2
0 x
2
4
4
Structure of Talk
• Introduction • Approaches to Variable Bandwidth Selection • A Cubic Spline Approach • MISE or ISE? • Numerical Results • Conclusions
Kernel Density Estimation
The kernel density estimator from univariate data X1, . . . , Xn defined by
n
˘ f (x) = n−1
i=1
Kh(x − Xi)
where • Kh = h−1K(x/h); • the kernel K is a symmetric pdf; ˘ • bandwidth h controls smoothness of f (x).
−5
−6
−7
−6
FIX
VK3
VK0
−5
−4
−3
−2
FIX
VK3
VK0
Density 4
Density 5
−3.0
−3.5
log(ISE)
log(ISE) FIX VK3 VK0
−4.0
−5.0
−4.5
−5.5
−5.0
−4.5
−4.0
−3.5
FIX
VK3
VK0
13
log(ISE) −6.5 −6.0 −5.5 −5.0 −4.5 −4.0 −8 −7 −6
18
References
Abramson, I. (1982). On bandwidth variation in kernel estimation – a square root law, Annals of Statistics 10, 1217–1223. Fan, J., Hall, P., Martin, M. and Patil, P. (1996). On local smoothing of nonparametric curve estimators, Journal of the American Statistical Association 91, 258–266. Sain, S. and Scott, D. (1996). On locally adaptive density estimation, Journal of the American Statistical Association 91, 1525–1534. Sheather, S. and Jones, M. (1991). A reliable data-based bandwidth selection method for kernel density estimation, Journal of the Royal Statistical Society, Series B 53, 683–690.
| | || |
| |
|
|
|
h
2 120
4
8
130
140 x
150
160
170
17
Conclusions
• Sample-point adaptive bandwidths have plenty to offer. • In implementing the cubic spline bandwidth selector, there was devil in the detail. Selecting tuning parameters required some work. • Can variable bandwiths and data sharpening be combined?
6
A Cubic Spline Approach
• log(α) is a cubic spline. • Regard α as interpolant for given set of (Xi, hi) pairs. • Choose ‘optimal’ α as minimizer of ˆ ˆ CV (α) = R{f (·; α)} − 2(n − 1)−1n−1
10
Simulation Study
• Three density estimators considered: – F IX: fixed bandwidth estimator, using Sheather-Jones (1991) plug-in h; – V K0: variable kernel using zero order spline α (Sain and Scott, 1996); – V K3: variable kernel using cubic spline log(α). • Sample sizes n = 100, n = 500. • Five test densities.
5
Approaches to Variable Bandwidth Selection
n
ˆ f = n−1
i=1
Khi (x − Xi)
where hi = α(Xi)
ˆ • Choose α(·) so that AMISE(f ) is small; e.g. Abramson (1982) suggested hi ∝ f (Xi)−1/2. • Restrict α(·) to some class (e.g. piecewise constant functions), then optimize with respect to some criterion; e.g. Sain and Scott (1996)
f(x) 0.0 −3 0.5
1.0
−2
−1
0 x
1
2
3
15
Example 2: Old Faithful Eruption Data, n = 107
0.5
0.6
f(x)
0.0
0.1
0.2
0.3
0.4
FIX VK3 VK0
| |||||||||||| || | | | | | | | | || | |||||||||||||||||||||||| |||||||| | | || |||||
0.0
0.5
0.0 −4
0.2
−2
0 x
2
4
Density 5
f(x)
0.00
0.10
0.20
−4
−2
0 x
2
4
12
Simulation Study Results for n = 100
Density 1
−3
Density 2
Density 3
−3
−4
−4
log(ISE)
log(ISE)
−5
log(ISE) FIX VK3 VK0
Density 5
Density 2
VK3 VK0
VK3 VK0 log(ISE) −6.0 −5.5 −5.0 −4.5 −4.0 −3.5 −3.0 FIX VK3 VK0
Density 3
14
Example 1: Data from Density 3, n = 500
1.5
True f VK3 FIX
9
MISE or ISE? (part 2)
• Cubic spline approach a natural and straightforward extension of Sain and Scott’s (1996) piecewise constant bandwidth method – but not if you try to work with MISE and (h1, . . . , hn) viewpoint. • Cubic spline approach can be regarded in case (ii) terms if you view interpolation spline as simple computational time saver to reduce number of nodes from n.
i=j
Khj (Xi − Xj )
where R(g) =
g 2.
• CV (α) is cross-validation estimate of MISE(α) − R(f ).
7
Remarks on Cubic Spline Approch
• Performance of method depends on tuning parameters: – number of nodes (i.e. interpolation points); – placement of nodes. • Computation may be facilitated by pre-binning data. • Cubic splines used previously to estimate local bandwidth function b(x); Fan, Hall, Martin and Patil (1996). Connections?