SiZer for smoothing splines

Abstract

Smoothing splines are an attractive method for scatterplot smoothing. The SiZer approach to statistical inference is adapted to this smoothing method, named SiZerSS. This allows quick and sure inference as to “which features in the smooth are really there” as opposed to “which are due to sampling artifacts”, when using smoothing splines for data analysis. Applications of SiZerSS to mode, linearity, quadraticity and monotonicity tests are illustrated using a real data example. Some small scale simulations are presented to demonstrate that the SiZerSS and the SiZerLL (the original local linear version of SiZer) often give similar performance in exploring data structure but they can not replace each other completely.

Appendix: Derivations of (5),(6), and (7)

First of all, assume X₁, X₂, ⋯, X_n have been sorted so that X₁ < X₂ < ⋯ < X_n. Write f_i = f(X_i) and γi = f″(X_i) to be the values of f(x) and f″(x) at X_i for i = 1, 2, ⋯, n. Define f = (f₁, ⋯, f_n)^T and γ = (γ₂, ⋯, γ_{n −1})^T. Let h_i = X_{i+ 1} − X_i, i = 1, 2, ⋯, n − 1. Let Q : n × (n − 2), R: (n − 2) × (n − 2) and K: n × n be the matrices as defined in Green and Silverman (1994, pages 12–13). According to Theorem 2.1 of Green and Silverman (1994, page 13), f is a natural cubic spline with knots at X_i, i = 1, 2, ⋯, n if and only if

$$K=Q R^{-1} Q^{T}, \quad \gamma=R^{-1} Q^{T} \mathbf{f}, \quad \int f^{\prime \prime}(x)^{2} d x=\mathbf{f}^{T} K \mathbf{f}.$$

((11))

Simple calculation then leads to the following desired formula:

$$\hat{\mathbf{f}}=(W+\lambda K)^{-1} W \mathbf{Y} \equiv A_{\lambda} \mathbf{Y},$$

((12))

with the weight matrix W = diag(w₁, w₂, ⋯, w_n), the hat matrix A_λ = (W + λK)⁻¹ W, and the response vector Y = (Y₁, Y₂, ⋯, Y_n)^T.

Using (11) and (12), we are now ready to give the matrix formulas for computing $\hat{f}_{\lambda}(x), \hat{f}_{\lambda}^{\prime}(x)$, and $\widehat{sd}\left\{\hat{f}_{\lambda}^{\prime}(x)\right\}$ at a given grid of locations x = [x₁, x₂, ⋯, x_N]^T. By Green and Silverman (1994, pages 22–23), for any x, we can write $\hat{f}(x)$ and $\hat{f}{}^{\prime}(x)$ as linear combinations of $\hat{\mathbf{f}}$ and $\hat{\gamma}$. Let h_i(x) = x − X_i, i = 1, 2, ⋯, n. When x < X₁,

$$\hat{f}(x)=\hat{f}_{1}+h_{1}(x)\left\{\frac{\hat{f}_{2}-\hat{f}_{1}}{h_{1}}-\frac{h_{1}}{6} \hat{\gamma}_{2}\right\}, \quad \hat{f}^{\prime}(x)=\frac{\hat{f}_{2}-\hat{f}_{1}}{h_{1}}-\frac{h_{1}}{6} \hat{\gamma}_{2}.$$

When X_i ≤ x ≤ X_{i+ 1}, let $\delta_{i}(x)=[1+\frac{h_{i}(x)}{h_{i}}] \hat{\gamma}_{i+1}+[1-\frac{h_{i+1}(x)}{h_{i}}] \hat{\gamma}_{i}$ for some i = 1, 2, ⋯, n,

$$\begin{aligned} \hat{f}(x) &=\frac{h_{i}(x) \hat{f}_{i+1}-h_{i+1}(x) \hat{f}_{i}}{h_{i}}+\frac{h_{i}(x) h_{i+1}(x) \delta_{i}(x)}{6}, \\ \hat{f}^{\prime}(x) &=\frac{\hat{f}_{i+1}-\hat{f}_{i}}{h_{i}}+\frac{h_{i}(x) h_{i+1}(x)(\hat{\gamma}_{i+1}-\hat{\gamma}_{i})}{6 h_{i}}+\frac{\left[h_{i}(x)+h_{i+1}(x)\right] \delta_{i}(x)}{6} \end{aligned}$$

When x > X_n,

$$\hat{f}(x)=\hat{f}_{n}+\frac{h_{n}(x)}{6}\left\{\frac{\hat{f}_{n}-\hat{f}_{n-1}}{h_{n-1}}+h_{n-1} \hat{\gamma}_{n-1}\right\},$$

$$\hat{f}^{\prime}(x)=\frac{1}{6}\left\{\frac{\hat{f}_{n}-\hat{f}_{n-1}}{h_{n-1}}+h_{n-1} \hat{\gamma}_{n-1}\right\}.$$

It follows that $\hat{f}(x)$ and $\hat{f}^{\prime}(x)$ can be written respectively as $c^{T} \hat{\mathbf{f}}-d^{T} \hat{\gamma}$ and $\tilde{c}^{T} \hat{\mathbf{f}}-\tilde{d}^{T} \hat{\gamma}$ where $c, \tilde{c}, d$and $\tilde{d}$ are coefficient vectors, depending on x and X₁, X₂, ⋯, X_n only. Let $\hat{f}\left(x_{i}\right)=c_{i}^{T} \hat{\mathbf{f}}-d_{i}^{T} \hat{\gamma}, \hat{f}^{\prime}\left(x_{i}\right)=\tilde{c}_{i}^{T} \hat{\mathbf{f}}-\tilde{d}_{i}^{T} \hat{\gamma}, i=1,2, \cdots, n$. Define C = (c₁, c₂, ⋯, c_n)^T, D = (d₁, d₂, ⋯, d_n)^T, and define $\tilde{C}$ and $\tilde{D}$ similarly. Set $\hat{\mathbf{f}}_{\mathbf{x}}=[\hat{f}(x_{1}), \cdots, \hat{f}(x_{N})]^{T}$ and $\hat{\mathbf{f}}_{\mathbf{x}}^{\prime}=[\hat{f}^{\prime}(x_{1}), \cdots, \hat{f}^{\prime}(x_{N})]^{T}$. Then, using (11) and (12), we have

$$\begin{array}{l}{\hat{\mathbf{f}}_{\mathbf{x}}\quad =\quad C \hat{\mathbf{f}}-D \hat{\gamma}=[C-D R^{-1} Q^{T}] \hat{\mathbf{f}}=M \hat{\mathbf{f}}=M A_{\lambda} \mathbf{Y}}, \\ {\hat{\mathbf{f}}{}^{\prime}_{\mathbf{x}}\quad =\quad \tilde{C} \hat{\mathbf{f}}-\tilde{D} \hat{\gamma}=[\tilde{C}-\tilde{D} R^{-1} Q^{T}] \hat{\mathbf{f}}=\tilde{M} \hat{\mathbf{f}}=\tilde{M} A_{\lambda} \mathbf{Y}},\end{array}$$

where M = C − DR^{− 1}Q^T and $\tilde{M}$ is similarly defined.