Most recent changepoint detection in censored panel data

Abstract

This study aims to detect the most recent changepoint in censored panel data by ignoring dependence within and between segments as well as taking into account the serial autocorrelation. A comparison of different methods to detect the most recent changepoint for censored data is presented. Different censoring rates such as 20%, 50%, and 90% in the case of right and left censoring while (10%, 10%), (25%, 25%) and (40%, 50%) for interval censoring are considered. Further, we use most recent changepoint (MRC), double cumulative sum binary segmentation, non parametric changepoint detection (ECP), multiple changepoints in multivariate time series, analyzing each series in the panel independently, and analyzing aggregated data (AGG) methods. It is observed that different censoring rates have a significant effect on the detection of changepoints in high dimensional data. It is also noticed that the MRC method outperforms the competing methods considered in this study. In addition to investigating the impact of penalties, the performance of MRC and AGG methods is also compared using water quality data of the Niagara River. Also, a data set related to survival time of stroke patients is also a part of this study. An R package “cpcens” is available in comprehensive R archive network to replicate the results of this article.

Appendices

Appendix

A. The double CUSUM binary segmentation algorithm

Let ${\mathcal {I}}_{s,d} = [s,s + e_{T}] \cup [d - e_{T},d]$ represent a fraction of the interval [s,d] and we do not search for the changepoints on this interval in order to account for possible bias. Let the index u is used to represent the level and v represents the location of the node at each level. Then repeat the following steps.

Step 1:

Set (u,v) = (1,1), $s_{u,v}$ = 1 and $d_{u,v}$ = T

Step 2:

At current level u, repeat the following steps for all v.

Step 2.1:

Let s = $s_{u,v}$ and d = $d_{u,v}$, obtain the CUSUMs series ${\chi ^{(k)}_{s,b,d}}$ for $b \in [s,d)$ and $k = 1,\ldots ,n$, on which $D_{m}^{\varphi }({\{|\chi ^{(k)}_{s,b,d}|}\} _{k=1}^{n})$ is computed over all b and m.

Step 2.2:

Obtain the test statistic

$$\begin{aligned} T_{s,d}^{\varphi } = \mathrm {{\underset{b\in [s,d] \ {\mathcal {I}}_{s,d} }{\max } \underset{1\le m\le n}{max}}}D_{m}^{\varphi }({\{|\chi ^{(k)}_{s,b,d}|}\}_{k=1}^{n}). \end{aligned}$$

where

$$\begin{aligned} D_{m}^{\varphi }({\{|\chi ^{(k)}_{s,b,d}|}\}_{k=1}^{n})&= \Big \{\frac{m(2n-m)}{2n}\Big \}^{\varphi }\Bigg (\frac{1}{m}\sum \limits _{k=1}^{m}|\chi _{s,b,d}^{(k)}|-\frac{1}{2n-m}\sum \limits _{k=m+1}^{n}|\chi _{s,b,d}^{(k)}|\Bigg ) \\&= \Big \{\frac{m(2n-m)}{2n}\Big \}^{\varphi }\frac{1}{m}\sum \limits _{k=1}^{m}\Bigg (|\chi _{s,b,d}^{(k)}|-\frac{1}{2n-m}\sum \limits _{k=m+1}^{n}|\chi _{s,b,d}^{(k)}|\Bigg ) \end{aligned}$$

the $D^{\varphi }_{m}$ (DC operator) takes the ordered CUSUM values $|\chi _{s,b,d}^{(1)}|\ge |\chi _{s,b,d}^{(2)}|\ge \cdots \ge |\chi _{s,b,d}^{(n)}|$ at each b, as its input for some $\varphi \in [0,1]$.

Step 2.3:

If $T^ {\varphi }_{s,d} \le \pi ^{\varphi }_{n,T}$, stop searching for changepoints on the interval [s,d]. On the other hand, if $T^ {\varphi }_{s,d} \ge \pi ^{\varphi }_{n,T}$, locate

$$\begin{aligned} {\widehat{\eta }} = \mathrm {{arg\underset{b\in [s,d] \ {\mathcal {I}}_{s,d} }{\max } \underset{1\le m\le n}{max}}}D_{m}^{\varphi }({\{|\chi ^{(k)}_{s,b,d}|}\}_{k=1}^{n}). \end{aligned}$$

and proceed to Step 2.4.

Step 2.4:

Add ${\widehat{\eta }}$ to the set of estimated changepoints and divide the interval $[s_{u,v},d_{u,v}]$ into two sub-intervals $[s_{u+1,2v-1},d_{u+1,2v-1}]$ and $[s_{u+1,2v},d_{u+1,2v}]$, where $s_{u+1,2v-1} = s_{u,v}, d_{u+1,2v-1}$ = ${\widehat{\eta }}$, $s_{u+1,2v} = {{\widehat{\eta }}} + 1$ and $d_{u+1,2v} = d_{u,v}$.

Step 3:

Once $[s_{u,v},d_{u,v}]$ for all v are examined at level u, set u $\leftarrow$ u + 1 and go to Step 2. Step 2.3 furnishes a stopping rule to the DCBS algorithm: quit the search for further changepoints when $T^ {\varphi }_{s,d} \le \pi ^{\varphi }_{n,T}$ on every [s,d] defined by two adjacent estimated changepoints.