Classification using sequential order statistics

Abstract

Whereas discrimination methods and their error probabilities were broadly investigated for common data distributions such as the multivariate normal or t-distributions, this paper considers the case when the recorded data are assumed to be observations from sequential order statistics. Random vectors of sequential order statistics describe, e.g., successive failures in a k-out-of-n system or in other coherent and load sharing systems allowing for changes of underlying lifetime distributions caused by component failures. Within this framework, the Bayesian two-class discrimination approach with known prior probabilities and class parameters is considered, and exact and asymptotic formulas for the error probabilities in terms of Erlang and hypoexponential distributions are derived. Since the Bayesian classifier is closely related to Kullback–Leibler’s information distance, this approach is extended by invoking other divergence measures such as Jeffreys and Rényi’s distance. While exact formulas for the misclassification rates of the resulting distance-based classifiers are not available, inequalities among the corresponding error probabilities are derived. The performance of the applied classifiers is illustrated by some simulation results.

Hypoexponential Distribution

In the literature, the distribution of a sum of independent exponentially distributed random variables is considered in a variety of applications. The name hypoexponential distribution is used, e.g., in Ross (2011).

Definition A.1

(Hypoexponential Distribution) The hypoexponential distribution with shape parameters $s_1, \dots , s_m \in {\mathbb {N}}$ and pairwise distinct rate parameters $\beta _1, \dots , \beta _m > 0$ (notation: $HExp(s_1, \dots , s_m, \beta _1, \dots , \beta _m )$) has Lebesgue density function:

and distribution function:

$$\begin{aligned} F(x) = {\left\{ \begin{array}{ll} 0, \quad x \le 0 \\ 1 - \prod \nolimits _{j=1}^m \beta _j^{s_j} \sum \limits _{k=1}^m \sum \limits _{l=1}^{s_k} \frac{ \varPhi _{k,l}( - \beta _k ) }{ ( l - 1 )! ~ \beta _k^{ s_k - l + 1 } } \sum \nolimits _{ j = 0 }^{ s_k - l } \frac{ \exp ( - \beta _k x ) (\beta _k x) ^j }{ j! }, \quad x>0 \end{array}\right. } \end{aligned}$$

where $\varPhi _{k,l} (t) := \frac{ d^{ l-1 } }{ dt^{ l-1 } } \underset{ j \ne k }{ \prod \limits _{ j=1 } ^ m } ( \beta _j + t )^{ -s_j }$.

Theorem A.6

Let $X_1, \dots , X_m$ be independent Erlang distributed random variables with shape parameter $s_i \in {\mathbb {N}}$, $1 \le i \le m$, and pairwise distinct rate parameters $\beta _1 , \dots , \beta _m$, respectively. Then the sum $\mathcal {X} := \sum \nolimits _{j=1}^m X_i$ follows a hypoexponential distribution with shape parameters $s_1, \dots , s_m \in {\mathbb {N}}$ and rate parameters $\beta _1, \dots , \beta _m > 0$.

Proof

See Scheuer (1988). $\square$

Lemma A.8

• (i) The cdf of an hypoexponential distribution with shape parameters $s_1, \dots , s_m \in {\mathbb {N}}$ and pairwise distinct rate parameters $\beta _1, \dots , \beta _m > 0$ is given by:

  $$\begin{aligned} F(x) = {\left\{ \begin{array}{ll} 0, \quad x \le 0 \\ 1 - \prod \nolimits _{j=1}^m \beta _j^{s_j} \sum \nolimits _{k=1}^m \sum \nolimits _{l=1}^{s_k} \frac{ \varPsi _{k,l}( - \beta _k ) x^{s_k - l } }{ (s_k - l)! ( l - 1 )! } \exp ( - \beta _k x ), \quad x>0 \end{array}\right. } \end{aligned}$$

  where $\varPsi _{k,l} ( t ) = - \frac{ d^{l-1} }{ dt^{l-1} } \underset{ j \ne k }{ \prod \nolimits _{j=0}^m } ( \beta _j + t ) ^{ -s_j }$, $s_0 := 1$ and $\beta _0 := 0$.

• (ii) $\varPhi _{k,l} ( t ) = ( -1 )^{l-1} ( l-1 )! \sum \nolimits _{(i_1, \dots , i_m) \in \varOmega _{\varPhi ,k}} \underset{ j \ne k }{\prod \nolimits _{j=1}^m } \left( {\begin{array}{c} i_j + s_j - 1 \\ i_j \end{array}}\right) \tau _j$

  where $\tau _j := ( \beta _j + t )^{ -( s_j + i_j ) }$ and $\varOmega _{\varPhi ,k} := \bigg \{ \varvec{i} \in {\mathbb {N}}_0^m:\underset{ j \ne k }{ \sum \nolimits _{j = 1} ^m } i_j = l - 1,~ i_k = 0\bigg \}$.

• (iii) $\varPsi _{k,l} ( t ) = ( -1 )^{l-1} ( l-1 )! \sum \nolimits _{(i_1, \dots , i_m) \in \varOmega _{\varPsi ,k }} \underset{ j \ne k }{\prod \nolimits _{j=0}^m} \left( {\begin{array}{c} i_j + s_j - 1 \\ i_j \end{array}}\right) \tau _j$

  where $\tau _j := ( \beta _j + t )^{ -( s_j + i_j ) }$ and $\varOmega _{\varPsi ,k} := \bigg \{ \varvec{i} \in {\mathbb {N}}_0^m:\underset{ j \ne k }{ \sum \nolimits _{j = 0} ^m } i_j = l - 1,~ i_k = 0\bigg \}$.

Proof

See Amari and Misra (1997). $\square$

Lemma A.9

Let $\mathcal {Z}^+$ and $\mathcal {Z}^-$ be independent hypoexponential random variables with shape parameters $s_1^+, \dots , s_m^+$ and $s_1^-, \dots , s_n^-$ and rate parameters $\beta _1^+, \dots , \beta _m^+$ and $\beta _1^-, \dots , \beta _n^-$, respectively. Then the pdf of $\mathcal {Z}^+ - \mathcal {Z}^-$ is given by

$$\begin{aligned}&f^{\mathcal {Z}^+ - \mathcal {Z}^-} ( t ) = {\left\{ \begin{array}{ll} B^+ B^- \sum \limits _{i = 1}^m \sum \limits _{ j = 1 }^{ s_i^+ } \sum \limits _{k = 1}^n \sum \limits _{ l = 1 }^{ s_k^- } \frac{ \varPhi ^+_{i,j}( - \beta _i^+ ) \varPhi ^-_{k,l}( - \beta _k^- ) }{ ( j - 1 )! ( l - 1 )! } e^{ - \beta _i^+ t }\\ \times \sum \limits _{ r = 0 } ^{ s_i^+ - j } \left( {\begin{array}{c} s_k^- - l + r \\ r \end{array}}\right) \frac{ 1 }{ ( s_i^+ - j - r )! } t^{ s_i ^+ - j - r } ( \beta _i^+ + \beta _k^- )^{ l - s_k^- - r - 1 } , ~&{}t \ge 0 \\ B^+ B^- \sum \limits _{i = 1}^m \sum \limits _{ j = 1 }^{ s_i^+ } \sum \limits _{k = 1}^n \sum \limits _{ l = 1 }^{ s_k^- } \frac{ \varPhi ^+_{i,j}( - \beta _i^+ ) \varPhi ^-_{k,l}( - \beta _k^- ) }{ ( s_i^+ - j )! ( j - 1 )! ( s_k^- - l )! ( l - 1 )! } e^{ \beta _k^- t }\\ \times \sum \limits _{ r = 0 } ^{ s_i^+ - j } \left( {\begin{array}{c} s_i^+ - j \\ r \end{array}}\right) t^{ s_i ^+ - j - r } \\ \times \sum \limits _{ p = 0 }^{ s_k^- - l + r } \frac{ ( s_k^- - l + r )! }{ ( s_k^- - l + r - p )! } ( \beta _i^+ + \beta _k^- )^{ - ( p + 1 ) } ( - t ) ^ {s_k^- - l + r - p} ,~&{} t < 0 \\ \end{array}\right. } \end{aligned}$$

where $B^+ = \prod \nolimits _{j=1}^m { \left( \beta _j^+ \right) }^{s_j^+}$ and $B^- = \prod \nolimits _{j=1}^n { \left( \beta _j^- \right) }^{s_j^-}$.

Proof

See Katzur (2015), lemma A.9. $\square$