Probabilistic system opacity in discrete event systems

Abstract

In many emerging security applications, a system designer frequently needs to ensure that a certain property of a given system (that may reveal important details about the system’s operation) be kept secret (opaque) to outside observers (eavesdroppers). Motivated by such applications, several researchers have formalized, analyzed, and described methods to verify notions of opacity in discrete event systems of interest. This paper introduces and analyzes a notion of opacity in systems that can be modeled as probabilistic finite automata or hidden Markov models. We consider a setting where a user needs to choose a specific hidden Markov model (HMM) out of m possible (different) HMMs, but would like to “hide” the true system from eavesdroppers, by not allowing them to have an arbitrary level of confidence as to which system has been chosen. We describe necessary and sufficient conditions (that can be checked with polynomial complexity), under which the intruder cannot distinguish the true HMM, namely, the intruder cannot achieve a level of certainty about its decision, which is above a certain threshold that we can a priori compute.

Appendix A: Proof of Theorem 1

In order to simplify the notation, we present a proof that is appropriate for a decision rule that we introduce. We name this rule, “empirical rule” (A.1) which is based on the total number of single events. The empirical rule is useful only when we can distinguish statistically the two HMMs, counting only single events. This rule is illustrated for the case of single events, but can also be applied for (finite) event sequences that can be produced by at least one of the two HMMs. This is equivalent to the statement “ S ⁽¹⁾ and S ⁽²⁾ are not probabilistically equivalent from steady–state” in Theorem 1. The statistical measure that we use is the distance in variation, which compares the expected frequency of an event, against the measured frequency. The expected frequency of an event can be computed easily and is equivalent to what we call “steady-state emission probability” for a single event. In order to prove the asymptotical tightness of the upper bound on the probability of misclassification, we use a generalisation of “Hoeffding’s inequality” (Glynn and Ormoneit 2002), for functions of Markov chains. We apply the generalised Hoeffding’s inequality, to distinguish two enhanced HMM models defined in 1, and we prove that these enhanced models, are also irreducible and aperiodic, as long as the given HMMs are irreducible and aperiodic. The expected frequency for event sequences can be computed without any state explosion in the enhanced models (compared to the initial HMMs) something that is established in Lemma 6. Finally, we use the generalised Hoeffding’s inequality, combined with the empirical rule, in order to establish Theorem 1. Most of the material in this appendix was presented in our previous work in Keroglou and Hadjicostis (2014) where we explored an emprirical frequency rule for stochastic fault diagnosis. The new material in this paper is related to important proofs that were omitted in our previous work due to space limitations. Specifically, we provide a complete proof in three important Lemmas (Lemmas 4, 5, and 6), and in Section 5 that uses Hoeffding’s inequality to obtain an upper bound on the probability of misclassification.

1.1 A.1 Empirical rule

We define a suboptimal rule for HMM classification, which compares the total number of occurrences of each event (see Definition 10) against their frequencies (Definition 11) expected in each of the two HMMs.

Definition 10

(Fraction of times event e _i appears (m _n (e _i ))).Suppose we are given an observation sequence of length n (ω = ω[1]⋯ω[n]). Wedefine $m_{n}(e_{i})= \frac {1}{n}\displaystyle \sum \limits \limits _{t = 1}^{n} g_{e_{i}}(\omega [t])$,where

$$g_{e_{i}}(\omega[t])= \left\{ \begin{array}{ll} 1, & \text{if } \omega[t]=e_{i}, \\ 0, & \text{ otherwise}. \end{array} \right. $$

In other words, m _n (e _i )is the fraction of timesevent e _i appears inobservation sequence ω.

Definition 11

(Distance in Variation d _V (v, v ^′)BetweenTwo Probability Vectors v, v ^′).The distance in variation (Dembo and Zeitouni 1998) between two |E|-dimensional probabilityvectors v, v ^′is definedas

$$d_{V}(v, v^{\prime})=\frac{1}{2}\displaystyle\sum\limits_{j = 1}^{|E|}|v(j)-v^{\prime}(j)|\geq 0 \; , $$

where v(j)(v ^′(j)) is the j th entryof vector v(v ^′).

Let the stationary emission probabilities in Definition 6 for HMM S ⁽¹⁾ (S ⁽²⁾) be denoted by the |E|-dimensional vector $\pi ^{(1)}_{e}=\left [\pi ^{(1)}_{e_{1}},...,\pi ^{(1)}_{e_{|E|}}\right ]^{T}$ (respectively, by $\pi ^{(2)}_{e}=[\pi ^{(2)}_{e_{1}},...,\pi ^{(2)}_{e_{|E|}}]^{T}$). Then, we have $d_{V}(\pi ^{(1)}_{e},\pi ^{(2)}_{e})=\frac {1}{2}\displaystyle \sum \limits _{j = 1}^{|E|}|\pi ^{(1)}_{e_{j}}-\pi ^{(2)}_{e_{j}}|$.

Definition 12

(Empirical Rule). Given two irreducible and aperiodic HMMs, S ⁽¹⁾and S ⁽²⁾, and a sequenceof observations ω = ω[1]ω[2]⋯ω[n], we perform classification using the following suboptimal rule.

• We first compute m _n = [m _n (e ₁),m _n (e ₂),⋯ ,m _n (e _|E|)]^Tas in Definition 10.

• We then set $\theta = \frac {1}{2} d_{V}(\pi ^{(1)}_{e}, \pi ^{(2)}_{e})$,where $\pi ^{(j)}_{e}$, j ∈{1, 2},is the stationary emission probability vector for S ^(j),and compare

  $$ d_{V}(m_{n},\pi^{(1)}_{e})_<^> \theta \; . $$

  (4)

• We decide in favor of S ⁽¹⁾ (S ⁽²⁾)ifthe right (left) quantity is larger.

1.2 A.2 Enhanced HMM model

In this section we define a function of the states of the underlying Markov chain of the two HMMs S ⁽¹⁾ and S ⁽²⁾, that counts the occurrences of each event e _i ∈ E, with which we arrive at that state. This is not necessarily possible in S ^(j), j ∈{1, 2}, because in general we can reach a state via different events. The reason we need to define a function of the states is so that we can analyze the empirical rule (Definition 12) using existing techniques for Markov chain analysis.

First, we obtain, for each of the given HMMs, an enhanced construction that allows us to discriminate the transition to the same state but via different events. We prove that our enhanced construction inherits the properties of irreducibility and aperiodicity (the two conditions needed to apply Theorem 2) from the corresponding original HMM. The two enhanced HMM models are denoted by $\widetilde {S}^{(j)}= \{\widetilde {Q}^{(j)}, E, \widetilde {\Delta }^{(j)}, \widetilde {\Lambda }^{(j)}, \widetilde {\pi _{0}}^{(j)} \}$, j ∈{1, 2}. The enhanced construction creates replicas of each state, depending on the event via which one reaches this state. Thus, for each state $q^{}_{h} \in Q^{(j)}$, we create states $q^{}_{h,e_{i}} \in \widetilde {Q^{(j)}}$, e _i ∈ E, to represent that we reach state $q^{}_{h} \in Q^{(j)}$ under the output symbol $e^{}_{i}$. Clearly, we end up with at most $|\widetilde {Q}^{(j)}|=|Q|\times |E_{}|$ states.

The following discussion applies to each original HMM and its enhanced model (we drop j, j ∈{1, 2}, to simplify notation). In the state probability vectors π[k], $\widetilde {\pi }[t]$, where t is the current state epoch, states are indexed in the order shown below

$${\pi[t]}^{}_{} = \left[ \begin{array}{c} \pi[t](q_{1}) \\ \pi[t](q_{2}) \\ \vdots\\ \pi[t](q_{|Q|}) \end{array} \right] ,~~~ {\widetilde{\pi}[t]}^{}_{} = \left[ \begin{array}{c} \widetilde{\pi}[t](q_{1,e_{1}}) \\ \widetilde{\pi}[t](q_{1,e_{2}}) \\ \vdots\\ \widetilde{\pi}[t](q_{1,e_{|E|}}) \\ \widetilde{\pi}[t](q_{2,e_{1}}) \\ \vdots\\ \widetilde{\pi}[t](q_{|Q|,e_{|E|}}) \\ \end{array} \right]. $$

The matrix $\widetilde {A}_{e_{i}}$, e _i ∈ E, satisfies $\widetilde {A}_{e_{i}}(q_{h,e_{i}},q^{\prime }_{h,e^{\prime }_{i}})=A_{e_{i}}(q_{h},q^{\prime }_{h})$, $\forall e^{\prime }_{i} \in E$ and ∀q _h , q h′∈ Q (zero otherwise). We also have for, $e^{\prime }_{i^{\prime }}\in E$ and $q_{h,e_{i}}, q^{\prime }_{h,e^{\prime }_{i}}\in \widetilde {Q}$, $\widetilde {\Lambda }(q^{\prime }_{h,e^{\prime }_{i}}, e_{i}, q_{h,e_{i}})=\widetilde {A}_{e_{i}}(q_{h,e_{i}}, e_{i}, q^{\prime }_{h,e^{\prime }_{i}})$ (zero otherwise). We observe that matrix $\widetilde {A}_{e_{i}}$ is constructed by blocks of matrix $A_{e_{i}}$. If we define row-vector $R_{|E|}= \underbrace {[1 1 {\cdots } 1]}_{|E|- times}$ and let

$$R_{i,|E|}= \underbrace{[0 {\cdots} 0\text{ 1 }0 {\cdots} 0],}_{\text{single one at } i \text{ th position}} $$

then the state transition matrix $\widetilde {A}^{(j)}_{e_{i}}$ for the enhanced model $\widetilde {S}^{(j)}$ can be written as⁵

$$\widetilde{A}^{(j)}_{e_{i}}=A^{(j)}_{e_{i}}\otimes \left( R^{T}_{i,|E|} \otimes R_{|E|}\right). $$

Example 2 We create the enhanced HMM models $\widetilde {S}^{(1)}$ (shown in Fig. 3) and $\widetilde {S}^{(2)}$ for S ⁽¹⁾ and S ⁽²⁾ respectively (shown in Fig. 1). We note that the underlying state transition matrix, for each enhanced model, is irreducible and aperiodic (as we will see, $\widetilde {S}^{(j)}$ will be irreducible and aperiodic as long as S ^(j) is irreducible and aperiodic). The corresponding $\widetilde {A}^{(1)}_{\alpha },\widetilde {A}^{(1)}_{\beta },\widetilde {A}^{(2)}_{\alpha },\widetilde {A}^{(2)}_{\beta }$ are as follows:

$${ \widetilde{A}}^{(1)}_{\alpha} = \left[ \begin{array}{cccc} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0.50 & 0.50 & 0.50 & 0.50 \\ 0&0 &0 &0 \end{array} \right] \text{, } {\widetilde{A}}^{(1)}_{\beta} = \left[ \begin{array}{cccc} 0&0 &0 &0 \\ 0.50& 0.50 &0.25 &0.25 \\ 0& 0 &0 &0 \\ 0& 0 &0.25 &0.25 \end{array} \right], $$

$${ \widetilde{A}}^{(2)}_{\alpha} = \left[ \begin{array}{cccc} 0&0 &0 &0 \\ 0& 0 &0 &0 \\ 0.60& 0.60 & 0.06 & 0.06 \\ 0& 0 &0 &0 \end{array} \right]\text{, } { \widetilde{A}}^{(2)}_{\beta} = \left[ \begin{array}{cccc} 0&0 &0 &0 \\ 0.40& 0.40 &0.04 &0.04 \\ 0& 0 &0 &0 \\ 0& 0 &0.90 &0.90 \end{array} \right]. $$

Fig. 3

Enhanced model $\widetilde {S}^{(1)}$ for HMM model S ⁽¹⁾ in Fig. 1

1.3 A.3 Required conditions for using Hoeffding’s inequality on enhanced models

Proposition 1 (Hoeffding’s Inequality on Enhanced HMM Model)

Consider enhanced HMMs, $\widetilde {S}^{(j)}= \{\widetilde {Q}^{(j)}, E, \widetilde {\Delta }^{(j)},\allowbreak \widetilde {\Lambda }^{(j)},\allowbreak \widetilde {\pi _{0}}^{(j)} \}$ , j ∈{1, 2},with |E|events and transition matrix $\widetilde {A}^{(j)}$ . Assuming the Markov chains that correspond to the enhanced models $\widetilde {S}^{(j)}$ , j ∈{1, 2}, are irreducible and aperiodic, we denote their stationary distributions by $\widetilde {\pi }^{(j)}>0$ and stationary emission distribution for events e _i ∈ E by $\widetilde {\pi }^{(j)}_{e}>0$ .

Using the enhanced models ( $\widetilde {S}^{(1)}$ and $\widetilde {S}^{(2)}$ ) for each e _i ∈ E , we define the indicator functions $f_{e_{i}}(q_{h,e_{j}})$ , $\forall q_{h, e_{j}} \in \widetilde {Q}$ ,as

$$f_{e_{i}}(q_{h, e_{j}})= \left\{ \begin{array}{ll} 1, & \text{if } e_{j}=e_{i}, \\ 0, & \text{ otherwise}. \end{array} \right. $$

Let $m_{n}(e_{i})= \frac {1}{n}\displaystyle \sum \limits _{t = 1}^{n} f_{e_{i}}(q[t])$, i.e., the |E|-dimensional vector m _n = [m _n (e ₁), m _n (e ₂),...,m _n (e _|E|)]^T denotes the empirical frequencies with which each event appears in the given observation window of length n. Let M _j , be the smallest integer such that ${(\widetilde {A}^{(j)})}^{M_{j}}>0$, element-wise, and $\lambda _{j}=\min _{l,l^{\prime }} \left \{ \frac {{(\widetilde {A}^{(j)})}^{M_{j}}(l,l^{\prime })}{\widetilde {\pi }^{(j)}(l)} \right \}$, where $\widetilde {\pi }^{(j)}(l)$ is the stationary distribution of $\widetilde {S}^{(j)}$. As long as the enhanced model $\widetilde {S}^{(j)}$ is irreducible and aperiodic, it can be shown Glynn and Ormoneit (2002) and Hadjicostis (2005) that the following is true for $n>\frac {2M_{j}}{\lambda _{j}\epsilon }$, for each event e _i (1 ≤ i ≤|E|), and for $F^{(j)}(n)=\exp \left (-\frac {{\lambda _{j}}^{2}\left (n\epsilon - \frac {2M_{j}}{\lambda _{j}}\right )^{2}}{2n{M_{j}}^{2}}\right )$:

$$\begin{array}{@{}rcl@{}} Pr(|m_{n}(e_{i}) - \widetilde{\pi}^{(j)}_{e}(e_{i})|\geq \epsilon) \leq {F^{(j)}(n)}. \end{array} $$

(5)

In order to use Eq. 5, we need $\widetilde {S}^{(j)}$ to correspond to an irreducible (Definition 3) and aperiodic (Definition 4) Markov chain. We now show that $\widetilde {S}^{(j)}$ is irreducible and aperiodic if S ^(j) is irreducible and aperiodic. Also, we establish that $\widetilde {\pi }^{(j)}_{e}=\pi ^{(j)}_{e}$.

Lemma 4

If HMM $S^{(j)}=(Q^{(j)}, E^{(j)}, \Delta ^{(j)}, \Lambda ^{(j)}, \pi ^{(j)}_{0})$ is irreducible (Definition 3), then the enhanced HMM $\widetilde {S}^{(j)}= \{\widetilde {Q}^{(j)}, E, \widetilde {\Delta }^{(j)},\allowbreak \widetilde {\Lambda }^{(j)}, \widetilde {\pi _{0}}^{(j)} \}$ is also irreducible.

Proof

We prove irreducibility by establishing the property that any state$q_{h,e_{i}} \in \widetilde {Q}^{(j)}$thatdoes not belong to a set of strongly connected states (Definition 3), may exhibit outgoingtransitions but will have no incoming transition. Consider in the enhanced model the set ofstates

$$ \widetilde{Q}_{ss}=\{q_{m,e} \in \widetilde{Q}| \exists q_{m^{\prime}} \in Q, \exists e \in E \text{ s.t. } \Lambda(q_{m^{\prime}},e,q_{m})>0\}. $$

Sincethe set of states Q in the original system is strongly connected, we can easily show that the states in$\widetilde {Q}_{ss}$are stronglyconnected: given q _{m, e} ,$q_{m^{\prime },e^{\prime }}$ $\in \widetilde {Q}$we can find a path to connectthem as follows: Let $q_{m^{\prime \prime }}$be such that $\Lambda (q_{m^{\prime \prime }},e^{\prime },q_{m^{\prime }})>0$. Then, we can find a path

$$q_{m}\xrightarrow{e_{i_{1}}} q_{i_{1}}\xrightarrow{e_{i_{2}}} q_{i_{2}}\rightarrow {\cdots} \xrightarrow{e_{i_{t}}} q_{i_{t}} = q_{m^{\prime\prime}} \; .$$

(because the originalHMM is irreducible). Therefore

$$q_{m,e}\xrightarrow{e_{i_{1}}} q_{i_{1},e_{1}}\xrightarrow{e_{i_{2}}} q_{i_{2},e_{i_{2}}}\rightarrow {\cdots} \xrightarrow{e_{i_{t}}} q_{i_{t}}= q_{m^{\prime\prime}} \xrightarrow{e^{\prime}} q_{m^{\prime},e^{\prime}}$$

is a path thatconnects $q_{m,e} \in \widetilde {Q}$to$q_{m^{\prime },e^{\prime }} \in \widetilde {Q}$. We finallyconclude that the states that do not belong to the set of strongly connected states, have only outgoingtransitions. Therefore, by choosing an appropriate initial distribution function that excludes all these transientstates,⁶we can ensure that all of these transient states will never be visited. □

Lemma 5

If HMM $S^{(j)}=(Q^{(j)}, E^{(j)}, \Delta ^{(j)}, \Lambda ^{(j)}, \pi ^{(j)}_{0})$ is aperiodic (Definition 4), then the enhanced HMM $\widetilde {S}^{(j)}= \{\widetilde {Q}^{(j)}, E, \widetilde {\Delta }^{(j)},\allowbreak \widetilde {\Lambda }^{(j)}, \widetilde {\pi _{0}}^{(j)} \}$ is also aperiodic.

Proof

We show that if the enhanced model $\widetilde {S}^{(j)}$is periodic with period k, this contradicts the fact that S ^(j)is aperiodic. Suppose that $\widetilde {S}^{(j)}$is periodic with period k (Definition 4 and Lemma2). This means we can group all possible states of$\widetilde {S}^{(j)}$to k groups ($\widetilde {C}_{1},\widetilde {C}_{2},\cdots ,\widetilde {C}_{k}$)such that for a state $q_{l,e} \in \widetilde {C}_{m}$,there exist one-step transitions only to states in$\widetilde {C}_{m^{\prime }}$,where m ^′ = m + 1 mod k.

Due to the construction of enhanced models, the outgoing behaviour of q _{l, e} states ∀e ∈ E are copies of the outgoing behaviour of q _l ∈ Q.We can easily see that if there exists $q_{l,e} \in \widetilde {Q}$,that belongs to $\widetilde {C}_{m}$,then also $q_{l,e^{\prime }} \in \widetilde {Q}$belongs to $\widetilde {C}_{m}$,for all e, e ^′∈ E(due to the same outgoing behaviour). Thus, we can also group q ∈ Q into C _i , i ∈{1, 2,...,k},classes. Thus, S ^(j)is periodic, with period k, which is a contradiction. □

1.4 A.4 Consistency of stationary emission probabilities for S ^(j) and $\widetilde {S}^{(j)}$

We now show that in the enhanced model $\widetilde {S}^{(j)}$, the stationary emission probabilities of each event are consistent with the original model S ^(j) for j = 1, 2.

Lemma 6

The computed stationary emission probabilities for symbols in the enhanced model $\widetilde {S}^{(j)}$ , j ∈{1, 2}which is denoted respectively by $\widetilde {\pi }^{(j)}_{e}$ ,is identical to $\pi ^{(j)}_{e}$ corresponding to S ^(j) .

Proof

Let $\widetilde {\pi }^{(j)}_{s}$ denote the steady-state distribution vector in the enhanced model j. Then, we have that under each model

$$\pi^{(j)}_{s}=(I_{n}\otimes R_{|E|})\times \widetilde{\pi}^{(j)}_{s},\ j \in \{1,2\}. $$

For S ^(j) and ∀e _i ∈ E, the stationary emission probability $\pi ^{(j)}_{e}(e_{i})$can be expressed as

$$\pi^{(j)}_{e}(e_{i})= R_{n}\times\left( A^{(j)}_{e_{i}} \times \pi^{(j)}_{s}\right), $$

where as for the enhanced model⁷ $\widetilde {S}^{(j)}$.

$$\begin{array}{@{}rcl@{}} \widetilde{\pi}^{(j)}_{e}(e_{i})&=& R_{n|E|} \times \widetilde{A}^{(j)}_{e_{i}} \times \widetilde{\pi}^{(j)}_{s}\\ &=& R_{n|E|} \times \left( A^{(j)}_{e_{i}} \otimes \left( R^{T}_{i,|E|} \otimes R_{|E|}\right)\right)\times \widetilde{\pi}^{(j)}_{s}\\ &=& (R_{n} \otimes R_{|E|})\times \left( A^{(j)}_{e_{i}} \otimes \left( R^{T}_{i,|E|} \otimes R_{|E|}\right)\right)\times \widetilde{\pi}^{(j)}_{s}\\ &=& (R_{n} \times A^{(j)}_{e_{i}})\otimes \left( R_{|E|} \times \left( R^{T}_{i,|E|}\otimes R_{|E|}\right)\right)\times \widetilde{\pi}^{(j)}_{s}\\ &=& \left( \left( R_{n} \times A^{(j)}_{e_{i}}\right)\otimes R_{|E|}\right)\times \widetilde{\pi}^{(j)}_{s}\\ &=& \left( R_{n} \times A^{(j)}_{e_{i}}\right)\otimes (R_{1}\times R_{|E|})\times \widetilde{\pi}^{(j)}_{s}\\ &=& R_{n} \times \left( A^{(j)}_{e_{i}}\otimes R_{|E|}\right)\times \widetilde{\pi}^{(j)}_{s}. \end{array} $$

Moreover, we have

$$\begin{array}{@{}rcl@{}} \pi^{(j)}_{e}(e_{i})&=& R_{n} \times \left( A^{(j)}_{e_{i}}\times \pi^{(j)}_{s}\right)\\ &=& R_{n}\times \left( A^{(j)}_{e_{i}}\times (I_{n}\otimes R_{|E|})\times \widetilde{\pi}^{(j)}_{s}\right)\\ &=& R_{n}\times \left( A^{(j)}_{e_{i}}\otimes R_{1}\right)\times (I_{n} \otimes R_{|E|})\times \widetilde{\pi}^{(j)}_{s} \\ &=& R_{n}\times \left( A^{(j)}_{e_{i}}\times I_{n}\right)\otimes (R_{1} \times R_{|E|})\times \widetilde{\pi}^{(j)}_{s} \\ &=& R_{n}\times \left( A^{(j)}_{e_{i}}\otimes R_{|E|}\right)\times \widetilde{\pi}^{(j)}_{s} \\ &=& \widetilde{\pi}^{(j)}_{e}(e_{i}), \end{array} $$

which allows us to conclude that $\pi ^{(j)}_{e}(e_{i})=\widetilde {\pi }^{(j)}_{e}(e_{i})$, ∀e _i ∈ E. □

1.5 A.5 Upper bound on the probability of error

Given two HMMs S ⁽¹⁾ and S ⁽²⁾ (each irreducible and aperiodic), we construct the corresponding enhanced HMM models ($\widetilde {S}^{(1)}$, $\widetilde {S}^{(2)}$) with underlying irreducible and a periodic Markov chains $\widetilde {MC}^{(1)}=(\widetilde {Q}^{(1)},\widetilde {A}^{(1)},\widetilde {\pi _{0}}^{(1)})$ and $\widetilde {MC}^{(2)}=(\widetilde {Q}^{(2)},\widetilde {A}^{(2)},\widetilde {\pi _{0}}^{(2)})$ (i.e., this means that $\widetilde {A}^{(1)}$ and $\widetilde {A}^{(2)}$ are primitive matrices). Suppose we have⁸ $d_{V}(\widetilde {\pi }^{(1)}_{e}, \widetilde {\pi }^{(2)}_{e})>0$. Then, if we apply the empirical rule and use Hoeffding’s inequality (Proposition 1), we obtain the upper bound on the probability of error using the empirical rule where F(n) is given by

$$ F(n)=\max\{F^{(1)}(n),F^{(2)}(n)\} \; . $$

(6)

Proof

We consider two error cases:

Case 1::

Decide S ⁽¹⁾when the system is S ⁽²⁾;

Case 2::

Decide S ⁽²⁾when the system is S ⁽¹⁾.

Case 1:

The decision of S ⁽¹⁾is equivalent to the event

$$H^{(1)}: d_{V}(m_{n},\pi^{(1)}_{e})<\theta,$$

which necessarily implies d _V (m _n , π e(2)) ≥ 𝜃 $\left (\text {for } \theta =\frac {1}{2}d_{V}\left (\pi ^{(1)}_{e},\pi ^{(2)}_{e}\right )\right )$. Otherwise, we reach acontradiction, because d _V (m _n , π e(2)) < 𝜃 and $d_{V}(m_{n},\pi ^{(1)}_{e})< \theta$imply

$$\begin{array}{@{}rcl@{}} d_{V}\left( \pi^{(1)}_{e},\pi^{(2)}_{e}\right) & < & d_{V}\left( m_{n},\pi^{(1)}_{e}\right)+ d_{V}\left( m_{n},\pi^{(2)}_{e}\right) \\ & = & 2\theta \\ & = & d_{V}\left( \pi^{(1)}_{e},\pi^{(2)}_{e}\right) \; . \end{array} $$

Thus, H ⁽¹⁾implies$d_{V}(m_{n},\pi ^{(1)}_{e})\geq \theta$, whichimplies

$$H^{(1)}_{k}: \left\{ \exists e_{k} \in E \text{ such that } \left|m_{n}(e_{k})-\pi^{(2)}_{e}(e_{k})\right|>\frac{\theta}{|E|} \right\}. $$

Therefore, we have to consider two different subcases:

a):

$m_{n}(e_{k})-\pi ^{(2)}_{e}(e_{k})>0$,

b):

$m_{n}(e_{k})-\pi ^{(2)}_{e}(e_{k})<0$.

The probability of error for Case 1 and subcase a), after n observations, is

$$\begin{array}{@{}rcl@{}} \Pr(\text{error at } n,\text{ Case 1})&=&\Pr(H^{(1)}| S^{(2)})P(S^{(2)})\\ &\leq& \Pr\left( H^{(1)}_{k}|S^{(2)}\right)P(S^{(2)})\\ &\leq& F^{(2)}(n)P(S^{(2)}) \; , \end{array} $$

where $\Pr \left (H^{(1)}_{k}| S^{(2)}\right )\equiv \Pr (m_{n}(e_{k})-\pi ^{(2)}_{e}(e_{k}))>\frac {\theta }{|E|})\leq F^{(2)}_{n}$,for $\epsilon =\frac {\theta }{|E|}$.

In Case 1a) we can immediately apply Eq. 5, but in Case 1b) in order to find apositive measure we choose to count the number of appearances of all elements in k ^c = {e ∈ E | s.t. e≠e _k }, i.e., all possibleevents except e _k ∈ E.Then $m_{n}(e_{k^{c}})-\pi ^{(2)}_{e}(e_{k^{c}})=(1-m_{n}(e_{k}))-(1-\pi ^{(2)}_{e}(e_{k}))>0$,and we can apply (5), which leads us to the same bound.

Case 2:

With the same reasoning as in Case 1, we establish the following inequality

$$\begin{array}{@{}rcl@{}} \Pr(\text{error at } n,\text{ Case 2})&=&\Pr(H^{(2)}| S^{(1)})P(S^{(1)})\\ &\leq& \Pr\left( H^{(2)}_{k^{\prime}}|S^{(1)}\right)P(S^{(1)}))\\ &\leq &F^{(1)}(n)P(S^{(1)}) , \end{array} $$

where $H^{(2)}: d_{V}(m_{n},\pi ^{(1)}_{e})>\theta$, which implies that $H^{(2)}_{k^{\prime }}$: {There existsat least one $e_{k^{\prime }}$such that $|m_{n}(e_{k^{\prime }})-\pi ^{(1)}_{e}(e_{k^{\prime }})|>\frac {\theta }{|E|}$,where $e_{k^{\prime }}$ ∈ E }. Theclaim follows using similar arguments as in Case 1.

Finally, we prove that

$$\begin{array}{@{}rcl@{}} \Pr(\text{error at } n)&=&\Pr(\text{error at } n,\text{ Case 1})+ \Pr(\text{error at } n,\text{ Case 2})\\ &=&\Pr(H^{(1)}|S^{(2)})P(S^{(2)}) + \Pr(H^{(2)}|S^{(1)})P(S^{(1)})) \\ &\leq& \Pr\left( H^{(1)}_{k}|S^{(2)}\right)P(S^{(2)}) + \Pr\left( H^{(2)}_{k^{\prime}}|S^{(1)}\right)P(S^{(1)}))\\ &\leq& F(n)(P(S^{(1)})+ P(S^{(2)}))\equiv F(n) \end{array} $$

In other words, if two HMMs have different expected frequencies for at least one single event, thisallows to asymptotically distinguish the two HMMs using the empirical rule. Without loss ofgenerality, we can extend this result to event sequences, that have different expected frequencies.Knowing that the expected frequency of an event sequence can be computed as the probability ofoccurrence of an event sequence, generated by an HMM that starts at the steady-state, then if thereis at least one event sequence with different expected frequencies, is equivalent to the fact that thetwo HMMs, are not probabilistically equivalent from the steady-state. This concludes the proof. □