Alerting patients via health information system considering trust-dependent patient adherence

Abstract

The internet of things has ushered in a world of possibilities in chronic disease management. Connected to the health information network, a health device can monitor and provide intervention recommendations to patients in real time. However, this new health information system may face the risk of patients not following the system’s recommendations depending on their perception of the system. In this paper, we consider patients’ trust in the system a key factor driving their adherence to the system’s recommendation and develop an analytical model to design the optimal alerting strategy in the context of asthma management. Our method acknowledges that patient’s trust may change over time based on their experience of using the system, which may influence their future adherence behavior. We derive a set of structural properties of our solution and demonstrate that our approach can significantly improve patients’ quality of life compared to the current practice of asthma management. Furthermore, we investigate various real-world scenarios, such as the case that patients may have different level of tolerance for receiving alerts. Based on our findings, valuable insights can be shared with patients, healthcare practitioners, and companies in the technology-enabled healthcare business sector.

Appendices

Appendix A: Proofs of analytical results

The following results are used throughout Appendix.

Lemma A.1

[25] For two probability mass functions ${\varvec{x}}$ and ${\varvec{x}}'$ with the same dimension $\left| X\right|$, ${\varvec{x}}\le _s{\varvec{x}}'$ iff $\sum _{i\in X}{{\varvec{x}}\left( i\right) f\left( i\right) }\ge \sum _{i\in X}{{\varvec{x}}'\left( i\right) f\left( i\right) }$ for every non-increasing f in $i\in X$.

Lemma A.2

[47] For two probability mass functions ${\varvec{x}}$ and ${\varvec{x}}'$ with the same dimension $\left| X\right|$, if ${\varvec{x}}\le _r{\varvec{x}}'$ then ${\varvec{x}}\le _s{\varvec{x}}'$.

Lemma A.3

[26] If ${\varvec{P}}\in {TP}_2$ and ${\varvec{x}}\le _r{\varvec{x}}'$ are two probability mass functions with the same dimension $\left| X\right|$, then ${\varvec{x}}{\varvec{P}}\le _r{\varvec{x}}'{\varvec{P}}$ provided that ${\varvec{P}}$ have appropriate dimension.

Lemma A.4

[43] For ${\varvec{x}}\le _r{\varvec{x}}'$ in Definition 2, ${\varvec{x}}\le _r\left( 1-\lambda \right) {\varvec{x}}+\lambda {\varvec{x}}'\le _r{\varvec{x}}'$ for any arbitrary $\lambda \in \left[ 0, 1\right]$.

Lemma A.5

[48] The optimality equation $V^*_t\left( \varvec{\pi } \right)$ is piecewise linear convex hence can be written in terms of the maximum of a finite number of linear functions as:

$$\begin{aligned} V^*_t\left( \varvec{\pi } \right)= & {} \max _k \left[ \sum _{s\in S}{\pi (s)\alpha ^k_t(s)}\right] \\&\text { for some }\alpha _t=\left\{ {\alpha }^0_t,{\alpha }^1_t,{\alpha }^2_t,\dots \right\} , \end{aligned}$$

for all $t\le t_E$ where the $\left| S\right|$-dimensional vector $\alpha ^i_t=\left[ {\alpha }^i_t\left( s\right) \right]$ for $s\in S$ called the $\alpha$-vectors.

Proof of analytical results 1 and 2

Analytical Results 1 and 2 are crucial because, based on them, we can claim a monotone optimal value function nonincreasing in $\varvec{\pi }\in {{\varvec{\Pi }}}$. We first show Lemma 1 as follows:

Lemma 1

Suppose (C1)–(C3) hold, then the state transition probability matrix ${{\varvec{\Gamma }}}_t^{a,o}$ has ${TP}_2$ property for all $a\in {\mathcal {A}}$ and $o\in {\mathbf {O}}$ where (C1)–(C3) are

$$\begin{aligned} \text {(C1) }&v^{loss}_{HL}\ge v^{none}_{HL}\ge v^{gain}_{HL} \text { and } c^{loss}_{LL}=c^{none}_{LL}=c^{gain}_{LL},&\\ \text {(C2) }&v^{none}_{HH}\ge 1-v^{none}_{LL} \text { and } v^{loss}_{HL}\le v^{loss}_{LL},&\\ \text {(C3) }&c^0_{BB}\ge c^1_{BB}, c^0_{GG}=c^1_{GG}\text {, and } c^1_{GG}c^1_{BB}-c^1_{GB}c^1_{BG}\ge 0.&\end{aligned}$$

Proof of Lemma 1

First, we consider $o=y\in \varvec{Y}$ for $a\in {\mathcal {A}}$. In this case, the transition probability matrices are

$$\begin{aligned} {\varvec{{\Gamma }}}^{W,y}_t={\varvec{{\Gamma }}}^{A,y}_t=\left[ \begin{array}{ccc} p^{0,none}_{00}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{0j}} &{} p^{0,none}_{01}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{0j}} &{} p^{0,none}_{02}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{0j}} \\ p^{0,none}_{10}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{1j}} &{} p^{0,none}_{11}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{1j}} &{} p^{0,none}_{12}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{1j}} \\ p^{0,none}_{20}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{2j}} &{} p^{0,none}_{21}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{2j}} &{} p^{0,none}_{22}/\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{2j}} \end{array} \right] \ , \end{aligned}$$

where ${{\Gamma }}^{W,y}_{0j}$, ${{\Gamma }}^{W,y}_{1j}$, and ${{\Gamma }}^{W,y}_{2j}$ can be replaced with ${{\Gamma }}^{A,y}_{0j}$, ${{\Gamma }}^{A,y}_{1j}$, and ${{\Gamma }}^{A,y}_{2j}$.

Let us ignore the denominator and focus on the numerator for each entry in ${\varvec{{\Gamma }}}^{a,y}_t$. Note that $[p_{ij}^{x,z}]=c_{kl}^x\times v_{qu}^z$ denotes a matrix of transition probabilities from state i to state j for $i,j\in \{0,1,2\}$, where $x\in \{1,0\}$, $z\in \{gain,loss,none\}$, $k=G$ if $i\in \{0,1\}$, $k=B$ if $i=2$, $l=G$ if $j\in \{0,1\}$, $l=B$ if $j=2$, $q=H$ if $i=0$, $q=L$ if $i\in \{1,2\}$, $u=H$ if $j=0$, and $u=L$ if $j\in \{1,2\}$. Then, we have

$$\begin{aligned} {\widetilde{\varvec{{\Gamma }}}}^{a,y}_t= & {} \left[ \begin{array}{c} {\widetilde{\varvec{{\Gamma }}}}^{a,y}_t\left( \cdot |0\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{a,y}_t\left( \cdot |1\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{a,y}_t\left( \cdot |2\right) \end{array} \right] =\left[ \begin{array}{ccc} p^{0,none}_{00} &{} p^{0,none}_{01} &{} p^{0,none}_{02} \\ p^{0,none}_{10} &{} p^{0,none}_{11} &{} p^{0,none}_{12} \\ p^{0,none}_{20} &{} p^{0,none}_{21} &{} p^{0,none}_{22} \end{array} \right] \\= & {} \left[ \begin{array}{ccc} c^0_{GG}v^{none}_{HH} &{} c^0_{GG}v^{none}_{HL} &{} c^0_{GB}v^{none}_{HL} \\ c^0_{GG}v^{none}_{LH} &{} c^0_{GG}v^{none}_{LL} &{} c^0_{GB}v^{none}_{LL} \\ c^0_{BG}v^{none}_{LH} &{} c^0_{BG}v^{none}_{LL} &{} c^0_{BB}v^{none}_{LL} \end{array} \right] . \end{aligned}$$

The first, second, and third rows have the following relationship:

$$\begin{aligned}&\frac{c^0_{GG}v^{none}_{HH}}{c^0_{GG}v^{none}_{LH}}\ge \frac{c^0_{GG}v^{none}_{HL}}{c^0_{GG}v^{none}_{LL}}=\frac{c^0_{GB}v^{none}_{HL}}{c^0_{GB}v^{none}_{LL}}\\&\quad \mathrm { and }\frac{c^0_{GG}v^{none}_{LH}}{c^0_{BG}v^{none}_{LH}}=\frac{c^0_{GG}v^{none}_{LL}}{c^0_{BG}v^{none}_{LL}}\ge \frac{c^0_{GB}v^{none}_{LL}}{c^0_{BB}v^{none}_{LL}}{\ ,}\end{aligned}$$

where the first inequality holds because of (C2) and the second inequality holds due to (C3). It can be shown that $c^0_{GB}v^{none}_{HH}=1-\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{0j}}$, $c^0_{GB}v^{none}_{LH}=1-\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{1j}}$, and $c^0_{BB}v^{none}_{LH}=1-\sum ^2_{j=0}{{{\Gamma }}^{W,y}_{2j}}$. Because both $\left( 1-c^0_{GB}v^{none}_{LH}\right) /\left( 1-c^0_{GB}v^{none}_{HH}\right)$ and $\left( 1-c^0_{BB}v^{none}_{LH}\right) /\left( 1-c^0_{GB}v^{none}_{LH}\right)$ are non-negative, we obtain

$$\begin{aligned} \frac{c^0_{GG}v^{none}_{HH}\left( 1-c^0_{GB}v^{none}_{LH}\right) }{c^0_{GG}v^{none}_{LH}\left( 1-c^0_{GB}v^{none}_{HH}\right) }\ge&\frac{c^0_{GG}v^{none}_{HL}\left( 1-c^0_{GB}v^{none}_{LH}\right) }{c^0_{GG}v^{none}_{LL}\left( 1-c^0_{GB}v^{none}_{HH}\right) }\\ =&\frac{c^0_{GB}v^{none}_{HL}\left( 1-c^0_{GB}v^{none}_{LH}\right) }{c^0_{GB}v^{none}_{LL}\left( 1-c^0_{GB}v^{none}_{HH}\right) } \ \mathrm { and } \\ \frac{c^0_{GG}v^{none}_{LH}\left( 1-c^0_{BB}v^{none}_{LH}\right) }{c^0_{BG}v^{none}_{LH}\left( 1-c^0_{GB}v^{none}_{LH}\right) }=&\frac{c^0_{GG}v^{none}_{LL}\left( 1-c^0_{BB}v^{none}_{LH}\right) }{c^0_{BG}v^{none}_{LL}\left( 1-c^0_{GB}v^{none}_{LH}\right) }\\ \ge&\frac{c^0_{GB}v^{none}_{LL}\left( 1-c^0_{BB}v^{none}_{LH}\right) }{c^0_{BB}v^{none}_{LL}\left( 1-c^0_{GB}v^{none}_{LH}\right) }, \end{aligned}$$

which directly implies ${\varvec{{\Gamma }}}^{a,y}_t\left( \cdot |0\right) {\le }_r{\varvec{{\Gamma }}}^{a,y}_t\left( \cdot |1\right)$ and ${\varvec{{\Gamma }}}^{a,y}_t\left( \cdot |1\right) {\le }_r{\varvec{{\Gamma }}}^{a,y}_t\left( \cdot |2\right)$ from Definition 2. Thus, from Definition 3, ${\varvec{{\Gamma }}}^{a,y}_t\in {TP}_2$ for $a\in {\mathcal {A}}$ and $t\le t_E$.

Now, for $o\mathrm {=}{\mathrm {s}}^H_t$ and $a\mathrm {=}W$, the transition matrix without the normalizing denominators is

$$\begin{aligned} {\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t= & {} \left[ \begin{array}{c} {\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |0\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |1\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |2\right) \end{array} \right] =\left[ \begin{array}{ccc} p^{1,gain}_{00} &{} p^{1,gain}_{01} &{} p^{1,gain}_{02} \\ p^{1,gain}_{10} &{} p^{1,gain}_{11} &{} p^{1,gain}_{12} \\ p^{1,loss}_{20} &{} p^{1,loss}_{21} &{} p^{1,loss}_{22} \end{array} \right] \\= & {} \left[ \begin{array}{ccc} c^1_{GG}v^{gain}_{HH} &{} c^1_{GG}v^{gain}_{HL} &{} c^1_{GB}v^{gain}_{HL} \\ c^1_{GG}v^{gain}_{LH} &{} c^1_{GG}v^{gain}_{LL} &{} c^1_{GB}v^{gain}_{LL} \\ c^1_{BG}v^{loss}_{LH} &{} c^1_{BG}v^{loss}_{LL} &{} c^1_{BB}v^{loss}_{LL} \end{array} \right] {\ ,}\end{aligned}$$

and we can derive following relationships

$$\begin{aligned}&\frac{c^1_{GG}v^{gain}_{HH}}{c^1_{GG}v^{gain}_{LH}}\ge \frac{c^1_{GG}v^{gain}_{HL}}{c^1_{GG}v^{gain}_{LL}}=\frac{c^1_{GB}v^{gain}_{HL}}{c^1_{GB}v^{gain}_{LL}}\\&\quad \mathrm { and }\frac{c^1_{GG}v^{gain}_{LH}}{c^1_{BG}v^{loss}_{LH}}=\frac{c^1_{GG}v^{gain}_{LL}}{c^1_{BG}v^{loss}_{LL}}\ge \frac{c^1_{GB}v^{gain}_{LL}}{c^1_{BB}v^{loss}_{LL}}{\ ,}\end{aligned}$$

where the first inequality holds because of (C1) and (C2), the second inequality holds due to (C3), and the second equality is based on (C1). Therefore, ${\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |0\right) {\le }_r{\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |1\right) {\le }_r{\widetilde{\varvec{{\Gamma }}}}^{W,s^C_t}_t\left( \cdot |2\right)$ which implies that ${\varvec{{\Gamma }}}^{W,s^C_t}_t\in {TP}_2$ from Definition 3. Similarly, for $o\mathrm {=}{\mathrm {s}}^H_t$ and $a\mathrm {=}A$, we have

$$\begin{aligned} {\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t= & {} \left[ \begin{array}{c} {\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |0\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |1\right) \\ {\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |2\right) \end{array} \right] =\left[ \begin{array}{ccc} p^{1,loss}_{00} &{} p^{1,loss}_{01} &{} p^{1,loss}_{02} \\ p^{1,loss}_{10} &{} p^{1,loss}_{11} &{} p^{1,loss}_{12} \\ p^{1,gain}_{20} &{} p^{1,gain}_{21} &{} p^{1,gain}_{22} \end{array} \right] \\= & {} \left[ \begin{array}{ccc} c^1_{GG}v^{loss}_{HH} &{} c^1_{GG}v^{loss}_{HL} &{} c^1_{GB}v^{loss}_{HL} \\ c^1_{GG}v^{loss}_{LH} &{} c^1_{GG}v^{loss}_{LL} &{} c^1_{GB}v^{loss}_{LL} \\ c^1_{BG}v^{gain}_{LH} &{} c^1_{BG}v^{gain}_{LL} &{} c^1_{BB}v^{gain}_{LL} \end{array} \right] {\ ,} \end{aligned}$$

and we derive following relationships

$$\begin{aligned}&\frac{c^1_{GG}v^{loss}_{HH}}{c^1_{GG}v^{loss}_{LH}}\ge \frac{c^1_{GG}v^{loss}_{HL}}{c^1_{GG}v^{loss}_{LL}}=\frac{c^1_{GB}v^{loss}_{HL}}{c^1_{GB}v^{loss}_{LL}}\\&\quad \mathrm { and }\frac{c^1_{GG}v^{loss}_{LH}}{c^1_{BG}v^{gain}_{LH}}=\frac{c^1_{GG}v^{loss}_{LL}}{c^1_{BG}v^{gain}_{LL}}\ge \frac{c^1_{GB}v^{loss}_{LL}}{c^1_{BB}v^{gain}_{LL}}{\ ,} \end{aligned}$$

where the first inequality holds because (C2) and the second inequality holds due to (C3). The second equality is based on (C1). Therefore, ${\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |0\right) {\le }_r{\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |1\right) {\le }_r{\widetilde{\varvec{{\Gamma }}}}^{A,s^C_t}_t\left( \cdot |2\right)$ which implies that ${\varvec{{\Gamma }}}^{A,s^C_t}_t\in {TP}_2$ from Definition 3. Based on all results above, we see that ${\varvec{{\Gamma }}}^{a,o}_t\in {TP}_2$ for $a\in {\mathcal {A}}$, $o\in \varvec{O}$, and $t\le t_E$. $\square$

Lemma 1 shows that the state transition probability matrix ${{\varvec{\Gamma }}}_t^{a,o}$ has the $TP_2$ property. We can also show that the $TP_2$ property can be retained after simplifying the state transition probability matrix to ${{\varvec{\Gamma }}}_t^{a}$ as follows:

Proposition 1

Suppose (C1)–(C3) hold, then the ${{\varvec{\Gamma }}}_t^{a,o}\in {TP}_2$ for all $a\in {\mathcal {A}}$ and $t\le t_E$.

Proof of Proposition 1

To recall, the overall transition probability for $a_t=W$ is expressed as

$$\begin{aligned} {\varvec{{\Gamma }}}^W_t=\left[ \begin{array}{ccc} \left( 1-q^{0,W}_t\right) p^{0,none}_{00}+q^{0,W}_tp^{1,gain}_{00} &{} \left( 1-q^{0,W}_t\right) p^{0,none}_{01}+q^{0,W}_tp^{1,gain}_{01} &{} \left( 1-q^{0,W}_t\right) p^{0,none}_{02}+q^{0,W}_tp^{1,gain}_{02} \\ \left( 1-q^{1,W}_t\right) p^{0,none}_{10}+q^{1,W}_tp^{1,gain}_{10} &{} \left( 1-q^{1,W}_t\right) p^{0,none}_{11}+q^{1,W}_tp^{1,gain}_{11} &{} \left( 1-q^{1,W}_t\right) p^{0,none}_{12}+q^{1,W}_tp^{1,gain}_{12} \\ \left( 1-q^{2,W}_t\right) p^{0,none}_{20}+q^{2,W}_tp^{1,loss}_{20} &{} \left( 1-q^{2,W}_t\right) p^{0,none}_{21}+q^{2,W}_tp^{1,loss}_{21} &{} \left( 1-q^{2,W}_t\right) p^{0,none}_{22}+q^{2,W}_tp^{1,loss}_{22} \end{array} \right] , \end{aligned}$$

Now, let $\varvec{p}^{x,z}_{i\cdot }$ denote the a vector of transition probabilities from state i to $j\in \left\{ 0,1,2\right\}$ for any given $x\in \left\{ 1,0\right\}$ and $z\in \left\{ none,gain,loss\right\}$. Then, from Definition 2, we obtain

$$\begin{aligned} \frac{c^0_{GG}v^{none}_{HH}}{c^1_{GG}v^{gain}_{HH}} \le \frac{c^0_{GG}v^{none}_{HL}}{c^1_{GG}v^{gain}_{HL}}=\frac{c^0_{GB}v^{none}_{HL}}{c^1_{GG}v^{gain}_{HL}}\ \Leftrightarrow \ \varvec{p}^{1,gain}_{0\cdot }{\varvec{\le }}_r \varvec{p}^{0,none}_{0\cdot }\ , \end{aligned}$$

where the inequality holds because of condition (C1) and the equality holds due to condition (C3). Similarly, we show

$$\begin{aligned} \frac{c^0_{BG}v^{none}_{LH}}{c^1_{BG}v^{loss}_{LH}}=\frac{c^0_{BG}v^{none}_{LL}}{c^1_{BG}v^{loss}_{LL}}\le \frac{c^0_{BB}v^{none}_{LL}}{c^1_{BB}v^{loss}_{LL}}\ \Leftrightarrow \ \varvec{p}^{1,loss}_{2\cdot }{\varvec{\le }}_r\varvec{p}^{0,none}_{2\cdot }\ , \end{aligned}$$

where the equality and inequality hold due to conditions (C1) and (C3), respectively. Furthermore, from the same conditions (C1) and (C3), it is easy to show that $\varvec{p}^{1,gain}_{1\cdot }\varvec{=}\varvec{p}^{0,none}_{1\cdot }$ because $c^0_{GG}=c^1_{GG}$, $c^0_{GB}=c^1_{GG}$, and $v^{none}_{LL}=v^{gain}_{LH}$.

Let ${\varvec{{\varGamma }}}^W_t\left( \cdot |i\right)$ denote a $\left| S\right|$-dimensional vector of state transition probabilities from state i to $j\in \left\{ 0,1,2\right\}$. Then, from Lemma A.4, we get

$$\begin{aligned}\left\{ \ \ \ \ \begin{array}{l} \varvec{p}^{1,gain}_{0\cdot }{\le }_r{\varvec{{\varGamma }}}^W_t\left( 0|\cdot \right) {\le }_r\varvec{p}^{0,none}_{0\cdot }\ \\ \varvec{p}^{0,none}_{1\cdot }={\varvec{{\varGamma }}}^W_t\left( 1|\cdot \right) =\varvec{p}^{1,gain}_{1\cdot }\ \ \\ \varvec{p}^{1,loss}_{2\cdot }{\le }_r{\varvec{{\varGamma }}}^W_t\left( 2|\cdot \right) {\le }_r\varvec{p}^{0,none}_{2\cdot }\ \end{array} \right. \ .\end{aligned}$$

Now we show

$$\begin{aligned}\left\{ \ \ \ \ \begin{array}{l} \frac{c^0_{GG}v^{none}_{HH}}{c^1_{GG}v^{none}_{LH}}\ge \frac{c^0_{GG}v^{none}_{HL}}{c^1_{GG}v^{none}_{LL}}=\frac{c^0_{GB}v^{none}_{HL}}{c^1_{GG}v^{none}_{LL}}\ \Leftrightarrow \ \varvec{p}^{0,none}_{0\cdot }{\le }_r\varvec{p}^{0,none}_{1\cdot } \\ \frac{c^1_{GG}v^{gain}_{LH}}{c^1_{BG}v^{loss}_{LH}}=\frac{c^1_{GG}v^{gain}_{LL}}{c^1_{BG}v^{loss}_{LL}}\ge \frac{c^1_{GB}v^{gain}_{LL}}{c^1_{BB}v^{loss}_{LL}}\ \Leftrightarrow \ \varvec{p}^{1,gain}_{1\cdot }{\le }_r\varvec{p}^{1,loss}_{2\cdot } \end{array} \right. \ ,\end{aligned}$$

where the inequality in the first expression holds because of condition (C2) and the rest is due to condition (C3). Based on this result, we obtain following relationship:

$$\begin{aligned}&\varvec{p}^{1,gain}_{0\cdot }{\le }_r{\varvec{{\varGamma }}}^W_t\left( \cdot |0\right) {\le }_r\varvec{p}^{0,none}_{0\cdot }{\le }_r\varvec{p}^{0,none}_{1\cdot }={\varvec{{\varGamma }}}^W_t\left( \cdot |1\right) \\&\quad =\varvec{p}^{1,gain}_{1\cdot }{\le }_r\varvec{p}^{1,loss}_{2\cdot }{\le }_r{\varvec{{\varGamma }}}^W_t\left( \cdot |2\right) {\le }_r\varvec{p}^{0,none}_{2\cdot }\\&\quad \Longleftrightarrow \ {\varvec{{\varGamma }}}^W_t\left( \cdot |0\right) {\le }_r{\varvec{{\varGamma }}}^W_t\left( \cdot |1\right) {\le }_r{\varvec{{\varGamma }}}^W_t\left( \cdot |2\right) \ .\end{aligned}$$

Therefore, based on Definition 3, ${\varvec{{\varGamma }}}^W_t\in {TP}_2$.

For $a_t=A$, we follow the identical procedures and get following relationships:

$$\begin{aligned}\left\{ \ \ \ \ \begin{array}{l} \frac{c^0_{GG}v^{none}_{HH}}{c^1_{GG}v^{loss}_{HH}}\ge \frac{c^0_{GG}v^{none}_{HL}}{c^1_{GG}v^{loss}_{HH}}=\frac{c^0_{GB}v^{none}_{HL}}{c^1_{GB}v^{loss}_{HL}}\ \Leftrightarrow \ \varvec{p}^{0,none}_{0\cdot }{\varvec{\le }}_r\varvec{p}^{1,loss}_{0\cdot }\ \\ \frac{c^0_{GG}v^{none}_{LH}}{c^1_{GG}v^{loss}_{LH}}=\frac{c^0_{GG}v^{none}_{LL}}{c^1_{GG}v^{loss}_{LL}}=\frac{c^0_{GB}v^{none}_{LL}}{c^1_{GB}v^{loss}_{LL}}\ \Leftrightarrow \ \varvec{p}^{0,none}_{1\cdot }=\varvec{p}^{1,loss}_{1\cdot }\ \ \\ \frac{c^0_{BG}v^{none}_{LH}}{c^1_{BG}v^{gain}_{LH}}\le \frac{c^0_{BG}v^{none}_{LL}}{c^1_{BG}v^{gain}_{LL}}=\frac{c^0_{BB}v^{none}_{LL}}{c^1_{BB}v^{gain}_{LL}}\ \Leftrightarrow \ \varvec{p}^{1,gain}_{2\cdot }{\varvec{\le }}_r\varvec{p}^{0,none}_{2\cdot }\ \end{array} \right. \ ,\end{aligned}$$

where (C1) and (C3) were used and, using (C2) and (C3), we further obtain

$$\begin{aligned}\left\{ \ \ \ \ \begin{array}{l} \frac{c^1_{GG}v^{loss}_{HH}}{c^1_{GG}v^{loss}_{LH}}\ge \frac{c^1_{GG}v^{loss}_{HL}}{c^1_{GG}v^{loss}_{LL}}=\frac{c^1_{GB}v^{loss}_{HL}}{c^1_{GB}v^{loss}_{LL}}\ \Leftrightarrow \ \varvec{p}^{1,loss}_{0\cdot }{\varvec{\le }}_r\varvec{p}^{1,loss}_{1\cdot } \\ \frac{c^1_{GG}v^{loss}_{LH}}{c^1_{BG}v^{gain}_{LH}}=\frac{c^1_{GG}v^{loss}_{LL}}{c^1_{BG}v^{gain}_{LL}}\ge \frac{c^1_{GB}v^{loss}_{LL}}{c^1_{BB}v^{gain}_{LL}}\ \Leftrightarrow \ \varvec{p}^{1,loss}_{1\cdot }{\varvec{\le }}_r\varvec{p}^{1,gain}_{2\cdot } \end{array} \right. \ . \end{aligned}$$

Now, from Lemma A.4, we show

$$\begin{aligned}&\varvec{p}^{0,none}_{0\cdot }{\varvec{\le }}_r{\varvec{{\varGamma }}}^A_t\left( \cdot |0\right) {\varvec{\le }}_r\varvec{p}^{1,loss}_{0\cdot }{\varvec{\le }}_r\varvec{p}^{1,loss}_{1\cdot }\\&\quad ={\varvec{{\varGamma }}}^A_t\left( \cdot |1\right) {\varvec{\le }}_r\varvec{p}^{1,gain}_{2\cdot }{\varvec{\le }}_r{\varvec{{\varGamma }}}^A_t\left( \cdot |2\right) {\varvec{\le }}_r\varvec{p}^{0,none}_{2\cdot }\\&\quad \Longleftrightarrow \ {\varvec{{\varGamma }}}^A_t\left( \cdot |0\right) {\varvec{\le }}_r{\varvec{{\varGamma }}}^A_t\left( \cdot |1\right) {\varvec{\le }}_r{\varvec{{\varGamma }}}^A_t\left( \cdot |2\right) \ .\end{aligned}$$

Therefore, based on Definition 3, ${\varvec{{\varGamma }}}^A_t\in {TP}_2$ hence ${\varvec{{\varGamma }}}^a_t\in {TP}_2$ for all $a\in {\mathcal {A}}$ and $t\le t_E$. $\square$

Furthermore, from Lemma 1, we get

Proposition 2

For any two beliefs $\varvec{\pi }$,$\varvec{\pi }^{'}\in {{\varvec{\Pi }}}$ such that $\varvec{\pi }\le _r\varvec{\pi }^{'}$, suppose (C1)–(C3) hold, then $\varvec{\tau }_{\varvec{\pi }}^{a,o} \le _r {\varvec{\tau }^{a,o}_{\varvec{\pi }^{'}}}$ for any $a\in {\mathcal {A}}$ and $o\in {\mathbf {O}}$.

Proof of Proposition 2

Let ${\varvec{{\Psi }}}_t\left( o\right)$ denote a 3-by-3 matrix defined as

$$\begin{aligned}{\varvec{{\Psi }}}_t\left( o\right) =\left[ \begin{array}{ccc} {{\Lambda }}^{0,a}_t\left( o\right) &{} {{\Lambda }}^{0,a}_t\left( o\right) &{} {{\Lambda }}^{0,a}_t\left( o\right) \\ {{\Lambda }}^{1,a}_t\left( o\right) &{} {{\Lambda }}^{1,a}_t\left( o\right) &{} {{\Lambda }}^{1,a}_t\left( o\right) \\ {{\Lambda }}^{2,a}_t\left( o\right) &{} {{\Lambda }}^{2,a}_t\left( o\right) &{} {{\Lambda }}^{2,a}_t\left( o\right) \end{array} \right] \ ,\end{aligned}$$

which has three identical columns for $o\in \varvec{O}$ and $a\in {\mathcal {A}}$. Now, based on Definition 3, it is easy to show that the following matrix has ${TP}_2$ property

$$\begin{aligned}{\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t=\left[ \begin{array}{ccc} {{\Lambda }}^{0,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 0|0\right) &{} {{\Lambda }}^{0,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 1|0\right) &{} {{\Lambda }}^{0,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 2|0\right) \\ {{\Lambda }}^{1,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 0|1\right) &{} {{\Lambda }}^{1,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 1|1\right) &{} {{\Lambda }}^{1,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 2|1\right) \\ {{\Lambda }}^{2,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 0|2\right) &{} {{\Lambda }}^{2,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 1|2\right) &{} {{\Lambda }}^{2,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 2|2\right) \end{array} \right] \ , \end{aligned}$$

where the operator $\circ$ indicates element-wise multiplication (Hadamard product). For each belief, we get

$$\begin{aligned}\varvec{\pi }\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right) \varvec{=}{\left[ \sum _{{\varvec{s}}\in S}{\pi \left( s'\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) }\right] }_{s'\in \varvec{S}}\varvec{\ ,}\end{aligned}$$

and

$$\begin{aligned}\varvec{\pi }\varvec{'}\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right) \varvec{=}{\left[ \sum _{{\varvec{s}}\in S}{\pi '\left( s'\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) }\right] }_{s'\in \varvec{S}}\varvec{\ .}\end{aligned}$$

From Lemma A.3, $\varvec{\pi }\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right) {\varvec{\le }}_r\varvec{\pi }\varvec{'}\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right)$ holds because ${\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\in {TP}_2$. Based on Definition 3, we rewrite the expression as

$$\begin{aligned}&\varvec{\pi }\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right) {\varvec{\le }}_r{\varvec{\pi }'}\left( {\varvec{{\Psi }}}_t\left( o\right) \circ {\varvec{{\Gamma }}}^{a,o}_t\right) \nonumber \\&\quad \Longrightarrow \ \frac{\sum _{{\varvec{s}}\in S}{\pi \left( 0\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 0|s\right) }}{\sum _{{\varvec{s}}\in S}{\pi '\left( 0\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 0|s\right) }}\\&\quad \ge \frac{\sum _{{\varvec{s}}\in S}{\pi \left( 1\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 1|s\right) }}{\sum _{{\varvec{s}}\in S}{\pi '\left( 1\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 1|s\right) }}\\&\quad \ge \frac{\sum _{{\varvec{s}}\in S}{\pi \left( 2\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 2|s\right) }}{\sum _{{\varvec{s}}\in S}{\pi '\left( 2\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( 2|s\right) }}. \end{aligned}$$

From the definition of our belief updating function, we get

$$\begin{aligned}&\frac{{\tau }^{a,o}_{\pi }\left( 0\right) \sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}{{\tau }^{a,o}_{\pi '}\left( 0\right) \sum _{s\in S}{\pi '\left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}\\&\quad \ge \frac{{\tau }^{a,o}_{\pi }\left( 1\right) \sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}{{\tau }^{a,o}_{\pi '}\left( 1\right) \sum _{s\in S}{\pi '\left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}\\&\quad \ge \frac{{\tau }^{a,o}_{\pi }\left( 2\right) \sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}{{\tau }^{a,o}_{\pi '}\left( 2\right) \sum _{s\in S}{\pi '\left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}\ .\end{aligned}$$

Multiplying a non-negative quantity $\sum _{s\in S}{\pi '\left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }/\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }$ yields

$$\begin{aligned} \frac{\tau _\pi ^{a,o}(0)}{\tau _{\pi '}^{a,o}(0)}\ge \frac{\tau _{\pi }^{a,o} (1)}{\tau _{\pi '}^{a,o} (1)}\ge \frac{\tau _\pi ^{a,o} (2)}{\tau _{\pi '}^{a,o} (2)}, \end{aligned}$$

which implies ${\varvec{\tau }}^{a,o}_{\pi }{\le }_r{\varvec{\tau }}^{a,o}_{\pi '}$ for any $a\in {\mathcal {A}}$ and $o\in \varvec{O}$ based on Definition 3. $\square$

The Analytical Result 1 is based on Lemma 1 and Proposition 1, and the Analytical Result 2 is equivalent to Proposition 2.

Proof of analytical result 3

The Analytical Result 3 shows that our optimal value function is nonincreasing in $\varvec{\pi }\in {{\varvec{\Pi }}}$ without assuming an unrealistic $TP_2$ assumption on the observation probability matrix. First, we give Lemma 2 as follows:

Lemma 2

For $y_t\in \varvec{Y}$ and $s^C_t\in \{G,B\}$, let $\varvec{\xi }_{s^C_t}=\left[ \xi _{s^C_t}(y_t)\right]$ be a |Y|-dimensional probability vector where $\xi _{s^C_t}(y_t)=P(y_t|s^C_t)$ and define $\gamma =\xi _G(y^M)/\xi _B(y^M)$ where $0<\gamma \le 1$. Suppose following conditions are satisfied for all $a\in {\mathcal {A}}$, then the observation probability matrix $\varvec{\varLambda }^a_t={\left[ {{\Lambda }}^{s,a}_t(o)\right] }_{s\in \varvec{S},o\in \varvec{O}}\in {TP}_2$ for all $a\in {\mathcal {A}}$ and $t\le t_E$.

$$\begin{aligned} \text {(C4) }&\varvec{\xi }_G {\le }_r {\varvec{\xi }}_B,&\\ \text {(C5) }&q^{0,a}_t\le q^{1,a}_t\le \delta \left( a\right) =q^{1,a}_t/\left\{ \gamma \left( 1-q^{1,a}_t\right) +q^{1,a}_t\right\} \le q^{2,a}_t.&\end{aligned}$$

Proof of Lemma 2

We give proof only for $a_t=W$ because the proof for $a_t=A$ is identical. For $a_t=W$, the observation matrix for $o_t\in \varvec{O}=\left\{ 0,\ 1,\ 2^+,s^C_t\right\}$ is defined as

$$\begin{aligned} {\varvec{{\Lambda }}}^W_t=\left[ \begin{array}{c} {\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }0\right) \\ {\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }1\right) \\ {\varvec{{\Lambda }}}^W_t\left( \cdot \left| \right. 2\right) \end{array} \right] =\left[ \begin{array}{cc} \begin{array}{cc} \left( 1-q^{0,W}_t\right) {\xi }_G\left( 0\right) &{} \left( 1-q^{0,W}_t\right) {\xi }_G\left( 1\right) \\ \left( 1-q^{1,W}_t\right) {\xi }_G\left( 0\right) &{} \left( 1-q^{1,W}_t\right) {\xi }_G\left( 1\right) \\ \left( 1-q^{2,W}_t\right) {\xi }_B\left( 0\right) &{} \left( 1-q^{2,W}_t\right) {\xi }_B\left( 1\right) \end{array} &{} \begin{array}{cc} \left( 1-q^{0,W}_t\right) {\xi }_G\left( 2^+\right) &{} q^{0,W}_t \\ \left( 1-q^{1,W}_t\right) {\xi }_G\left( 2^+\right) &{} q^{1,W}_t \\ \left( 1-q^{2,W}_t\right) {\xi }_B\left( 2^+\right) &{} q^{2,W}_t \end{array} \end{array} \right] \ ,\end{aligned}$$

because ${{\Lambda }}^{s,a}_t\left( o=s^C_t\right) =q^{s,a}_t$ and ${{\Lambda }}^{s,a}_t\left( o=y_t\right) =\left( 1-q^{s,a}_t\right) \times P\left( y_t|s^C_t\right)$ for $y_t\in \varvec{Y}$. Each row in ${\varvec{{\Lambda }}}^W_t$, $\varvec{{\Lambda }}^W_t(\cdot |i)$, denotes an observation probability vector with dimension of $\left| \varvec{O}\right|$ for state i.

For the first and second rows in ${\varvec{{\Lambda }}}^W_t$, we see

$$\begin{aligned} \frac{\left( 1-q^{0,W}_t\right) {\xi }_G\left( 0\right) }{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 0\right) }= & {} \frac{\left( 1-q^{0,W}_t\right) {\xi }_G\left( 1\right) }{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 1\right) }\\= & {} \frac{\left( 1-q^{0,W}_t\right) {\xi }_G\left( 2^+\right) }{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 2^+\right) }\ge \frac{q^{0,W}_t}{q^{1,W}_t}\ ,\end{aligned}$$

where the last inequality holds because of (C2). Thus, from Definition 2, ${\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }0\right) {\varvec{\le }}_r{\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }1\right)$.

Similarly, for the second and third rows in ${\varvec{{\Lambda }}}^W_t$, we get

$$\begin{aligned} \frac{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 0\right) }{\left( 1-q^{2,W}_t\right) {\xi }_B\left( 0\right) }\ge & {} \frac{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 1\right) }{\left( 1-q^{2,W}_t\right) {\xi }_B\left( 1\right) }\\\ge & {} \frac{\left( 1-q^{1,W}_t\right) {\xi }_G\left( 2^+\right) }{\left( 1-q^{2,W}_t\right) {\xi }_B\left( 2^+\right) }\ge \frac{q^{1,W}_t}{q^{2,W}_t}\ ,\end{aligned}$$

where the last inequality holds due to (C2).

From (C1), ${\varvec{\xi }}_G{\varvec{\le }}_r{\varvec{\xi }}_B$ which implies ${\xi }_G\left( 0\right) /{\xi }_B\left( 0\right)$ $\ge$ ${\xi }_G\left( 1\right) /{\xi }_B\left( 1\right)$ $\ge$ ${\xi }_G\left( 2\right) /{\xi }_B\left( 2\right)$. Because $\left( 1-q^{1,W}_t\right) /\left( 1-q^{2,W}_t\right)$ is always positive, the inequalities do not change after multiplication. Therefore, the first and second inequalities hold. Now, based on Definition 2, ${\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }1\right) {\varvec{\le }}_r{\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }2\right)$. Because ${\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }0\right) {\varvec{\le }}_r{\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }1\right)$ and ${\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }1\right) {\varvec{\le }}_r{\varvec{{\Lambda }}}^W_t\left( \cdot {\left| \right. }2\right)$, from Definition 3, ${\varvec{{\Lambda }}}^W_t={\left[ {{\Lambda }}^{s,W}_t\left( o\right) \right] }_{s\in \varvec{S},o\in \varvec{O}}\in {TP}_2$ for $t\le t_E$. Following the same procedure for $a=A$, it is easy to show that ${\varvec{{\Lambda }}}^A_t={\left[ {{\Lambda }}^{s,A}_t\left( o\right) \right] }_{s\in \varvec{S},o\in \varvec{O}}\in {TP}_2$ for $t\le t_E$. $\square$

We stated that (C5) is not a viable assumption in the SAM application. Therefore, we need to find a way to ensure monotonic nonincreasing value function in $\varvec{\pi }\in {{\varvec{\Pi }}}$ without depending on Lemma 2. To do so, we give Lemma 3 and Proposition 3 as below:

Lemma 3

The value function for $a\in {\mathcal {A}}$ is $V^a_t(\varvec{\pi })=\sum _{s\in \varvec{S}}{\pi (s)\left[ r_t(s,a)+\sum _{s'\in S}{\varGamma ^a_t(s'|s){\widetilde{\alpha }}^{\kappa (\varvec{\pi },a)}_{t+1}(s')}\right] }$ where $\kappa (\varvec{\pi },a)=\mathrm {argmax}_k \left[ \sum _{s\in \varvec{S}}{\pi (s)\sum _{s'\in \varvec{S}}{\varGamma ^a_t(s'|s)\varvec{\alpha }^k_{t+1}(s')}}\right]$ and $\alpha ^k_{t+1}(s')$ is $\alpha$-vector defined by Lemma A.5. Then, based on the revised $\alpha$-vector ${\widetilde{\alpha }}_t^\kappa (\varvec{\pi },a)$, the optimizing $\alpha$-vector for a given belief $\varvec{\pi }\in \varvec{\Pi }$ is denoted as $\alpha ^{k^*(\varvec{\pi })}_t$ where $k^*(\varvec{\pi })=\mathrm {argmax}_{\left\{ \kappa \left( \varvec{\pi },W\right) ,\kappa \left( \varvec{\pi },A\right) \right\} } \left[ \sum _{s\in \varvec{S}}{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi },W\right) }_t\left( s\right) },\sum _{s\in \varvec{S}}{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi },A\right) }_t\left( s\right) }\right]$.

Proof of Lemma 3

First, we express the optimal value function for the updated belief ${\varvec{\tau } }^{a,o}_{\pi }$ in terms of $\alpha$-vectors introduced in Lemma A.5 as

$$\begin{aligned} V^*_{t+1}({\varvec{\tau } }^{a,o}_{\pi })&=\max _{k}\left[ \sum _{s'\in S}{\tau ^{a,o}_{\pi }(s')\alpha ^k_{t+1}(s')}\right] \\&=\max _{k} \left[ \sum _{s'\in S}{\left( \frac{\sum _{s\in S}{\pi (s)\varLambda ^{s,a}_t(o)\varGamma ^{a,o}_t(s'|s)}}{\sum _{s\in S}{\pi (s)\varLambda ^{s,a}_t(o)}}\right) {\alpha }^k_{t+1}\left( s'\right) }\right] \\&=\max _{k} \left[ \frac{1}{\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}\right. \\&\quad \left. \sum _{s'\in S}{\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }}\right] , \end{aligned}$$

where ${\alpha }^k_{t+1}\left( s'\right)$ is the $\alpha$-vector in Lemma A.5.

Now, the optimal value function for an action a is

$$\begin{aligned} \begin{aligned} V^a_t\left( \varvec{\pi }\right)&=\mathrm {max} \left[ \sum _{s\in S}{\sum _{o\in O}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) r_t\left( s,a,o\right) }}\right. \\&\quad \left. +\sum _{s\in S}{\sum _{o\in O}{\sum _{s'\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) V^*_{t+1}\left( {\varvec{\tau }}^{a,o}_{\pi }\right) }}}\right] \ \\&=\mathrm {max} \left[ \sum _{s\in S}{\sum _{o\in O}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) r_t\left( s,a,o\right) }}+ \right. \\&\quad \left. \sum _{s\in S}{\sum _{o\in O}{\sum _{s'\in S}{\frac{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) \left\{ \sum _{s'\in S}{\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) {{\Gamma }}^{a,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }}\right\} \ }{\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) }}}}}\right] \\&=\mathrm {max} \left[ \sum _{s\in S}{\sum _{o\in O}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) r_t\left( s,a,o\right) }}\right. \\&\quad \left. +\sum _{o\in O}{\sum _{s\in S}{\pi \left( s\right) {{\Lambda }}^{s,a}_t\left( o\right) \sum _{s'\in S}{{{\Gamma }}^{a,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }}}\right] \ \\&=\mathrm {max} \left[ \sum _{s\in S}\pi \left( s\right) \left\{ \sum _{o\in O}{{\Lambda }}^{s,a}_t\left( o\right) \left( r_t\left( s,a,o\right) \right. \right. \right. \\&\quad \left. \left. \left. +\sum _{s'\in S}{{{\Gamma }}^{a,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }\right) \right\} \right] \ \\&={\mathrm {max} \left[ \sum _{s\in S}{\pi \left( s\right) \left( r_t\left( s,a\right) +\sum _{s'\in S}{{{\Gamma }}^a_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }\right) }\right] }. \end{aligned} \end{aligned}$$

Then, the optimal value function for a given action a becomes $V^a_t\left( \varvec{\pi }\right) =\sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,a\right) }_t\left( s\right) }$, where $\kappa \left( \varvec{\pi } ,a\right) ={{\mathrm {argmax}}_k \left[ \sum _{s\in S}{\pi \left( s\right) \sum _{s'\in S}{{{\Gamma }}^a_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }}\right] \ }$.

Based on this formulation, we can see that the overall optimal value function is expressed as

$$\begin{aligned} V^*_t\left( \varvec{\pi }\right) =\mathrm {max}\left\{ V^W_t\left( \varvec{\pi }\right) ,V^A_t\left( \varvec{\pi }\right) \right\} =\sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{k^*\left( \pi \right) }_t}\ ,\end{aligned}$$

where $k^*\left( \varvec{\pi } \right) ={\mathop {\mathrm {argmax}}_{\left\{ \kappa \left( \varvec{\pi } ,W\right) ,\kappa \left( \varvec{\pi } ,A\right) \right\} } \left[ \sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,W\right) }_t\left( s\right) },\sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }\right] \ }$. $\square$

Proposition 3

Suppose the following conditions on disutility hold in addition to (C1)–(C3), then the optimizing revised $\alpha$-vector is non-increasing in $s\in S$ for an arbitrary belief $\varvec{\pi }\in \varvec{\Pi }$: that is, for any $s_1,s_2\in S$ such that $s_1<s_2$, ${{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi }\right) }_t\left( s_1\right) \ge {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi }\right) }_t\left( s_2\right)$ for all $t\le t_E$.

$$\begin{aligned} \text {(C6) }&\phi ^{o=y}_{s=0} = \phi ^{o=y}_{s=1}=\phi ^{o=y}_{s=2}=\phi _{a=W}=\phi _{s=0}=0,&\\ \text {(C7) }&\phi _{a=A}+{\phi }^{o=s^C}_{s=0}\le {\phi }_{s=1},&\\ \text {(C8) }&\phi _{s=1}+{\phi }_{a=A}+{\phi }^{o=s^C}_{s=1}\le {\phi }_{s=2}\le 1-{\phi }_{a=A}-{\phi }^{o=s^C}_{s=2}.&\end{aligned}$$

Proof of Proposition 3

From Lemma 3, we know that

$$\begin{aligned} {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,a\right) }_t=\mathrm {max}\left[ \sum _{o\in O}{{{\Lambda }}^{s,a}_t\left( o\right) \left( r_t\left( s,a,o\right) +\sum _{s'\in S}{{{\Gamma }}^{a,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }\right) }\right] {\ .} \end{aligned}$$

First, we can show that $r_t\left( s,a,o\right)$ is a non-increasing function in $s\in S$ for any $a{\in }{\mathcal {A}}$ and $o\in O$. Based on (C6), we can derive following minimums and maximums.

$$\begin{aligned} {\mathop {\mathrm {min}}_{a,o} \left[ r_t\left( s=0,a,o\right) \right] \ }&=1-{\phi }_{a=A}-{\phi }^{o=s^C}_{s=0}{\ ,} \\ {\mathop {\mathrm {max}}_{a,o} \left[ r_t\left( s=1,a,o\right) \right] \ }&=1-{\phi }_{s=1}{\ ,} \\ {\mathop {\mathrm {min}}_{a,o} \left[ r_t\left( s=1,a,o\right) \right] \ }&=1-{\phi }_{a=A}-{\phi }^{o=s^C}_{s=1}-{\phi }_{s=1}{\ ,} \text { and} \\ {\mathop {\mathrm {max}}_{a,o} \left[ r_t\left( s=2,a,o\right) \right] \ }&=1-{\phi }_{s=2}{\ .} \end{aligned}$$

From (C7)–(C8), it is straightforward to show that ${\mathop {\mathrm {min}}_{a,o} \left[ r_t\left( s=0,a,o\right) \right] \ }\ge {\mathop {\mathrm {max}}_{a,o} \left[ r_t\left( s=1,a,o\right) \right] \ }$ and ${\mathop {\mathrm {min}}_{a,o} \left[ r_t\left( s=1,a,o\right) \right] \ }\ge {\mathop {\mathrm {max}}_{a,o} \left[ r_t\left( s=2,a,o\right) \right] \ }$. Therefore, $r_t\left( s,a,o\right)$ is non-increasing in $s\in \varvec{S}$.

At the last time epoch $t=t_E$, the optimal value function is defined as

$$\begin{aligned} V^*_{t_E}\left( \varvec{\pi }\right) \mathrm {=}{\mathop {\mathrm {max}}_{a} \left[ \sum _{s\in \varvec{S}}{\pi \left( s\right) }\sum _{o\in \varvec{O}}{{{\Lambda }}^{s,a}_{t_E}\left( o\right) r_{t_E}\left( s,a,o\right) }\right] \ }{\ ,} \end{aligned}$$

which implies that the$\ \alpha$-vector can be defined as ${\alpha }_{t_E}\left( s\right) =r_{t_E}\left( s,a\right) =\sum _{o\in O}{{{\Lambda }}^{s,a}_{t_E}\left( o\right) r_{t_E}\left( s,a,o\right) }$. From the result above, $r_{t_E}\left( s,a,o\right)$ is non-increasing in $s\in \varvec{S}$ for any $a{\in }{\mathcal {A}}$ and $o\in \varvec{O}$ hence the optimizing $\alpha$-vector at time ${t_E}$ is non-increasing in $s\in \varvec{S}$. In other words, the assertion holds when $t={t_E}$. Now, assume inductively that the assertion holds at $t+1,\ t+2,\dots ,\ {t_E}$ and assume the optimizing $\alpha$-vector ${{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t$ is associated with action $a^*$. Based on (C1)–(C3) and Proposition 1, ${\varvec{{\Gamma }}}^a_t\in {TP}_2$ and, from Definition 3 and Lemma A.2, we get

$$\begin{aligned} {\varvec{{\Gamma }}}^a_t\left( \cdot |0\right) {\varvec{\le }}_r{\varvec{{\Gamma }}}^a_t\left( \cdot |1\right) {\varvec{\le }}_r{\varvec{{\Gamma }}}^a_t\left( \cdot |2\right) {\ }{\Rightarrow }{\varvec{{\Gamma }}}^a_t\left( \cdot |0\right) {\varvec{\le }}_s{\varvec{{\Gamma }}}^a_t\left( \cdot |1\right) {\varvec{\le }}_s{\varvec{{\Gamma }}}^a_t\left( \cdot |2\right) {\ ,}\end{aligned}$$

From the above inequalities and Lemma A.1,

$$\begin{aligned}&{{\Gamma }}^{a^*}_t\left( 0|0\right) {\alpha }^k_{t+1}\left( 0\right) +{{\Gamma }}^{a^*}_t\left( 1|0\right) {\alpha }^k_{t+1}\left( 1\right) +{{\Gamma }}^{a^*}_t\left( 2|0\right) {\alpha }^k_{t+1}\left( 2\right) \\&\quad {\ge }{{\Gamma }}^{a^*}_t\left( 0|1\right) {\alpha }^k_{t+1}\left( 0\right) +{{\Gamma }}^{a^*}_t\left( 1|1\right) {\alpha }^k_{t+1}\left( 1\right) +{{\Gamma }}^{a^*}_t\left( 2|1\right) {\alpha }^k_{t+1}\left( 2\right) {\ ,}\end{aligned}$$

where the inequality holds due to the induction assumption and similarly we get

$$\begin{aligned}&{{\Gamma }}^{a^*}_t\left( 0|1\right) {\alpha }^k_{t+1}\left( 0\right) +{{\Gamma }}^{a^*}_t\left( 1|1\right) {\alpha }^k_{t+1}\left( 1\right) +{{\Gamma }}^{a^*}_t\left( 2|1\right) {\alpha }^k_{t+1}\left( 2\right) \\&\quad {\ge }{{\Gamma }}^{a^*}_t\left( 0|2\right) {\alpha }^k_{t+1}\left( 0\right) +{{\Gamma }}^{a^*}_t\left( 1|2\right) {\alpha }^k_{t+1}\left( 1\right) +{{\Gamma }}^{a^*}_t\left( 2|2\right) {\alpha }^k_{t+1}\left( 2\right) {\ .}\end{aligned}$$

Therefore, $\sum _{s'}{{{\Gamma }}^{a^*}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }$ is non-increasing in $s\in S$.

Because both $r_t\left( s,a^*,o\right)$ and $\sum _{s'}{{{\Gamma }}^{a^*,o}_t\left( s'|s\right) {\alpha }^k_{t+1}\left( s'\right) }$ are non-increasing in $s\in S$, we have

$$\begin{aligned}&{{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t\left( s_1\right) \\&\quad =\sum _{o\in O}{{{\Lambda }}^{s_1,a^*}_t\left( o\right) \left( r_t\left( s_1,a^*,o\right) +\sum _{s'\in S}{{{\Gamma }}^{a^*,o}_t\left( s'|s_1\right) {\alpha }^k_{t+1}\left( s'\right) }\right) } \\&\quad =\sum _{o\in O}{{{\Lambda }}^{s_1,a^*}_t\left( o\right) r_t \left( s_1,a^*,o\right) }+\sum _{s'\in S}{{{\Gamma }}^{a^*}_t\left( s'|s_1\right) {\alpha }^k_{t+1}\left( s'\right) } \\&\quad \ge \sum _{o\in O}{{{\Lambda }}^{s_2,a^*}_t\left( o\right) r_t\left( s_2,a^*,o\right) }+\sum _{s'\in S}{{{\Gamma }}^{a^*}_t\left( s'|s_2\right) {\alpha }^k_{t+1}\left( s'\right) }={{\widetilde{\alpha }}}^{k^*\left( \pi \right) }_t\left( s_2\right) \ , \end{aligned}$$

where $\sum _{o\in O}{{{\Lambda }}^{s,a^*}_t\left( o\right) }\mathrm {=1}$ for any given $s\in S$. Based on the results above, for any $s_1,s_2\in S$ such that $s_1<s_2$, ${{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t\left( s_1\right) \ge {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t\left( s_2\right)$ for all $t\le {t_E}$. $\square$

Finally, based on Lemma 3 and Proposition 3, we give Theorem 1.

Theorem 1

Suppose (C1)–(C3) and (C6)–(C8) hold. Then, for any belief vectors $\varvec{\pi },\varvec{\pi }'\in \varvec{{\varPi }}$ such that $\varvec{\pi }{\le }_s\varvec{\pi }'$, $V^*_t\left( \varvec{\pi }\right) \ge V^*_t\left( \varvec{\pi }'\right)$ for all $t\le t_E$.

Proof of Theorem 1

Based on Lemma 3, the optimal value function is

$$\begin{aligned} V^*_t\left( \varvec{\pi }\right) ={\mathop {\mathrm {max}}_{k} \left[ \sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^k_t\left( s|\varvec{\pi } \right) }\right] \ }=\sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t\left( s\right) }{\ .}\end{aligned}$$

From Lemma A.1 and Proposition 3, we get

$$\begin{aligned} V^*_t\left( \varvec{\pi }\right) =\sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi } \right) }_t\left( s\right) }\ge \sum _{s\in S}{\pi \left( s\right) {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi }'\right) }_t\left( s\right) }\ge \sum _{s\in S}{\pi '\left( s\right) {{\widetilde{\alpha }}}^{k^*\left( \varvec{\pi }'\right) }_t\left( s\right) }=V^*_t\left( \varvec{\pi }'\right) {\ ,}\end{aligned}$$

where the first inequality holds because of the definition of the optimal value function. $\square$

As we see in Theorem 1, we do not need the ${TP}_2$ property on the observation probability matrix. Therefore, it is permissible to violate (C5) in Lemma 2. The Analytical Result 3 is a summary of Theorem 1, Lemma 3, and Proposition 3.

Proof of analytical result 4

The Analytical Result 4 is based on Theorem 2 and Corollary 1.

Theorem 2

Let $a^*_t\left( \varvec{\pi }\right)$ denote the optimal action at time t for a given belief $\varvec{\pi }\in \varvec{{\varPi }}$. Suppose (C1)–(C4) and (C6)–(C8) hold. Furthermore, suppose (C5) holds for$\ a=W$ and $\sum _s{\left( \pi \left( s\right) -\pi '\left( s\right) \right) \sum _{s'}{\varGamma ^A_t(s'|s){{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi },A\right) }_{t+1}\left( s'\right) }}\ge 0$ holds. Then, if $a^*_t\left( \varvec{\pi }\varvec{'}\right) =W$ then $a^*_t\left( \varvec{\pi }\right) =W$ and if $a^*_t\left( \varvec{\pi }\right) =A$, then $a^*_t\left( \varvec{\pi }\varvec{'}\right) =A$ for any $\varvec{\pi },\varvec{\pi }\varvec{'}\in \varvec{{\varPi }}$ such that $\varvec{\pi }{\le }_s\varvec{\pi }\varvec{'}$.

Proof of Theorem 2

Consider the case of $a^*_t\left( \varvec{\pi }\varvec{'}\right) =W$. Theorem 2 says that $V^W_t\left( \varvec{\pi }'\right) \ge V^A_t\left( \varvec{\pi }'\right)$ and $V^W_t\left( \varvec{\pi }\right) \ge V^A_t\left( \varvec{\pi }\right)$. Now, suppose the converse is true which means $V^W_t\left( \varvec{\pi }'\right) \ge V^A_t\left( \varvec{\pi }'\right)$ and $V^W_t\left( \varvec{\pi }\right) <V^A_t\left( \varvec{\pi }\right)$. In this case, we get

$$\begin{aligned} V^W_t\left( \varvec{\pi }'\right) -V^W_t\left( \varvec{\pi }\right) >V^A_t\left( \varvec{\pi }'\right) -V^A_t\left( \varvec{\pi }\right) . \end{aligned}$$

(1)

Because (C4) holds and (C5) holds for $a=W$, based on Lemma 2, ${\varvec{{\Lambda }}}^W_t\in {TP}_2$. Also, because (C1)–(C3) hold, ${\varvec{{\Gamma }}}^W_t\in {TP}_2$ based on Proposition 1. When both ${\varvec{{\Lambda }}}^W_t$ and ${\varvec{{\Gamma }}}^W_t$ have ${TP}_2$ property, we can show that $V^W_t\left( \varvec{\pi }\varvec{'}\right) -V^W_t\left( \varvec{\pi }\right) \le 0$ because $V^W_t\left( \varvec{\pi }\right) \ge V^W_t\left( \varvec{\pi }\varvec{'}\right)$ for $\varvec{\pi }{\varvec{\le }}_s\varvec{\pi }\varvec{'}$. We omit this proof which depends on (C6)–(C8), Lemma A.1, and Proposition 2.

Now, from (1), we obtain

$$\begin{aligned} V^A_t\left( {\varvec{\pi }'}\right)<V^A_t\left( \varvec{\pi }\right) \Rightarrow \sum _s{\pi '\left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ',A\right) }_t\left( s\right) }<\sum _s{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }\\ \Rightarrow \sum _s{\pi '\left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ',A\right) }_t\left( s\right) }-\sum _s{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }<0\ . \end{aligned}$$

Because ${{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t$ is the optimizing $\alpha$-vector, we get

$$\begin{aligned}&\sum _s{\pi '\left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }-\sum _s{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }\\&\quad \le \sum _s{{\pi }'{\left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( {\varvec{\pi }}',A\right) }_t\left( s\right) }}-\sum _s{\pi \left( s\right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }<0 \\&\quad \Rightarrow \sum _s{\left( \pi \left( s\right) -\pi '\left( s\right) \right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }<0, \end{aligned}$$

which contradicts to the condition $\sum _s{\left( \pi \left( s\right) -\pi '\left( s\right) \right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }\ge 0$.

Now, consider the case of $a^*_t\left( \varvec{\pi }\varvec{'}\right) =A$. Theorem 2 shows that $V^A_t\left( \varvec{\pi }\right) \ge V^W_t\left( \varvec{\pi }\right)$ and $V^A_t\left( \varvec{\pi }\varvec{'}\right) \ge V^W_t\left( \varvec{\pi }\varvec{'}\right)$. Suppose the converse is true which means $V^A_t\left( \varvec{\pi }\right) \ge V^W_t\left( \varvec{\pi }\right)$ and $V^A_t\left( \varvec{\pi }\varvec{'}\right) <V^W_t\left( \varvec{\pi }\varvec{'}\right)$. Then, we obtain

$$\begin{aligned}V^A_t\left( \varvec{\pi }\right) -V^A_t\left( \varvec{\pi }\varvec{'}\right) >V^W_t\left( \varvec{\pi }\right) -V^W_t\left( \varvec{\pi }\varvec{'}\right) \ge 0\ .\end{aligned}$$

Therefore, $V^A_t\left( {\varvec{\pi }}'\right) <V^A_t\left( \varvec{\pi }\right)$ and, as shown before, $\sum _s{\left( \pi \left( s\right) -\pi '\left( s\right) \right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }<0$ which contradicts to the condition $\sum _s{\left( \pi \left( s\right) -\pi '\left( s\right) \right) {{\widetilde{\alpha }}}^{\kappa \left( \varvec{\pi } ,A\right) }_t\left( s\right) }\ge 0$. $\square$

From Theorem 2, we get Corollary 1 as follows:

Corollary 1

Define two probabilities as ${\pi }^*_{GH}=\mathrm {max}\left\{ \pi \left( 0\right) :\ \pi \left( 1\right) =0,a^*\left( \varvec{\pi }\right) =A\right\}$ and ${\pi }^*_{GL}=\mathrm {max}\left\{ \pi \left( 1\right) :\ \pi \left( 0\right) =0,a^*\left( \varvec{\pi }\right) =A\right\}$. Suppose the conditions in Theorem 2 hold, then ${\pi }^*_{GH}\le {\pi }^*_{GL}$.

Proof of Corollary 1

Suppose the converse is true, ${\pi }^*_{GH}>{\pi }^*_{GL}$, and specify two beliefs ${\varvec{\pi }}_1$ and ${\varvec{\pi }}_2$ as ${\varvec{\pi }}_1=[ \begin{array}{ccc} {\pi }^*_{GH}&0&1-{\pi }^*_{GH} \end{array} ]$ and ${\varvec{\pi }}_2=[ \begin{array}{ccc} 0&{\pi }^*_{GH}&1-{\pi }^*_{GH} \end{array} ]$. Then, by definition, it is easy to see that $a^*\left( {\varvec{\pi }}_1\right) =A$ and ${\varvec{\pi }}_1{\le }_s{\varvec{\pi }}_2$. Therefore, based on Theorem 2, $a^*\left( {\varvec{\pi }}_2\right) =A$. Now, because $a^*\left( {\varvec{\pi }}_2\right) =A$ and ${\varvec{\pi }}_2\left( 0\right) =0$, by definition, we get ${\pi }^*_{GL}\ge {\pi }^*_{GH}$ which contradicts to ${\pi }^*_{GH}>{\pi }^*_{GL}$. $\square$

Appendix B: Sensitivity analysis

To check the robustness of our model and to quantify the impact of potential violation of our assumption on trust transition, we conduct a series of numerical experiments assuming that the patients are enrolled to the SAM program for a year (365 days). In our study, we initially assume that the trust state evolves according to the trust state transition probability matrix (Table 2 in the manuscript). There are two probability matrices: one for the case when the patient visited a clinic and went through a clinical diagnosis and another for the case when there was no diagnosis performed. The matrices are defined based on the assumption that trust mainly depends on the performance of the system (e.g., false alarm and misdetection reduce the trust level whereas correct alert/no-alert should increase the trust level). To forcefully create a hypothetical scenario where trust is affected by other unknown factors, we assume that a random trust state transition occurs with a probability $\theta$. In other words, instead of following the specific trust state transition function, for $365\times \theta$ days within a year, the trust level of the patient is determined by flipping a coin (50/50% chance of being in high/low trust state). By increasing $\theta$, to some extent, we can show how robust (or vulnerable) the method is when our assumption on trust transition was violated. Table 5 summarizes the numerical experiment results. For each simulation run, we assumed 1000 patients are using the SAM system. The end of decision time is 365 ($t_E$=365).

Table 5 Sensitivity analysis on random trust state transition

As expected, larger $\theta$ (i.e., trust depends more heavily on other factors than the performance of the system) reduces the performance (mean QALD) of our method. Under the perfect scenario where trust depends solely on the performance of the system, the average QALD per year is about 344 (as reported in Fig. 6 in the manuscript). The average QALD decreases by 6% when 10% of the trust state transition is triggered by other factors unknown to the model.