How patient compliance impacts the recommendations for colorectal cancer screening

Abstract

Colorectal cancer (CRC) is one of the most common cancers and the second leading cause of cancer death in the U.S. As a measure of prevention, timely screening is necessary for patients as it can spell the difference between life and death. Fecal occult blood tests (FOBTs) is used for the average-risk patient. Colonoscopy is used for the high-risk and the average-risk patient whose outcome of FOBTs is positive. While colonoscopy is considered to be the most accurate test for detecting colorectal cancer, its side-effects have serious consequences that could result in intestinal perforation and even death. As a result, some patients do not follow the physicians’ advices. It is therefore important to design a good screening schedule that balances the risk of CRC and the side-effects of colonoscopy, and at the same time to take the patient’s personal characteristics and compliance into account. We formulate a finite-horizon, partially observable Markov decision process model to optimize the CRC screening program for both average and high-risk patients. Our model incorporates information of prior screening history, patient compliance and more personal risk factors. We find that the patients with low compliance rate should be recommended to undergo colonoscopy more frequently.

Appendices

Appendix A: proof of Markov property

We illustrate the Markov property and how the historic information correlate current belief state for the average-risk patient in the following: To make this explicit, we define $h(t)$ as the total available information about the process at the end of control interval $t$. We assume that the time variable $t$ increases with increasing time. For the process as defined in this paper, there are two update processes (observations) during a control interval. One is after FOBT, the other is after colonoscopy. Let $n(t+1)$ and $o(t+1)$ denote the observation of FOBT and colonoscopy respectively, and $a(t+1)$ denotes the control alternative. During control interval $t+1$, we can write

$$\begin{aligned} h(t+)= & {} [o(t+1), a(t+1), h(t)]\\ h(t+1)= & {} [n(t+1), h(t+)] \end{aligned}$$

That is, $h(t+1)$ represents our state of information prior to control interval $t+1$ plus the additional information. $h(t+)$ represents state of information prior to FOBT. The Fig. 3 depicts the decision sequence.

Fig. 3

Decision sequence

By the definition of the information state vector,

$$\begin{aligned} b_{t+}(j)=Pr(s(t+)=j|h(t)). \end{aligned}$$

From Bayes’ rule, we obtain

$$\begin{aligned} b_{t+}(j)= & {} Pr(s(t+) = j, o(t)=\theta | a(t), h(t))/Pr(o(t)=\theta |a(t),h(t))\nonumber \\= & {} \sum _{i}Pr(s(t) = i|a(t),h(t))Pr(s(t+)=j|s(t)=i,a(t),h(t))\nonumber \\&Pr(o(t)=\theta |s(t+)=j,s(t)=i,a(t),h(t))/Pr(o(t)=\theta |a(t),h(t))\nonumber \\= & {} \sum _{i}b_{t}(j)p^{a}_{ij}R^{a}_{i\theta }/\sum _{ij}b_{t}(i) p^{a}_{ij}R^{a}_{i\theta } \end{aligned}$$

(2)

where $s(t+)$ and $s(t)$ are discrete-valued random variable equal to the internal state of the process at time $t+$ and $t$ respectively. Equation (1) indicates that $b_{t+}$ only depend on $b_{t}$, $a$, $o$. That is, $b_{t}$ summarizes all the information gained prior to control interval $t$ and represents a sufficient statistic for the complete past history of the process $h(t)$.

$$\begin{aligned} b_{t}(j)= & {} Pr(s(t)=j|h(t))\nonumber \\= & {} Pr(s(t)=j|n(t+1),h(t+))\nonumber \\= & {} \sum _{i}Pr(s(t)=j|s(t+)=i,n(t\!+\!1),h(t+))Pr(s(t+)\!=\!i|n(t+1),h(t\!+\!))\nonumber \\= & {} \sum _{i}p^{n}_{ij}b_{t+}(i) \end{aligned}$$

(3)

From Eqs. 2 and 3, the result is obtained.

Similarly, we can also obtain the Markov property for high-risk patients.

(note: $p^{a}_{ij}$ represents the transition probability from state $i$ to state $j$, given that action $a$ is chosen; $R^{a}_{i\theta }$ represents observation probability of $o=\theta$ when the action chosen is $a$ and the true health state is $i$. $p^{n}_{ij}$ represents the transition probability from state $i$ to state $j$, given that the number of positive outcome of FOBTs is $n$)

Appendix B: proofs of results in section 3

Proof of Theorem 1

Recall that

$$\begin{aligned}&W_{t}^{*}\left( b,\beta \right) \nonumber \\&\quad = \underset{a_{t}\in A}{\max }\beta \sum _{i\in S}b\left( i\right) \biggl \{\biggl [\sum _{o\in \left\{ T,TP\right\} }R_{t}^{a_{t}}\left( o\mid i\right) \left[ r_{t}(i,a_{t},o)+W_{t+1}^{*}(\bar{\sigma }_{t}[b,a,o]),\beta \right] \biggl ]\\&\qquad + \, R_{t}^{a_{t}}\left( TC\mid i\right) H_{t}\biggr \}\\&\qquad + \, (1-\beta )\sum _{i\in S}b\left( i\right) \biggl \{\biggl [R_{t}^{W}\left( T\mid i\right) \left[ r_{t}(i,W,T)+W_{t+1}^{*} (\bar{\sigma }_{t}[b,W,T]),\beta \right] \biggl ]\biggr \} \end{aligned}$$

We prove Theorem 1 by induction. By our boundary assumptions,

$$\begin{aligned} W_{T}^{*}\left( b_{T},\beta \right) =\sum _{i\in S^{b}}b_{T}\left( i\right) r_{T}\left( i\right) . \end{aligned}$$

Assume that $W_{t+1}^{*}(b,\beta )$ is a piecewise linear and convex function, i.e.,

$$\begin{aligned} W_{t+1}^{*}(b,\beta )=\max _{k}\sum _{i\in S^{b}}b(i)\bar{\alpha }_{t+1}^{k}(i), \end{aligned}$$

where $\bar{\alpha }_{t+1}^{k}(i)$ are some constants. We next show that $W_{t}^{*}(b,\beta )$ is a piecewise linear and convex function.

The $W_{t}(b,a_{t},\beta )$ means the total quality-adjusted life years from time $t$ to last decision period for a high-risk patient whose belief state is $b$, compliance rate is $\beta$ and action is $a_{t}$ at time $t$.

$$\begin{aligned}&W_{t}(b,a_{t},\beta )\nonumber \\&\quad = \beta \sum _{i\in S^{b}}\Biggl [b(i)\biggl \{\sum _{o\in \left\{ T,TP\right\} }R_{t}^{a_{t}}(o\mid i)\biggl [r_{t}(i,a_{t},o)\\&\qquad + \, W_{t+1}^{*}(\bar{\sigma }[b,a_{t},o],\beta )\biggl ]+R_{t}^{a_{t}}(TC\mid i)H_{t}\biggr \}\Biggl ]\\&\qquad +(1-\beta \, )\sum _{i\in S^{b}}\Biggl [b(i)R_{t}^{W}(T\mid i)\biggl [r_{t}(i,W,T)+W_{t+1}^{*}(\bar{\sigma }[b,W,T],\beta )\biggl ]\Biggl ] \end{aligned}$$

Similar with proofs of Li et al., by expanding the $W_{t+1}^{*}(\bar{\sigma }[b,a_{t},o],\beta )$, we can obtain the following result:

$$\begin{aligned}&W_{t}(b,a_{t},\beta ) \nonumber \\&\quad = \sum _{i\in S^{b}}b(i)\Biggl \{\beta \Biggr [\sum _{o\in \left\{ T,TP\right\} }R_{t}^{a_{t}}(o\mid i)\biggl [r_{t}(i,a_{t},o)\\ \nonumber \\&\qquad + \, \sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,a_{t},o)\bar{\alpha }_{t+1}^{\bar{l}(b,a_{t},o,\beta )}(i')\biggr ]+R_{t}^{a_{t}}(TC\mid i)H_{t}\Biggl ]\\ \nonumber \\&\qquad + \, (1-\beta )\Biggr [R_{t}^{W}(T\mid i)\biggr [r_{t}(i,W,T)+\sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,W,T)\bar{\alpha }_{t+1}^{\bar{l}(b,W,T,\beta )}(i')\biggl ]\Biggl ]\Biggl \} \end{aligned}$$

where

$$\begin{aligned} \bar{l}(b,a_{t},o,\beta )=\arg \max _{k}\left\{ \sum _{i\in S_{b}}b(i)R_{t}^{a_{t}}(o\mid i)\sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,a_{t},o)\bar{\alpha }_{t+1}^{k}(i')\right\} . \end{aligned}$$

Then,

$$\begin{aligned} W_{t}^{*}(b,\beta )= & {} \max _{a_{t}\in A}\sum _{i\in S^{b}}b(i)\Biggl \{\beta \Biggl [\sum _{o\in \left\{ T,TP\right\} }R_{t}^{a_{t}}(o\mid i)\biggl [r_{t}(i,a_{t},o)\\&\quad + \sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,a_{t},o)\bar{\alpha }_{t+1}^{\bar{l}(b,a_{t},o,\beta )}(i')\biggr ]+ R_{t}^{a_{t}}(TC\mid i)H_{t}\Biggr ]\\&\quad + \, (1-\beta )\Biggr [R_{t}^{W}(T\mid i)\biggr [r_{t}(i,W,T)+\sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,W,T)\bar{\alpha }_{t+1}^{\bar{l}(b,W,T,\beta )}(i')\biggl ]\Biggl ]\Biggl \}\\= & {} \max _{a_{t}\in A}\sum _{i\in S^{b}}b(i)\bar{\alpha }_{t}^{\bar{l}(b,a_{t},\beta )}(i)\\= & {} \sum _{i\in S^{b}}b(i)\bar{\alpha }_{t}^{\bar{l}(b,\beta )}(i) \end{aligned}$$

where

$$\begin{aligned}&\bar{\alpha }_{t}^{\bar{l}(b,a_{t},\beta )}(i) \nonumber \\&\quad = \beta \Biggl [\sum _{o\in \left\{ T,TP\right\} }R_{t}^{a_{t}}(o\mid i)\biggl [r_{t}(i,a_{t},o)+\sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,a_{t},o)\bar{\alpha }_{t+1}^{\bar{l}(b,a_{t},o,\beta )}(i')\biggr ]\\ \nonumber \\&\qquad + \, R_{t}^{a_{t}}(TC\mid i)H_{t}\Biggl ]\\ \nonumber \\&\qquad + \, (1-\beta _{t})\Biggl [R_{t}^{W}(T\mid i)\biggr [r_{t}(i,W,T)+\sum _{i'\in S^{b}}\bar{P}_{t}(i'\mid i,W,T)\bar{\alpha }_{t+1}^{\bar{l}(b,W,T,\beta )}(i')\biggl ]\Biggl ] \end{aligned}$$

and

$$\begin{aligned} \bar{l}(b,\beta )=\arg \max _{k}\sum _{i\in S_{b}}b(i)\alpha _{t}^{k}=\arg \max _{(\bar{l}(b,W,\beta ), \bar{l}(b,C,\beta ))}\sum _{i\in S_{b}}b(i)\bar{\alpha }_{t}^{\bar{l}(b,a_{t},\beta )}(i). \end{aligned}$$

Hence, the proof is done. $\square$

Proof of Theorem 2

Similar with Theorem 1, we also prove Theorem 2 by induction. By our boundary assumptions,

$$\begin{aligned} V_{T}^{*}\left( b_{T},\alpha \right) =\sum _{b_{T}\in S^{b}}b_{T}\left( i\right) r_{T}\left( i\right) . \end{aligned}$$

Assume that $V_{t+1}^{*}(b,\alpha )$ is piecewise linear and convex, i.e.,

$$\begin{aligned} V_{t+1}^{*}(b,\alpha )=\max _{k}\sum _{i\in S^{b}}b(i)\alpha _{t+1}^{k}(i), \end{aligned}$$

where $\alpha _{t+1}^{k}(i)$ are some constants. We next show that $V_{t}^{*}(b,\alpha )$ is a piecewise linear and convex function. First, we establish that $V_{t+,n}^{*}(b_{t}^{F},\alpha )$ is piecewise linear and convex. Let $V_{t+,n}(b_{t}^{F},a_{t},\alpha )$ represents the total quality-adjusted life years from time $t$ to last decision period for an average-risk patient whose belief state is $b$, compliance rate is $\alpha$ and action is $a_{t}$ at time $t$. From Theorem 1, we know that the $W_{t}^{*}(b,\beta )$ is piecewise linear and convex. We apply the result in the formulation of $V_{t+,n}^{*}(b_{t}^{F},a_{t},\alpha )$, and obtain

$$\begin{aligned}&V_{t+,n}(b_{t}^{F},a_{t},\alpha ) \nonumber \\&\quad = \alpha \sum _{i\in S^{b}}b_{t}^{F}(i)\biggl \{ R_{t}^{a_{t}}(T\mid i)\biggl [r_{t}(i,a_{t},T)+V_{t+1}^{*}\left( \sigma _{t}[b_{t}^{F},a_{t},T],\alpha \right) \biggl ]\nonumber \\&\qquad + \, R_{t}^{a_{t}}(TP\mid i)\biggl [r_{t}(i,a_{t},TP)+W_{t+1}^{*}\left( \bar{\sigma }_{t}[b_{t}^{F},a_{t},TP],\beta \right) \biggl ]+R_{t}^{a_{t}}(TC\mid i)H_{t}\biggl \}\nonumber \\&\qquad + \, (1-\alpha )\sum _{i\in S^{b}}b_{t}^{F}(i)\biggl \{ R_{t}^{W}(T\mid i)\biggl [r_{t}(i,W,T)+V_{t+1}^{*}\left( \sigma _{t}[b_{t}^{F},W,T],\alpha \right) \biggl ]\nonumber \\&\qquad + \, R_{t}^{W}(TC\mid i)H_{t}\biggl \}. \end{aligned}$$

By expanding the $V_{t+1}^{*}\left( \sigma _{t}[b_{t}^{F},a_{t},T],\alpha \right)$ and $W_{t+1}^{*}\left( \bar{\sigma }_{t}[b_{t}^{F},a_{t},TP],\beta \right)$, we obtain the following result:

$$\begin{aligned}&V_{t+,n}(b_{t}^{F},a_{t},\alpha ) \nonumber \\&\quad = \sum _{i\in S^{b}}b_{t}^{F}(i)\Biggl \{\alpha \biggl [R_{t}^{a_{t}}(T\mid i)\biggl [r_{t}(i,a_{t},T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,a_{t},T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},a_{t},T],a_{t},\alpha )}(i')\biggl ]\nonumber \\&\qquad + \, R_{t}^{a_{t}}(TP\mid i)\biggl [r_{t}(i,a_{t},TP)+\sum _{i''\in S^{b}}\bar{P}_{t}(i''\mid i,a_{t},TP)\bar{\alpha }_{t+1}^{\bar{l}(b_{t}^{F},a_{t},\beta )}(i'')\biggl ]+R_{t}^{a_{t}}(TC\mid i)H_{t}\biggr ]\nonumber \\&\qquad + \, (1-\alpha )R_{t}^{W}(T\mid i)\biggl [r_{t}(i,W,T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,W,T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},W,T],W,\alpha )}(i')\biggr ]\Biggl \} \end{aligned}$$

where

$$\begin{aligned} l(\sigma _{t}[b_{t}^{F},a_{t},T],a_{t},\alpha )=\arg \max _{k}\bigl \{\sum _{j\in S^{b}}\sigma _{t}[b_{t}^{F},a_{t},T](j)\alpha _{t+1}^{k}(i')\bigr \}, \end{aligned}$$

and

$$\begin{aligned} \bar{l}(b_{t}^{F},a_{t},\beta )=\arg \max _{k}\bigl \{\sum _{j'\in S^{b}}b_{t}^{F}(j')R_{t}^{a_{t}}(TP\mid j')\sum _{i''\in S^{b}}\bar{P}_{t}(i''\mid j',a_{t},TP)\bar{\alpha }_{t+1}^{k}(i'')\bigr \}. \end{aligned}$$

Hence, $V_{t+,n}^{*}(b_{t}^{F},\alpha )$ is a piecewise linear and convex function, i.e.,

$$\begin{aligned}&V_{t+,n}^{*}(b_{t}^{F},\alpha ) \nonumber \\&\quad = \max _{a_{t}\in A}\sum _{i\in S^{b}}b_{t}^{F}(i)\Biggl \{\alpha \biggl [R_{t}^{a_{t}}(T\mid i)\biggl [r_{t}(i,a_{t},T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,a_{t},T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},a_{t},T],a_{t},\alpha )}(i')\biggl ] \nonumber \\&\qquad + \, R_{t}^{a_{t}}(TP\mid i)\biggl [r_{t}(i,a_{t},TP)+\sum _{i''\in S^{b}}\bar{P}_{t}(i''\mid i,a_{t},TP)\bar{\alpha }_{t+1}^{\bar{l}(b_{t}^{F},a_{t},\beta )}(i'')\biggl ]+R_{t}^{a_{t}}(TC\mid i)H_{t}\biggr ]\nonumber \\&\qquad + \, (1-\alpha )R_{t}^{W}(T\mid i)\biggl [r_{t}(i,W,T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,W,T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},W,T],W,\alpha )}(i')\biggr ]\Biggl \}\nonumber \\&\quad = \max _{a_{t}\in A}\sum _{i\in S^{b}}b_{t}^{F}(i)\alpha _{t+}^{l_{+}(b_{t}^{F},a_{t},\alpha )}(i)\nonumber \\&\quad = \sum _{i\in S^{b}}b_{t}^{F}(i)\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i) \end{aligned}$$

where

$$\begin{aligned}&\alpha _{t+}^{l_{+}(b_{t}^{F},a_{t},\alpha )}(i)\nonumber \\&\quad = \alpha \Biggl [R_{t}^{a_{t}}(T\mid i)\biggl [r_{t}(i,a_{t},T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,a_{t},T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},a_{t},T],a_{t},\alpha )}(i')\biggl ]\nonumber \\&R_{t}^{a_{t}}(TP\mid i)\biggl [r_{t}(i,a_{t},TP)+\sum _{i''\in S^{b}}\bar{P}_{t}(i''\mid i,a_{t},TP)\bar{\alpha }_{t+1}^{\bar{l}([b_{t}^{F},a_{t},TP],\beta )}(i'')\biggl ]\nonumber \\&\qquad + \, R_{t}^{a_{t}}(TC\mid i)H_{t}\Biggr ]\nonumber \\&\qquad + \, (1-\alpha )R_{t}^{W}(T\mid i)\biggl [r_{t}(i,W,T)+\sum _{i'\in S^{b}}P_{t}(i'\mid i,W,T)\alpha _{t+1}^{l(\sigma _{t}[b_{t}^{F},W,T],W,\alpha )}(i')\biggr ] \end{aligned}$$

and

$$\begin{aligned} l_{+}(b_{t}^{F},\alpha )=\arg \max _{(l_{+}(b_{t}^{F},W,\alpha ), l_{+}(b_{t}^{F},C,\alpha ))}\sum _{i\in S^{b}}b_{t}^{F}(i)\alpha _{t+}^{l_{+}(b_{t}^{F},a_{t},\alpha )}(i). \end{aligned}$$

Then, we obtain

$$\begin{aligned} V_{t}^{*}(b,\alpha )= & {} \sum _{i\in S^{b}}b(i)\left( \sum _{n}R_{t}^{F}(n\mid i)V_{t+,n}^{*}(b_{t}^{F},\alpha )\right) \\= & {} \sum _{i\in S^{b}}b(i)\left( \sum _{n}R_{t}^{F}(n\mid i)\sum _{i'\in S^{b}}b_{t}^{F}(i')\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i')\right) \\= & {} \sum _{i\in S^{b}}b(i)\left( \sum _{n}R_{t}^{F}(n\mid i)\sum _{i'\in S^{b}}\frac{b(i')R_{t}^{F}(n\mid i')}{\sum _{j\in S^{b}}b(j)R_{t}^{F}(n\mid j)}\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i')\right) \\= & {} \sum _{n}\left( \left( \sum _{i\in S^{b}}b(i)R_{t}^{F}(n\mid i)\right) \sum _{i'\in S^{b}}\frac{b(i')R_{t}^{F}(n\mid i')}{\sum _{j\in S^{b}}b(j)R_{t}^{F}(n\mid j)}\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i')\right) \\= & {} \sum _{i'\in S^{b}}b(i')\left( \sum _{n}R_{t}^{F}(n\mid i')\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i')\right) . \end{aligned}$$

Let

$$\begin{aligned} \alpha _{t}^{l(b,\alpha )}(i')=\sum _{n}R_{t}^{F}(n\mid i')\alpha _{t+}^{l_{+}(b_{t}^{F},\alpha )}(i') , \end{aligned}$$

then

$$\begin{aligned} V_{t}^{*}(b,\alpha )=\sum _{i'\in S^{b}}b(i')\alpha _{t}^{l(b,\alpha )}(i'). \end{aligned}$$

$\square$