Practical interruptible conversations: distributed dynamic verification with multiparty session types and Python

Abstract

The rigorous and comprehensive verification of communication-based software is an important engineering challenge in distributed systems. Drawn from our industrial collaborations (Ocean Observatories Initative, http://www.oceanobservatories.org/, JBoss Savara Project, http://www.jboss.org/savara) on Scribble, a choreography description language based on multiparty session types, and its theoretical foundations (Honda et al., in POPL, pp 273–284, 2008), this article proposes a dynamic verification framework for structured interruptible conversation programming. We first present our extension of Scribble to support the specification of asynchronously interruptible conversations. We then implement a concise API for conversation programming with interrupts in Python that enables session types properties to be dynamically verified for distributed processes. Finally, we expose the underlying theory of our interrupt mechanism, studying its syntax and semantics, its integration in MPST theory and proving the correctness of our design. Our framework ensures the global safety of a system in the presence of asynchronous interrupts through independent runtime monitoring of each endpoint, checking the conformance of the local execution trace to the specified protocol. The usability of our framework for describing and verifying choreographic communications has been tested by integration into the large scientific cyberinfrastructure developed by the Ocean Observatories Initiative. Asynchronous interrupts have proven expressive enough to represent and verify their main classes of communication patterns, including asynchronous streaming and various timeout-based protocols, without introducing any implicit synchronisations. Benchmarks show conversation programming and monitoring can be realised with little overhead.

Appendix: Proofs

This appendix includes a full proof of Theorem 5.3.3.

Theorem 5.3

(Session fidelity) If $\varDelta$ corresponds to $G_1,\dots ,G_n$ and $\varDelta ,\varepsilon \rightarrow ^* \varDelta ',\varSigma '$, there exists $\varDelta ',\varSigma ' \rightarrow ^*\varDelta '',\varepsilon$ such that $\varDelta ''$ corresponds to $G''_1,\dots ,G''_n$ which is a derivative of $G_1,\dots ,G_n$.

Proof

We prove that if there is an intermediate correspondence between $\varDelta ,\varSigma$ and $G_1,\dots ,G_n$ and if $\varDelta , \varepsilon \rightarrow \varDelta ',\varSigma '$, then there is an intermediate correspondence $\varDelta ''$ and $G''_1,\dots ,G''_n$ which is a derivative of $G_1,\dots ,G_n$.

We use $\varOmega , \varTheta$ alongside $\varDelta$ to denote session environment. According to the derivative definition above, we extend the notion of evaluation contexts to global types.

Case $(\mathsf {Out})$ is trivial from the first rule of intermediate correspondence.

Case $(\mathsf {EOut})$. We have $\varDelta = \varTheta , s[\mathtt {r}]:E^{\mathtt {c}_0}[\{\!| T |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r} ? l \rangle ; T']$. Correspondence gives

• $\varDelta = \varTheta _0, s[\mathtt {r}]:E^{\mathtt {c}_0}[\{\!| T |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r} ? l \rangle ; T'], \prod _{1 \le i \le n} s[\mathtt {r}_i]:{E_{i}}^{\mathtt {c}_i}[\{\!| T_i |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r} ? l \rangle ; T'_i]$ and

• $\varSigma = \varSigma _1, \varSigma _0$ with $\varTheta _0;\varSigma _0$

corresponding to $G_2, \dots , G_n$ and $(\varDelta - \varTheta ');\varSigma _1$ corresponding to $G_1$. We know that $\varSigma ' = \varSigma _0, \prod _{1 \le i \le n} s[\mathtt {r}_i]:h_i.\mathtt {c}^{\mathtt {I}}[\mathtt {r},\mathtt {r}_i]\langle l \rangle$. Concluding is easy using the second rule of intermediate correspondence with $k = 0$.

Case $(\mathsf {In})$. We assume $\varDelta = \varTheta , s[\mathtt {r}']:E^{\mathtt {c}}[\mathtt {r}?\{l_i.T_i\}]$ and $\varSigma = \varSigma _0, s[\mathtt {r}']:h.\mathtt {c}[\mathtt {r},\mathtt {r}']\langle l_j \rangle$. We know there exists $G_1, \dots G_n$ and $\varDelta _0$ such that $\varDelta _0 = \varTheta _1, \dots , \varTheta _n$ with $\varTheta _i = \bigcup _{\mathtt {r}\in G_i} G_i \uparrow ^\mathtt {r}$.

Without loss of generality we have $\varTheta _1 = s[\mathtt {r}']:E^{\mathtt {c}}[\mathtt {r}?\{l_i.T_i\}], \varTheta '_1$. By the rules of projection, it means $G_1 = \mathtt {r}{\rightarrow } \mathtt {r}' \!:\! \{ l_j. G_j \} _{j\in {J}}$, implying $\varTheta '_1 = s[\mathtt {r}']:E^{\mathtt {c}'}[\mathtt {r}?\{l_i.T_i\}], \varTheta ''_1$. So we have

$$\begin{aligned} \varDelta ' = \varOmega , s[\mathtt {r}]:E^{\mathtt {c}}[T_j], s[\mathtt {r}']: E^{\mathtt {c}'}[\mathtt {r}?\{l_i.T_i\}], \varTheta '_1 \hbox { and } \varSigma ' = \varSigma _0, s[\mathtt {r}']:h.\mathtt {c}[\mathtt {r},\mathtt {r}']\langle l_j \rangle . \end{aligned}$$

We apply use $(\mathsf {In})$ to conclude, using the projection rule on $G_j$.

Case $(\mathsf {EIn_1})$. We pose $\mathtt {r}= \mathtt {r}_{k+1}$. We have $\varSigma = \varSigma ', \mathtt {c}^{\mathtt {I}}[\mathtt {r}_0,\mathtt {r}_{k+1}]\langle l \rangle .h$ and $\varphi (\varSigma ',\mathtt {c})$. Then let us define $\varDelta = \varTheta , s[\mathtt {r}_{k+1}]:E^{\mathtt {c}_0}[\{\!| T_{k+1} |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}_0 ? l \rangle ; T']$ and $\varDelta '= \varTheta ,\, s[\mathtt {r}_{k+1}]E^{\mathtt {c}_0}[\{\!| {\parallel } T {\parallel } |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}_0 ? l \rangle ; T']$. Without loss of generality we suppose $\varDelta ; \varSigma$ corresponds to $G$. We deduce that

• $G = F{\{\!| G_0 |\!\}^{\mathtt {c}} \langle l \ \mathrm {by}\ \mathtt {r} \rangle ; G'}{\_}$;

  $\varDelta = s[\mathtt {r}_0]: E^{\_}[\{\!| T |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_] , \prod _{1 \le i \le k} s[\mathtt {r}_i]: {E_{i}}^{\_}[\{\!| {\parallel } T_i {\parallel } |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_]$, $\prod _{k+1 \le j \le n} s[\mathtt {r}_{j}]: {E_{i}}^{\_}[\{\!| T_{j} |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_]$; and

• $\varSigma = \varSigma _0, \prod _{k+1 \le j \le n} s[\mathtt {r}_{j}]: \mathtt {c}^{\mathtt {I}}[\mathtt {r},\mathtt {r}_{j}]\langle l \rangle .h_j$.

Thus we have

• $\varDelta ' = s[\mathtt {r}_0]: E^{\_}[\{\!| T |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_] , \prod _{1\le i \le k+1} s[\mathtt {r}_i]: {E_{i}}^{\_}[\{\!| {\parallel } T_i {\parallel } |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_], \prod _{k+2 \le j \le n} s[\mathtt {r}_{j}]: {E_{i}}^{\_}[\{\!| T_{j} |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_]$ and

• $\varSigma ' = \varSigma _0, \prod _{k+2 \le j \le n} s[\mathtt {r}_{j}]: \mathtt {c}^{\mathtt {I}}[\mathtt {r},\mathtt {r}_{j}]\langle l \rangle .h_j$.

We conclude using the second definition of intermediate correspondence with $k=k+1$.

Case $(\mathsf {EIn_2})$. We pose $\mathtt {r}_n = \mathtt {r}$, we have $\varSigma = \varSigma ', \mathtt {c}^{\mathtt {I}}[\mathtt {r}_0,\mathtt {r}_{n}]\langle l \rangle .h$ and $\lnot \varphi (\varSigma ',\mathtt {c})$. Then $\varDelta = \varTheta , s[\mathtt {r}_{n}]:E^{\mathtt {c}_0}[\{\!| T_n |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}_0 ? l \rangle ; T']$ and $\varDelta '= \varTheta , s[\mathtt {r}_{k+1}]E^{\mathtt {c}_0}[\{\!| {\parallel } T {\parallel } |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}_0 ? l \rangle ; T']$. Without loss of generality we suppose $\varDelta ; \varSigma$ corresponds to $G$. We deduce that

• $G = F{\{\!| G_0 |\!\}^{\mathtt {c}} \langle l \ \mathrm {by}\ \mathtt {r} \rangle ; G'}{\_}$,

• $\varDelta = s[\mathtt {r}_0]: E^{\_}[\{\!| T |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_] , \prod _{1 \le i \le n-1} s[\mathtt {r}_i]: {E_{i}}^{\_}[\{\!| {\parallel } T_i {\parallel } |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_], s[\mathtt {r}_n]: {E_{n}}^{\_}[\{\!| T_n |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_]$ and

• $\varSigma = \varSigma _0, s[\mathtt {r}_{n}]: \mathtt {c}^{\mathtt {I}}[\mathtt {r},\mathtt {r}_n]\langle l \rangle .h$.

From the semantics, we also have

• $\varDelta ' = s[\mathtt {r}_0]: E^{\_}[\{\!| \mathtt {Eend} |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_] , \prod _{1 \le i \le n} s[\mathtt {r}_i]: {E_{i}}^{\_}[\{\!| \mathtt {Eend} |\!\}^{\mathtt {c}} \;\triangleright \; \langle \mathtt {r}' ? l \rangle ; \_]$ and

• $\varSigma = \varSigma _0, s[\mathtt {r}_{n}]:h$.

We use the hypothesis and the intermediate correspondence rule to prove that $\varDelta ';\varSigma '$ corresponds to $F{\{\!| \mathtt {Eend} |\!\}^{\mathtt {c}} \langle l \ \mathrm {by}\ \mathtt {r} \rangle ; G'}{\_}$ which is a derivative of $G$. We conclude.

Case $(\mathsf {Disc'})$ is easy using the first definition of the intermediate correspondence.

Case $(\mathsf {EDisc_1})$ is similar to $(\mathsf {EIn_1})$.

Case $(\mathsf {EDisc_2})$ is similar to $(\mathsf {EIn_2})$.

We then prove the following progress property: if $\varDelta ,\varSigma$ is in intermediate correspondence with $G_1,\dots ,G_n$, and $\varSigma \ne \varepsilon$ then there exist $\varDelta ',\varSigma '$ with $\varSigma '$ strictly smaller than $\varSigma$. We prove it as follows:

• if $\varSigma$ contains $\mathtt {c}[\mathtt {r},\mathtt {r}']\langle l \rangle$, we use the weak projection definitions to prove that $\varDelta$ contains either $s[\mathtt {r}']:E^{\mathtt {c}}[\mathtt {r}?\{l_i.T_i\}{}]$ or $s[\mathtt {r}']:E^{\mathtt {c}'}[\{\!| {\parallel } T {\parallel } |\!\}^{]} \;\triangleright \; \langle \mathtt {r} ? l \rangle ; \_$ with $T$ containing $\mathtt {r}?\{l_i.T_i\}$. We conclude by applying $(\mathsf {In})$ or $(\mathsf {Disc'})$.

• otherwise $\varSigma$ contains $\mathtt {c}^{\mathtt {I}}[\mathtt {r},\mathtt {r}']\langle l \rangle$ and we use the intermediate correspondence definition to discuss whether $\mathtt {c}$ is inside an interrupted scope or not and then whether $\varphi (\varSigma _0,$) or not, the we conclude by applying $(\mathsf {EIn1}),\, (\mathsf {EIn2}),\, (\mathsf {EDisc1})$ or $(\mathsf {EDisc2})$.

By using these properties, we conclude the proof.$\square$