Suture Looping Task Pose Planner in a Constrained Surgical Environment

Abstract

We present a task pose planner for a robustly safe automation of suture looping tasks in robotic assisted minimally invasive surgery (RAMIS). Our approach uses pre-planned spiral looping trajectories and computes the task pose using a combination of linear programming for position optimization and a brute force iterative orientation search. The optimization maximizes the minimum distance from constraints to account for any uncertainties. We verify our approach in a simulation to be effective in avoiding instrument-tissue collisions, suture entanglement and effective length constraints, and gripper joint limits. In addition, it allows us to create a map that provides the task pose quality at suture exit point locations. With the map, we are able to identify task poses for multiple suture exit points and suitable port placements.

Appendices

Appendix A: Surgical Frame Definition

The $\{\mathcal {S}\}$ frame is derived from the location of the PPCs and the suture exit points (see Fig. 13). The frame x and y-axis components belong to the plane P, which is tangent to the tissue where the suture exit point $\vec {x}_{ex}$ lies. This plane can be obtained from pre-operative models created from 3D images. If there are multiple suture exit points, P is the best fit plane to the points. The x-axis component is in the direction of the projection of the vector $\vec {x}_{{\mathscr{B}}_{ppc}} - \vec {x}_{\mathcal {A}_{ppc}}$ on to P. The z-axis component is collinear to the normal $\vec {n}_{P}$ of P in the direction of the PPCs. The origin of $\{\mathcal {S}\}$ is the projection of the point $\vec {x}_{m}$ on to P, where $\vec {x}_{m}$ is the mid-point of the vector $\vec {x}_{{\mathscr{B}}_{ppc}} - \vec {x}_{\mathcal {A}_{ppc}}$. The frame origin $\vec {x}_{\mathcal {S}}$ and unit direction vectors $\hat {X}$, $\hat {Y}$, and $\hat {Z}$ are

Fig. 13

Surgical frame definition

$$ \begin{array}{@{}rcl@{}} \vec{x}_{\mathcal{S}} & =& \vec{x}_{m} + (d_{mP})(\vec{n}_{P}) \\ \hat{X} & =& \frac{\vec{x}_{AB} - (\vec{x}_{AB} \cdot \vec{n}_{P})(\vec{n}_{P})}{||\vec{x}_{AB} - (\vec{x}_{AB} \cdot \vec{n}_{P})(\vec{n}_{P})||}\\ \hat{Y} & =& \hat{X} \times \hat{Z}\\ \hat{Z} & =& \vec{n}_{P}, \end{array} $$

(A1)

where $\vec {x}_{AB}$ is the vector $\vec {x}_{{\mathscr{B}}_{ppc}} - \vec {x}_{\mathcal {A}_{ppc}}$, and the parameter d_mP is the distance between $\vec {x}_{m}$ and P.

Appendix B: Accounting for Uncertainties

We here discuss briefly a way that the proposed method can account for uncertainties of the surgical environment in the reachable workspace (ζ_rw), suture effective length (ζ_el), suture entanglement (ζ_se) and joint limit (ζ_𝜃) constraints, which are explained in Section 3.

The convex constraints ζ_rw and ζ_se are from a snapshot of the surgical environment. During a pre-operative planning, the snapshot can be obtained using models and simulations that mimic accurately the operating field [43], while during a surgery, the snapshot can be obtained using tissue tracking techniques and supervised tissue key point feature selection [44,45,46,47,48]. To address the changes of the environment, the method we propose benefits from taking multiple snapshots, i.e., multiple detections. From [49], the snapshots can be performed with a frequency of as low as 257 Hz to capture tissue movements due to breathing and heart beat. We can enumerate detections with the index d = 1, 2,.... Therefore, after each detection d, we obtain a couple of the corresponding sets of convex constraints $\zeta ^{d}_{rw}$ and $\zeta ^{d}_{se}$. Since the intersection of convex volumes is convex [27], we can use convex constraints from all detections in a single optimization to obtain the solution that satisfies all constraints $\zeta ^{d}_{rw}$, $\zeta ^{d}_{se}$, d = 1, 2....

In addition to multiple detections, for each type of constraints, we can introduce positive margins 𝜖_rw, 𝜖_el, 𝜖_se and 𝜖_𝜃 for the constraints ζ_rw, ζ_el, ζ_se and ζ_𝜃, respectively. In summary, the uncertainties can be accounted by the following constraints:

$$ \begin{array}{@{}rcl@{}} \zeta^{d}_{rw}&:&\ \vec{n}^{rw}_{k} \cdot \vec{x} + r + \epsilon_{rw} \geq c^{rw}_{k}, k_{d}=1, {\ldots} m_{d} \end{array} $$

(B2)

$$ \begin{array}{@{}rcl@{}} \zeta_{el}&:&\ \vec{n}^{el}_{k} \cdot \vec{x} + \epsilon_{el} \geq c^{el}_{k}, k=1, {\ldots} m \end{array} $$

(B3)

$$ \begin{array}{@{}rcl@{}} \zeta^{d}_{se}&:&\ \vec{n}_{se} \cdot \vec{x} + \epsilon_{se} \geq \vec{n}_{se} \cdot \vec{x}_{ex} \end{array} $$

(B4)

$$ \begin{array}{@{}rcl@{}} \zeta_{\theta}&:&\ \vec{n}_{\theta} \cdot \vec{x} + \epsilon_{\theta} \geq \vec{n}_{\theta} \cdot \vec{x}_{ppc}, \end{array} $$

(B5)

where m is the number of inequalities in the ζ_el constraint, d in the superscript of $\zeta ^{d}_{rw}$ and $\zeta ^{d}_{se}$ denotes sets of constraints obtained from multiple detections, and m_d is the number of inequalities in $\zeta ^{d}_{rw}$, where d = 1, 2,...