High-efficiency online planning using composite bounds search under partial observation

Abstract

Motion planning in uncertain environments is a common challenge and essential for autonomous robot operations. Representatively, the determinized sparse partially observable tree (DESPOT) algorithm shows reasonable performance for planning under uncertainty. However, DESPOT may generate a low-quality solution due to inaccurate searches and low efficiencies in the belief tree construction. Therefore, this paper proposes a high-efficiency online planning method built upon the DESPOT algorithm, namely, the DESPOT with discounted upper and lower bounds (DESPOT-DULB) algorithm, to simultaneously improve the efficiency and performance of motion planning. Particularly, the node’s information is represented by combining the upper and lower bounds of the node (ULB) in the forward exploration of the action space to reasonably assist the optimal action selection. Then, a discounted factor based on the depth information of the belief tree is introduced to reduce the gap between the upper bound and lower bound both in the action space and observation space. As a result, the proposed method can comprehensively represent the information of the node to ensure a near-optimal forward search. The theoretical proofs of the proposed method are provided as well. The simulation results, including three representative scenario comparisons and a parameter sensitivity analysis, demonstrate that the proposed method exhibits favorable performances in many examples of interest.

Appendices

Appendix 1

Proof

Let CH(b, a^∗) be all the set of b^′ = τ(b, a, z) for some $z\in {Z}_{b,{a}^{\ast }}$, that is, the set of children nodes of b in the DESPOT-DULB tree. If E(b) > 0, then ε(b) > 0, that is, μ(b) − l(b) > 0, and thus μ(b) ≠ l₀(b). Hence we have

$$ {\displaystyle \begin{array}{c}\mu (b)=d\left(b,{a}^{\ast}\right)=\rho \left(b,{a}^{\ast}\right)\\ {}+\left(\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\mu \left({b}^{\prime}\right)+\omega \sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}l\left({b}^{\prime}\right)\right)/\beta \end{array}} $$

(A1)

and

$$ {\displaystyle \begin{array}{c}l(b)\ge m\left(b,{a}^{\ast}\right)\ge \rho \left(b,{a}^{\ast}\right)\\ {}+\left(\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}l\left({b}^{\prime}\right)+\omega \sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\mu \left({b}^{\prime}\right)\right)/\beta \end{array}} $$

(A2)

where 0 < ω < 1, and β > 1. Subtracting the Eq. (A1) by the Eq. (A2), we have

$$ \mu (b)-l(b)\le \left(1-\omega \right)\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]/\beta $$

(A3)

where

$$ \beta ={\kappa}^{\Delta (b)} $$

(A4)

Considering 0 < 1 − ω < 1, κ > 1, we have

$$ {\displaystyle \begin{array}{c}\left(1-\omega \right)\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]/\beta \\ {}<\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]\end{array}} $$

(A5)

That is

$$ \mu (b)-l(b)\le \sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right] $$

(A6)

Combining the Eq. (A4) and Eq. (A6), we have

$$ {\displaystyle \begin{array}{c}{\kappa}^{\varDelta (b)}\left[\mu (b)-l(b)\right]\le {\kappa}^{\varDelta (b)}\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]\\ {}\le {\kappa}^{\varDelta \left({b}^{\prime}\right)}\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]\end{array}} $$

(A7)

Note that

$$ \frac{\mid {\varPhi}_b\mid }{K}\xi \varepsilon \left({b}_0\right)=\sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\frac{\mid {\varPhi}_{b^{\prime }}\mid }{K}\xi \varepsilon \left({b}_0\right) $$

(A8)

Hence, we have

$$ {\displaystyle \begin{array}{c}{\kappa}^{\varDelta (b)}\left[\mu (b)-l(b)\right]-\frac{\mid {\varPhi}_b\mid }{K}\xi \varepsilon \left({b}_0\right)\\ {}\le \sum \limits_{b^{\prime}\in CH\left(b,{a}^{\ast}\right)}\left\{{\kappa}^{\varDelta \left({b}^{\prime}\right)}\left[\mu \left({b}^{\prime}\right)-l\left({b}^{\prime}\right)\right]-\frac{\mid {\varPhi}_{b^{\prime }}\mid }{K}\xi \varepsilon \left({b}_0\right)\right\}\end{array}} $$

(A9)

That is, $E(b)\le \sum \limits_{z\in {Z}_{b,{a}^{\ast }}}E\left({b}^{\prime}\right)$.

Appendix 2

Proof

Let U₀^′(b) = U₀(b) + δ, then U₀^′ is an exact upper bound. Let μ₀^′ be the corresponding initial upper bound, and μ^′ be the corresponding upper bound on ν^∗(b). Then μ₀^′ is a valid initial upper bound for ν^∗(b) and the backup equations ensure that μ^′(b) is a valid upper bound for ν^∗(b). On the other hand, it is easily shown by induction that

$$ \mu (b)+{\gamma}^{\varDelta (b)}\frac{\mid {\varPhi}_b\mid }{K}\delta \ge {\mu}^{\prime }(b) $$

(B10)

When a special case for b = b₀, we have

$$ \mu (b)+\delta \ge {\mu}^{\prime}\left({b}_0\right) $$

(B11)

Hence, when the algorithm terminates, we have

$$ \mu \left({b}_0\right)+\delta \ge {\mu}^{\prime}\left({b}_0\right)\ge {\nu}^{\ast}\left({b}_0\right) $$

(B12)

Equivalently,

$$ {\displaystyle \begin{array}{c}{\nu}_{\hat{\pi}}=l\left({b}_0\right)\ge {\nu}^{\ast}\left({b}_0\right)-\left(\mu \left({b}_0\right)-l\left({b}_0\right)\right)-\delta \\ {}={\nu}^{\ast}\left({b}_0\right)-\varepsilon \left({b}_0\right)-\delta \end{array}} $$

(B13)

The Eq. (B13) holds because the initialization and the computation of the lower bound l via the backup equations are exactly that for finding a regularized optimal policy value in the partial DESPOT-DULB.