Hybrid Dec-POMDP/PID Guidance System for Formation Flight of Multiple UAVs

Abstract

This paper proposes a guidance system with a new hybrid architecture for multiple fixed-wing unmanned aerial vehicles (UAVs) formation flight. The proposed architecture is hybrid in the sense of combining two control policies found in guidance systems: the Decentralized Partially Observable Markov Decision Process (Dec-POMDP) policy and the well-known PID controller. The Dec-POMDP policy part enables every UAV to avoid collision with any other UAV from the formation. The PID part is activated for regions far from the collision risk zone and is implemented in such a way that each UAV is completely controlled by only its own data, independent of its neighbors. The major novelty of the proposed system, compared to other hybrid approaches, is its decentralized nature which favors overcoming the usual problems that come with centralization such as the high dependence on a central decision node, a leader, or a ground station. Inherited from both parts, the overall system performance is robust in noisy and uncertain environments. Simulations were extensively performed to show the effectiveness of the proposed guidance system in distinct scenarios.

Appendix: A

1.1 A.1 Transition Matrices

The transition probability matrices used in Section 5, ${T^{i}_{v}}$ and $T^{i}_{\omega }$, are described as follows (the number in parentheses is the index β_v and β_ω that multiplies the base values α_vb or α_ωb respectively, of each matrix):

$$ {T^{1}_{v}}(-1) = \left[\begin{array}{llllllll} 0.2 & 0.7 & 0 & 0.05 & 0 & 0.05 & 0 & 0 \\ 0.1 & 0.2 & 0 & 0.6 & 0 & 0.1 & 0 & 0 \\ 0.6 & 0.1 & 0.2 & 0 & 0 & 0 & 0.1 & 0 \\ 0.2 & 0 & 0 & 0.1 & 0 & 0.6 & 0 & 0.1 \\ 0.1 & 0.05 & 0.6 & 0 & 0.1 & 0 & 0.1 & 0.05 \\ 0.05 & 0 & 0 & 0 & 0 & 0.2 & 0 & 0.75 \\ 0.05 & 0 & 0.05 & 0 & 0.6 & 0.05 & 0.2 & 0.05 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0.3 & 0.4 \end{array}\right] $$

$$ {T^{1}_{v}}(0) = 0.8I + \frac{0.2}{7}(1_{8\times8} - I), $$

in which I is the 8-th dimensional identity matrix and 1_8x8 is the 8-th dimensional square matrix completely filled with 1.

$$ {T^{1}_{v}}(1) = \left[\begin{array}{llllllll} 0.2 & 0 & 0.7 & 0 & 0.05 & 0 & 0.05 & 0 \\ 0.6 & 0.2 & 0.1 & 0 & 0 & 0.1 & 0 & 0 \\ 0.1 & 0 & 0.2 & 0 & 0.6 & 0 & 0.1 & 0 \\ 0.1 & 0.6 & 0.05 & 0.1 & 0 & 0.1 & 0 & 0.05 \\ 0.2 & 0 & 0 & 0 & 0.1 & 0 & 0.6 & 0.1 \\ 0.05 & 0.05 & 0 & 0.6 & 0.05 & 0.2 & 0 & 0.05 \\ 0.05 & 0 & 0 & 0 & 0 & 0 & 0.2 & 0.75 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0.3 & 0.4 \end{array}\right] $$

Due to the symmetry of the problem:

$$ {T^{2}_{v}}(-1) = {T^{1}_{v}}(1) $$

$$ {T^{2}_{v}}(0) = {T^{1}_{v}}(0) $$

$$ {T^{2}_{v}}(1) = {T^{1}_{v}}(-1) $$

$$ {T^{3}_{v}} = {T^{1}_{v}} $$

$$T^{1}_{\omega}(-1) = \left[\begin{array}{llllllll} 0.2 & 0.2 & 0.2 & 0.1 & 0.1 & 0.1 & 0.1 & 0 \\ 0 & 0.2 & 0.1 & 0.3 & 0 & 0.4 & 0 & 0 \\ 0 & 0.1 & 0.2 & 0 & 0.3 & 0 & 0.4 & 0 \\ 0 & 0 & 0 & 0.1 & 0 & 0.4 & 0.2 & 0.3 \\ 0 & 0 & 0 & 0 & 0.1 & 0.2 & 0.4 & 0.3 \\ 0 & 0 & 0 & 0 & 0 & 0.2 & 0.1 & 0.7 \\ 0 & 0 & 0 & 0 & 0 & 0.1 & 0.2 & 0.7 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0.3 & 0.4 \end{array}\right] $$

$$ T^{1}_{\omega}(0) = {T^{1}_{v}}(0) $$

$$ T^{1}_{\omega}(1) = \left[\begin{array}{llllllll} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0.8 & 0.1 & 0 & 0 & 0 & 0.1 & 0 & 0 \\ 0.8 & 0 & 0.1 & 0 & 0 & 0 & 0.1 & 0 \\ 0.5 & 0.3 & 0 & 0 & 0 & 0.2 & 0 & 0 \\ 0.5 & 0 & 0.3 & 0 & 0 & 0 & 0.2 & 0 \\ 0.1 & 0.5 & 0 & 0.1 & 0 & 0.2 & 0 & 0.1 \\ 0.1 & 0 & 0.5 & 0 & 0.1 & 0 & 0.2 & 0.1 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0.3 & 0.4 \end{array}\right] $$

$$ T^{2}_{\omega}(-1) = \left[\begin{array}{llllllll} 0.1 & 0.45 & 0.45 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0.2 & 0 & 0 & 0 & 0.8 & 0 & 0 \\ 0 & 0 & 0.2 & 0 & 0 & 0 & 0.8 & 0 \\ 0 & 0 & 0 & 0.2 & 0 & 0.8 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.2 & 0 & 0.8 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0 & 0.7 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.3 & 0.7 \\ 0 & 0 & 0 & 0 & 0 & 0.3 & 0.3 & 0.4 \end{array}\right] $$

$$ T^{2}_{\omega}(0) = T^{1}_{\omega}(0) $$

$$ T^{2}_{\omega}(1) = T^{2}_{\omega}(-1) $$

$$ T^{3}_{\omega}(-1) = T^{1}_{\omega}(1) $$

$$ T^{3}_{\omega}(0) = T^{1}_{\omega}(0) $$

$$ T^{3}_{\omega}(1) = T^{1}_{\omega}(-1) $$

1.2 A.2 Proof of Eq. 5

There is a total of N agents that get some observation z_i. However, a total of $N_{\zeta _{i}} = j$ of those agents that perceive $z_{i} = s^{\prime }$ can be allocated. Since these agents were fixed, there are another N − j agents that might observe any from the $N_{z_{i}} - 1$ observations in which $z_{i} \neq s^{\prime }$. In this arrangement, the number of combinations is given by

$$ N_{comb} = (N_{z_{i}} - 1)^{N-j}. $$

(1)

However, there are other arrangements for the j agents that observe $z_{i} = s^{\prime }$. The number of possible permutations to fix those agents is given by

$$ N_{arr} = \frac{N!}{j!(N-j)!}. $$

(2)

For each arrangement, there are N_comb combinations so the total number of combinations, D_j, is given by

$$ D_{j} = N_{arr} N_{comb}. $$

(3)

Replacing (1) and (2) in (3) results in Eq. 5.