Combinatorial Resampling Particle Filter: An Effective and Efficient Method for Articulated Object Tracking

Abstract

Particle filter (PF) is a method dedicated to posterior density estimations using weighted samples whose elements are called particles. In particular, this approach can be applied to object tracking in video sequences in complex situations and, in this paper, we focus on articulated object tracking, i.e., objects that can be decomposed as a set of subparts. One of PF’s crucial step is a resampling step in which particles are resampled to avoid degeneracy problems. In this paper, we propose to exploit mathematical properties of articulated objects to swap conditionally independent subparts of the particles in order to generate new particle sets. We then introduce a new resampling method called Combinatorial Resampling that resamples over the particle set resulting from all the “admissible” swappings, the so-called combinatorial set. In essence, combinatorial resampling (CR) is quite similar to the combination of a crossover operator and a usual resampling, but there exists a fundamental difference between CR and the use of crossover operators: we prove that CR is sound, i.e., in a Bayesian framework, it is guaranteed to represent without any bias the posterior densities of the states over time. By construction, the particle sets produced by CR better represent the density to estimate over the whole state space than the original set and, therefore, CR produces higher quality samples. Unfortunately, the combinatorial set is generally of an exponential size and, therefore, to be scalable, we show how it can be implicitly constructed and resampled from, thus resulting in both an efficient and effective resampling scheme. Finally, through experimentations both on challenging synthetic and real video sequences, we also show that our resampling method outperforms all classical resampling methods both in terms of the quality of its results and in terms of computation times.

Appendix: Proofs

Proof of Proposition 1

Proof by induction on $j$. Assume that, before processing the object subparts in $P_j$, particles estimate density $p( \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t )$. This is clearly the case for $j=1$ since $P_1$ are the first subparts processed. Remember that $P_j,Q_j,R_j$ are the set of object subparts processed at the $j$th loop step, those processed up to (including) the $j$th loop step and those still to be processed respectively. We will now examine sequentially the densities estimated by the particle set after applying the PF’s prediction step in parallel over the subparts in $P_j$, then after applying PF’s correction step and, finally, after resampling. Note that, for simplicity of notation, Algorithm 1 is stated slightly differently since it propagates and corrects each $k \in P_j$ one subpart after the other. But propagations are independent of the weights of the particles, hence the result of Algorithm 1 is equivalent to first propagating all the particles in $P_j$ and, then, correcting them.

• 1. Let us show that after the parallel propagations of the subparts in $P_j$ (prediction step), the particle set represents $p( \mathbf x^{Q_{j}}_t, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1},$ $\mathbf y^{Q_{j-1}}_t )$. For instance, on Fig. 3, after propagating the subparts in $P_2$, the particle set estimates $p(\mathbf x_t^{\{1,2,4,6\}},\mathbf x_{t-1}^{\{3,5\}} | \mathbf y_{1:t-1}, \mathbf y^{1}_t )$, i.e., only the positions of the forearms still refer to time $t-1$ and the only observation taken into account at time $t$ is that of the torso (subpart $P_1$). As the transition function of each subpart $k$ is equal to $p(\mathbf x_t^{k} | \text{ Pa }(\mathbf x_t^{k}))$, all these parallel prediction steps correspond to computing:

  $$\begin{aligned}&\!\int \! p\left( \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \!\nonumber \\&\quad \times \prod _{k \in P_j} p\left( \mathbf x_t^{k} | \text{ Pa }(\mathbf x_t^{k})\right) d\mathbf x_{t-1}^{P_j}. \end{aligned}$$

  (4)

  By $d$-separation, a node is conditionally independent of all of its non descendants given its parents.⁵ Hence, for every $k \in P_j, \mathbf x^{k}_t$ and $\mathbf x^{R_{j}}_{t-1} \cup \mathbf y_{1:t-1}$ are conditionally independent given $\text{ Pa }(\mathbf x_t^{k})$ since the nodes in $\mathbf x^{R_{j}}_{t-1} \cup \mathbf y_{1:t-1}$ belong to time slice $t-1$ and are therefore non descendants of $\mathbf x^{k}_t$. As an example, in Fig. 22a, let $\mathbf x^{k}_t$ be the doubly-circled node, then $\text{ Pa }(\mathbf x_t^{k})$ corresponds to the striped nodes and $\mathbf x^{R_{j}}_{t-1}$ and $\mathbf y_{1:t-1}$ are the gray and black nodes respectively. For the same reason, $\mathbf x^{k}_t$ and $\mathbf x^{P_{j} \backslash \{k\}}_{t-1}$ are conditionally independent given $\text{ Pa }(\mathbf x_t^{k})$. In Fig. 22b, $\mathbf x^{P_{j} \backslash \{k\}}_{t-1}$ are represented by gray nodes. Similarly, by definition of sets $P_j$ and by Property 1 of Definition 2, for any $h$, the ancestors in time slice $t$ of any node in set $\mathbf x^{P_{h}}_t$ belong to $\mathbf x_t^{P_1} \cup \cdots \cup \mathbf x^{P_{h-1}}_t = \mathbf x^{Q_{h-1}}_t$. Therefore, $\mathbf x^{Q_{j-1}}_t \backslash \text{ Pa }_{t}(\mathbf x_t^{k})$ and $\mathbf y^{Q_{j-1}}_t$ cannot be descendants of $\mathbf x^{k}_t$ and are thus conditionally independent of $\mathbf x^{k}_t$ given $\text{ Pa }(\mathbf x_t^{k})$. In Fig. 22c, $\mathbf y^{Q_{j-1}}_t$ is represented by the black node and $\mathbf x^{Q_{j-1}}_t$ is its parent. For the same reason, $\mathbf x^{P_j \backslash \{k\}}_t$ are non descendants of $\mathbf x^{k}_t$ and are thus conditionally independent of $\mathbf x^{k}_t$ given $\text{ Pa }(\mathbf x_t^{k})$. They are represented by gray nodes in Fig. 22d. Overall, $\mathbf x^{k}_t$ and $(\mathbf x^{Q_{j-1}}_t \backslash \text{ Pa }_{t}(\mathbf x_t^{k})) \cup \mathbf x^{P_j \backslash \{k\}}_{t-1} \cup \mathbf x^{R_{j}}_{t-1} \cup \mathbf y_{1:t-1} \cup \mathbf y^{Q_{j-1}}_t \cup \mathbf x^{P_j \backslash \{k\}}_t$ are conditionally independent given $\text{ Pa }(\mathbf x_t^{k})$.

Denote by $\{k_1,\ldots ,k_h\}$ the elements of $P_j$. Then, by the preceding paragraph, for any $r \in \{1,\ldots ,h\}$,

$$\begin{aligned}&p(\mathbf x_t^{k_r} | \text{ Pa }(\mathbf x_t^{k_r})) = p(\mathbf x_t^{k_r} | \text{ Pa }(\mathbf x_t^{k_r}),(\mathbf x^{Q_{j-1}}_t \backslash \text{ Pa }_{t}(\mathbf x_t^{k_r})), \\&\quad \mathbf x^{P_j \backslash \{k_r\}}_{t-1},\mathbf x^{R_{j}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t, \mathbf x_t^{k_1},\ldots , \mathbf x_t^{k_{r-1}}). \end{aligned}$$

By Properties 1 and 2 of Definition 2, $\text{ Pa }(\mathbf x_t^{k_r}) = \text{ Pa }_{t}(\mathbf x_t^{k_r}) \cup \{ \mathbf x_{t-1}^{k_r}\}$. So, the above equation is equal to:

$$\begin{aligned} p\left( \mathbf x_t^{k_r} | \mathbf x^{Q_{j-1}}_t, \mathbf x^{P_j}_{t-1}, \mathbf x^{R_{j}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t, \mathbf x_t^{k_1},\ldots , \mathbf x_t^{k_{r-1}}\right) . \end{aligned}$$

By definition of $R_j$ (i.e., $R_j$ is the set of subparts to be processed after the $j$th loop step), we have $R_{j-1} = P_j \cup R_{j}$. Therefore, the above equation is equivalent to:

$$\begin{aligned}&p\left( \mathbf x_t^{k_r} | \text{ Pa }(\mathbf x_t^{k_r})\right) \\&\quad = p\left( \mathbf x_t^{k_r} | \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t, \mathbf x_t^{k_1},..., \mathbf x_t^{k_{r-1}}\right) . \end{aligned}$$

Consequently, we have:

$$\begin{aligned}&\prod _{r=1}^h p\left( \mathbf x_t^{k_r} | \text{ Pa }(\mathbf x_t^{k_r})\right) \\&\quad = \displaystyle \prod _{r=1}^h p\left( \mathbf x_t^{k_r} | \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t, \mathbf x_t^{k_1},\ldots , \mathbf x_t^{k_{r-1}}\right) \\&\quad =p\left( \mathbf x_t^{k_1}, \ldots , \mathbf x_t^{k_h} | \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t\right) \\&\quad =p\left( \mathbf x_t^{P_j} | \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t\right) \end{aligned}$$

Consequently, the integral of Eq. (4) is equivalent to:

$$\begin{aligned}&\int p\left( \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \\&\qquad \times \, p\left( \mathbf x_t^{P_j} | \mathbf x^{Q_{j-1}}_t , \mathbf x^{R_{j-1}}_{t-1} , \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \ d\mathbf x_{t-1}^{P_j} \\&\quad = \int p\left( \mathbf x_t^{P_j}, \mathbf x^{Q_{j-1}}_t, \mathbf x^{R_{j-1}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \ d\mathbf x_{t-1}^{P_j}. \end{aligned}$$

As $Q_j = Q_{j-1} \cup P_j$ and $R_{j-1} = P_j \cup R_j$, the above equation is equivalent to $p( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t )$.

• 2. Let us show that after applying the parallel correction steps on the $P_j$ subparts, the particle set estimates $p( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j}}_t )$. These operations correspond to computing density

  $$\begin{aligned} p\left( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \times \prod _{k \in P_j} p\left( \mathbf y_t^{k} | \mathbf x_t^{k}\right) \end{aligned}$$

  and, then, normalizing it. By $d$-separation, nodes $\mathbf y_t^k$ are conditionally independent of the rest of the OTDBN given $\mathbf x_t^k$, so, if we denote by $\{k_1,\ldots ,k_h\}$ the elements of $P_j$, then, for any $r \in \{1,\ldots ,h\}, p(\mathbf y_t^{k_r} | \mathbf x_t^{k_r}) = p(\mathbf y_t^{k_r} | \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t$, $\mathbf y_t^{k_1},\ldots ,\mathbf y_t^{k_{r-1}})$ since $\mathbf x_t^{k_r} \in \mathbf x_t^{Q_j}$. Therefore:

  $$\begin{aligned} \prod _{k \in P_j} p\left( \mathbf y_t^{k} | \mathbf x_t^{k}\right) = p\left( \mathbf y_t^{P_j} | \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) . \end{aligned}$$

So, before normalization, the particle set estimates density

$$\begin{aligned}&p\left( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) \\&\qquad \times \, p\left( \mathbf y_t^{P_j} | \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1}, \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t\right) \\&\quad = p\left( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1}, \mathbf y_t^{P_j} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t \right) , \end{aligned}$$

which, when normalized, is equal to density

$$\begin{aligned}&p\left( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j-1}}_t, \mathbf y_t^{P_j}\right) \\&\quad = p\left( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j}}_t \right) . \end{aligned}$$

Finally, as resamplings do not alter densities, at the end of the algorithm, the particle set estimates $p( \mathbf x_t^{Q_K}, \mathbf x^{R_{K}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{K}}_t ) = p( \mathbf x_t | \mathbf y_{1:t} )$.$\square$

Proof of Proposition 2

If $j = 1$, the proposition trivially holds since $\sigma$ is applied to all the nodes of the connected component of $\mathbf x_t^k$. Assume now that $j \ne 1$. Let $\text{ Desc }_{t}^{\mathbf x}(\mathbf x_{t}^k)$ and $\text{ Desc }_{t}^{\mathbf y}(\mathbf x_{t}^k)$ denote the set of states and observation nodes respectively in $\text{ Desc }_{t}(\mathbf x_{t}^k)$. We shall now partition the object subparts as described on Fig. 23 to highlight which subparts shall be permuted, which ones shall be identical to enable permutations and which subparts are unconcerned by permutations: let $\mathbf x_{t-1}^D =\text{ Desc }_{t-1}^{\mathbf x}(\mathbf x_{t-1}^k), \mathbf x_t^{k'} = \text{ Pa }_{t}(\mathbf x_t^k), \mathbf x_t^V = \mathbf x_t^{Q_j} \backslash (\{\mathbf x_t^k,\mathbf x_t^{k'}\})$ and $\mathbf x_{t-1}^W = \mathbf x_{t-1}^{R_j} \backslash \mathbf x_{t-1}^D$. Thus, the permuted subparts are $\mathbf x_t^k \cup \mathbf x_{t-1}^D$ (see Fig. 23), the identical subparts are $\mathbf x_{1:t}^{k'}$, and the subparts unconcerned by permutations are $\mathbf x_t^V \cup \mathbf x_{t-1}^W$. Similarly, $\mathbf y_{t-1}^D,\mathbf y_t^V,\mathbf y_{t-1}^W$ denote their corresponding observation nodes. Before permutations, the particle set estimates:

$$\begin{aligned}&p\left( \mathbf x_{t}^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j}}_{t} \right) \\&\quad \propto p\left( \mathbf x_{t}^{Q_j}, \mathbf x^{R_{j}}_{t-1} , \mathbf y_{1:t-1}, \mathbf y^{Q_{j}}_{t} \right) \\&\quad = p\left( \mathbf x_t^{\{k,k'\} \cup V}, \mathbf x_{t-1}^{D\cup W}, \mathbf y_{1:t}^{\{k,k'\}\cup V}, \mathbf y_{1:t-1}^{D \cup W}\right) \\&\quad = \displaystyle \int p\left( \mathbf x_t^{\{k\} \cup V}, \mathbf x_{1:t}^{k'}, \mathbf x_{t-1}^{D\cup W}, \mathbf y_{1:t}^{\{k,k'\}\cup V}, \mathbf y_{1:t-1}^{D \cup W}\right) d\mathbf x_{1:t-1}^{k'} \end{aligned}$$

Fig. 23

$d$-separation analysis

Given $\{\mathbf x_{1:t}^{k'}\}, S = \{\mathbf x_t^k\} \cup \mathbf x_{t-1}^D \cup \mathbf y_{1:t}^k \cup \mathbf y_{1:t-1}^D$ is conditionally independent of the rest of the OTDBN because, by Definition 3, no active chain can pass through an arc outgoing from a node in a conditioning set and, removing from the OTDBN the arcs outgoing from $\{\mathbf x_{1:t}^{k'}\}, S$ is not connected anymore to the rest of the OTDBN. For the same reason, $\mathbf x_t^V \cup \mathbf x_{t-1}^W \cup \mathbf y_{1:t}^V \cup \mathbf y_{1:t-1}^W$ is conditionally independent of the rest of the OTDBN given $\{\mathbf x_{1:t}^{k'}\}$. Therefore, the above integral is equal to:

$$\begin{aligned} \begin{aligned}&\int p\left( \mathbf x_{1:t}^{k'},\mathbf y_{1:t}^{k'}\right) \ p\left( \mathbf x_t^{k}, \mathbf x_{t-1}^{D},\mathbf y_{1:t}^k, \mathbf y_{1:t-1}^D | \mathbf x_{1:t}^{k'}\right) \\&\quad \times p\left( \mathbf x_t^{V},\mathbf x_{t-1}^{W},\mathbf y_{1:t}^{V}, \mathbf y_{1:t-1}^{W} | \mathbf x_{1:t}^{k'}\right) \ d\mathbf x_{1:t-1}^{k'}. \end{aligned} \end{aligned}$$

(5)

For fixed values of $\mathbf x_{1:t}^{k'}$, permuting particles over subparts $\{\mathbf x_t^{k}\} \cup \mathbf x_{t-1}^D$ as well as their weights induced by $\{\mathbf y_{1:t}^k\} \cup \mathbf y_{1:t-1}^D$ cannot change the estimation of density $p(\mathbf x_t^{k}, \mathbf x_{t-1}^{D},\mathbf y_{1:t}^k, \mathbf y_{1:t-1}^D | \mathbf x_{1:t}^{k'})$ because estimations by samples are insensitive to the order of the elements in the samples. Moreover, it can neither affect the estimation of density $p(\mathbf x_t^{V},\mathbf x_{t-1}^{W},\mathbf y_{1:t}^{V}, \mathbf y_{1:t-1}^{W} | \mathbf x_{1:t}^{k'})$ since $\mathbf x_t^V \cup \mathbf x_{t-1}^W \cup \mathbf y_{1:t}^V \cup \mathbf y_{1:t-1}^W$ and $\{\mathbf x_t^{k},\mathbf y_{1:t}^k\} \cup \mathbf x_{t-1}^D \cup \mathbf y_{1:t-1}^D$ are conditionally independent given $\{\mathbf x_{1:t}^{k'}\}$. Consequently, applying permutation $\sigma$ on subparts $\{\mathbf x_t^{k}\} \cup \text{ Desc }_{t-1}^{\mathbf x}(\mathbf x_t^k) = \{\mathbf x_t^{k}\} \cup \mathbf x_{t-1}^D$ as well as on their weights cannot change the estimation of Eq. (5) and, therefore, of density $p( \mathbf x_t^{Q_j}, \mathbf x^{R_{j}}_{t-1} | \mathbf y_{1:t-1}, \mathbf y^{Q_{j}}_t )$.$\square$

Proof of Proposition 3

Denote ${\mathcal {S}}= \{(\mathbf x_t^{(i),Q_j}, \mathbf x_{t-1}^{(i),R_j})\}_{i=1}^N$ and ${\mathcal {S}}'$ its combinatorial set (see Definition 5). In lines 2–3, Algorithm 2 selects which central subpart $Q_{j-1}$ particle $\mathbf x_t''$ should have. Subpart $\mathbf x_t^{(z),Q_{j-1}}, z \in \mathbf i_h$, should be selected w.r.t. the sum of the weights of the particles in ${\mathcal {S}}'$ having the same central subpart $Q_{j-1}$. Let us show that this is achieved using weights $W_h$.

In Definition 5, $\Sigma ^k$ is the set of all the possible permutations of the $k\mathrm{th}$ subpart of the particles in ${\mathcal {S}}$. Clearly, within each set ${\mathcal {S}}_h^k$, all the $N_h^k!$ permutations of the $k\mathrm{th}$ subpart of the particles of this set are admissible. They form the cycles within the permutations of $\Sigma ^k$ and, as such, a given permutation $\sigma _h^k$ over subset ${\mathcal {S}}_h^k$ of ${\mathcal {S}}$ may appear many times within $\Sigma ^k$. For instance, on the particles of Fig. 7b, one permutation $\sigma \in \Sigma ^3$ may swap subpart $3$ of the first two particles (belonging to ${\mathcal {S}}_1^3$) and leave all the other particles of ${\mathcal {S}}$ untouched, and another permutation $\sigma ' \in \Sigma ^3$ may also swap subpart $3$ of the first two particles and, additionally, swap subpart $3$ of the last two particles (belonging to ${\mathcal {S}}_2^3$). As such, the permutation over ${\mathcal {S}}_1^3$, i.e., over the first two particles, appears in several permutations of $\Sigma ^3$ (at least in $\sigma$ and $\sigma '$). Let $\mathbb {S}^k$ denote the set of distinct sets ${\mathcal {S}}_h^k$ (for instance, in Fig. 7b, $\mathbb {S}^3 = \{{\mathcal {S}}_1^3,{\mathcal {S}}_2^3\} = \{{\mathcal {S}}_1^3,{\mathcal {S}}_3^3\}$). Then $|\Sigma ^k| = \prod _{{\mathcal {S}}_h^k \in \mathbb {S}^k} N_h^k!$ and, therefore, a given permutation $\sigma _h^k$ over ${\mathcal {S}}_h^k$ appears $|\Sigma ^k| / N_h^k! = \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! / N_h^k!$ times in $\Sigma ^k$.

For the moment, consider only the permutations over the $k\mathrm{th}$ subpart. Each time a permutation is applied on ${\mathcal {S}}_h^k$, this creates a new pack of $|{\mathcal {S}}_h| = N_h$ particles whose values on $Q_{j-1}$ are equal to those of ${\mathcal {S}}_h$. By definition, ${\mathcal {S}}_h \subseteq {\mathcal {S}}_h^k$ and, usually, this is a strict inclusion. For instance, on Fig. 7, ${\mathcal {S}}_2 \subset {\mathcal {S}}_2^3$ . Therefore, each aforementioned pack of $N_h$ particles appears several times in ${\mathcal {S}}'$. For instance, on Fig. 7, $|{\mathcal {S}}_2^3| = 3$, hence this set induces 6 permutations, but $|{\mathcal {S}}_2| = 1$, hence, applying the 6 permutations over ${\mathcal {S}}_2^3$ necessarily implies that all packs in ${\mathcal {S}}'_2$ appear twice in this set (as a matter of fact, values $3, 3'$ and $3''$ appear twice). Let us compute precisely how many times each pack appears. As $|{\mathcal {S}}_h| = N_h$ and $|{\mathcal {S}}_h^k| = N_h^k$, there are $A_{N_h^k}^{N_h}$ different possibilities to assign some $k$-part of ${\mathcal {S}}_h^k$ to the particles of ${\mathcal {S}}_h$, where $A_n^k = n! / (n-k)!$ stands for the number of $k$-permutations out of $n$ elements. Hence, there are $A_{N_h^k}^{N_h}$ distinct packs. As there are $N_h^k!$ permutations within ${\mathcal {S}}_h^k$, packs are repeated $(N_h^k! / A_{N_h^k}^{N_h})$ times. In addition, by the preceding paragraph, permutations within ${\mathcal {S}}_h^k$ were already duplicated $\prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! / N_h^k!$ times in $\Sigma ^k$, hence, overall, in ${\mathcal {S}}'$ (where only permutations over the $k$th subpart are performed), each pack shall appear $(\prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! / N_h^k!) \times (N_h^k! / A_{N_h^k}^{N_h}) = \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! / A_{N_h^k}^{N_h}$ times.

Now, select one particle, say $\mathbf x_t^{(i)}$, in ${\mathcal {S}}_h$. By symmetry, the aforementioned permutations can assign any value of the $k\mathrm{th}$ subpart of the particles of ${\mathcal {S}}_h^k$ to $\mathbf x_t^{(i),k}$. So the sum of the weights of the resulting particles over all those permutations is equal to $W_h^k$ times the number of times each value $\mathbf x_t^{(z),k}, , z \in \mathbf i_h^k$, is repeated. There are $N_h$ possibilities to choose the particle $\mathbf x_t^{(i)}$ of ${\mathcal {S}}_h$ to which $\mathbf x_t^{(z),k}$ is assigned. Once this is done, there remains $A^{N_h-1}_{N_h^k-1}$ possibilities to fill the other particles. Overall, the weight induced by the permutations over the $k\mathrm{th}$ object subpart on the $Q_{j-1}$-central subpart of ${\mathcal {S}}_h$ is thus:

$$\begin{aligned}&\frac{\prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k!}{A_{N_h^k}^{N_h}} \times N_h \times A_{N_h^k -1}^{N_h -1} \times W_h^k \nonumber \\&\quad = \left( \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! \right) \times \frac{N_h}{N_h^k} \times W_h^k. \end{aligned}$$

(6)

Let us now consider the permutations over all the subparts in $P_j$. Let $\mathcal {P}_{N}$ represent the set of all particle sets of size $N$. Similarly, let $2^{\mathcal {P}_{N}}$ denote the set of sets of particle sets of size $N$. Finally, let $f : P_j \times 2^{\mathcal {P}_{N}} \mapsto 2^{\mathcal {P}_{N}}$ be the function which i) given $k \in P_j$ and a single particle set ${\mathcal {S}}$ returns the union of the particle sets resulting from the application on ${\mathcal {S}}$ of all the admissible permutations w.r.t. subpart $k$; and ii) given $k \in P_j$ and a set $\{{\mathcal {S}}_1,\ldots ,{\mathcal {S}}_r\}$ of particle sets, returns $\bigcup _{i=1}^r f (k, \{{\mathcal {S}}_i\})$. Let $P_j = \{k_1,\ldots ,k_d\}$. Clearly, the combinatorial set ${\mathcal {S}}'$ can be obtained by a sequence of applications of $f$ over ${\mathcal {S}}$:

$$\begin{aligned} {\mathcal {S}}' = f(k_d, f(k_{d-1}, \ldots , f(k_2,f(k_1,\{{\mathcal {S}}\})) \ldots )). \end{aligned}$$

By the preceding paragraphs, the sum of the weights w.r.t. $k_1$ assigned to the particles of $f(k_1,\{{\mathcal {S}}\})$ whose $Q_{j-1}$-central subparts belong to ${\mathcal {S}}_h$ is given by Eq. (6). By definition of function $f$, if $f(k_1,\{{\mathcal {S}}\}) = \{{\mathcal {S}}_1,\ldots ,{\mathcal {S}}_r\}$, then $f(k_2,f(k_1,\{{\mathcal {S}}\})) = \bigcup _{i=1}^r f(k_2,\{{\mathcal {S}}_i\})$ and, by the preceding paragraphs, the sum of the weights w.r.t. subpart $k_2$ assigned to the particles of $f(k_2,\{{\mathcal {S}}_i\})$ whose $Q_{j-1}$-central subparts belong to ${\mathcal {S}}_h$ is also given by Eq. (6). Therefore, the sum of the weights w.r.t. both $k_1$ and $k_2$ of the particles of $f(k_2,f(k_1,\{{\mathcal {S}}\}))$ whose $Q_{j-1}$-central subparts belong to ${\mathcal {S}}_h$ is the product over $k_1,k_2$ of the formula given in Eq. (6). By induction, after the application of the all the permutations over all the subparts of $P_j$, the weight assigned to the particles whose central subpart belongs ${\mathcal {S}}_h$ is thus that equal to:

$$\begin{aligned}&\prod _{k \in P_j} \left( \left( \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! \right) \times \frac{N_h}{N_h^k} \times W_h^k \right) \\&\qquad = \displaystyle \left( \prod _{k \in P_j} \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! \right) \times \prod _{k \in P_j} \frac{N_h}{N_h^k} \times W_h^k. \end{aligned}$$

Note that the first product, $\prod _{k \in P_j} \prod _{{\mathcal {S}}_r^k \in \mathbb {S}^k} N_r^k! = \prod _{k \in P_j} |\Sigma ^k|$, is independent of $h$. Therefore, up to a proportional constant, the weight of the particles of ${\mathcal {S}}'$ whose $Q_{j-1}$-central subparts belong to ${\mathcal {S}}_h$ is equal to $\prod _{k \in P_j} \frac{N_h}{N_h^k} \times W_h^k$, which is precisely the quantity $W_h$ given in the proposition.

Overall, lines 2–3 select correctly the $Q_{j-1}$ subpart w.r.t. the weight they would have in ${\mathcal {S}}'$. Once this is done, by $d$-separation, all the subparts in $P_j$ are independent and should be sampled w.r.t. $p(\mathbf x_t^k | \text{ Pa }_{t}(\mathbf x_t^k))$, which is done in lines 5–8 since $p(\mathbf x_t^k | \text{ Pa }_{t}(\mathbf x_t^k))$ is proportional to weight $w_t^{(i),k}$. Consequently, Algorithm 2 produces particle sets similar to those resulting from a resampling on ${\mathcal {S}}'$. Finally, as shown previously, ${\mathcal {S}}'$ estimates the same distribution as ${\mathcal {S}}$, hence Proposition 3 holds.$\square$