Cautious Classification with Data Missing Not at Random Using Generative Random Forests

Abstract

Missing data present a challenge for most machine learning approaches. When a generative probabilistic model of the data is available, an effective approach is to marginalize missing values out. Probabilistic circuits are expressive generative models that allow for efficient exact inference. However, data is often missing not at random, and marginalization can lead to overconfident and wrong conclusions. In this work, we develop an efficient algorithm for assessing the robustness of classifications made by probabilistic circuits to imputations of the non-ignorable portion of missing data at prediction time. We show that our algorithm is exact when the model satisfies certain constraints, which is the case for the recent proposed Generative Random Forests, that equip Random Forest Classifiers with a full probabilistic model of the data. We also show how to extend our approach to handle non-ignorable missing data at training time.

Supported by CAPES Finance 001, CNPQ grant #304012/2019-0.

Proof of Theorem 1

Membership in coNP is trivial: given a configuration $\mathbf {u}$ we can compute $\mathtt {M}(y',\mathbf {o},\mathbf {u})$ and $\mathtt {M}(y'',\mathbf {o},\mathbf {u})$ in linear time and decide the sign of its difference in constant time. Hence we have a polynomial certificate that the problem is not in the language.

We show hardness by reduction from the subset sum problem: Given positive integers $z_1,\dotsc ,z_n$, decide

$$\begin{aligned} \exists u \in \{0,1\}^n: \sum _{i \in [n]} v_i u_i = 1\, , \quad \text {where } v_i = \frac{2z_i}{\sum _{i \in [n]} z_i}\,. \end{aligned}$$

(11)

To solve that problem, build a tree-shaped deterministic PC as shown above, where $U_i$ are binary variables, $P_1(u) = \prod _i e^{-2v_iu_i}$ and $P_2(u)=\prod _i e^{-v_iu_i}$. Note that the PC is not class-factorized. Use the PC to compute:

$$ \delta (y',y'') = \min _u \left[ a\exp \left( -2\sum _i v_iu_i \right) - b\exp \left( -\sum _i v_iu_i \right) \right] \,. $$

If we call $x:=\exp (-\sum _i v_i u_i)$, the above expression is the minimum for positive x of $f(x):=ax^2 - bx$. Function f is a strictly convex function minimized at $x=b/(2a)$. Selecting a and b such that $b/(2a)=e^{-1}$ makes the minimum occur at $\sum _i v_iu_i = 1$. Thus, there is a solution to (11) if and only if $\delta (y',y'') \le -ae^{-2}$. This proof is not quite valid because the distributions $P_1(u)$ and $P_2(u)$ use non-rational numbers. However, we can use the same strategy as used to prove Theorem 5 in [16] and exploit the rational gap between yes and no instances of the original problem to encode a rational approximation of $P_1$ and $P_2$ of polynomial size.    $\square$