Sparse precision matrix estimation with missing observations

Abstract

Sparse Gaussian graphical models have been extensively applied to detect the conditional independence structures from fully observed data. However, datasets with missing observations are quite common in many practical fields. In this paper, we propose a robust Gaussian graphical model with the covariance matrix being estimated from the partially observed data. We prove that the inverse of the Karush–Kuhn–Tucker mapping associated with the proposed model satisfies the calmness condition automatically. We also apply a linearly convergent alternating direction method of multipliers to find the solution to the proposed model. The numerical performance is evaluated on both the synthetic data and real data sets.

Appendix

1.1 Proof of Lemma 2.1

Lemma 6.1

(Yu 2013, Theorem 1) Let f and g be two closed convex proper functions. A sufficient condition for $\mathrm{Prox}_{f+g}=\mathrm{Prox}_f\circ \mathrm{Prox}_g$ is

$$\begin{aligned} \partial g(x) \subseteq \partial g\left( \mathrm{Prox}_f(x)\right) ,\,\,\forall \,x \end{aligned}$$

Let $p:{\mathbb {S}}^n\rightarrow {\mathbb {R}}\cup \{+\infty \}$ be defined by

$$\begin{aligned} p(x)=\lambda \Vert x\Vert _{1,\mathrm{off}}+\varphi (x),\,\,\varphi (x):={\mathbb {I}}_{{\mathcal {C}}}(x), \,\,\, {\mathcal {C}}:=\left\{ x:\,\Vert x\Vert \le \alpha \right\} . \end{aligned}$$

(6.1)

It follows from the definition of proximal mapping that

$$\begin{aligned} \mathrm{Prox}_{\varphi }(x)=\Pi _{{\mathcal {C}}}(x)=\left\{ \begin{array}{ll} x,&{}\Vert x\Vert \le \alpha ,\\ \displaystyle \frac{\alpha x}{\Vert x\Vert },&{}\Vert x\Vert >\alpha . \end{array} \right. \end{aligned}$$

(6.2)

Lemma 6.2

Let function p be defined by (6.1), it holds that

$$\begin{aligned} \mathrm{Prox}_{p}= \Pi _{{\mathcal {C}}}\circ \mathrm{Prox}_{\lambda \Vert \cdot \Vert _{1,\mathrm{off}}}. \end{aligned}$$

Proof

From Lemma 6.1, it is sufficient to show that

$$\begin{aligned} \partial \mathrm{Prox}_{\lambda \Vert \cdot \Vert _{1,\mathrm{off}}}(x) \subseteq \partial \mathrm{Prox}_{\lambda \Vert \cdot \Vert _{1,\mathrm{off}}}(y),\,\,y=\Pi _{{\mathcal {C}}}(x),\,\,\forall x\in {\mathbb {R}}^m. \end{aligned}$$

(6.3)

From (6.2), it is sufficient to consider the following cases:

• (a) If $\Vert x\Vert \le \alpha$, then $y=x$. Therefore, the relationship (6.3) holds.

• (b) If $\Vert x\Vert >\alpha$, then $\displaystyle y={\alpha x}/{\Vert x\Vert }$, which means $\mathrm{sgn}(y)=\mathrm{sgn}(x)$. Therefore, the relationship (6.3) also holds in this case.

The proof is completed. □

1.2 Proof of Proposition 2.1

Lemma 6.3

(Zhang et al. 2020, Lemma 3.1) Let $f(x):=-\mathop {\mathrm {log\,det}}\limits \,x$. Then all ${\mathcal {G}}_f\in \partial \mathrm{Prox}_f(Z)$ are self-adjoint and positive definite with $\lambda _{\max }({{\mathcal {G}}}_f)<1$.

Lemma 6.4

Let $x\in {\mathbb {S}}^n$ and ${\mathcal {B}}:{\mathbb {S}}^n\rightarrow {\mathbb {S}}^n$ be any self-adjoint positive definite operator, p is the function defined in Lemma2.1. Then, for any chosen ${\mathcal {G}}_{p}\in \partial \mathrm{Prox}_{p}(x)$, the linear operator $I-{\mathcal {G}}_{p}+{\mathcal {G}}_{p}{\mathcal {B}}$ is nonsingular.

Proof

It follows Lemma 6.1 that $\mathrm{Prox}_p$ is the projection onto the closed convex set ${\mathcal {C}}$. Therefore, we know from Sun and Qi (2001, Theorem 2.3) that any element ${\mathcal {G}}_p\in \partial \mathrm{Prox}_{p}(x)$ is self-adjoint, positive definite, and $\lambda _{\max }({\mathcal {G}}_p)\in [0,1]$. The proof can be completed by Zhang et al. (2020, Lemma 3.2). □

Lemma 6.5

Let ${\mathcal {K}}_{pert}$ be the KKT mapping defined by (2.3), $({\bar{x}},{\bar{y}},{\bar{z}})$ be the KKT point of problem (1.4). Then, Any element in $\partial _{(x,y,z)}{\mathcal {K}}_{pert}(({\bar{x}},{\bar{y}},{\bar{z}}),(0,0,0))$ is nonsingular.

Proof

Since $\mathrm{Prox}_p$ is directionally differentiable, it follows from the chain rule presented in Sun (2006, Lemma 2.1) that for any ${\mathcal {G}}\in \partial _{(x,y,z)}{\mathcal {K}}_{pert}(({\bar{x}},{\bar{y}},{\bar{z}}),(0,0,0))$, there exist ${\mathcal {G}}_{f}\in \partial \mathrm{Prox}_{f}({\bar{x}}-{\bar{z}}-{\hat{s}})$ and ${\mathcal {G}}_{p}\in \partial \mathrm{Prox}_p({\bar{y}}+{\bar{z}})$ such that

$$\begin{aligned} {{\mathcal {G}}}(\Delta x,\Delta y,\Delta z)= \left( \begin{array}{c} \Delta x-{\mathcal {G}}_{f}(\Delta z+\Delta x)\\ \Delta y-{\mathcal {G}}_{p}(\Delta y+\Delta z)\\ \Delta x-\Delta y \end{array} \right) ,\,\,\,\forall \, (\Delta x,\Delta y,\Delta z)\in {\mathbb {S}}^n\times {\mathbb {S}}^n\times {\mathbb {S}}^n. \end{aligned}$$

Suppose that there exists $(\Delta x,\Delta y,\Delta z)\in {\mathbb {S}}^n\times {\mathbb {S}}^n\times {\mathbb {S}}^n$ such that ${{\mathcal {G}}}(\Delta x,\Delta y,\Delta z)=0$, i.e.,

$$\begin{aligned} \left\{ \begin{array}{l} \Delta x-{\mathcal {G}}_{f}(\Delta z+\Delta x)=0,\\ \Delta y-{\mathcal {G}}_{p}(\Delta y+\Delta z)=0,\\ \Delta x-\Delta y=0. \end{array} \right. \end{aligned}$$

(6.4)

It follows from Lemma 6.3 that both ${\mathcal {G}}_{f}$ and ${\mathcal {G}}^{-1}_{f}-I$ are self-adjoint and positive definite. This, together with (6.4), implies that

$$\begin{aligned} \Delta z = \left( {\mathcal {G}}^{-1}_{f}-I\right) \Delta x\,\,\,\hbox {and}\,\,\, \left( I-{\mathcal {G}}_p+{\mathcal {G}}_p\left( {\mathcal {G}}^{-1}_f-I\right) \right) \Delta x=0. \end{aligned}$$

(6.5)

We know from Lemma 6.4 that $(I-{\mathcal {G}}_p+{\mathcal {G}}_p({\mathcal {G}}^{-1}_f-I))$ is nonsingular. This, together with (6.5), implies that

$$\begin{aligned} \Delta x=0,\,\,\Delta y =0,\,\,\hbox {and}\,\, \Delta z =0. \end{aligned}$$

Therefore, ${\mathcal {G}}$ is nonsingular. The proof is completed. □

In order to give the proof of Proposition 2.1, we recall the implicit theorem from Clarke et al. (1998). Let ${\mathbb {X}}$ be a Hilbert space and ${\mathbb {M}}$ be a metric space. Consider the equation

$$\begin{aligned} {\mathcal {H}}(x,\alpha ) =0, \end{aligned}$$

where ${\mathcal {H}}$ is a mapping from ${\mathbb {X}}\times {\mathbb {M}}$ to ${\mathbb {X}}$. Assume that $V\subseteq {\mathbb {X}}$ is an open set such that ${\mathcal {H}}$ is continuous on $V\times {\mathbb {M}}$ and such that the partial derivative $\partial _x{\mathcal {H}}(x,\alpha )$ exists for all $(x,\alpha )\in V\times {\mathbb {M}}$, and is continuous jointly in $(x,\alpha )\in V\times {\mathbb {M}}$.

The following result is from Clarke et al. (1998, Theorem 3.6), which is usually named as Clarke’s implicit function theorem.

Lemma 6.6

Let $(x_0,\alpha _0)\in V\times {\mathbb {M}}$ be a point satisfying ${\mathcal {H}}(x_0,\alpha _0)=0$. Then one has

• (a) If $\partial _x{\mathcal {H}}(x_0,\alpha _0)$ is onto and one to one, then there exist neighborhoods ${\mathcal {N}}_x$ of $x_0$ and ${\mathcal {N}}_{\alpha }$ of $\alpha _0$ and a unique continuous function ${\hat{x}}(\cdot ):{\mathcal {N}}_{\alpha }\rightarrow {\mathcal {N}}_{x}$ with ${\hat{x}}(\alpha _0)=x_0$ such that ${\mathcal {H}}({\hat{x}}(\alpha ),\alpha )=0,\,\,\forall \alpha \in {\mathcal {N}}_{\alpha }$.

• (b) If in addition ${\mathcal {H}}$ is Lipschitz in a neighborhood of $(x_0,\alpha _0)$, then ${\hat{x}}$ is Lipschitz.

Now, we are ready to present the proof of Proposition 2.1.

Proof

The global Lipschitz continuities of the proximal mappings $\mathrm{Prox}_{f}$ and $\mathrm{Prox}_{p}$ imply that the mapping ${\mathcal {K}}_{pert}$ defined by (2.3) is Lipschitz continuous. Therefore, the proof can be completed by Lemmas 6.5, 6.6, and the fact that for any (u, v, w), the set $\mathsf{Sol}(u,v,w)$ must be a singleton if it is nonempty. □