Misspecified nonconvex statistical optimization for sparse phase retrieval

Abstract

Existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying “true” statistical models. To address this issue, we take a first step towards taming model misspecification by studying the high-dimensional sparse phase retrieval problem with misspecified link functions. In particular, we propose a simple variant of the thresholded Wirtinger flow algorithm that, given a proper initialization, linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions. We further provide extensive numerical experiments to support our theoretical findings.

Appendix

1.1 A Further simulations

We further investigate how the sparsity level of the estimator changes over iterations empirically. For the three link function $h_1$, $h_2$, and $h_3$ defined in Eq. (4.1), we plot the sparsity of the TWF algorithm in Fig. 5 up to $T = 1000$ iterations. Similar to our experimental study of the optimization error, here we set $p = 1000$, $s= 5$, and $n = 864 \approx 5 s^2 \log p$. Moreover, the tuning parameters are set as $\gamma = 2$, $\kappa = 15$, and $\eta = 0.005$. In this case, as we show in Sect. 4.1, our TWF algorithm converges geometrically to an estimator with high statistical accuracy. As shown in Fig. 5, our algorithm tends to over-estimate the sparsity. However, in all of these figures, the estimated sparsity gradually decreases as the algorithm proceeds. Furthermore, combining with the experiments in Sect. 4.1, although the estimator has more nonzero entries than the true parameter, the statistical error is satisfactory.

Fig. 5

Plots of the sparsity of the TWF updates $\beta ^{(t)}$. Here the link function is one of $h_1$, $h_2$, and $h_3$. In addition, we set $p = 1000$, $s = 5$, $n = 864 \approx 5 s^2\cdot \log p$, and run the TWF algorithm for $T = 1000$ iterations. These figures are generated based on 50 independent trials for each link function

Fig. 6

Plots of the statistical errors for the cubic model $Y = (X^\top \beta ^*)^3 + \epsilon$ and the phase retrieval model $Y = (X^\top \beta ^*)^2 + \epsilon$. We set $p = 1000$, $s= 5$, and let n vary. The plots are generated based on 100 independent trials for each (n, p, s). In a, we plot the two error curves together, which shows that the TWF algorithm incurs much larger error on the cubic model, whereas the error for the phase retrieval model becomes rather negligible in comparison. In b and c, we plot the two curves in a separately for presentation

In addition, we show a failed example, which violates Assumption 3.1, for the readers to better understand to what extent the proposed algorithm is robust to model misspecification. In particular, consider the cubic link function $f(u, v) = u^3 + v$, which violates Assumption 3.1 since in this case we have $\text {Cov} [ Y, (X ^\top \beta ^*)^2 ] = 0$. We compare this cubic model $Y = (X^\top \beta ^*)^3 + \epsilon$ with the phase retrieval model $Y = (X^\top \beta ^*)^2 + \epsilon$, where the link function is quadratic. We set $p = 1000$, $s = 5$, and let n vary. For each setting, we report the estimation error based on 100 independent trials. The statistical error of the TWF algorithm for these two models are plotted in Fig. 6a, which shows that our algorithm has nondecreasing estimation error for the cubic model even when n is very large. This is in sharp contrast with the phase retrieval model, where the estimation error is negligible. Moreover, we plot the two error curves separately in Fig. 6b and c to better see the details.