Skip to main content
SpringerLink
Log in

Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Low-rank matrix approximation has applications in many fields, such as 3D reconstruction from an image sequence and 2D filter design. In this paper, one issue with low-rank matrix approximation is re-investigated: the missing data problem. Much effort was devoted to this problem, and the Wiberg algorithm or the damped Newton algorithm were recommended in previous studies. However, the Wiberg or damped Newton algorithms do not suit for large (especially “long”) matrices, because one needs to solve a large linear system in every iteration. In this paper, we revitalize the usage of the Levenberg-Marquardt algorithm for solving the missing data problem, by utilizing the property that low-rank approximation is a minimization problem on subspaces. In two proposed implementations of the Levenberg-Marquardt algorithm, one only needs to solve a much smaller linear system in every iteration, especially for “long” matrices. Simulations and experiments on real data show the superiority of the proposed algorithms. Though the proposed algorithms achieve a high success rate in estimating the optimal solution by random initialization, as illustrated by real examples; it still remains an open issue how to properly do the initialization in a severe situation (that is, a large amount of data is missing and with high-level noise).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aguiar, P. M. Q., & Moura, J. M. F. (2003). Rank 1 weighted factorization for 3D structure recovery: algorithms and performance analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1134–1149.

    Article  Google Scholar 

  • Aguiar, P. M. Q., & Moura, J. M. F. (1999). Factorization as a rank 1 problem. In Proc. conf. computer vision and pattern recognition (pp. 178–184).

  • Aguiar, P. M. Q., & Moura, J. M. F. (2000). Weighted factorization. In ICIP (pp. 549–552).

  • Anandan, P., & Irani, M. (2002). Factorization with uncertainty. International Journal of Computer Vision, 49(2/3), 101–116.

    Article  MATH  Google Scholar 

  • Brand, M. (2002). Incremental singular value decomposition of uncertain data with missing values. In Proc. European conf. computer vision (pp. 707–720).

  • Brandt, S. (2002). Closed-form solutions for affine reconstruction under missing data. In Statistical methods for video processing workshop, in conjunction with ECCV02 (pp. 109–114).

  • Buchanan, A. M., & Fitzgibbon, A. W. (2005). Damped Newton algorithms for matrix factorization with missing data. In Proceedings of IEEE computer vision and pattern recognition.

  • Chen, P. (2004). An investigation of statistical aspects of linear subspace analysis for computer vision applications. Ph.D. thesis, Monash University.

  • Chen, P. (2008). Heteroscedastic low-rank matrix approximation by the Wiberg algorithm. IEEE Transactions on Signal Processing 56(4), 1429–1439.

    Article  Google Scholar 

  • Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: Application to sfm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1051–1063.

    Article  Google Scholar 

  • Chen, P., & Suter, D. (2007). A bilinear approach to the parameter estimation of a general heteroscedastic linear system, with application to conic fitting. Journal of Mathematical Imaging and Vision, 28(3), 191–208.

    Article  MathSciNet  Google Scholar 

  • Edelman, A., Arias, T. A., & Smith, S. T. (1998). The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2), 303–353.

    Article  MATH  MathSciNet  Google Scholar 

  • Golub, G. H., & Loan, C. F. V. (1996). Matrix computations (3rd ed.). Baltimore: Johns Hopkins Press.

    MATH  Google Scholar 

  • Guerreiro, R. F. C., & Aguiar, P. M. Q. (2002). 3D structure from video streams with partially overlapping images. In IEEE international conference on image processing ICIP’02.

  • Guerreiro, R. F. C., & Aguiar, P. M. Q. (2003). Estimation of rank deficient matrices from partial observations: Two-step iterative algorithms. In Energy minimization methods in computer vision and pattern recognition.

  • Hartley, R. I., & Zisserman, A. (2003). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.

    Google Scholar 

  • Heyden, A., & Kahl, F. (1998). Reconstruction from affine cameras using closure constraints. In ICPR (pp. 47–50).

  • Hua, Y., Nikpour, M., & Stoica, P. (2001). Optimal reduced-rank estimation and filtering. IEEE Transactions on Signal Processing, 49(3), 457–469.

    Article  Google Scholar 

  • Irani, M., & Anandan, P. (2000). Factorization with uncertainty. In Proc. European conf. computer vision (Vol. 49, pp. 539–553).

  • Jacobs, D. (1997). Linear fitting with missing data: Applications to structure-from-motion and to characterizing intensity images. In Proc. conf. computer vision and pattern recognition (pp. 206–212).

  • Jacobs, D. (2001). Linear fitting with missing data for structure-from-motion. Computer Vision and Image Understanding, 82, 57–81.

    Article  MATH  Google Scholar 

  • Kahl, F., & Heyden, A. (1998). Structure and motion from points, lines and conics with affine cameras. In Proc. European conf. computer vision (pp. 327–341).

  • Kahl, F., & Heyden, A. (1999). Affine structure and motion from points, lines and conics. International Journal of Computer Vision, 33(3), 163–180.

    Article  Google Scholar 

  • De la Torre, F., & Black, M. J. (2003). A framework for robust subspace learning. International Journal of Computer Vision, 54(1–3), 117–142.

    Article  MATH  Google Scholar 

  • Lu, W. S., Pei, S. C., & Wang, P. H. (1997). Weighted low-rank approximation of general complex matrices and its application in the design of 2D digital filter. IEEE Transactions on Circuits and Systems I, 44, 650–655.

    Article  MATH  MathSciNet  Google Scholar 

  • Manton, J. H., Mahony, R., & Hua, Y. (2003). The geometry of weighted low-rank approximations. IEEE Transactions on Signal Processing, 51(2), 500–514.

    Article  MathSciNet  Google Scholar 

  • Martinec, D., & Pajdla, T. (2002). Structure from many perspective images with occlusion. In Proc. European conf. computer vision (pp. 355–369).

  • Martinec, D., & Pajdla, T. (2005). 3D reconstruction by fitting low-rank matrices with missing data. In Proc. conf. computer vision and pattern recognition (pp. 198–205).

  • Morris, D. D., & Kanade, T. (1998). A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proc. int. conf. computer vision (pp. 696–702).

  • Nocedal, J., & Wright, S. J. (1999). Numerical optimization. Berlin: Springer.

    MATH  Google Scholar 

  • Okatani, T., & Deguchi, K. (2006). On the Wiberg algorithm for matrix factorization in the presence of missing components. International Journal of Computer Vision, 72(3), 329–337.

    Article  Google Scholar 

  • Poelman, C., & Kanade, T. (1997). A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 206–219.

    Article  Google Scholar 

  • Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in C (2nd ed.). Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Shum, H., Ikeuchi, K., & Reddy, R. (1995). Principal component analysis with missing data and its applications to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9), 854–867.

    Article  Google Scholar 

  • Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2), 137–154.

    Article  Google Scholar 

  • Triggs, B., McLauchlan, P., Hartley, R., & Fitzgibbon, A. (2000). Bundle adjustment—A modern synthesis. In W. Triggs, A. Zisserman, & R. Szeliski (Eds.), LNCS. Vision algorithms: theory and practice (pp. 298–375). Berlin: Springer.

    Chapter  Google Scholar 

  • Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data using power factorization and gpca. In Proc. conf. computer vision and pattern recognition.

  • Wiberg, T. (1976). Computation of principle components when data are missing. In 2nd symposium computational statistics.

Download references

Author information

Authors and Affiliations

  1. Shenzhen Institute of Advanced Integration Technology, Chinese Academy of Sciences/Chinese University of Hong Kong, Shenzhen, China

    Pei Chen

Authors
  1. Pei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pei Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, P. Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix. Int J Comput Vis 80, 125–142 (2008). https://doi.org/10.1007/s11263-008-0135-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0135-7

Keywords

Search

Navigation