Abstract
Several techniques have been proposed for tackling the Structure from Motion problem through factorization in the case of missing data. However, when the percentage of unknown data is high, most of them may not perform as well as expected. Focussing on this problem, an iterative multiresolution scheme, which aims at recovering missing entries in the originally given input matrix, is proposed. Information recovered following a coarse-to-fine strategy is used for filling in the missing entries. The objective is to recover, as much as possible, missing data in the given matrix. Thus, when a factorization technique is applied to the partially or totally filled in matrix, instead of to the originally given input one, better results will be obtained. An evaluation study about the robustness to missing and noisy data is reported. Experimental results obtained with synthetic and real video sequences are presented to show the viability of the proposed approach.
Similar content being viewed by others
References
Aanaes, H., Fisker, R.: Robust factorization. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1215–1225 (2002)
Buchanan, A., Fitzgibbon, A.W.: Damped Newton algorithms for matrix factorization with missing data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 316–322. University of Oxford, Oxford (2005)
Chen, P., Suter, D.: Recovering the missing components in a large noisy low-rank matrix: Application to SFM. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1051–1063 (2004)
Costeira, J.P., Kanade, T.: A multibody factorization method for independently moving objects. Int. J. Comput. Vis. 3, 159–179 (1998)
Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Graph. Image Process. 24, 381–395 (1981)
Golub, G.H., Van Loan, C.F. (eds.): Matrix Computations. Johns Hopkins Univ. Press, Baltimore (1989)
Guerreiro, R.F.C., Aguiar, P.M.Q.: Estimation of rank deficient matrices from partial observations: two-step iterative algorithms. In: EMMCVPR, pp. 450–466 (2003)
Guilbert, N., Bartoli, A., Heyden, A.: Affine approximation for direct batch recovery of Euclidean structure and motion from sparse data. Int. J. Comput. Vis. 69, 317–333 (2006)
Han, M., Kanade, T.: Reconstruction of a scene with multiple linearly moving objects. Int. J. Comput. Vis. 53, 285–300 (2000)
Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the Fourth Alvey Vision Conference, pp. 147–151 (1988)
Hartley, R., Schaffalitzky, F.: Powerfactorization: 3D reconstruction with missing or uncertain data. In: Australian-Japan Advanced Workshop on Computer Vision (2003)
Hartley, R., Zisserman, A. (eds.): Multiple View Geometry in Computer Vision. Cambridge University Press. Book web page at: http://www.robots.ox.ac.uk/~vgg/hzbook/index.html, accessed February 2007 (2000)
Huynh, D.Q., Hartley, R., Heyden, A.: Outlier correction in image sequences for the affine camera. In: IEEE ICCV, vol. 1, pp. 585–590 (2003)
Huynh, D.Q., Heyden, A.: Outlier detection in video sequences under affine projection. In: IEEE CVPR, vol. 1, pp. 695–701 (2001)
Jacobs, D.W.: Linear fitting with missing data for structure-from-motion. Comput. Vis. Image Underst. 1, 57–81 (2001)
Jia, H., Fortuna, J., Martinez, A.: Perturbation estimation of the subspaces for structure from motion with noisy and missing data. In: Third International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 1101–1007 (2006)
Julià, C., Sappa, A., Lumbreras, F., Serrat, J., López, A.: Factorization with missing and noisy data. In: Computational Science ICCS 2006: 6th International Conference. LNCS, vol. 1, pp. 555–562 (2006)
Julià, C., Sappa, A., Lumbreras, F., Serrat, J., López, A.: An iterative multiresolution scheme for SFM. In: ICIAR 2006. LNCS, vol. 4141, pp. 804–815 (2006)
Ma, Y., Soatto, J., Kosecká, J., Sastry, S.: An Invitation to 3D Vision: From Images to Geometric Models. Springer, New York (2004)
Martinec, D., Pajdla, T.: 3D reconstruction by fitting low-rank matrices with missing data. In: IEEE CVPR, vol. 1, pp. 198–205 (2005)
Morita, T., Kanade, T.: A sequential factorization method for recovering shape and motion from image streams. IEEE Trans. Pattern Anal. Mach. Intell. 19(8), 858–867 (1997)
Morris, D., Kanade, T.: A unified factorization algorithm for points, line segments and planes with uncertainty models. In: International Conference on Computer Vision, pp. 696–702 (1998)
Okatani, T., Deguchi, K.: On the Wiberg algorithm for matrix factorization in the presence of missing components. Int. J. Comput. Vis. 72, 329–337 (2007)
Poelman, C.J., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. Pattern Anal. Mach. Intell. 19, 206–218 (1997)
Quan, L., Kanade, T.: A factorization method for affine structure from line correspondences. In: CVPR (1996)
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)
Triggs, B.: Linear projective reconstruction from matching tensors. Image Vis. Comput. 15, 617–625 (1997)
Wiberg, T.: Computation of principal components when data is missing. In: Second Symposium of Computational Statistics, pp. 229–326 (1976)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Julià, C., Sappa, A.D., Lumbreras, F. et al. An Iterative Multiresolution Scheme for SFM with Missing Data. J Math Imaging Vis 34, 240–258 (2009). https://doi.org/10.1007/s10851-009-0144-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-009-0144-3