Abstract
Bundle adjustment is a least squares method-based algorithm for minimizing the global reprojection error and has provided an effective solution for structure from motion (SfM). The Levenberg–Marquardt algorithm provides a feasible and convenient way for bundle adjustment and creates a system of linear equations which are normal equations. For the special sparsity structure of an augmented Hessian matrix, a Schur complement trick is introduced to reduce computation complexity. However, the general sparse matrix storage formats are not optimized for the augmented Hessian matrix and consume too much computation time. According to the Schur complement trick, this paper divides the arrow-like augmented Hessian matrix into a structure matrix, a camera matrix and an observation matrix, and then proposes a new compressed matrix sequence (CMS) method to reduce time complexity for matrix operations. Under the definition of CMS, all of the matrices are stored in a dense form to accelerate the matrix operations, of which the matrix operations are redefined as well. CMS costs little computation time to build sparse matrices or access sparse matrices. The experimental results show that CMS achieves a significant speedup over general sparse matrix storage formats. Also, CMS being insensitive to the data input is more stable.
Similar content being viewed by others
Change history
13 June 2021
A Correction to this paper has been published: https://doi.org/10.1007/s00138-021-01222-5
References
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Chapter 21: Bundle Adjustment-A Modern Synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, vol. 1883, pp. 298–372. Springer, Berlin (2000)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)
Jeong, Y., Nister, D., Steedly, D., Szeliski, R., Kweon, I.: Pushing the envelope of modern methods for bundle adjustment. In: CVPR10, pp. 1474-1481 (2010)
Atkinson, K.B. (ed.): Close Range Photogrammetry and Machine Vision. Whittles Publishing, Roseleigh House, Latheronwheel, Caithness, Scotland (1996)
Granshaw, S.: Bundle adjustment methods in engineering photogrammetry. Photogramm. Record 10(56), 181–207 (1980)
Heinly, J., Schonberger, J. L., Dunn, E., Frahm, J. M.: Reconstructing the world* in six days*(as captured by the yahoo 100 million image dataset). In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3287-3295 (2015)
Çalışkan, A., Çevik, U.: Three-dimensional modeling in medical image processing by using fractal geometry. J. Comput. 12(5), 479–485 (2017)
Dellaert, F., Kaess, M.: Square Root SAM: Simultaneous localization and mappingvia square root information smoothing. Int. J. Robot. Res. 25(12), 1181–1203 (2006)
Kaess, M., Ranganathan, A., Dellaert F.: iSAM: Incremental Smoothing and Mapping, IEEE Trans Robot (TRO), 24(6), 1365–1378 (2008)
Kaess, M., Johannsson, H., Roberts, R. et al.: iSAM2: Incremental smoothing and mapping using the Bayes tree. Int. J. Robot. Res. 31(2), 216–235 (2012)
Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2000)
Moré, J. J.: The levenberg-marquardt algorithm: implementation and theory. In: Numerical analysis, pp. 105-116. Springer (1978)
Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Madsen, K., Nielsen, H., Tingleff, O.: Methods for non-linear least squares problems (2004)
Lourakis, M., Argyros, A.A.: SBA: A software package for generic sparse bundle adjustment. TOMS 36, 2 (2009)
Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Croz, J. Du, Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, Third Edition. SIAM Press, Philadelphia, (1999). LAPACK home page: http://www.netlib.org/lapack
Lawson, C.L., Hanson, R.J., Kincaid, D., Krogh, F.T.: Subprograms, basic linear algebra., for Fortran Usage, ACM Trans. Math Soft 5, 308–323 (1979)
Agarwal, S., Snavely, N., Seitz, S. M., Szeliski, R.: “Bundle adjustment in the large,” In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 29-42 (2010)
Byröd, M., Åström, K.: Conjugate Gradient Bundle Adjustment. In: ECCV10, pp. II: 114–127 (2010)
Bell, N., Garland, M.: Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: SC –09, pp. 1-11 (2009)
Li, R., Saad, Y.: GPU-accelerated preconditioned iterative linear solvers. Technical Report UMSI-2010-xx3, Minnesota Supercomputer Institute, University of Minnesota (2010)
Frahm, J., Georgel, P. Fite, Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y., Dunn, E., Clipp, B., Lazebnik, S., Pollefeys, M.: Building rome on a cloudless day. In: ECCV10, pp. IV: 368-381 (2010)
Gupta, S., Choudhary, S., Narayanan, P.J.: Practical time bundle adjustment for 3d reconstruction on gpu. In: ECCV Workshop on Computer Vision on GPUs (2010)
Wu, C., Agarwal, S., Curless, B., Seitz, S. M.: Multicore bundle adjustment. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 3057-3064. IEEE (2011)
Li, Y., Xie, P., Chen, X., Liu, J., Yang, B., Li, S., Gong, C., Gan, X., Xu, H.: VBSF: a new storage format for SIMD sparse matrixvector multiplication on modern processors. J. Supercomput. 76, 2063–2081 (2020)
Montagne, E., Ekambaram, A.: An optimal storage format for sparse matrices. Inf. Process. Lett. 90(1), 87–92 (2004)
Vassiliadis, S., Cotofana, S. D., Stathis, P.: Block based compression storage expected performance. In: Proceedings of 14th International Conference on High Performance Computing Systems and Applications (HPCS 2000) June (2000)
Saad, Y.: Iterative methods forsparse linear systems. 2nd edn. Society for Industrial and Applied Mathematics, pp. 95–226 (2003)
Chen, Y., Davis, T.,Hager, W., Rajamanickam, S.: Algorithm 887: CHOLMOD, Supernodal sparse Cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35(3), 22(2008). https://doi.org/10.1145/1391989.1391995
Davis, T., Hager, W.: Dynamic supernodes in sparse Cholesky update/downdate and triangular solves. ACM Trans. Math. Softw. 35(4) (2009)
Rotkin, V., Toledo, S.: The design and implementation of a new out-of-core sparse Cholesky factorization method. ACM Trans. Math. Softw. 30(1), 19–46 (2004)
Acknowledgements
This research work was supported by the National Key Research and Development Program of China(2018YFB0204301). We would also like to thank Dr. Haiyan Chen for her helpful suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: Error in affiliation for Jiaxin Peng corrected.
Rights and permissions
About this article
Cite this article
Peng, J., Liu, J. & Wei, H. A compressed matrix sequence method for solving normal equations of bundle adjustment. Machine Vision and Applications 32, 81 (2021). https://doi.org/10.1007/s00138-021-01212-7
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-021-01212-7