Hostname: page-component-8448b6f56d-jr42d Total loading time: 0 Render date: 2024-04-23T14:15:41.341Z Has data issue: false hasContentIssue false
  • English
  • Français

Graph-Based Registration and Blending for Undersea Image Stitching

Published online by Cambridge University Press:  31 May 2019

Xu Yang Open the ORCID record for Xu Yang [Opens in a new window] ,
Zhi-Yong Liu ,
Hong Qiao ,
Jian-Hua Su ,
Da-Xiong Ji ,
Ai-Yun Zang  and
Hai Huang
Xu Yang
Affiliation:
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China, E-mails: xu.yang@ia.ac.cn, zhiyong.liu@ia.ac.cn, hong.qiao@ia.ac.cn, jianhua.su@ia.ac.cn
Zhi-Yong Liu
Affiliation:
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China, E-mails: xu.yang@ia.ac.cn, zhiyong.liu@ia.ac.cn, hong.qiao@ia.ac.cn, jianhua.su@ia.ac.cn
Hong Qiao
Affiliation:
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China, E-mails: xu.yang@ia.ac.cn, zhiyong.liu@ia.ac.cn, hong.qiao@ia.ac.cn, jianhua.su@ia.ac.cn
Jian-Hua Su
Affiliation:
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China, E-mails: xu.yang@ia.ac.cn, zhiyong.liu@ia.ac.cn, hong.qiao@ia.ac.cn, jianhua.su@ia.ac.cn
Da-Xiong Ji
Affiliation:
Ocean College, Zhejiang University, Zhoushan 316000, People’s Republic of China. E-mail: 0015108@zju.edu.cn
Ai-Yun Zang
Affiliation:
College of Engineering, Ocean University of China, Qingdao 266100, People’s Republic of China. E-mail: zaysi@sina.com
Hai Huang*
Affiliation:
National Key Laboratory of Science and Technology of Underwater Vehicle, Harbin Engineering University, Harbin 150001, People’s Republic of China
*
*Corresponding author. E-mail: haihus@163.com
Get access Rights & Permissions [Opens in a new window]

Summary

Image stitching is important for the perception and manipulation of undersea robots. In spite of a well-developed technique, it is still challenging for undersea images because of their inevitable appearance ambiguity caused by the limited light in the undersea environment, and local disturbance caused by moving objects, ocean current, etc. To get a clean and stable background panorama in the undersea environment, this paper proposes an undersea image-stitching method by introducing graph-based registration and blending procedures. Specifically, in the registration procedure, matching the features in each undersea image pair is formulated and solved by graph matching, to incorporate the structural information between features. In the blending procedure, an energy function on the indirect graph Markov random field is proposed, which takes both image consistency and neighboring consistency into consideration. Coincidentally, both graph matching and energy minimization can be mathematically formulated by integer quadratic programming problems with different constraints; the recently proposed graduated nonconvexity and concavity procedure is used to optimize both problems. Experiments on both synthetic images and real-world undersea images witness the effectiveness of the proposed method.

Keywords

Undersea image stitchingFeature correspondenceGraph matchingEnergy minimizationNonsubmodular function
Type
Articles
Information
Robotica , Volume 38 , Issue 3 , March 2020 , pp. 396 - 409
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bay, H., Tuytelaars, T. and Van Gool, L., “SURF: Speeded Up Robust Features,” Proceedings of the 7th European Conference on Computer Vision, Austria (2006) pp. 404–417.Google Scholar
Belongie, S. and Malik, J., “Matching with Shape Contexts,” Proceedings of the IEEEWorkshop on Contentbased Access of Image and Video Libraries, Hilton Head Island, SC, USA (2000) pp. 20–26.Google Scholar
Boyd, S. and Vandenberghe, L., Convex Optimization (Cambridge University Press, Cambridge, UK, 2004).CrossRefGoogle Scholar
Boykov, Y., Veksler, O. and Zabih, R., “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001).CrossRefGoogle Scholar
Brown, M. and Lowe, D. G., “Automatic panoramic image stitching using invariant features,” Int’l. J. Comput. Vision. 74(1), 59–73 (2007).CrossRefGoogle Scholar
Calonder, M., Lepetit, V., Strecha, C. and Fua, P., “BRIEF: Binary Robust Independent Elementary Features,” European Conference on Computer Vision (Springer, Berlin, Heidelberg, 2010) pp. 778–792.Google Scholar
Cour, T. and Shi, J. B., “Solving Markov Random Fields with Spectral Relaxation,” Proceedings of the International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico (2007)Google Scholar
Davis, J., “Mosaics of Scenes with Moving Objects,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA (1998) pp. 354–360.Google Scholar
Egozi, A., Keller, Y. and Guterman, H., “A probabilistic approach to spectral graph matching,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 18–27 (2013).Google Scholar
Elibol, A., Garcia, R. and Gracias, N., “A new global alignment approach for underwater optical mapping,” Ocean Eng. 38(10), 1207–1219 (2011).Google Scholar
Ferreira, F., Veruggio, G., Caccia, M., Zereik, E. and Bruzzone, G., “A Real-Time Mosaicking Algorithm Using Binary Features for ROVs,” Proceedings of the Mediterranean Conference on Control and Automation, Chania (2013) pp. 1267–1273.CrossRefGoogle Scholar
Finlayson, G., Hordley, S., Schaefer, G. and Tian, G. Y., “Illuminant and device invariant colour using histogram equalisation,” Pattern Recogn. 38(2), 179–190 (2005).Google Scholar
Fischler, M. A. and Bolles, R. C., “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM. 24(6), 381–395 (1981).Google Scholar
Frank, M. and Wolfe, P., “An algorithm for quadratic programming,” Naval Res. Logis. Q. 3(1–2), 95–110 (1956).Google Scholar
Garcia-Fidalgo, E., Ortiz, A., Bonnin-Pascual, F. and Company, J. P., “Fast Image Mosaicing Using Incremental Bags of Binary Words,” Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden (2016) pp. 1174–1180.Google Scholar
Jaggi, M., “Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization,” Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA (2013) pp. 427–435.Google Scholar
Kolmogorov, V., “Convergent tree-reweighted message passing for energy minimization,” IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1568–1583 (2006).Google Scholar
Kolmogorov, V. and Rother, C., “Minimizing nonsubmodular functions with graph cuts a review,” IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1274–1279 (2007).Google Scholar
Leone, A., Distante, C., Mastrolia, A. and Indiveri, G.,” A Fully Automated Approach for Underwater Mosaicking,” OCEANS (2006) pp. 1–6.Google Scholar
Leordeanu, M. and Hebert, M., “A Spectral Technique for Correspondence Problems Using Pairwise Constraints,” Proceedings of the IEEE International Conference on Computer Vision, Beijing, China (2005) pp. 1482–1489.Google Scholar
Leordeanu, M. and Hebert, M., “Efficient MAP Approximation for Dense Energy Functions,” Proceedings of the International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA (2006).Google Scholar
Li, C., Liu, Z. Y., Yang, X., Qiao, H. and Su, J. H., “Stitching contaminated images,” Neurocomputing 214, 829–836 (2016).Google Scholar
Liu, Z. Y. and Qiao, H., “GNCCP - graduated nonconvexity and concavity procedure,” IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1258–1267 (2014).Google Scholar
Loera, J. D., Rambau, J. and Santos, F., Triangulations: Structures for Algorithms and Applications (Springer-Verlag, Berlin, Heidelberg, 2010).CrossRefGoogle Scholar
Lowe, D. G., “Object Recognition from Local Scale-Invariant Features,” Proceedings of IEEE International Conference on Computer Vision, Kerkyra, Greece, vol. 2 (1999) pp. 1150–1157.Google Scholar
Mills, A. and Dudek, G., “Image stitching with dynamic elements,” Image Vision Comput. 27(10), 1593–1602 (2009).Google Scholar
Minka, T. P., “Old and New Matrix Algebra Useful for Statistics,” Technical report (2001).Google Scholar
Ravikumar, P. and Lafferty, J., “Quadratic Programming Relaxations for Metric Labeling and Markov Random Field Map Estimation,” Proceedings of the International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA (2006) pp. 737–744.Google Scholar
Sun, J., Zheng, N. N. and Shum, H. Y., “Stereo matching using belief propagation,” IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003).Google Scholar
Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M. and Rother, C., “A comparative study of energy minimization methods for Markov random fields with smoothness-based priors,” IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008).Google Scholar
Tang, C. Q., Tian, G. Y., Chen, X. T., Wu, J. B., Li, K. and Meng, H., “Infrared and visible images registration with adaptable local-global feature integration for rail inspection,” Infrared Phys. Technol. 87, 31–39 (2017).Google Scholar
Tian, G. Y., Gledhill, D. and Taylor, D., “Comprehensive interest points based imaging mosaic,” Pattern Recogn. Lett. 24(9), 1171–1179 (2003).Google Scholar
Torr, P. H. S., “Solving Markov Random Fields Using Semi Definite Programming,” Proceedings of the International Conference on Artificial Intelligence and Statistics, Key West, Florida, USA (2003).Google Scholar
Torr, P. H. S. and Zisserman, A., “MLESAC: A new robust estimator with application to estimating image geometry,” Comput. Vision Image Understand. 78, 138–156 (2000).Google Scholar
Uyttendaele, M., Eden, A. and Skeliski, R., “Eliminating Ghosting and Exposure Artifacts in Image Mosaics,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA (2001) pp. II509–II516.Google Scholar
Yang, X., Liu, Z. Y., Qiao, H., Song, Y. B., Ren, S. N. and Zheng, S. W., “Underwater image matching by incorporating structural constraints,” Int’l. J. Adv. Robot. Syst. to appear, https://doi.org/10.1177/1729881417738100.CrossRefGoogle Scholar