Skip to main content
SpringerLink
Log in

A fast and robust homography scheme for real-time planar target detection

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The present paper is concerned with the problem of robust pose estimation for planar targets in the context of real-time mobile vision. For robust recognition of targets at very low computational costs, we employ feature-based methods which are based on local binary descriptors allowing fast feature matching at run time. The matching set is then fed to a robust parameter estimation algorithm to obtain a reliable estimate of homography. The robust estimation of model parameters, which in our case is a 2D homographic transformation, constitutes an essential part of the whole recognition process. We present a highly optimized and device-friendly implementation of homography estimation through a unified hypothesize-and-verify framework. This framework is specifically designed to meet the growing demand for fast and robust estimation on power-constrained platforms. The focus of the approach described in this paper is not only on developing fast algorithms for the recognition framework, but also on the optimized implementation of such algorithms by accounting for the computing capacity of modern CPUs. The experimentations show that the resulting homography estimation implementation proposed in this paper brings a speedup of 25\(\times\) over the regular OpenCV RANSAC homography estimation function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Available online at www.eecs.uottawa.ca/~laganier/projects/mobilevision.

  2. Available online at https://github.com/camilosw/ofxVideoMapping.

References

  1. Akhoury, S.S., Laganière, R.: Training binary descriptors for improved robustness and efficiency in real-time matching. In: Alfredo, P., (ed.), ICIAP 2013, volume 8157 of Lecture Notes in Computer Science, pp. 288–298 (2013)

  2. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, pp. 510–517 (2012)

  3. Barclay, A., Kaufmann, H.: FT-RANSAC: towards robust multi-modal homography estimation. In: Pattern recognition in remote sensing (PRRS), 2014 8th IAPR workshop on, pp. 1–4 (2014)

  4. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Understand. 110(3), 346–359 (2008)

    Article  Google Scholar 

  5. Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Computer vision and pattern recognition, 1997, Proceedings 1997 IEEE Computer society conference on, pp. 1000–1006, IEEE (1997)

  6. Bilaniuk, O., Bazargani, H., Laganiere, R.: Fast target recognition on mobile devices: revisiting gaussian elimination for the estimation of planar homographies. In: Computer vision and pattern recognition workshops (CVPRW), 2014 IEEE conference on, pp. 119–125, (2014)

  7. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: Brief: binary robust independent elementary features. ECCV 2010, volume 6314 of Lecture Notes in Computer Science, pp. 778–792. Springer, Berlin Heidelberg (2010)

  8. Chum, O., Matas, J.: Randomized ransac with Td,d test. Image Vis Comput, pp. 448–457 (2002)

  9. Chum, O., Matas, J.: Matching with PROSAC—progressive sample consensus. In: Computer vision and pattern recognition, 2005. CVPR 2005. IEEE computer society conference on, vol. 1, pp. 220–226 (2005)

  10. Chum, O., Matas, J.: Optimal randomized RANSAC. IEEE Trans. Pattern Anal. Mach. Intell. 30(8), 1472–1482 (2008)

    Article  Google Scholar 

  11. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  12. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  13. Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the thirtieth annual acm symposium on theory of computing, STOC ’98, pp. 604–613, New York, NY, USA, (1998) ACM. ISBN 0-89791-962-9

  14. Ke, Y., Sukthankar, R.: PCA-SIFT: A more distinctive representation for local image descriptors. In: Computer vision and pattern recognition, 2004. CVPR 2004, Proceedings of the 2004 IEEE computer society conference on, volume 2, pp. II-506, IEEE (2004)

  15. Lepetit, V., Fua, P.: Keypoint recognition using randomized trees. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006)

    Article  Google Scholar 

  16. Lepetit, V., Pilet, J., Fua, P.: Point matching as a classification problem for fast and robust object pose estimation. In: Computer vision and pattern recognition, 2004. CVPR 2004, Proceedings of the 2004 IEEE computer society conference on, vol 2, pp. II-244-II-250, vol. 2 (2004)

  17. Leutenegger, S., Chli, M., Siegwart, R.Y.: Brisk: binary robust invariant scalable keypoints. In Computer vision (ICCV), 2011 IEEE international conference on, pp. 2548–2555 (2011)

  18. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  19. Matas, J., Chum, O.: Randomized ransac with sequential probability ratio test. In Computer vision, 2005. ICCV 2005. Tenth IEEE international conference on, vol. 2, pp. 1727–1732 (2005)

  20. Matas, J., Chum, O.: Randomized RANSAC, Vienna University of Technology, pp. 49–58 (2002)

  21. Mondragòn, I.F., Campoy, P., Martinez, C., Olivares-Mèndez, M.A.: 3d pose estimation based on planar object tracking for uavs control. In: Robotics and automation (ICRA), 2010 IEEE international conference on, pp. 35–41 (2010)

  22. Mrquez-Neila, P., Lpez-Alberca, J. Buenaposada, J.M., Baumela, L.: Speeding-up homography estimation in mobile devices. J. Real Time Image Process., pp 1–14 (2013)

  23. Niitsuma, H., Rangarajan, P., Kanatani, K.: High accuracy homography computation without iterations. In: Proceedings of 16th symposium sensing imaging inf (SSII10) (2010)

  24. Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)

    Article  Google Scholar 

  25. Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 2022–2038 (2013)

    Article  Google Scholar 

  26. Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: Computer vision, 2005. ICCV 2005. Tenth IEEE international conference on, vol. 2, pp. 1508–1515 (2005)

  27. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. ECCV 2006, volume 3951 of Lecture Notes in Computer Science, pp. 430–443. Springer, Berlin Heidelberg (2006)

  28. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: Computer vision (ICCV), 2011 IEEE international conference on, pp. 2564–2571 (2011)

  29. Skrypnyk, I., Lowe, D.G.: Scene modelling, recognition and tracking with invariant image features. In: Mixed and augmented reality, 2004. ISMAR 2004. Third IEEE and ACM International symposium on, pp. 110–119 (2004)

  30. Taylor, S., Rosten, E., Drummond, T.: Robust feature matching in 2.3 \(\mu\)s. In: Computer vision and pattern recognition workshops, 2009. CVPR Workshops 2009. IEEE computer society conference on, pp. 15–22 (2009)

  31. Taylor, S., Drummond, T.: Binary histogrammed intensity patches for efficient and robust matching. Int. J. Comput. Vis. 94(2), 241–265 (2011)

    Article  Google Scholar 

  32. Torr, P.H.S., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Understand. 78, 2000 (2000)

    Article  Google Scholar 

  33. Vincent, E., Laganiere, R.: An empirical study of some feature matching strategies. In: Procedings of conference vision interface, Calgary, Canada, pp. 139–145 (2002)

  34. Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., Schmalstieg, D.: Pose tracking from natural features on mobile phones. In: Proceedings of the 7th IEEE/ACM international symposium on mixed and augmented reality, pp. 125–134. IEEE Computer Society (2008)

  35. Wald, A.: Sequential Analysis, 1st edn. Wiley, Ney York (1947)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON, K1N 6N5, Canada

    Hamid Bazargani, Olexa Bilaniuk & Robert Laganière

Authors
  1. Hamid Bazargani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olexa Bilaniuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Laganière
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Bazargani.

Appendix: Homography estimation using Gaussian elimination

Appendix: Homography estimation using Gaussian elimination

This appendix shows precisely the row operations we use for reduction to reduced-row-echelon form of the matrix shown in Eq. 4. The Gaussian elimination is done in two parts. In the first, identical row operations are applied to the top and bottom halves of the matrix, while in the second, row operations are applied to the whole matrix.

$$\begin{aligned} \left( \begin{array}{ccccccccc} x_{0} &{} y_{0} &{} 1 &{} 0 &{} 0 &{} 0 &{} -x_{0}X_{0} &{} -y_{0}X_{0} &{} X_{0} \\ x_{1} &{} y_{1} &{} 1 &{} 0 &{} 0 &{} 0 &{} -x_{1}X_{1} &{} -y_{1}X_{1} &{} X_{1} \\ x_{2} &{} y_{2} &{} 1 &{} 0 &{} 0 &{} 0 &{} -x_{2}X_{2} &{} -y_{2}X_{2} &{} X_{2} \\ x_{3} &{} y_{3} &{} 1 &{} 0 &{} 0 &{} 0 &{} -x_{3}X_{3} &{} -y_{3}X_{3} &{} X_{3} \\ 0 &{} 0 &{} 0 &{} x_{0} &{} y_{0} &{} 1 &{} -x_{0}Y_{0} &{} -y_{0}Y_{0} &{} Y_{0} \\ 0 &{} 0 &{} 0 &{} x_{1} &{} y_{1} &{} 1 &{} -x_{1}Y_{1} &{} -y_{1}Y_{1} &{} Y_{1} \\ 0 &{} 0 &{} 0 &{} x_{2} &{} y_{2} &{} 1 &{} -x_{2}Y_{2} &{} -y_{2}Y_{2} &{} Y_{2} \\ 0 &{} 0 &{} 0 &{} x_{3} &{} y_{3} &{} 1 &{} -x_{3}Y_{3} &{} -y_{3}Y_{3} &{} Y_{3} \end{array}\right) \end{aligned}$$

Starting from the above, we subtract rows 2 and 6 from the rows 0, 1, 3 and 4, 5, 7, respectively, thus eliminating almost all 1s in columns 2 and 5. Since we choose not to scale the rows containing the said 1s, they will remain unaffected throughout the remainder of the computation and therefore no storage needs to be reserved for them.

$$\begin{aligned} \sim \left( \begin{array}{ccccccccc} x_{0}-x_{2} &{} y_{0}-y_{2} &{} 0 &{} 0 &{} 0 &{} 0 &{} x_{2}X_{2}-x_{0}X_{0} &{} y_{2}X_{2}-y_{0}X_{0} &{} X_{0}-X_{2} \\ x_{1}-x_{2} &{} y_{1}-y_{2} &{} 0 &{} 0 &{} 0 &{} 0 &{} x_{2}X_{2}-x_{1}X_{1} &{} y_{2}X_{2}-y_{1}X_{1} &{} X_{1}-X_{2} \\ x_{2} &{} y_{2} &{} 1 &{} 0 &{} 0 &{} 0 &{} -x_{2}X_{2} &{} -y_{2}X_{2} &{} X_{2} \\ x_{3}-x_{2} &{} y_{3}-y_{2} &{} 0 &{} 0 &{} 0 &{} 0 &{} x_{2}X_{2}-x_{3}X_{3} &{} y_{2}X_{2}-y_{3}X_{3} &{} X_{3}-X_{2} \\ 0 &{} 0 &{} 0 &{} x_{0}-x_{2} &{} y_{0}-y_{2} &{} 0 &{} x_{2}Y_{2}-x_{0}Y_{0} &{} y_{2}Y_{2}-y_{0}Y_{0} &{} Y_{0}-Y_{2} \\ 0 &{} 0 &{} 0 &{} x_{1}-x_{2} &{} y_{1}-y_{2} &{} 0 &{} x_{2}Y_{2}-x_{1}Y_{1} &{} y_{2}Y_{2}-y_{1}Y_{1} &{} Y_{1}-Y_{2} \\ 0 &{} 0 &{} 0 &{} x_{2} &{} y_{2} &{} 1 &{} -x_{2}Y_{2} &{} -y_{2}Y_{2} &{} Y_{2} \\ 0 &{} 0 &{} 0 &{} x_{3}-x_{2} &{} y_{3}-y_{2} &{} 0 &{} x_{2}Y_{2}-x_{3}Y_{3} &{} y_{2}Y_{2}-y_{3}Y_{3} &{} Y_{3}-Y_{2} \end{array}\right) \end{aligned}$$

We note here that at this stage, of the 72 potential floating-point values in the matrix, only 32 (excluding the two remaining 1s) are distinct and non-zero. This neatly fits in half of a vector register file with 16 four-lane registers, a common configuration in most modern architectures.

For brevity, after this point only the row operations are given. They were designed to delay the use of reciprocals as long as possible. The first part is duplicated on both the top and bottom half.

First, we eliminate column 0 of rows 1 and 3:

$$\begin{aligned} {\mathbf {R}}_{1} = r_{0,x} * {\mathbf {R}}_{1} - r_{1,x} * {\mathbf {R}}_{0}, \qquad idem\,\, on\;\, {\mathbf {R}}_{5}\\ {\mathbf {R}}_{3} = r_{0,x} * {\mathbf {R}}_{3} - r_{3,x} * {\mathbf {R}}_{0}, \qquad idem\,\, on \,\;{\mathbf {R}}_{7} \end{aligned}.$$

We eliminate column 1 of rows 0 and 3.

$$\begin{aligned} {\mathbf {R}}_{0} = r_{1,y} * {\mathbf {R}}_{0} - r_{0,y} * {\mathbf {R}}_{1}, \qquad idem\,\, on\;\, {\mathbf {R}}_{4} \\ {\mathbf {R}}_{3} = r_{1,y} * {\mathbf {R}}_{3} - r_{3,y} * {\mathbf {R}}_{1}, \qquad idem\,\, on \,\;{\mathbf {R}}_{7} \end{aligned}.$$

We eliminate columns 0 and 1 of row 2.

$$\begin{aligned} {\mathbf {R}}_{0}& = \frac{1}{r_{0,x}} * {\mathbf {R}}_{0}, \qquad idem\,\, on \;\,{\mathbf {R}}_{4} \\ {\mathbf {R}}_{1}& = \frac{1}{r_{1,y}} * {\mathbf {R}}_{1}, \qquad idem\,\, on \,\;{\mathbf {R}}_{5} \\ {\mathbf {R}}_{2}& = {\mathbf {R}}_{2} - (r_{2,x} * {\mathbf {R}}_{0} + r_{2,y} * {\mathbf {R}}_{1}), \qquad idem\,\, on\; \,{\mathbf {R}}_{6} \end{aligned}.$$

Columns 0–5 of rows 3 and 7 are zero, and the matrix now resembles:

$$\begin{aligned} \left( \begin{array}{ccccccccc} 1 &{} &{} &{} &{} &{} 0 &{} a_{06} &{} a_{07} &{} a_{08} \\ &{} \ddots &{} &{} &{} &{} &{} a_{16} &{} a_{17} &{} a_{18} \\ &{} &{} 1 &{} 0 &{} &{} &{} a_{26} &{} a_{27} &{} a_{28} \\ &{} &{} 0 &{} 0 &{} &{} &{} a_{36} &{} a_{37} &{} a_{38} \\ &{} &{} 0 &{} 1 &{} &{} &{} a_{46} &{} a_{47} &{} a_{48} \\ &{} &{} &{} &{} \ddots &{} &{} a_{56} &{} a_{57} &{} a_{58} \\ &{} &{} &{} &{} &{} 1 &{} a_{66} &{} a_{67} &{} a_{68} \\ 0 &{} &{} &{} &{} &{} 0 &{} a_{76} &{} a_{77} &{} a_{78} \end{array}\right) , \end{aligned}$$

which concludes the first part. We now cease treating the matrix as two independent \(4 \times 9\) halves and now consider the rightmost three columns as one \(8 \times 3\) matrix for the second part. We use the barren rows 3 and 7 to eliminate columns 6 and 7 as follows: first, we normalize row 7.

$$\begin{aligned} {\mathbf {R}}_{7}= & {} \frac{1}{r_{76}} * {\mathbf {R}}_{7} \end{aligned}.$$

We eliminate column 6 of rows 0–6.

$$\begin{aligned} {\mathbf {R}}_{0} & = \,{\mathbf {R}}_{0} - r_{06} * {\mathbf {R}}_{7} \qquad {\mathbf {R}}_{1} = {\mathbf {R}}_{1} - r_{16} * {\mathbf {R}}_{7} \\ {\mathbf {R}}_{2}& =\, {\mathbf {R}}_{2} - r_{26} * {\mathbf {R}}_{7} \qquad {\mathbf {R}}_{3} = {\mathbf {R}}_{3} - r_{36} * {\mathbf {R}}_{7} \\ {\mathbf {R}}_{4}&=\, {\mathbf {R}}_{4} - r_{46} * {\mathbf {R}}_{7} \qquad {\mathbf {R}}_{5} = {\mathbf {R}}_{5} - r_{56} * {\mathbf {R}}_{7} \\ {\mathbf {R}}_{6 }&=\, {\mathbf {R}}_{6} - r_{66} * {\mathbf {R}}_{7} \end{aligned}.$$

We normalize row 3.

$$\begin{aligned} {\mathbf {R}}_{3}&= \frac{1}{r_{37}} * {\mathbf {R}}_{3} \end{aligned}.$$

We eliminate column 7 of rows 0–2 and 4–6.

$$\begin{aligned} {\mathbf {R}}_{0}& =\, {\mathbf {R}}_{0} - r_{07} * {\mathbf {R}}_{3} \qquad {\mathbf {R}}_{1} = {\mathbf {R}}_{1} - r_{17} * {\mathbf {R}}_{3} \\ {\mathbf {R}}_{2} &=\, {\mathbf {R}}_{2} - r_{27} * {\mathbf {R}}_{3} \qquad {\mathbf {R}}_{4} = {\mathbf {R}}_{4} - r_{47} * {\mathbf {R}}_{3} \\ {\mathbf {R}}_{5} &= \, {\mathbf {R}}_{5} - r_{57} * {\mathbf {R}}_{3} \qquad {\mathbf {R}}_{6} = {\mathbf {R}}_{6} - r_{67} * {\mathbf {R}}_{3} \\ \end{aligned}.$$

This leaves the last column of the matrix, which contains the homography, normalized to \(h_{22} = 1\):

$$\begin{aligned} \sim \left( \begin{array}{ccccccccc} 1 &{} &{} &{} &{} &{} &{} &{} &{} h_{00} \\ &{} \ddots &{} &{} &{} &{} &{} &{} &{} h_{01} \\ &{} &{} 1 &{} 0 &{} &{} &{} &{} &{} h_{02} \\ &{} &{} 0 &{} 0 &{} &{} &{} &{} 1 &{} h_{21} \\ &{} &{} 0 &{} 1 &{} &{} &{} &{} &{} h_{10} \\ &{} &{} &{} &{} \ddots &{} &{} &{} &{} h_{11} \\ &{} &{} &{} &{} &{} 1 &{} &{} &{} h_{12} \\ 0 &{} &{} &{} &{} &{} 0 &{} 1 &{} &{} h_{20} \end{array}\right) \rightarrow \left( \begin{array}{ccc} h_{00} &{} h_{01} &{} h_{02} \\ h_{10} &{} h_{11} &{} h_{12} \\ h_{20} &{} h_{21} &{} 1 \end{array} \right) \end{aligned}.$$

This concludes the second part.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bazargani, H., Bilaniuk, O. & Laganière, R. A fast and robust homography scheme for real-time planar target detection. J Real-Time Image Proc 15, 739–758 (2018). https://doi.org/10.1007/s11554-015-0508-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-015-0508-4

Keywords

Search

Navigation