Skip to main content
SpringerLink
Log in

Corner Detection Using Multi-directional Structure Tensor with Multiple Scales

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Corners are important features for image analysis and computer vision tasks. Local structure tensors with multiple scales are widely used in intensity-based corner detectors. In this paper, the properties of intensity variations of a step edge, L-type corner, Y- or T-type corner, X-type corner, and star-type corner are investigated. The properties that we obtained indicate that the image intensity variations of a corner are not always large in all directions. The properties also demonstrate that existing structure tensor-based corner detection methods cannot depict the differences of intensity variations well between edges and corners which result in wrong corner detections. We present a new technique to extract the intensity variations from input images using anisotropic Gaussian directional derivative filters with multiple scales. We prove that the new extraction technique on image intensity variation has the ability to accurately depict the characteristics of edges and corners in the continuous domain. Furthermore, the properties of the intensity variations of step edges and corners enable us to derive a new multi-directional structure tensor with multiple scales, which has the ability to depict the intensity variation differences well between edges and corners in the discrete domain. The eigenvalues of the multi-directional structure tensor with multiple scales are used to develop a new corner detection method. Finally, the criteria on average repeatability (under affine image transformation, JPEG compression, and noise degradation), region repeatability based on the Oxford dataset, repeatability metric based on the DTU dataset, detection accuracy, and localization accuracy are used to evaluate the proposed detector against ten state-of-the-art methods. The experimental results show that our proposed detector outperforms all the other tested detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Aanæs, H., Dahl, A. L., & Pedersen, K. S. (2012). Interesting interest points. International Journal of Computer Vision, 97(1), 18–35.

    Google Scholar 

  • Alcantarilla, P., Bartoli, A., Davison, A. (2012). KAZE features. In European conference on computer vision (pp. 214–227). Springer.

  • Awrangjeb, M., & Lu, G. (2008). Robust image corner detection based on the chord-to-point distance accumulation technique. IEEE Transactions on Multimedia, 10(6), 1059–1072.

    Google Scholar 

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). SURF: Speeded up robust features. In European conference on computer vision (pp. 404–417). Springer.

  • Bowyer, K., Kranenburg, C., & Dougherty, S. (1999). Edge detector evaluation using empirical ROC curves. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 354–359).

  • Brox, T., Weickert, J., Burgeth, B., & Mrázek, P. (2006). Nonlinear structure tensors. Image and Vision Computing, 24(1), 41–55.

    Google Scholar 

  • Cornelis, N., & Van Gool, L. (2008). Fast scale invariant feature detection and matching on programmable graphics hardware. In Computer vision and pattern recognition workshops (pp. 1–8).

  • Deriche, R., & Giraudon, G. (1993). A computational approach for corner and vertex detection. International Journal of Computer Vision, 10(2), 101–124.

    Google Scholar 

  • DeTone, D., Malisiewicz, T., Rabinovich, A. (2018). Superpoint: Self-supervised interest point detection and description. In IEEE conference on computer vision and pattern recognition (pp. 224–236).

  • Duval-Poo, M. A., Odone, F., & De Vito, E. (2015). Edges and corners with shearlets. IEEE Transactions on Image Processing, 24(11), 3768–3780.

    MathSciNet  MATH  Google Scholar 

  • Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A-Optics Image Science and Vision, 4(12), 2379–2394.

    Google Scholar 

  • Gao, X., Sattar, F., & Venkateswarlu, R. (2007). Multiscale corner detection of gray level images based on LoG-Gabor wavelet transform. IEEE Transactions on Circuits and Systems for Video Technology, 17(7), 868–875.

    Google Scholar 

  • Gårding, J., & Lindeberg, T. (1996). Direct computation of shape cues using scale-adapted spatial derivative operators. International Journal of Computer Vision, 17(2), 163–191.

    Google Scholar 

  • Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Alvey vision conference (pp. 147–151).

  • Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Huang, F. C., Huang, S. Y., Ker, J. W., & Chen, Y. C. (2012). High-performance SIFT hardware accelerator for real-time image feature extraction. IEEE Transactions on Circuits and Systems for Video Technology, 22(3), 340–351.

    Google Scholar 

  • Kenney, C. S., Manjunath, B., Zuliani, M., Hewer, G. A., & Van Nevel, A. (2003). A condition number for point matching with application to registration and postregistration error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1437–1454.

    Google Scholar 

  • Koenderink, J. J. (1984). The structure of images. Biological Cybernetics, 50(5), 363–370.

    MathSciNet  MATH  Google Scholar 

  • Laptev, I. (2005). On space–time interest points. International Journal of Computer Vision, 64(2–3), 107–123.

    Google Scholar 

  • Lee, J. S., Sun, Y. N., & Chen, C. H. (1995). Multiscale corner detection by using wavelet transform. IEEE Transactions on Image Processing, 4(1), 100–104.

    Google Scholar 

  • Lenc, K., & Vedaldi, A. (2016). Learning covariant feature detectors. In European conference on computer vision (pp. 100–117). Springer.

  • Lepetit, V., & Fua, P. (2006). Keypoint recognition using randomized trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1465–1479.

    Google Scholar 

  • Lindeberg, T. (1998). Feature detection with automatic scale selection. International Journal of Computer Vision, 30(2), 79–116.

    Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Google Scholar 

  • Marimon, D., Bonnin, A., Adamek, T., & Gimeno, R. (2010). DARTs: Efficient scale-space extraction of DAISY keypoints. In IEEE conference on computer vision and pattern recognition (pp. 2416–2423).

  • Maver, J. (2010). Self-similarity and points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(7), 1211–1226.

    Google Scholar 

  • Miao, Z., & Jiang, X. (2013). Interest point detection using rank order LoG filter. Pattern Recognition, 46(11), 2890–2901.

    Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.

    Google Scholar 

  • Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., et al. (2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1–2), 43–72.

    Google Scholar 

  • Mokhtarian, F., & Suomela, R. (1998). Robust image corner detection through curvature scale space. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1376–1381.

    Google Scholar 

  • Moravec, H. P. (1979). Visual mapping by a robot rover. In Proceedings of the 6th international joint conference on artificial intelligence (Vol. 1, pp. 598–600).

  • Noble, J. A. (1988). Finding corners. Image and Vision Computing, 6(2), 121–128.

    Google Scholar 

  • Olson, C. F. (2000). Adaptive-scale filtering and feature detection using range data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9), 983–991.

    Google Scholar 

  • Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 629–639.

    Google Scholar 

  • Pham, T. A., Delalandre, M., Barrat, S., & Ramel, J. Y. (2014). Accurate junction detection and characterization in line-drawing images. Pattern Recognition, 47(1), 282–295.

    Google Scholar 

  • Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1), 81–106.

    Google Scholar 

  • Rattarangsi, A., & Chin, R. T. (1990). Scale-based detection of corners of planar curves. In 10th international conference on pattern recognition (Vol. 1, pp. 923–930).

  • Rosten, E., Porter, R., & Drummond, T. (2010). Faster and better: A machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 105–119.

    Google Scholar 

  • Ruzon, M. A., & Tomasi, C. (2001). Edge, junction, and corner detection using color distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1281–1295.

    Google Scholar 

  • Schmid, C., Mohr, R., & Bauckhage, C. (2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2), 151–172.

    MATH  Google Scholar 

  • Shui, P., & Zhang, W. (2012). Noise-robust edge detector combining isotropic and anisotropic Gaussian kernels. Pattern Recognition, 45(2), 806–820.

    MATH  Google Scholar 

  • Shui, P., & Zhang, W. (2013). Corner detection and classification using anisotropic directional derivative representations. IEEE Transactions on Image Processing, 22(8), 3204–3218.

    Google Scholar 

  • Smith, S. M., & Brady, J. M. (1997). SUSAN—A new approach to low level image processing. International Journal of Computer Vision, 23(1), 45–78.

    Google Scholar 

  • Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics, 26(3), 835–846.

    Google Scholar 

  • Su, R., Sun, C., & Pham, T. D. (2012). Junction detection for linear structures based on Hessian, correlation and shape information. Pattern Recognition, 45(10), 3695–3706.

    Google Scholar 

  • Teh, C. H., & Chin, R. T. (1989). On the detection of dominant points on digital curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(8), 859–872.

    Google Scholar 

  • Trujillo, L., Olague, G. (2006). Synthesis of interest point detectors through genetic programming. In Proceedings of the 8th annual conference on genetic and evolutionary computation (pp. 887–894).

  • Verdie, Y., Yi, V., Fua, P., & Lepetit, V. (2015). TILDE: A temporally invariant learned detector. In IEEE conference on computer vision and pattern recognition (pp. 5279–5288).

  • Wang, Y.-P. (1999). Image representations using multiscale differential operators. IEEE Transactions on Image Processing, 8(12), 1757–1771.

    MathSciNet  MATH  Google Scholar 

  • Weickert, J., Romeny, B. T. H., & Viergever, M. A. (1998). Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Transactions on Image Processing, 7(3), 398–410.

    Google Scholar 

  • Widynski, N., & Mignotte, M. (2014). A multiscale particle filter framework for contour detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10), 1922–1935.

    Google Scholar 

  • Wilson, K., & Snavely, N. (2014). Robust global translations with 1DSfM. In European conference on computer vision (pp. 61–75). Springer.

  • Witkin, A. (1984). Scale-space filtering: A new approach to multi-scale description. In IEEE international conference on acoustics, speech, and signal processing (Vol. 9, pp. 150–153).

  • Xia, G., Delon, J., & Gousseau, Y. (2014). Accurate junction detection and characterization in natural images. International Journal of Computer Vision, 106(1), 31–56.

    MathSciNet  MATH  Google Scholar 

  • Yi, K. M., Trulls, E., Lepetit, V., & Fua, P. (2016). LIFT: Learned invariant feature transform. In European conference on computer vision (pp. 467–483).

  • Zhang, X., Qu, Y., Yang, D., Wang, H., & Kymer, J. (2015). Laplacian scale-space behavior of planar curve corners. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(11), 2207–2217.

    Google Scholar 

  • Zhang, W., & Shui, P. (2015). Contour-based corner detection via angle difference of principal directions of anisotropic Gaussian directional derivatives. Pattern Recognition, 48(9), 2785–2797.

    Google Scholar 

  • Zhang, W., Sun, C., Breckon, T., & Alshammari, N. (2019). Discrete curvature representations for noise robust image corner detection. IEEE Transactions on Image Processing, 28(9), 4444–4459.

    MathSciNet  MATH  Google Scholar 

  • Zhang, W., Wang, F., Zhu, L., & Zhou, Z. (2014). Corner detection using Gabor filters. IET Image Processing, 8(11), 639–646.

    Google Scholar 

  • Zhang, W.-C., Zhao, Y., Breckon, T. P., & Chen, L. (2017). Noise robust image edge detection based upon the automatic anisotropic Gaussian kernels. Pattern Recognition, 63(2), 193–205.

    Google Scholar 

  • Zhang, X., Yu, F. X., Karaman, S., Chang, S.-F. (2017). Learning discriminative and transformation covariant local feature detectors. In IEEE conference on computer vision and pattern recognition (pp. 6818–6826).

  • Zhong, B., & Liao, W. (2007). Direct curvature scale space: Theory and corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 100–108.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61401347). We thank the anonymous reviewers for their detailed comments that substantially improved the paper.

Author information

Authors and Affiliations

  1. College of Electrical and Information, Xi’an Polytechnic University, Xi’an, 710048, China

    Weichuan Zhang

  2. CSIRO Data61, PO Box 76, Epping, NSW, 1710, Australia

    Changming Sun

Authors
  1. Weichuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changming Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weichuan Zhang.

Additional information

Communicated by D. Scharstein.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Sun, C. Corner Detection Using Multi-directional Structure Tensor with Multiple Scales. Int J Comput Vis 128, 438–459 (2020). https://doi.org/10.1007/s11263-019-01257-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-019-01257-2

Keywords

Search

Navigation