Skip to main content
SpringerLink
Log in

Morphologically Invariant Matching of Structures with the Complete Rank Transform

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Invariances are one of the key concepts to render computer vision algorithms robust against severe illumination changes. However, there is no free lunch: With any invariance comes an unavoidable loss of information. The goal of our paper is to introduce two novel descriptors which minimise this loss: the complete rank transform and the complete census transform. They are invariant under monotonically increasing intensity rescalings, while containing a maximum possible amount of information. To analyse our descriptors, we embed them as constancy assumptions into a variational framework for optic flow computation. As a suitable regularisation term, we choose total generalised variation that favours piecewise affine solutions. Our experiments focus on the KITTI benchmark where robustness w.r.t. illumination changes is one of the main issues. The results demonstrate that our descriptors yield state-of-the-art accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://www.dagm.de/symposien/special-sessions/.

  2. To save space, we omit the full ranking table which is available at http://vision.middlebury.edu/flow/.

References

  • Alvarez, L., Guichard, F., Lions, P. L., & Morel, J. M. (1993). Axioms and fundamental equations in image processing. Archive for Rational Mechanics and Analysis, 123, 199–257.

    Article  MATH  MathSciNet  Google Scholar 

  • Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011). A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92(1), 1–31.

    Article  Google Scholar 

  • Bhat, D. N., & Nayar, S. K. (1998). Ordinal measures for image correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(4), 415–423.

    Article  Google Scholar 

  • Braux-Zin, J., Dupont, R., & Bartoli, A.. (2013). A general dense image matching framework combining direct and feature-based costs. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney (pp 185–192).

  • Bredies, K., Kunisch, K., & Pock, T. (2010). Total generalized variation. SIAM Journal on Imaging Sciences, 3(3), 492–526.

    Article  MATH  MathSciNet  Google Scholar 

  • Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In T. Pajdla & J. Matas (Eds.), Computer vision—ECCV 2004, Part IV (Vol. 3024, pp. 25–36)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Bruhn, A., & Weickert, J. (2005). Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: Proceedings of the IEEE international conference on computer vision (ICCV), Beijing (vol. 1, pp. 749–755).

  • Calonder, M., Lepetit, V., Ozuysal, M., Trzcinski, T., Strecha, C., & Fua, P. (2012). BRIEF: Computing a local binary descriptor very fast. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1281–1298.

    Article  Google Scholar 

  • Chan, C. H., Goswami, B., Kittler, J., & Christmas, W. (2012). Local ordinal contrast pattern histograms for spatiotemporal, lip-based speaker authentication. IEEE Transactions on Information Forensics and Security, 7(2), 602–612.

    Article  Google Scholar 

  • Charbonnier, P., Blanc-Féraud, L., Aubert, G., & Barlaud, M. (1994). Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of the IEEE international conference on image processing, IEEE Computer Society Press, Austin, TX (vol. 2, pp. 168–172).

  • Chen, J., Kellokumpu, V. P., Zhao, G., & Pietikinen, M. (2013). RLBP: Robust local binary pattern. In T. Burghardt, D. Damen, W. Mayol-Cuevas, & M. Mirmehdi (Eds.), Proceedings of the British machine vision conference. Bristol: BMVA.

    Google Scholar 

  • Demetz, O., Hafner, D., & Weickert, J. (2013). The complete rank transform: A tool for accurate and morphologically invariant matching of structures. In T. Burghardt, D. Damen, W. Mayol-Cuevas, & M. Mirmehdi (Eds.), Proceedings of the British machine vision conference. Bristol: BMVA.

    Google Scholar 

  • Demetz, O., Stoll, M., Voltz, S., Weickert, J., & Bruhn, A. (2014). Learning brightness transfer functions for the joint recovery of illumination changes and optical flow. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer vision—ECCV 2014 (Vol. 8689, pp. 455–471). Switzerland: Springer.

    Google Scholar 

  • Fröba, B., & Ernst, A. (2004). Face detection with the modified census transform. In: Proceedings of the IEEE international conference on automatic face and gesture recognition (FGR), Seoul, pp. 91–96.

  • Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, pp. 3354–3361.

  • Gelfand, I. M., & Fomin, S. V. (2000). Calculus of variations. New York: Dover.

    MATH  Google Scholar 

  • Gennert, M. A., & Negahdaripour, S. (1987). Relaxing the brightness constancy assumption in computing optical flow. Technical Report 975, Artificial Intelligence Laboratory, Massachusetts Institiute of Technology.

  • Grewenig, S., Weickert, J., Schroers, C., & Bruhn, A. (2013). Cyclic schemes for PDE-based image analysis. Technical Report 327, Department of Mathematics, Saarland University, Saarbrücken.

  • Hafner, D., Demetz, O., & Weickert, J. (2013). Why is the census transform good for robust optic flow computation? In A. Kuijper, T. Pock, K. Bredies, & H. Bischof (Eds.), Scale-space and variational methods in computer vision (Vol. 7893, pp. 210–221)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Hermann, S., & Klette, R. (2013). Hierarchical scan-line dynamic programming for optical flow using semi-global matching. In J. I. Park & J. Kim (Eds.), Computer vision—ACCV 2012 workshops (Vol. 7729, pp. 556–567)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Hewer, A., Weickert, J., Scheffer, T., Seibert, H., & Diebels, S. (2013). Lagrangian strain tensor computation with higher order variational models. In T. Burghardt, D. Damen, W. Mayol-Cuevas, & M. Mirmehdi (Eds.), Proceedings of the British machine vision conference. Bristol: BMVA.

    Google Scholar 

  • Horn, B., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–203.

    Article  Google Scholar 

  • Kim, T. H., Lee, H. S., & Lee, K. M. (2013). Optical flow via locally adaptive fusion of complementary data costs. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney, pp. 3344–3351.

  • Liu, C., Yuen, J., & Torralba, A. (2011). SIFT flow: Dense correspondence across scenes and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), 978–994.

    Article  Google Scholar 

  • Lowe, D. L. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., & Zhang, X. (2011). On building an accurate stereo matching system on graphics hardware. In: Proceedings of the IEEE international conference on computer vision workshops (ICCV workshops), Barcelona, pp. 467–474.

  • Mileva, Y., Bruhn, A., & Weickert, J. (2007). Illumination-robust variational optical flow with photometric invariants. In F. A. Hamprecht, C. Schnör, & B. Jähne (Eds.), Pattern recognition., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Mittal, A., & Ramesh, V. (2006). An intensity-augmented ordinal measure for visual correspondence. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), New York, NY (vol 1, pp. 849–856).

  • Mohamed, M., Rashwan, H., Mertsching, B., Garcia, M., & Puig, D. (2014). Illumination-robust optical flow using a local directional pattern. IEEE Transactions on Circuits and Systems for Video Technology, 24(9), 1499–1508.

    Article  Google Scholar 

  • Müller, T., Rabe, C., Rannacher, J., Franke, U., & Mester, R. (2011). Illumination robust dense optical flow using census signatures. In R. Mester & M. Felsberg (Eds.), Pattern recognition., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Otte, M., & Nagel, H. H. (1994). Optical flow estimation: Advances and comparisons. In J. O. Eklundh (Ed.), Computer vision—ECCV ’94 (Vol. 800, pp. 49–60)., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.

    Article  Google Scholar 

  • Pietikäinen, M., Hadid, A., Zhao, G., & Ahonen, T. (2011). Computer vision using local binary patterns. London: Springer.

    Book  Google Scholar 

  • Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007). Numerical recipes: The art of scientific computing (3rd ed.). New York, NY: Cambridge University Press.

    Google Scholar 

  • Puxbaum, P., & Ambrosch, K. (2010). Gradient-based modified census transform for optical flow. In G. Bebis, R. D. Boyle, B. Parvin, D. Koracin, R. Chung, R. I. Hammoud, M. Hussain, K. H. Tan, R. Crawfis, D. Thalmann, D. Kao, & L. Avila (Eds.), Advances in visual computing, Part I (Vol. 6453)., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Ranftl, R., Gehrig, S., Pock, T., & Bischof, H. (2012). Pushing the limits of stereo using variational stereo estimation. In: Proc. IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain, pp. 401–407.

  • Ranftl, R., Bredies, K., & Pock, T. (2014). Non-local total generalized variation for optical flow estimation. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer vision ECCV 2014 (Vol. 8689, pp. 439–454)., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Rashwan, H., Mohamed, M., Garcia, M., Mertsching, B., & Puig, D. (2013). Illumination robust optical flow model based on histogram of oriented gradients. In J. Weickert, M. Hein, & B. Schiele (Eds.), Pattern recognition (Vol. 8142, pp. 354–363)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Sloane, N. J. A., & Plouffe, S. (1995). The encyclopedia of integer sequences. San Diego, CA: Academic Press.

    MATH  Google Scholar 

  • Soatto, S. (2009). Actionable information in vision. In: Proceedings of the IEEE international conference on computer vision, IEEE Computer Society Press (pp. 2138–2145).

  • Stein, F. (2004). Efficient computation of optical flow using the census transform. In C. E. Rasmussen, H. H. Bülthoff, B. Schölkopf, & M. A. Giese (Eds.), Pattern recognition (Vol. 3175, pp. 79–86)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Steinbrücker, F., Pock, T., & Cremers, D. (2009). Advanced data terms for variational optic flow estimation. In: M. A. Magnor, B. Rosenhahn, H. Theisel (Eds.) Proceedings of the vision, modeling, and visualization workshop (VMV), DNB (pp. 155–164).

  • Sun, D., Roth, S., & Black, M. (2014). A quantitative analysis of current practices in optical flow estimation and the principles behind them. International Journal of Computer Vision, 106(2), 115–137.

    Article  Google Scholar 

  • Tang, F., Lim, S. H., Chang, N. L., & Tao, H. (2009). A novel feature descriptor invariant to complex brightness changes. In: Proceedings of the IEEE Computer Society conference on computer vision and pattern recognition (CVPR), Miami, FL (pp. 2631–2638).

  • Tukey, J. W. (1971). Exploratory data analysis. Menlo Park, CA: Addison–Wesley.

    Google Scholar 

  • Uras, S., Girosi, F., Verri, A., & Torre, V. (1988). A computational approach to motion perception. Biological Cybernetics, 60, 79–87.

    Article  Google Scholar 

  • van de Weijer, J., & Gevers, T. (2004). Robust optical flow from photometric invariants. In: Proceedings of the IEEE international conference on image processing (ICIP), Singapore (pp. 1835–1838).

  • Vogel, C., Roth, S., & Schindler, K. (2013). An evaluation of data costs for optical flow. In J. Weickert, M. Hein, & B. Schiele (Eds.), Pattern recognition (Vol. 8142, pp. 343–353)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Vogel, C. R., & Oman, M. E. (1996). Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 17(1), 227–238.

  • Wang, Z., Fan, B., & Wu, F. (2011). Local intensity order pattern for feature description. In: Proceedings of the IEEE International conference on computer vision (ICCV), Barcelona (pp. 603–610).

  • Wedel, A., Pock, T., Zach, C., Cremers, D., & Bischof, H. (2008). An improved algorithm for TV-L1 optical flow. In D. Cremers, B. Rosenhahn, A. L. Yuille, & F. R. Schmidt (Eds.), Statistical and geometrical approaches to visual motion analysis (Vol. 5604)., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Wei, D., Liu, C., & Freeman, W. T. (2014). A data-driven regularization model for stereo and flow. In: Proceedings of the IEEE international conference on 3D vision, Tokyo.

  • Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013). Deepflow: Large displacement optical flow with deep matching. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney (pp. 1385–1392).

  • Werlberger, M., Pock, T., & Bischof, H. (2010). Motion estimation with non-local total variation regularization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Francisco, CA (pp. 2464–2471).

  • Xu, L., Jia, J., & Matsushita, Y. (2010). Motion detail preserving optical flow estimation. In: Proceedings IEEE conference on computer vision and pattern recognition (CVPR), IEEE Computer Society Press (pp. 1293–1300).

  • Zabih, R., & Woodfill, J. (1994). Non-parametric local transforms for computing visual correspondence. In J. O. Eklundh (Ed.), Computer vision—ECCV ’94, Part II (Vol. 801, pp. 151–158)., Lecture Notes in Computer Science Berlin: Springer.

    Google Scholar 

  • Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., et al. (2009). Complementary optic flow. In D. Cremers, Y. Boykov, A. Blake, & F. R. Schmidt (Eds.), Energy minimization methods in computer vision and pattern recognition (Vol. 5681, pp. 207–220)., Lecture Notes in Computer Science Berlin: Springer.

    Chapter  Google Scholar 

  • Zimmer, H., Bruhn, A., & Weickert, J. (2011). Optic flow in harmony. International Journal of Computer Vision, 3(93), 368–388.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

Our research is partly funded by the Cluster of Excellence Multimodal Computing and Interaction within the Excellence Initiative of the German Federal Government, and by the Deutsche Forschungsgemeinschaft through a Gottfried Wilhelm Leibniz Prize for Joachim Weickert. This is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Mathematical Image Analysis Group, Saarland University, Saarbrücken, Germany

    Oliver Demetz, David Hafner & Joachim Weickert

Authors
  1. Oliver Demetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Hafner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver Demetz.

Additional information

Communicated by Michael Valstar, Andrew French, and Tony Pridmore.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Demetz, O., Hafner, D. & Weickert, J. Morphologically Invariant Matching of Structures with the Complete Rank Transform. Int J Comput Vis 113, 220–232 (2015). https://doi.org/10.1007/s11263-015-0800-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0800-6

Keywords

Search

Navigation