Skip to main content
SpringerLink
Log in

Phase-correlation guided area matching for realtime vision and video encoding

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In computer vision and video encoding applications, one of the first and most important steps is to establish a pixel-to-pixel correspondence between two images of the same scene obtained at slightly different times or points of view. One of the most popular methods to find these correspondences, known as Area Matching, consists in performing a computationally intensive search for each pixel in the first image, around a neighborhood of the same pixel in the second image. In this work we propose a method which significantly reduces the search space to only a few candidates, and permits the implementation of real-time vision and video encoding algorithms which do not require specialized hardware such as GPU’s or FPGA’s. Theoretical and experimental support for this method is provided. Specifically, we present results from the application of the method to the realtime video compression and transmission, as well as the realtime estimation of dense optical flow and stereo disparity maps, where a basic implementation achieves up to 100 fps in a typical dual-core PC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aguilar-Ponce, R.M., Tecpanecatl-Xihuitl, J.L., Alba-Cadena, A., Arce-Santana, E.: Poc-guided area matching algoritm for video encoding. In: Proceedings of the 53rd IEEE International Midwest Symposium on Circuits and Systems, Seattle, Washington, pp. 503–508 (2010)

  2. Alba, A., Arce-Santana, E.: Phase-correlation guided search for realtime stereo vision. In: 13th International Workshop on Combinatorial Image Analysis. Lecture Notes in Computer Science, vol. 5852, pp. 212–223 (2009)

  3. Alba, A., Arce-Santana, E., Rivera, M.: Optical flow estimation with prior models obtained from phase correlation. In: 6th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 6453, pp. 417–426 (2010)

  4. Arce, E., Marroquin, J.L.: High-precision stereo disparity estimation using HMMF models. Image Vis. Comput. 25, 623–636 (2007)

    Article  Google Scholar 

  5. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Tech. Rep. MSR-TR-2009-179. Microsoft Research (2009)

  6. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)

    Article  Google Scholar 

  7. Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  8. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2005)

    Article  MathSciNet  Google Scholar 

  9. Chien, L.H., Aoki, T.: Robust motion estimation for video sequences based on phase-only correlation. In: 6th IASTED International Conference on Signal and Image Processing, pp. 441–446 (2004)

  10. Darabiha, A., MacLean, W.J., Rose, J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. Mach. Vis. Appl. 17(2), 116–132 (2006)

    Article  Google Scholar 

  11. Di Stefano, L., Marchionni, M., Mattoccia, S.: A fast area-based stereo matching algorithm. Image Vis. Comput. 22, 983–1005 (2004)

    Article  Google Scholar 

  12. Díaz, J., Ros, E., Mota, S., Carrillo, R., Agis, R.: Real time optical flow processing system. Field programmable logic and application. Lecture Notes in Computer Science, vol. 3203, pp. 617–626 (2004)

  13. Ernst, I., Hirschmüller, H.: Mutual information based semi-global stereo matching on the GPU. In: 4th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 5358, pp. 228–239 (2008)

  14. Essannouni, F., Haj Thami, R.O., Salam, A., Aboutajdine, D.: An efficient fast full search block matching algorithm using FFT algorithms. Int. J. Comput. Sci. Network Security 6(3B), 130–133 (2006)

    Google Scholar 

  15. Frigo, M., Johnson, S.G.: FFTW Web Page. http://www.fftw.org (2009)

  16. Gong, M.: Enforcing temporal consistency in real-time stereo estimation. In: 9th European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 3953, pp. 564–577 (2006)

  17. Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vis. 47(1-3), 229–246 (2002)

    Article  MATH  Google Scholar 

  18. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  19. Keller, Y., Averbuch, A., Moshe, I.: Pseudopolar-based estimation of large translations, rotations, and scalings in images. IEEE Trans. Image Process. 14(1), 12–22 (2005)

    Article  MathSciNet  Google Scholar 

  20. Kim, H., Min, D.B., Sohn, K.: Real-time stereo using foreground segmentation and hierarchical disparity estimation. In: 6th Pacific-Rim Conference on Multimedia. Lecture Notes in Computer Science, vol. 3767, pp. 384–395 (2005)

  21. Koga, T., Iinuma, K., Hirano, A., Iijima, Y.: Motion compensated interframe coding for video conferencing. NTC’81 Conference Record, pp. G5.3.1–G5.3.5 (1981)

  22. Kollnig, H., Nagel, H.H., Otte, M.: Association of motion verbs with vehicle movements extracted from dense optical flow fields. In: Proceedings of 3rd European Conference on Computer Vision (ECCV ’94). Lecture Notes in Computer Science, vol. 801/II, pp. 338–347 (1994)

  23. Kopp, M., Purgathofer, W.: Efficient 3 × 3 Median Filter Computations. Technical University, Vienna (1994)

    Google Scholar 

  24. Li, R., Zeng, B., Liou, M.L.: A new three-step search algorithm for block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 4(4), 438–442 (1994)

    Article  Google Scholar 

  25. López, S., Callico, G.M., López, J.F., Roberto, S.: A high quality/low computational cost technique for block matching motion estimation. In: Design, Automation and Test in Europe Conference and Exhibition (DATE 05), pp. 2–7 (2005)

  26. Lu, J., Liou, M.L.: A simple and efficient search algorithm for block matching motion estimation. IEEE Trans. Circuits Syst. Video Technol. 7(2), 429–433 (1997)

    Article  Google Scholar 

  27. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of Imaging Understanding Workshop, pp. 121–130 (1981)

  28. Marroquin, J.L., Mitter, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. J. Am. Stat. Assoc. 82, 76–89 (1987)

    Article  MATH  Google Scholar 

  29. Marzat, J., Dumortier, Y., Ducrot, A.: Real-time dense and accurate parallel optical flow using cuda. In: WSCG 2009 Proceedings, pp. 105–111 (2009)

  30. Nie, Y., Ma, K.K.: Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans. Image Process. 11(12), 1442–1448 (2002)

    Article  Google Scholar 

  31. Pauwels, K., Van Hulle, M.M.: Realtime phase-based optical flow on the GPU. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008

  32. Po, L., Ma, W.: A novel four-step search algorithm for fast block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 6(3), 313–317 (1996)

    Article  Google Scholar 

  33. Rannacher, J.: Realtime 3D motion estimation on graphics hardware. Bachelor thesis, Heidelberg University (2009)

  34. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)

    Article  MATH  Google Scholar 

  35. Shimizu, M., Okutomi, M.: Sub-pixel estimation error cancellation on area-based matching. Int. J. Comput. Vis. 63(3), 207–224 (2005)

    Article  Google Scholar 

  36. Takita, K., Muquit, M.A., Aoki, T., Higuchi, T.: A sub-pixel correspondence search technique for computer vision applications. IECIE Trans. Fundam. E 87-A(8), 1913–1923 (2004)

    Google Scholar 

  37. Toyama, K., Krumm, J., Brummit, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Seventh International Conference on Computer Vision, pp. 255–261 (1999)

  38. Veksler, O.: Reducing search space for stereo correspondence with graph cuts. In: British Machine Vision Conference, vol. 2, pp. 709–719 (2006)

  39. Wei, Z., Lee, D.J., Nelson, B., Martineau, M.: A fast and accurate tensor-based optical flow algorithm implemented in FPGA. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV’07) (2007)

  40. Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000)

    Article  MathSciNet  Google Scholar 

  41. Zinner, C., Humenberger, M., Ambrosch, K., Kubinger, W.: An optimized software-based implementation of a census-based stereo matching algorithm. In: 4th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 5358, pp. 216–227 (2008)

Download references

Acknowledgments

This work was supported by grant PROMEP/UASLP/10/CA06. A. Alba was partially supported by grant PROMEP/103.5/09/573.

Author information

Authors and Affiliations

  1. Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, Av. Salvador Nava Mtz. S/N, Zona Universitaria, 78290, San Luis Potosí, SLP, Mexico

    Alfonso Alba, Edgar Arce-Santana, Ruth M. Aguilar-Ponce & Daniel U. Campos-Delgado

Authors
  1. Alfonso Alba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edgar Arce-Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruth M. Aguilar-Ponce
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel U. Campos-Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alfonso Alba.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Alba, A., Arce-Santana, E., Aguilar-Ponce, R.M. et al. Phase-correlation guided area matching for realtime vision and video encoding. J Real-Time Image Proc 9, 621–633 (2014). https://doi.org/10.1007/s11554-012-0243-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-012-0243-z

Keywords

Search

Navigation