Abstract
Optical flow estimation is still an important task in computer vision with many interesting applications. However, the results obtained by most of the optical flow techniques are affected by motion discontinuities or illumination changes. In this paper, we introduce a brightness correction field combined with a gradient constancy constraint to reduce the sensibility to brightness changes between images to be estimated. The advantage of this brightness correction field is its simplicity in terms of computational complexity and implementation. By analyzing the deficiencies of the traditional total variation regularization term in weakly textured areas, we also adopt a structure-adaptive regularization based on the robust Huber norm to preserve motion discontinuities. Finally, the proposed energy functional is minimized by solving its corresponding Euler-Lagrange equation in a more effective multi-resolution scheme, which integrates the twice downsampling strategy with a support-weight median filter. Numerous experiments show that our method is more effective and produces more accurate results for optical flow estimation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, S., Roth, S., Scharstein, D., Black, M., Lewis, J., Szeliski, R., 2007. A Database and Evaluation Methodology for Optical Flow. IEEE 11th Int. Conf. on Computer Vision, p.1–8. [doi:10.1109/ICCV.2007.4408903]
Barron, J., Fleet, D., Beauchemin, S., 1994. Performance of optical flow techniques. Int. J. Comput. Vis., 12(1): 43–77. [doi:10.1007/BF01420984]
Black, M.J., Anandan, P., 1996. The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Understand., 63(1):75–104. [doi:10.1006/cviu.1996.0006]
Brox, T., Malik, J., 2011. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):500–513. [doi:10.1109/TPAMI.2010.143]
Brox, T., Bruhn, A., Papenberg, N., Weickert, J., 2004. High Accuracy Optical Flow Estimation Based on a Theory for Warping. European Conf. on Computer Vision, p.25–36.
Bruhn, A., Weickert, J., Schnörr, C., 2005. Lucas/Kanade meets Horn/Schunk: combining local and global optical flow methods. Int. J. Comput. Vis., 61(3):211–231. [doi:10.1023/B:VISI.0000045324.43199.43]
Cassisa, C., Simoens, S., Prinet, V., 2009. Two-frame optical flow formulation in an unwarped multiresolution scheme. LNCS, 5856:790–797. [doi:10.1007/978-3-642-10268-4_93]
Dessauer, M.P., Dua, S., 2010. Optical flow object detection, motion estimation, and tracking on moving vehicles using wavelet decompositions. SPIE, 7694:76941J. [doi:10.1117/12.853281]
Efros, A., Berg, A., Mori, G., Malik, J., 2003. Recognizing Action at a Distance. Proc. 9th IEEE Int. Conf. on Computer Vision, p.726–733. [doi:10.1109/ICCV.2003.1238420]
Fakih, A., Zelek, J., 2008. Structure from Motion: Combining Features Correspondences and Optical Flow. 19th Int. Conf. on Pattern Recognition, p.1–4. [doi:10.1109/ICPR.2008.4761007]
Gennert, M.A., Negahdaripour, S., 1987. Relaxing the Brightness Constancy Assumption in Computing Optical Flow. Technical Report, Massachusetts Institute of Technology, Cambridge, MA, USA.
Gilland, D.R., Mair, B.A., Parker, J.G., 2008. Motion estimation for cardiac emission tomography by optical flow methods. Phys. Med. Biol., 53(11):2991–3006. [doi:10.1088/0031-9155/53/11/016]
Gray, R.M., 2006. Toeplitz and circulant matrices: a review foundations and trends. Commun. Inform. Theory, 2(3):155–239. [doi:10.1561/0100000006]
Haussecker, H.W., Fleet, D.J., 2001. Computing optical flow with physical models of brightness variation. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):661–673. [doi:10.1109/34.927465]
Horn, B., Schunck, B., 1981. Determining optical flow. Artif. Intell., 17(1–3):185–203. [doi:10.1016/0004-3702(81)90024-2]
Hsiao, I., Rangarajan, A., Gindi, G., 2003. A new convex edge-preserving median prior with applications to tomography. IEEE Trans. Med. Imag., 22(5):580–585. [doi:10.1109/TMI.2003.812249]
Huber, P.J., 1973. Robust regression: asymptotics, conjectures and Monte Carlo. Ann. Stat., 1(5):799–821. [doi:10.1214/aos/1176342503]
Kim, Y.H., Martinez, A.M., Kak, C.A., 2005. Robust motion estimation under varying illumination. Image Vis. Comput., 23(4):365–375. [doi:10.1016/j.imavis.2004.05.010]
Lempitsky, V., Roth, S., Rother, C., 2008. FusionFlow: Discrete Continuous Optimization for Optical Flow Estimation. IEEE Conf. on Computer Vision and Pattern Recognition, p.1–8. [doi:10.1109/CVPR.2008.4587751]
Li, Y., Osher, S., 2009. A new median formula with applications to PDE based denoising. Commun. Math. Sci., 7(3):741–753.
Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T., 2008. SIFT flow: dense correspondence across different scenes. LNCS, 5304:28–42. [doi:10.1007/978-3-540-88690-7_3]
Myronenko, A., Song, X., 2009. Image Registration by Minimization of Residual Complexity. IEEE Conf. on Computer Vision and Pattern Recognition, p.49–56. [doi:10.1109/CVPR.2009.5206571]
Negahdaripour, S., 1998. Revised definition of optical flow: integration of radiometric and geometric cues for dynamic scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(9):961–979. [doi:10.1109/34.713362]
Sand, P., Teller, S., 2008. Particle video: long-range motion estimation using point trajectories. Int. J. Comput. Vis., 80(1):72–91. [doi:10.1007/s11263-008-0136-6]
Shulman, D., Herve, J.Y., 1989. Regularization of Discontinuous Flow Fields. Proc. Workshop on Visual Motion, p.81–86. [doi:10.1109/WVM.1989.47097]
Steinbrücker, F., Pock., T., 2009. Large Displacement Optical Flow Computation without Warping. Int. Conf. on Computer Vision, p.1609–1614.
Strang, G., 1999. The discrete cosine transform. SIAM Rev., 41(1):135–147. [doi:10.1137/S0036144598336745]
Sun, D., Roth, S., Black, M.J., 2010. Secrets of Optical Flow Estimation and Their Principles. IEEE Conf. on Computer Vision and Pattern Recognition, pp.2432–2439. [doi:10.1109/CVPR.2010.5539939]
Teng, C.H., Lai, S.H., Chen, Y.S., Hsu, W.H., 2005. Accurate optical flow computation under non-uniform brightness variations. Comput. Vis. Image Understand., 97(3):315–346. [doi:10.1016/j.cviu.2004.08.002]
Wang, M.Y., Hu, H.B., Qin, B.J., 2007. Robust Deformable Medical Image Registration Using Optical Flow and Multilevel Free Form Deformation. Nuclear Science Symp. Conf. Record, p.4552–4555.
Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D., 2009a. An improved algorithm for TV-L 1 optical flow. LNCS, 5604:23–45. [doi:10.1007/978-3-642-03061-1_2]
Wedel, A., Cremers, D., Pock, T., Bischof, H., 2009b. Structure- and Motion-Adaptive Regularization for High Accuracy Optic Flow. IEEE 12th Int. Conf. on Computer Vision, p.1663–1668. [doi:10.1109/ICCV.2009.5459375]
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H., 2009. Anisotropic Huber-L 1 Optical Flow. British Machine Vision Conf., p.1–11.
Yoon, K.J., Kweon, I.S., 2006. Adaptive supportweight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell., 28(4):650–656. [doi:10.1109/TPAMI.2006.70]
Zach, C., Pock, T., Bischof, H., 2007. A duality based approach for realtime TV-L 1 optical flow. LNCS, 4713:214–223. [doi:10.1007/978-3-540-74936-3_22]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Natural Science Foundation of China (No. U0935004) and an IDeA Network of Biomedical Research Excellence (INBRE) grant from the National Institutes of Health (NIH) (No. 5P20RR01647206)
Rights and permissions
About this article
Cite this article
Wang, W., Su, Zx., Pan, Js. et al. Robust optical flow estimation based on brightness correction fields. J. Zhejiang Univ. - Sci. C 12, 1010–1020 (2011). https://doi.org/10.1631/jzus.C1100062
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1100062