Pravin Bhat
Abstract
We present an optimization framework for exploring gradient-domain solutions for image and video processing. The proposed framework unifies many of the key ideas in the gradient-domain literature under a single optimization formulation. Our hope is that this generalized framework will allow the reader to quickly gain a general understanding of the field and contribute new ideas of their own.
We propose a novel metric for measuring local gradient saliency that identifies salient gradients that give rise to long, coherent edges, even when the individual gradients are faint. We present a general weighting scheme for gradient constraints that improves the visual appearance of results. We also provide a solution for applying gradient-domain filters to videos and video streams in a coherent manner.
Finally, we demonstrate the utility of our formulation in creating effective yet simple to implement solutions for various image-processing tasks. To exercise our formulation we have created a new saliency-based sharpen filter and a pseudo image-relighting application. We also revisit and improve upon previously defined filters such as nonphotorealistic rendering, image deblocking, and sparse data interpolation over images (e.g., colorization using optimization).
Supplemental Material
- Agrawal, A. and Raskar, R., 2007. Gradient domain manipulation techniques vision and graphics. ICCV 2007 Courses.Google Scholar
- Agrawal, A., Raskar, R., Nayar, S., and Li, Y., 2005. Removing photography artifacts using gradient projection and flash exposure sampling. ACM Trans. Graph. Google ScholarDigital Library
- Agrawal, A., Raskar, R., and Chellappa, R. 2006. Edge suppression by gradient field transformation using cross-projection tensors. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'06). 2301--2308. Google ScholarDigital Library
- Agrawal, A., Raskar, R., and Chellappa, R. 2006. What is the range of surface reconstructions from a gradient field. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 578--591. Google ScholarDigital Library
- Attneave, F. 1954. Some informational aspects of visual perception. Psychol. Rev. 61, 3, 183--193.Google ScholarCross Ref
- Averbuch, A., Schclar, A., and Donoho, D. 2005. Deblocking of block-transform compressed images using weighted sums of symmetrically aligned pixels. IEEE Trans. Image Process. 14, 2, 200--212. Google ScholarDigital Library
- Barten, P. G. 1999. Contrast Sensitivity of the Human Eye and Its Effects on Image Quality. International Society for Optical Engineering.Google Scholar
- Beaudot, W., and Mullen, K. 2003. How long range is contour integration in human color vision? In Visual Neuroscience, vol. 15, 51--64.Google Scholar
- Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Proceedings of the Eurographics Symposium on Rendering Techniques. Eurographics, 327--338. Google ScholarDigital Library
- Bhat, P., Curless, B., Cohen, M., and Zitnick, L. 2008. Fourier Analysis of the 2D screened poisson equation for gradient domain problems. In Proceedings of the European Conference on Computer Vision (ECCV'08). Google ScholarDigital Library
- Bhat, P., Curless, B., Cohen, M., and Zitnick, L. 2008. Gradientshop: Gradient-domain image and video processing. http://www.GradientShop.com.Google Scholar
- Black, M. J., Sapiro, G., Marimont, D. and Heeger, D., 1998. Robust anisotropic diffusion. IEEE Trans. Image Process. Google ScholarDigital Library
- Castagno, R. and Ramponi, G. 1996. A rational filter for the removal of blocking artifacts in image sequences coded at low bitrate. In Proceedings of the European Signal Processing Conference (EUSIPC).Google Scholar
- Drori, I., Leyvand, T., Fleishman, S., Cohen-Or, D., and Yeshurun., H. 2004. Video operations in the gradient domain. Tech. rep., Tel-Aviv University.Google Scholar
- Elder, J. H., and Goldberg, R. M. 2001. Image editing in the contour domain. IEEE Trans. Pattern Anal. Mach. Intell. 23, 3, 291--296. Google ScholarDigital Library
- Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'02). ACM Press, New York, 249--256. Google ScholarDigital Library
- Freeman, W. T., and Adelson, E. H. 1991. The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13, 9, 891--906. Google ScholarDigital Library
- Georgiev, T. 2006. Covariant derivatives and vision. In Proceedings of the 9th European Conference on Computer Vision (ECCV'06). Google ScholarDigital Library
- Gooch, A. A., Olsen, S. C., Tumblin, J., and Gooch, B. 2005. Color2gray: Salience-preserving color removal. ACM Trans. Graph. 24, 3, 634--639. Google ScholarDigital Library
- Hong, S., Chan, Y., and Siu, W. 1996. A practical real-time post-processing technique for block effect elimination. In Proceedings of the IEEE International Conference on Image Processing (ICIP'96). II: 21--24.Google Scholar
- Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarDigital Library
- Kim, Tae-Hoon, Ahn, Jongwoo, Choi, and Gyu, M. 2007. Image dequantization: Restoration of quantized colors. Comput. Graph. Forum 26, 3, 619--626.Google ScholarCross Ref
- Levin, A., Zomet, A., Peleg, S., and Weiss, Y. 2004. Seamless image stitching in the gradient domain. In Hebrew University Tech. rep. 2003-82.Google Scholar
- Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. In ACM SIGGRAPH Papers (SIGGRAPH'04). ACM Press, New York, 689--694. Google ScholarDigital Library
- Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. In ACM SIGGRAPH Papers (SIGGRAPH'06). ACM Press, New York, 646--653. Google ScholarDigital Library
- Marschner, S. R. and Greenberg, D. P. 1997. Inverse lighting for photography. In Proceedings of the 5th Color Imaging Conference. Society for Imaging Science and Technology.Google Scholar
- McCann, J. and Pollard, N. S. 2008. Real-time gradient-domain painting. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
- Orzan, A., Bousseau, A., Barla, P., and Thollot, J. 2007. Structure-preserving manipulation of photographs. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR). Google ScholarDigital Library
- Paris, S. 2008. Edge-preserving smoothing and mean-shift segmentation of video streams. In Proceedings of the 10th European Conference on Computer Vision (ECCV'08). Springer, 460--473. Google ScholarDigital Library
- Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. In ACM SIGGRAPH Papers (SIGGRAPH'03). ACM Press, New York, 313--318. Google ScholarDigital Library
- Rempel, A. G., Trentacoste, M., Seetzen, H., Young, H. D., Heidrich, W., Whitehead, L., and Ward, G. 2007. Ldr2hdr: On-the-Fly reverse tone mapping of legacy video and photographs. ACM Trans. Graph. 26, 3, 39. Google ScholarDigital Library
- Sand, P., and Teller, S. 2006. Particle video: Long-range motion estimation using point trajectories.In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). IEEE Computer Society, 2195--2202. Google ScholarDigital Library
- Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf.Google Scholar
- Szeliski, R. 2006. Locally adapted hierarchical basis preconditioning. In ACM SIGGRAPH Papers (SIGGRAPH'06). ACM Press, New York, 1135--1143. Google ScholarDigital Library
- Tomar, S. 2006. Converting video formats with ffmpeg. Linux J. 146, 10. Google ScholarDigital Library
- Wang, H., Raskar, R., and Ahuja, N. 2004. Seamless video editing. In Proceedings of the 17th International Conference on Pattern Recognition, (ICPR'04) vol. 3, IEEE Computer Society, 858--861. Google ScholarDigital Library
- Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Real-time video abstraction. ACM Trans. Graph. 25, 3, 1221--1226. Google ScholarDigital Library
- Zeng, Y., Chen, W., and Peng, Q. 2006. A novel variational image model: Towards a unified approach to image editing. J. Comput. Sci. Technol.Google ScholarCross Ref
Index Terms
- GradientShop: A gradient-domain optimization framework for image and video filtering
Recommendations
Interactive relighting with dynamic BRDFs
We present a technique for interactive relighting in which source radiance, viewing direction, and BRDFs can all be changed on the fly. In handling dynamic BRDFs, our method efficiently accounts for the effects of BRDF modification on the reflectance ...
Interactive relighting with dynamic BRDFs
SIGGRAPH '07: ACM SIGGRAPH 2007 papersWe present a technique for interactive relighting in which source radiance, viewing direction, and BRDFs can all be changed on the fly. In handling dynamic BRDFs, our method efficiently accounts for the effects of BRDF modification on the reflectance ...
Relighting with 4D incident light fields
SIGGRAPH '03: ACM SIGGRAPH 2003 PapersWe present an image-based technique to relight real objects illuminated by a 4D incident light field, representing the illumination of an environment. By exploiting the richness in angular and spatial variation of the light field, objects can be relit ...
Comments