Iddo Drori
Abstract
We present a new method for completing missing parts caused by the removal of foreground or background elements from an image. Our goal is to synthesize a complete, visually plausible and coherent image. The visible parts of the image serve as a training set to infer the unknown parts. Our method iteratively approximates the unknown regions and composites adaptive image fragments into the image. Values of an inverse matte are used to compute a confidence map and a level set that direct an incremental traversal within the unknown area from high to low confidence. In each step, guided by a fast smooth approximation, an image fragment is selected from the most similar and frequent examples. As the selected fragments are composited, their likelihood increases along with the mean confidence of the image, until reaching a complete image. We demonstrate our method by completion of photographs and paintings.
Supplemental Material
- ASHIKHMIN, M. 2001. Synthesizing natural textures. In ACM Symposium on Interactive 3D Graphics, 217--226. Google Scholar
Digital Library
- BAKER, S., AND KANADE, T. 2000. Limits on super-resolution and how to break them. In IEEE Conference on Computer Vision and Pattern Recognition, 372--379.Google ScholarCross Ref
- BERTALMIO, M., SAPIRO, G., CASELLES, V., AND BALLESTER, C. 2000. Image inpainting. In Proceedings of ACM SIGGRAPH 2000, ACM Press, 417--424. Google ScholarDigital Library
- BERTALMIO, M., VESE, L., SAPIRO, G., AND OSHER, S. 2003. Simultaneous structure and texture image inpainting. In IEEE Conference on Computer Vision and Pattern Recognition, to appear.Google ScholarCross Ref
- BORENSTEIN, E., AND ULLMAN, S. 2002. Class-specific, top-down segmentation. In European Conference on Computer Vision, 109--124. Google ScholarDigital Library
- BROOKS, S., AND DODGSON, N. 2002. Self-similarity based texture editing. ACM Transactions on Graphics, 21, 3, 653--656. Google ScholarDigital Library
- BURT, P. J., AND ADELSON, E. H. 1985. Merging images through pattern decomposition. Applications of Digital Image Processing VIII 575, 173--181.Google Scholar
- CHAN, T., AND SHEN, J. 2001. Mathematical models for local nontexture inpainting. SIAM Journal on Applied Mathematics 62, 3, 1019--1043.Google Scholar
- CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics, 21, 3, 243--248. Google ScholarDigital Library
- EFROS, A. A., AND FREEMAN, W. T. 2001. Image quilting for texture synthesis and transfer. In Proceedings of ACM SIGGRAPH 2001, ACM Press, 341--346. Google ScholarDigital Library
- EFROS, A., AND LEUNG, T. 1999. Texture synthesis by non-parametric sampling. In IEEE International Conference on Computer Vision, 1033--1038. Google ScholarDigital Library
- FREEMAN, W. T., PASZTOR, E. C., AND CARMICHAEL, O. T. 2000. Learning low-level vision. International Journal of Computer Vision 40, 1, 25--47. Google ScholarDigital Library
- FREEMAN, W. T., JONES, T. R., AND PASZTOR, E. 2002. Example-based super-resolution. IEEE Computer Graphics and Applications, 56--65. Google ScholarDigital Library
- GONZALEZ, R. C., AND WOODS, R. E. 2002. Digital Image Processing. Prentice Hall. Google ScholarDigital Library
- GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. 1996. The lumigraph. In Proceedings of ACM SIGGRAPH 96, ACM Press, 43--54. Google ScholarDigital Library
- GUY, G., AND MEDIONI, G. 1996. Inferring global perceptual contours from local features. IEEE International Journal of Computer Vision, 1--2, 113--133. Google ScholarDigital Library
- HAEBERLI, P. 1990. Paint by numbers: Abstract image representations. In Computer Graphics (Proceedings of ACM SIGGRAPH 90), ACM Press, 207--214. Google ScholarDigital Library
- HEEGER, D. J., AND BERGEN, J. R. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of ACM SIGGRAPH 95, ACM Press, 229--238. Google ScholarDigital Library
- HERTZMANN, A., JACOBS, C. E., OLIVER, N., CURLESS, B., AND SALESIN, D. H. 2001. Image analogies. In Proceedings of ACM SIGGRAPH 2001, ACM Press, 327--340. Google ScholarDigital Library
- HIRANI, A. N., AND TOTSUKA, T. 1996. Combining frequency and spatial domain information for fast interactive image noise removal. In Proceedings of ACM SIGGRAPH 96, ACM Press, 269--276. Google ScholarDigital Library
- IGEHY, H., AND PEREIRA, L. 1997. Image replacement through texture synthesis. In IEEE International conference on Image Processing, vol. 3, 186--189. Google ScholarDigital Library
- KOFFKA, K. 1935, 1967. Principles of Gestalt Psychology. New York, Hartcourt, Brace and World.Google Scholar
- NOE, A., PESSOA, L., AND THOMPSON, E. 1998. Finding out about filling-in: A guide to perceptual completion for visual science and the philosophy of perception. Behavioral and Brain Sciences, 6, 723--748, 796--802.Google Scholar
- OH, B. M., CHEN, M., DORSEY, J., AND DURAND, F. 2001. Image-based modeling and photo editing. In Proceedings of ACM SIGGRAPH 2001, ACM Press, 433--442. Google ScholarDigital Library
- PALMER, S. 1999. Vision Science. MIT Press.Google Scholar
- PORTER, T., AND DUFF, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of ACM SIGGRAPH 84), 253--259. Google ScholarDigital Library
- SHARON, E., BRANDT, A., AND BASRI, R. 2000. Completion energies and scale. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 10, 1117--1131. Google ScholarDigital Library
- SOLER, C., CANI, M.-P., AND ANGELIDIS, A. 2002. Hierarchical pattern mapping. ACM Transactions on Graphics, 21, 3, 673--680. Google ScholarDigital Library
- WEI, L. Y., AND LEVOY, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Proceedings of ACM SIGGRAPH 2000, ACM Press, 479--488. Google ScholarDigital Library
- WELSH, T., ASHIKHMIN, M., AND MUELLER, K. 2002. Transferring color to greyscale images. ACM Transactions on Graphics, 21, 3, 277--280. Google ScholarDigital Library
- WILLIAMS, L., AND JACOBS, D. W. 1997. Stochastic completion fields: A neural model of illusory contour shape and salience. Neural Computation 9, 4, 837--858. Google ScholarDigital Library
Index Terms
- Fragment-based image completion
Recommendations
Fragment-based image completion
SIGGRAPH '03: ACM SIGGRAPH 2003 PapersWe present a new method for completing missing parts caused by the removal of foreground or background elements from an image. Our goal is to synthesize a complete, visually plausible and coherent image. The visible parts of the image serve as a ...
Sample-based image completion using structure synthesis
Image completion technique is widely used in image processing applications such as textural recovery, object removal, image edit, etc. When filling in the missing areas of an image, it is often a challenge to keep local consistency of image structures ...
Image completion based on views of large displacement
This paper presents an algorithm for image completion based on the views of large displacement. A distinct from most existing image completion methods, which exploit only the target image’s own information to complete the damaged regions, our algorithm ...
Comments