Skip to main content
SpringerLink
Log in

Joint Contour Filtering

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Edge/structure-preserving operations for images aim to smooth images without blurring the edges/structures. Many exemplary edge-preserving filtering methods have recently been proposed to reduce the computational complexity and/or separate structures of different scales. They normally adopt a user-selected scale measurement to control the detail smoothing. However, natural photos contain objects of different sizes, which cannot be described by a single scale measurement. On the other hand, contour analysis is closely related to edge-preserving filtering, and significant progress has recently been achieved. Nevertheless, the majority of state-of-the-art filtering techniques have ignored the successes in this area. Inspired by the fact that learning-based edge detectors significantly outperform traditional manually-designed detectors, this paper proposes a learning-based edge-preserving filtering technique. It synergistically combines the differential operations in edge-preserving filters with the effectiveness of the recent edge detectors for scale-aware filtering. Unlike previous filtering methods, the proposed filters can efficiently extract subjectively meaningful structures from natural scenes containing multiple-scale objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Notes

  1. The structure-preserving filtering can be considered as a special design of edge-preserving filtering to deal with its limitation in handling textures. In most cases, this paper adopts the phrase “edge-preserving” for a broader concept.

  2. The implementations with the default parameters published by the authors were employed.

  3. Experiments conducted in this study use two iterations, except for the large-scale texture removal task in Sect. 4.1, which requires more iterations to smooth large-scale highly-textured images.

  4. Please note that when a spatial parameter is a fractional number, it represents the percentage of width/height of the image.

References

  • Adams, A., Baek, J., & Davis, A. (2010). Fast high-dimensional filtering using the permutohedral lattice. CGF, 29(2), 753–762.

    Google Scholar 

  • Adams, A., Gelfand, N., Dolson, J., & Levoy, M. (2009). Gaussian kd-trees for fast high-dimensional filtering. ACM TOG (SIGGRAPH), 28, 21:1–21:12.

    Google Scholar 

  • An, X., & Pellacini, F. (2008). Appprop: All-pairs appearance-space edit propagation. ACM TOG (SIGGRAPH Asia), 27(3), 40:1–40:9.

    Google Scholar 

  • Arbelaez, P., Maire, M., Fowlkes, C., & Malik, J. (2011). Contour detectionand hierarchical image segmentation. IEEE TPAMI, 33, 898–916.

    Article  Google Scholar 

  • Arbeláez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014). Multiscale combinatorial grouping. In CVPR.

  • Arnheim, R. (1956). Art and visual perception: A psychology of the creative eye. Berkeley: University of California Press.

    Google Scholar 

  • Aujol, J., Gilboa, G., Chan, T., & Osher, S. (2006). Structure–texture image decomposition-modeling, algorithms, and parameter selection. IJCV, 67(1), 111–136.

    Article  MATH  Google Scholar 

  • Avidan, S., & Shamir, A. (2007). Seam carving for content-aware image resizing. ACM TOG (SIGGRAPH), 26(3), 10.

    Article  Google Scholar 

  • Bertasius, G., Shi, J., & Torresani, L. (2015a). Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In CVPR.

  • Bertasius, G., Shi, J., & Torresani, L. (2015b). High-for-low and low-for-high: Efficient boundary detection from deep object features and its applications to high-level vision. In ICCV.

  • Bousseau, A., Paris, S., & Durand, F. (2009). User-assisted intrinsic images. ACM TOG (SIGGRAPH Asia), 28, 130:1–130:10.

    Google Scholar 

  • Boyadzhiev, I., Bala, K., Paris, S., & Durand, F. (2012). User-guided white balance for mixed lighting conditions. ACM TOG (SIGGRAPH Asia), 31(6), 200:1–200:10.

    Google Scholar 

  • Buades, A., & Lisani, J. L. (2016). Directional filters for color cartoon+texture image and video decomposition. Journal of Mathematical Imaging and Vision, 55(1), 125–135.

    Article  MathSciNet  Google Scholar 

  • Canny, J. (1986). A computational approach to edge detection. In IEEE TPAMI.

  • Catanzaro, B., Su, B. Y., Sundaram, N., Lee, Y., Murphy, M., & Keutzer, K. (2009). Efficient, high-quality image contour detection. In ICCV.

  • Chambolle, A., & Darbon, J. (2009). On total variation minimization and surface evolution using parametric maximum flows. IJCV, 84(3), 288–307.

    Article  MATH  Google Scholar 

  • Chen, J., Paris, S., & Durand, F. (2007). Real-time edge-aware image processing with the bilateral grid. ACM TOG (SIGGRAPH), 26(3), 103.

    Article  Google Scholar 

  • Cho, H., Lee, H., Kang, H., & Lee, S. (2014). Bilateral texture filtering. ACM TOG (SIGGRAPH), 33(4), 128:1–128:8.

    Google Scholar 

  • Criminisi, A., Sharp, T., Rother, C., & Perez, P. (2010). Geodesic image and video editing. ACM TOG, 29(5), 134.

    Article  Google Scholar 

  • Dani, A. L., Lischinski, D., & Weiss, Y. (2004). Colorization using optimization. ACM TOG (SIGGRAPH), 23, 689–694.

    Article  Google Scholar 

  • Dollár, P., Tu, Z., & Belongie, S. (2006). Supervised learning of edges and object boundaries. In CVPR.

  • Dollár, P., & Zitnick, C. L. (2013). Structured forests for fast edge detection. In ICCV.

  • Dollár, P., & Zitnick, C. L. (2015). Fast edge detection using structured forests. IEEE TPAMI.

  • Donoho, D., Chui, C., Coifman, R. R., & Lafon, S. (2006). Diffusion maps. Applied and Computational Harmonic Analysis, 21(1), 5–30.

    Article  MathSciNet  MATH  Google Scholar 

  • Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: Wiley.

    MATH  Google Scholar 

  • Durand, F., & Dorsey, J. (2002). Fast bilateral filtering for the display of high-dynamic-range images. ACM TOG (SIGGRAPH), 21(3), 257–266.

    Google Scholar 

  • Eisemann, E., & Durand, F. (2004). Flash photography enhancement via intrinsic relighting. ACM TOG (SIGGRAPH), 23(3), 673–678.

    Article  Google Scholar 

  • Farbman, Z., Fattal, R., & Lischinski, D. (2010). Diffusion maps for edge-aware image editing. ACM TOG (SIGGRAPH Asia), 29(6), 145:1–145:10.

    Google Scholar 

  • Farbman, Z., Fattal, R., Lischinski, D., & Szeliski, R. (2008). Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM TOG (SIGGRAPH), 27(3), 67.

    Google Scholar 

  • Fattal, R. (2009). Edge-avoiding wavelets and their applications. ACM TOG (SIGGRAPH), 28(3), 1–10.

    Article  Google Scholar 

  • Gastal, E., & Oliveira, M. (2011). Domain transform for edge-aware image and video processing. ACM TOG (SIGGRAPH), 30(4), 69:1–69:12.

    Google Scholar 

  • Gastal, E., & Oliveira, M. (2012). Adaptive manifolds for real-time high-dimensional filtering. ACM TOG (SIGGRAPH), 31(4), 33:1–33:13.

    Google Scholar 

  • Gilboa, G. (2014). A total variation spectral framework for scale and texture analysis. SIAM Journal of Imaging Sciences, 7(4), 1937–1961.

    Article  MathSciNet  MATH  Google Scholar 

  • Gupta, S., Arbeláez, P. A., Girshick, R. B., & Malik, J. (2015). Indoor scene understanding with RGB-D images: Bottom-up segmentation, object detection and semantic segmentation. IJCV, 112(2), 133–149.

    Article  MathSciNet  Google Scholar 

  • He, K., Sun, J., & Tang, X. (2013). Guided image filtering. IEEE TPAMI, 35, 1397–1409.

    Article  Google Scholar 

  • Karacan, L., Erdem, E., & Erdem, A. (2013). Structure-preserving image smoothing via region covariances. ACM TOG (SIGGRAPH Asia), 32(6), 176:1–176:11.

    Google Scholar 

  • Kivinen, J. J., Williams, C. K., & Heess, N. (2014). Visual boundary prediction: A deep neural prediction network and quality dissection. In AISTATS.

  • Kyprianidis, J. E., & Döllner, J. (2008). Image abstraction by structure adaptive filtering. In Proceedings of EG UK theory and practice of computer graphics, Manchester, United Kingdom, 2008 (pp. 51–58).

  • Kyprianidis, J. E., & Kang, H. (2011). Image and video abstraction by coherence-enhancing filtering. Computer Graphics Forum, 30(2), 593–602.

    Article  Google Scholar 

  • Levin, A., Lischinski, D., & Weiss, Y. (2006). A closed form solution to natural image matting. In CVPR.

  • Lim, J., Zitnick, C. L., & Dollár, P. (2013). Sketch tokens: A learned mid-level representation for contour and object detection. In CVPR.

  • Lischinski, D., Farbman, Z., Uyttendaele, M., & Szeliski, R. (2006). Interactive local adjustment of tonal values. ACM TOG (SIGGRAPH), 25(3), 646–653.

    Article  Google Scholar 

  • Margolin, R., Zelnik-Manor, L., & Tal, A. (2014). How to evaluate foreground maps. In CVPR.

  • Meyer, Y. (2001). Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. Providence: American Mathematical Society.

    Book  MATH  Google Scholar 

  • Min, D., Choi, S., Lu, J., Ham, B., Sohn, K., & Do, M. N. (2014). Fast global image smoothing based on weighted least squares. IEEE TIP, 23(12), 5638–5653.

    MathSciNet  MATH  Google Scholar 

  • Paris, S., & Durand, F. (2009). A fast approximation of the bilateral filter using a signal processing approach. IJCV, 81, 24–52.

    Article  Google Scholar 

  • Paris, S., Kornprobst, P., Tumblin, J., & Durand, F. (2009). Bilateral filtering: Theory and applications. Foundations and Trends in Computer Graphics and Vision, 4(1), 1–73.

    Article  MATH  Google Scholar 

  • Parisand, S., Hasinoff, S. W., & Kautz, J. (2011). Local laplacian filters: Edge-aware image processing with a Laplacian pyramid. ACM TOG (SIGGRAPH), 30(4), 68:1–68:12.

    Google Scholar 

  • Perazzi, F., Krahenbuhl, P., Pritch, Y., & Hornung, A. (2012). Saliency filters: Contrast based filtering for salient region detection. In CVPR (pp. 733–740).

  • Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. IEE TPAMI, 12, 629–639.

    Article  Google Scholar 

  • Perreault, S., & Hbert, P. (2007). Median filtering in constant time. IEEE TIP, 16(9), 2389–2394.

    MathSciNet  Google Scholar 

  • Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004). Digital photography with flash and no-flash image pairs. ACM TOG (SIGGRAPH), 23(3), 664–672.

    Article  Google Scholar 

  • Pham, T. Q., & van Vliet, L. J. (2005). Separable bilateral filtering for fast video preprocessing. In ICME.

  • Porikli, F. (2008). Constant time o(1) bilateral filtering. In CVPR.

  • Rhemann, C., Hosni, A., Bleyer, M., Rother, C., & Gelautz, M. (2011). Fast cost-volume filtering for visual correspondence and beyond. In CVPR.

  • Ren, X., & Liefeng, B. (2012). Discriminatively trained sparse code gradients for contour detection. In NIPS.

  • Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60(1–4), 259–268.

    Article  MathSciNet  MATH  Google Scholar 

  • Shen, W., Wang, X., Wang, Y., Bai, X., & Zhang, Z. (2015). Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In CVPR (pp. 3982–3991).

  • Subr, K., Soler, C., & Durand, F. (2009). Edge-preserving multiscale image decomposition based on local extrema. ACM ToG (SIGGRAPH Asia), 28(5), 147.

    Google Scholar 

  • Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In ICCV (pp. 839–846).

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE TIP, 13(4), 600–612.

    Google Scholar 

  • Weickert, J. (1999). Coherence-enhancing diffusion filtering. IJCV, 31(2–3), 111–127.

    Article  Google Scholar 

  • Weiss, B. (2006). Fast median and bilateral filtering. ACM TOG (SIGGRAPH), 25(3), 519–526.

    Article  Google Scholar 

  • Xie, S., & Tu, Z. (2015). Holistically-nested edge detection. In Proceedings of IEEE international conference on computer vision.

  • Xu, K., Li, Y., Ju, T., Hu, S. M., & Liu, T. Q. (2009). Efficient affinity-based edit propagation using k-d tree. ACM ToG (SIGGRAPH Asia), 28(5), 118:1–118:6.

    Google Scholar 

  • Xu, L., Lu, C., Xu, Y., & Jia, J. (2011). Image smoothing via l0 gradient minimization. ACM TOG (SIGGRAPH Asia), 36(6), 174.

    Google Scholar 

  • Xu, L., Yan, Q., & Jia, J. (2013). A sparse control model for image and video editing. ACM TOG (SIGGRAPH Asia), 32(6), 197.

    Google Scholar 

  • Xu, L., Yan, Q., Xia, Y., & Jia, J. (2012). Structure extraction from texture via relative total variation. ACM TOG (SIGGRAPH Asia), 31(6), 139.

    Google Scholar 

  • Yan, Q., Xu, L., Shi, J., & Jia, J. (2013). Hierarchical saliency detection. In CVPR.

  • Yang, J., Price, B., Cohen, S., Lee, H., & Yang, M. H. (2016). Object contour detection with a fully convolutional encoder-decoder network. In CVPR.

  • Yang, Q. (2012). Recursive bilateral filtering. In ECCV (pp. 399–413).

  • Yang, Q. (2016). Semantic filtering. In CVPR.

  • Yang, Q., Ahuja, N., & Tan, K. (2015). Constant time median and bilateral filtering. IJCV, 112(3), 307–318.

    Article  Google Scholar 

  • Yang, Q., Tan, K. H., & Ahuja, N. (2009). Real-time o(1) bilateral filtering. In CVPR (pp. 557–564).

  • Yang, Q., Wang, S., & Ahuja, N. (2010). Svm for edge-preserving filtering. In CVPR (pp. 1775–1782).

  • Yin, W., Goldfarb, D., & Osher, S. (2005). Image cartoon-texture decomposition and feature selection using the total variation regularized l1 functional. In VLSM (pp. 73–84).

  • Yoon, K. J., & Kweon, I. S. (2006). Adaptive support-weight approach for correspondence search. IEEE TPAMI, 28(4), 650–656.

    Article  Google Scholar 

  • Zeune, L., van Dalum, G., Terstappen, L. W. M. M., van Gils, S. A., & Brune, C. (2016). Multiscale segmentation via Bregman distances and nonlinear spectral analysis. CoRR arXiv:1604.06665.

  • Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., & Mech, R. (2015). Minimum barrier salient object detection at 80 fps. In ICCV.

  • Zhang, Q., Shen, X., Xu, L., & Jia, J. (2014). Rolling guidance filter. In ECCV.

  • Zheng, S., Tu, Z., & Yuille, A. (2007). Detecting object boundaries using low-,mid-, and high-level information. In CVPR.

  • Ziou, D., & Tabbone, S. (1998). Edge detection techniques: An overview. IEEE TPAMI, 8, 537–559.

    Google Scholar 

  • Zitnick, C. L., & Dollár, P. (2014). Edge boxes: Locating object proposals from edges. In ECCV.

  • Zitnick, C. L., & Parikh, D. (2012). The role of image understanding in contour detection. In CVPR.

Download references

Acknowledgements

We thank all the reviewers for valuable comments. This work was supported by the National Basic Research Program of China (Grant No. 2015CB351705), the State Key Program of National Natural Science Foundation of China (Grant No. 61332018).

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, Shannxi, China

    Xing Wei & Yihong Gong

  2. JingChi Corp., 330 Gibraltar Drive, Sunnyvale, CA, 94089, USA

    Qingxiong Yang

Authors
  1. Xing Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingxiong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yihong Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingxiong Yang.

Additional information

Communicated by S. Soatto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, X., Yang, Q. & Gong, Y. Joint Contour Filtering. Int J Comput Vis 126, 1245–1265 (2018). https://doi.org/10.1007/s11263-018-1091-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-018-1091-5

Keywords

Search

Navigation