skip to main content
research-article

Fast median filters using separable sorting networks

Published: 19 July 2021 Publication History

Abstract

Median filters are a widely-used tool in graphics, imaging, machine learning, visual effects, and even audio processing. Currently, very-small-support median filters are performed using sorting networks, and large-support median filters are handled by O(1) histogram-based methods. However, the constant factor on these O(1) algorithms is large, and they scale poorly to data types above 8-bit integers. On the other hand, good sorting networks have not been described above the 7 X 7 case, leaving us with no fast way to compute integer median filters of modest size, and no fast way to compute floating point median filters for any size above 7 X 7.
This paper describes new sorting networks that efficiently compute median filters of arbitrary size. The key idea is that these networks can be factored to exploit the separability of the sorting problem - they share common work across scanlines, and within small tiles of output. We also describe new ways to run sorting networks efficiently, using a sorting-specific instruction set, compiler, and interpreter.
The speed-up over prior work is more than an order of magnitude for a wide range of data types and filter sizes. For 8-bit integers, we describe the fastest median filters for all sizes up to 25 X 25 on CPU, and up to 33 X 33 on GPU. For higher-precision types, we describe the fastest median filters at all sizes tested on both CPU and GPU.

Supplementary Material

VTT File (3450626.3459773.vtt)
ZIP File (a70-adams.zip)
a70-adams.zip
MP4 File (3450626.3459773.mp4)
Presentation.

References

[1]
Kenneth E Batcher. 1968. Sorting networks and their applications. In Proceedings of the April 30--May 2, 1968, spring joint computer conference. 307--314.
[2]
G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000). Michael Codish and Moshe Zazon-Ivry. 2010. Pairwise cardinality networks. In International Conference on Logic for Programming Artificial Intelligence and Reasoning. Springer, 154--172.
[3]
Derry Fitzgerald. 2010. Harmonic/percussive separation using median filtering. In Proceedings of the International Conference on Digital Audio Effects (DAFx), Vol. 13.
[4]
William T Freeman. 1988. Median filter for reconstructing missing color samples. US Patent 4,724,395.
[5]
Oded Green. 2017. Efficient scalable median filtering using histogram-based operations. IEEE Transactions on Image Processing 27, 5 (2017), 2217--2228.
[6]
Wenzel Jakob. 2019. Enoki: structured vectorization and differentiation on modern processor architectures. https://github.com/mitsuba-renderer/enoki.
[7]
Michał Karpiński and Marek Piotrów. 2019. Encoding cardinality constraints using multiway merge selection networks. Constraints 24, 3--4 (2019), 234--251.
[8]
Michael Kass and Justin Solomon. 2010. Smoothed local histogram filters. In ACM SIGGRAPH 2010 papers. 1--10.
[9]
Minsik Kim, Deokho Kim, Minyong Sung, and Won Woo Ro. 2015. An accelerated separable median filter with sorting networks. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 803--807.
[10]
Donald E. Knuth. 1998. The Art of Computer Programming (3rd ed.). Fundamental Algorithms, Vol. 1. Addison Wesley Longman Publishing Co., Inc. (book).
[11]
Morgan McGuire. 2008. A Fast, Small-Radius GPU Median Filter. In Published in ShaderX6. https://casual-effects.com/research/McGuire2008Median/index.html ShaderX6.
[12]
NVIDIA. 2020. NVIDIA Performance Primitives. https://developer.nvidia.com/NPP.
[13]
Ian Parberry. 1992. The pairwise sorting network. Parallel Processing Letters 2, 02n03 (1992), 205--211.
[14]
Grégoire Pau, Florian Fuchs, Oleg Sklyar, Michael Boutros, and Wolfgang Huber. 2010. EBImage---an R package for image processing with applications to cellular phenotypes. Bioinformatics 26, 7 (03 2010), 979--981. arXiv:https://academic.oup.com/bioinformatics/article-pdf/26/7/979/559050/btq046.pdf
[15]
Simon Perreault and Patrick Hébert. 2007. Median filtering in constant time. IEEE transactions on image processing 16, 9 (2007), 2389--2394.
[16]
Gilles Perrot, Stéphane Domas, and Raphaël Couturier. 2014. Fine-tuned High-speed Implementation of a GPU-based Median Filter. Journal of Signal Processing Systems 75, 3 (2014), 185--190.
[17]
Fatih Porikli. 2005. Integral histogram: A fast way to extract histograms in cartesian spaces. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 1. IEEE, 829--836.
[18]
Jonathan Ragan-Kelley, Andrew Adams, Sylvain Paris, Marc Levoy, Saman Amarasinghe, and Frédo Durand. 2012. Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--12.
[19]
Gabriel Salvador, Juan M Chau, Jorge Quesada, and Cesar Carranza. 2018. Efficient GPU-based implementation of the median filter based on a multi-pixel-per-thread framework. In 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI). IEEE, 121--124.
[20]
Ricardo M Sánchez and Paul A Rodríguez. 2013. Highly parallelable bidimensional median filter for modern parallel programming models. Journal of Signal Processing Systems 71, 3 (2013),221--235.
[21]
E. Stewart. 2004. Intel Integrated Performance Primitives: How to Optimize Software Applications Using Intel IPP. Intel Press.
[22]
Deqing Sun, Stefan Roth, and Michael J Black. 2010. Secrets of optical flow estimation and their principles. In 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2432--2439.
[23]
JW Tukey. 1974. Nonlinear (nonsuperposable) methods for smoothing data. Proc. Cong. Rec. EASCOM'74 (1974), 673--681.
[24]
Vaibhav Vavilala. 2019. Lightpruning on Toy Story 4. In ACM SIGGRAPH 2019 Talks. 1--2.
[25]
Frederick M Waltz. 1989. Fast implementation of ranked filters on general-purpose image processing hardware. In Automated Inspection and High-Speed Vision Architectures II, Vol. 1004. International Society for Optics and Photonics, 25--32.
[26]
Frederick M Waltz, Ralf Hack, and Bruce G Batchelor. 1998. Fast efficient algorithms for 3x3 ranked filters using finite-state machines. In Machine Vision Systems for Inspection and Metrology VII, Vol. 3521. International Society for Optics and Photonics, 278--287.
[27]
Ben Weiss. 2006. Fast median and bilateral filtering. In ACM SIGGRAPH 2006 Papers. 519--526.
[28]
Pavan Yalamanchili, Umar Arshad, Zakiuddin Mohammed, Pradeep Garigipati, Peter Entschev, Brian Kloppenborg, James Malcolm, and John Melonakos. 2015. ArrayFire - A high performance software library for parallel computing with an easy-to-use API. https://github.com/arrayfire/arrayfire
[29]
Qingxiong Yang, Narendra Ahuja, and Kar-Han Tan. 2015. Constant time median and bilateral filtering. International Journal of Computer Vision 112, 3 (2015), 307--318.
[30]
Jure Žbontar and Yann LeCun. 2016. Stereo matching by training a convolutional neural network to compare image patches. The journal of machine learning research 17, 1 (2016), 2287--2318.

Cited By

View all
  • (2023)Local Adaptive Image Filtering Based on Recursive Dilation SegmentationSensors10.3390/s2313577623:13(5776)Online publication date: 21-Jun-2023
  • (2023)Detection of peanut seed vigor based on hyperspectral imaging and chemometricsFrontiers in Plant Science10.3389/fpls.2023.112710814Online publication date: 27-Feb-2023
  • (2023)Fast Grayscale Morphology for Circular WindowComputer Graphics Forum10.1111/cgf.1498342:7Online publication date: 30-Oct-2023
  • Show More Cited By

Index Terms

  1. Fast median filters using separable sorting networks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 40, Issue 4
    August 2021
    2170 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3450626
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 July 2021
    Published in TOG Volume 40, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. median filters
    2. sorting networks

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)99
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Local Adaptive Image Filtering Based on Recursive Dilation SegmentationSensors10.3390/s2313577623:13(5776)Online publication date: 21-Jun-2023
    • (2023)Detection of peanut seed vigor based on hyperspectral imaging and chemometricsFrontiers in Plant Science10.3389/fpls.2023.112710814Online publication date: 27-Feb-2023
    • (2023)Fast Grayscale Morphology for Circular WindowComputer Graphics Forum10.1111/cgf.1498342:7Online publication date: 30-Oct-2023
    • (2023)Fast Guided Median FilterIEEE Transactions on Image Processing10.1109/TIP.2022.323291632(737-749)Online publication date: 2023
    • (2022)Constant Time Median Filter Using 2D Wavelet MatrixACM Transactions on Graphics10.1145/3550454.355551241:6(1-10)Online publication date: 30-Nov-2022
    • (2022)Better Fixed-Point Filtering with Averaging TreesProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35438695:3(1-8)Online publication date: 27-Jul-2022
    • (2022)Design of High-Speed Multiway Merge Sorting Networks Using Fast Single-Stage N-Sorters and N-FiltersIEEE Access10.1109/ACCESS.2022.319337010(77980-77992)Online publication date: 2022
    • (2022)How separable median filters can get better results than full 2D versionsThe Journal of Supercomputing10.1007/s11227-021-04233-178:7(10118-10148)Online publication date: 1-May-2022

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media