Skip to main content
SpringerLink
Log in

Ordered Hypothesis Machines

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Stack Filters are a class of non-linear filter typically used for noise suppression. Advantages of Stack Filters are their generality and the existence of efficient optimization algorithms under mean absolute error (Wendt et al. in IEEE Trans. Acoust. Speech Signal Process. 34:898–910, 1986). In this paper we describe our recent efforts to use the class of Stack Filters for classification problems. This leads to a novel class of continuous domain classifiers which we call Ordered Hypothesis Machines (OHM). We develop convex optimization based learning algorithms for Ordered Hypothesis Machines and highlight their relationship to Support Vector Machines and Nearest Neighbor classifiers. We report on the performance on synthetic and real-world datasets including an application to change detection in remote sensing imagery. We conclude that OHM provides a novel way to reduce the number of exemplars used in Nearest Neighbor classifiers and achieves competitive performance to the more computationally expensive K-Nearest Neighbor method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arce, G.: A general weighted median filter structure admitting negative weights. In: Proc. 11th Int. Joint Conf. on Artifical Intelligence, vol. 46, pp. 3195–3205 (1998)

    Google Scholar 

  2. Barner, K.: C-stack filters. In: ICASSP-91, International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 2005–2008 (1991). doi:10.1109/ICASSP.1991.150796

    Chapter  Google Scholar 

  3. Blanchard, G., Schafer, C., Rozenholc, Y., Muller, K.R.: Optimal dyadic decision trees. Mach. Learn. 66(2–3), 209–241 (2007)

    Article  Google Scholar 

  4. Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools (2000)

  5. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001). doi:10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  6. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth, Belmont (1984)

    MATH  Google Scholar 

  7. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001). Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm

  8. Fitch, J., Coyle, E., Gallagher, N.: Median filtering by threshold decomposition. IEEE Trans. Acoust. Speech Signal Process. ASSP-32(6), 1183–1189 (1984)

    Article  Google Scholar 

  9. Han, C.C.: A supervised classification scheme using positive boolean function. In: 16th Int. Conf. on Pattern Recognition, vol. 2, pp. 100–103 (2002)

    Google Scholar 

  10. Hastie, T., Tibshirani, R.: Discriminant adaptive nearest neighbor classification. IEEE Trans. Pattern Anal. Mach. Intell. 18, 607–616 (1996). doi:10.1109/34.506411. URL http://portal.acm.org/citation.cfm?id=232678.232681

    Article  Google Scholar 

  11. Jet Propulsion Laboratory (JPL), N.A., (NASA), S.A.: Airborne visible/infrared imaging spectrometer (aviris). http://aviris.jpl.nasa.gov/

  12. Kaburlasos, V., Petridis, V.: Fuzzy lattice neurocomputing (fln) models. Neural Netw. 13(10), 1145–1170 (2000). doi:10.1016/S0893-6080(00)00074-5. URL http://www.sciencedirect.com/science/article/B6T08-41XM6GH-K/2/8961210cefa265a6d93967722c577d29

    Article  Google Scholar 

  13. Kim, Y.T., Arce, G.: Permutation filter lattices: a general order-statistic filtering framework. IEEE Trans. Signal Process. 42(9), 2227–2241 (1994). doi:10.1109/78.317846

    Article  Google Scholar 

  14. Lin, J., Coyle, E.J.: Minimum mean absolute error estimation over the class of generalized stack filters. IEEE Trans. Acoust. Speech Signal Process. 38, 663–678 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  15. Lin, J., Sellke, T., Coyle, E.: Adaptive stack filtering under the mean absolute error criterion. In: Porter, W., Kak, S. (eds.) Advances in Communications and Signal Processing. Lecture Notes in Control and Information Sciences, vol. 129, pp. 263–276. Springer, Berlin (1989). doi:10.1007/BFb0042738

    Chapter  Google Scholar 

  16. Muselli, M.: Approximation properties of positive boolean functions. In: Apolloni, B., Marinaro, M., Nicosia, G., Tagliaferri, R. (eds.) Neural Nets. 16th Italian Workshop on Neural Nets, WIRN 2005 and International Workshop on Natural and Artificial Immune Systems, NAIS 2005. Revised Selected Papers. Lecture Notes in Computer Science, vol. 3931. Springer, Berlin (2006)

    Google Scholar 

  17. Muselli, M.: Switching Neural Networks: A New Connectionist Model for Classification. Lecture Notes in Computer Science, vol. 3931, pp. 23–30 (2006)

    Google Scholar 

  18. Paredes, J.L., Arce, G.R.: Optimization of stack filters based on mirrored threshold decomposition. IEEE Trans. Signal Process. 49, 1179–1188 (2001)

    Article  Google Scholar 

  19. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods: Support Vector Learning, pp. 185–208 (1999)

    Google Scholar 

  20. Porter, R., Eads, D., Hush, D., Theiler, J.: Weighted order statistic classifiers with large rank-oder margin. In: Proc. 20th Int. Conf. on Machine Learning (2003)

    Google Scholar 

  21. Porter, R., Hush, D., Zimmer, B.: Error minimizing algorithms for nearest neighbor classifiers. In: Proceedings of the SPIE (2011)

    Google Scholar 

  22. Porter, R.B., Zimmer, G.B., Hush, D.: Stack filter classifiers. In: Wilkinson, M.F., Roerdink, J. (eds.) ISMM 2009, 9th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing. Lecture Notes in Computer Science, vol. 5720, pp. 282–294. Springer, Berlin (2009)

    Chapter  Google Scholar 

  23. Ritter, G.X., Sussner, P.: An introduction to morphological neural networks. In: 13th Int. Conf. on Pattern Recognition, vol. 4, pp. 709–717 (1996)

    Chapter  Google Scholar 

  24. Sussner, P.: Morphological perceptron learning. In: International Symposium on Intelligent Systems and Semiotics, pp. 477–482 (1998)

    Google Scholar 

  25. Sussner, P., Esmi, E.L.: Morphological perceptrons with competitive learning: Lattice-theoretical framework and constructive learning algorithm. Inf. Sci. 181(10), 1929–1950 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Theiler, J.: Quantitative comparison of quadratic covariance-based anomalous change detector. Appl. Opt. 47, F12–F26 (2008)

    Article  Google Scholar 

  27. Theiler, J., Perkins, S.: Proposed framework for anomalous change detection. In: ICML Workshop on Machine Learning Algorithms for Surveillance and Event Detection, pp. 7–14 (2006)

    Google Scholar 

  28. Tumer, K., Ghosh, J.: Linear and order statistics combiners for pattern classification. In: Combining Artificial Neural Nets, pp. 127–162 (1999)

    Google Scholar 

  29. Valle, M.E., Sussner, P.: A general framework for fuzzy morphological associative memories. Fuzzy Sets Syst. 159, 747–768 (2008). doi:10.1016/j.fss.2007.10.010. URL http://portal.acm.org/citation.cfm?id=1344840.1344937

    Article  MathSciNet  MATH  Google Scholar 

  30. Wang, J., Neskovic, P., Cooper, L.N.: Improving nearest neighbor rule with a simple adaptive distance rule. Pattern Recognit. Lett. 28, 207–213 (2006)

    Article  Google Scholar 

  31. Wendt, P., Coyle, E., Gallagher, N.: Stack filters. IEEE Trans. Acoust. Speech Signal Process. 34, 898–910 (1986)

    Article  Google Scholar 

  32. Wilson, D., Martinez, T.: Reduction techniques for instance-based learning algorithms. Mach. Learn. 38, 257–286 (2000)

    Article  MATH  Google Scholar 

  33. Yang, P., Maragos, P.: Min-max classifiers: Learnability, design and application. Pattern Recognit. 28, 879–899 (1995)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, Texas A&M University-Corpus Christi, Corpus Christi, TX, USA

    G. Beate Zimmer

  2. Los Alamos National Lab, Los Alamos, NM, USA

    Don Hush & Reid Porter

Authors
  1. G. Beate Zimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Don Hush
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Reid Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reid Porter.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zimmer, G.B., Hush, D. & Porter, R. Ordered Hypothesis Machines. J Math Imaging Vis 43, 121–134 (2012). https://doi.org/10.1007/s10851-011-0293-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-011-0293-z

Keywords

Search

Navigation