Skip to main content
SpringerLink
Log in

Approximate Functions in a Problem of Sets Separation

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, problems of mathematical diagnostics are considered. The most popular approach to these problems is based on statistical methods. In this paper, the author treats the mentioned problems by means of optimization. This approach can be useful in the case where statistical characteristics of the database are unknown or the database is not sufficiently large. In this paper, a nonsmooth model is used where it is required to separate two sets, whose convex hulls may intersect. A linear classifier is used to identify the points of two sets. The quality of identification is evaluated by the so-called natural functional, based on the number of misclassified points. It is required to find the optimal hyperplane, which minimizes the number of misclassified points by means of the translation and rotation operations. Since the natural functional (number of misclassified points) is discontinuous, it is suggested to approximate it by some surrogate functional possessing at least the continuity property. In this paper, two surrogate functionals are introduced and studied. It is shown that one of them is subdifferentiable, and the second one is continuously differentiable. It is also demonstrated that the theory of exact penalization can be employed to reduce the given constrained optimization problems to an unconstrained one. Numerical methods are constructed, where the steepest descent directions of the surrogate functionals are used to minimize the natural one. Necessary conditions for a minimum are formulated for both surrogate functionals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning: Neural and Statistical Classification. Ellis Horwood, Chichester (1994)

    MATH  Google Scholar 

  2. Quinlan, J.R.: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  3. Astorino, A., Fuduli, A.: Support vector machine polyhedral separability in semisupervised learning. J. Opt. Theory Appl. 164(3), 1039–1050 (2015)

  4. Bradley, P.S., Mangasarian, O.L.: Feature selection via concave minimization and support vector machines, machine learning. In: Proceedings of the Fifteenth International Conference (ICML’98), Morgan Kaufmann, San Francisco, California, pp. 82–90 (1998)

  5. Bradley, P.S., Mangasarian, O.L.: Massive data discrimination via linear support vector machines. Opt. Methods Softw. 13, 1–10 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  6. De Coste, D., Scholkopf, B.: Training invariant support vector machines. Mach. Learn. 46, 161–190 (2002)

    Article  MATH  Google Scholar 

  7. Lee, Y.-J., Mangasarian, O.L.: SSVM: a smooth support vector machine for classification. Comput. Optim. Appl. 20(1), 5–22 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  8. Mangasarian, O.L.: Linear and nonlinear separation of patterns by linear programming. Oper. Res. 13, 444–452 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  9. Mangasarian, O.L.: Misclassification minimization. J. Glob. Optim. 5, 309–323 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  10. Mangasarian, O.L.: Mathematical programming in data mining. Data Min Knowl Discov 1, 183–201 (1997)

    Article  Google Scholar 

  11. Vapnik, V.: The Nature of Statistical Learning Theory, 334 p. Springer, New York (2000)

  12. Anderberg, M.R.: Cluster Analysis for Applications. Academic Press, New York (1973)

    MATH  Google Scholar 

  13. Bhuyan, N.J., Raghavan, V.V., Venkatesh, K.E.: Genetic algorithms for clustering with an ordered representation. In: Proceedings of the Fourth International Conference on Genetic Algorithms, pp. 408–415 (1991)

  14. Hansen, P., Jaumard, B.: Cluster analysis and mathematical programming. Math. Program. 79, 191–215 (1997)

    MathSciNet  MATH  Google Scholar 

  15. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31, 264–323 (1999)

    Article  Google Scholar 

  16. Mirkin, B.: Mathematical Classification and Clustering, 448 p. Kluwer Academic Publishers, Dordrecht (1996)

  17. Fisher, R.A.: Contributions to Mathematical Statistics. Wiley, New York (1950)

  18. Nagy, G.: State of the art in pattern recognition. Proce. IEEE 56, 836–862 (1968)

    Article  Google Scholar 

  19. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel Based Methods, 168 p. Cambridge University Press, Cambridge (2000)

  20. Scholkopf, B., Smola, A.: Learning with Kernels. The MIT Press, Cambridge (2002)

    MATH  Google Scholar 

  21. Bagirov, A.M., Rubinov, A.M., Yearwood, J.: A global optimization approach to classification. Optim. Eng. 3, 129–155 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  22. Bagirov, A.M., Rubinov, A.M., Yearwood, J.: A heuristic algorithm for feature selection based on optimization techniques. In: Abbas, H., Newton, C., Sarker, R. (eds.) Heuristic and Optimization for Knowledge Discovery, pp. 13–26. Idea Publishing Group, Hershey (2002)

    Chapter  Google Scholar 

  23. Bennett, K.P., Mangasarian, O.L.: Robust linear programming discrimination of two linearly inseparable sets. Optim. Method Softw. 1, 23–34 (1992)

    Article  Google Scholar 

  24. Highleyman, W.H.: Linear decision functions with applications to pattern recognition. Proc. IRE 50, 1501–1514 (1962)

    Article  MathSciNet  Google Scholar 

  25. Demyanov, V.F.: Identification of points of two convex sets. Vestnik S.-Petersburg Univ. Ser. I 3(17), 14–20 (2001)

  26. Demyanov, V.F.: Extremum Conditions and Variational Problems, 136 p. SPbSU, St. Petersburg (In Russian) (2000)

  27. Demyanov, V.F., Rubinov, A.M.: Foundation of Nonsmooth Analyses and Quasidifferential Calculus, 431 p. Nauka, Moscow (In Russian) (1990)

  28. Demyanov, V.F., Vasiliev, L.V.: Nondifferential Optimization. Springer, New York (1986)

    Google Scholar 

  29. Karmanov, V.G.: Mathematical Programming. Mir Publishers, Moscow (1989)

    MATH  Google Scholar 

  30. Polyak, B.T.: Introduction to Optimization, 384 p. Nauka, Moscow (In Russian) (1983)

  31. Pshenichny, B.N., Danilin, Y.M.: Numerical Methods in Extremal Problems, 2nd edn, 320 p. Mir Publishers, Moscow (2009)

    Google Scholar 

  32. Vasil’ev, F.P.: Numerical Method for Solving Extremal Problems, 550 p. Nauka, Moscow (In Russian) (1988)

  33. Anan’ev, K.I., Demyanova, V.V., Dem’yanov, V.F., Kokorina, A.V., Svistun, S.Ya., Stegalin, I.S.: Optimization methods in diagnostics problems. Vestnik S.-Petersburg Univ. Ser. Prikl. Mat. Inform. Prots. Upr. 10(3), 312 (2011)

  34. Bagirov, A.M., Rubinov, A.M., Soukhoroukova, N.V., Yearwood, J.: Unsupervised and supervised data classification via nonsmooth and global optimization. Top, Sociedad de Estadistica e Investigacion Operativa, Madrid, Spain 11(1), 1–75 (2003)

  35. Demyanov, V.F.: Mathematical diagnostics via nonsmooth analysis. OMS 20, 197–218 (2005)

    MathSciNet  MATH  Google Scholar 

  36. Demyanov, V.F., Astorino, A., Gaudioso, M.: Nonsmooth problems in mathematical diagnostics. In: Hadjisavvas, N., Pardalos, P.M. (eds.) Advances in Convex Analysis and Global Optimization; Nonconvex Optimization and Its Applications, vol. 54, pp. 11–30. Kluwer Academic Publishers, Dordrecht (2001)

    Chapter  Google Scholar 

  37. Demyanova, V.V., Demyanov, V.F.: One-dimensional identification problem and ranking parameters. J. Glob. Optim. 48, 29–40 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  38. Demyanova, V.V., Demyanov, V.F.: The method of virtual experts in mathematical diagnostics. J. Glob. Optim. 35(2), 215–234 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  39. Grigor’eva, X. V.: Surrogate functionals in diagnosis problems. Vestnik S.-Petersburg Univ. Ser. 10. Prikl. Mat. Inform. Prots. Upr., Issue 1, 34–43 (2009)

  40. Kokorina, A. V.: Unsupervised and supervised Data Classification Via Nonsmooth and Global Optimization. Top, Volume 11, Number 1, Sociedad de Estadistica e Tnvestigacion Operativa, Madrid, Spain, pp. 86–89 (2003)

  41. Malozemov, V.N., Cherneutsanu, E.K.: The best linear separation of two sets. Constructive nonsmooth analysis and related topics. Spring. Optim. Appl. 87, 175–183 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Bache, K., Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2013). http://archive.ics.uci.edu/ml

Download references

Acknowledgments

I would like to thank Dr. N. Sukhorukova (Australia, Melbourne, Swinburne University of Technology) for useful comments.

Author information

Authors and Affiliations

  1. Saint-Petersburg State University, Saint-Petersburg, Russia

    Xeniya Vladimirovna Grigor’eva

Authors
  1. Xeniya Vladimirovna Grigor’eva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xeniya Vladimirovna Grigor’eva.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Grigor’eva, X.V. Approximate Functions in a Problem of Sets Separation. J Optim Theory Appl 171, 550–572 (2016). https://doi.org/10.1007/s10957-015-0766-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-015-0766-0

Keywords

Mathematics Subject Classification

Search

Navigation