Skip to main content
Springer Nature Link
Log in

On the optimization properties of the correntropic loss function in data analysis

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

Similarity measures play a critical role in the solution quality of data analysis methods. Outliers or noise often taint the solution, hence, practical data analysis calls for robust measures. The correntropic loss function is a smooth and robust measure. In this paper, we present the properties of the correntropic loss function that can be utilized in optimization based data analysis methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms. Wiley, London (2006)

    Book  Google Scholar 

  2. Ben-Israel, A., Mond, B.: What is invexity. J. Aust. Math. Soc. Ser. B 28(1), 1–9 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  3. Cambini, A., Martein, L.: Generalized Convexity and Optimization: Theory and Applications, vol. 616. Springer, Berlin (2008)

    Google Scholar 

  4. Craven, B.D.: Duality for generalized convex fractional programs. In: Schaible, S., Ziemba, W.T. (eds.) Generalized Concavity in Optimization and Economics, pp. 473–489. Academic Press, New York (1981)

  5. Eddington, S.A.S.: Stellar Movements and the Structure of the Universe. Macmillan and Company, limited, London (1914)

    Google Scholar 

  6. Fisher, R.A., et al.: A mathematical examination of the methods of determining the accuracy of an observation by the mean error, and by the mean square error. Mon. Not. R. Astron. Soc. 80, 758–770 (1920)

    Google Scholar 

  7. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions, vol. 114. Wiley, London (2011)

    Google Scholar 

  8. Hanson, M.A.: On sufficiency of the kuhn-tucker conditions. J. Math. Anal. Appl. 80(2), 545–550 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  9. He, R., Zheng, W.S., Hu, B.G., Kong, X.W.: A regularized correntropy framework for robust pattern recognition. Neural. Comput. 23(8), 2074–2100 (2011)

    Article  MATH  Google Scholar 

  10. Huber, P.J., Ronchetti, E.M.: Robust Statistics, Wiley Series in Probability and Statistics, New Jersey (2009)

  11. Huber, P.J.: Robust Statistical Procedures. Number 27. SIAM, Philadelphia, USA (1997)

  12. Khanh, P.Q.: Invex-convexlike functions and duality. J. Optim. Theory. Appl. 87(1), 141–165 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  13. Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: a localized similarity measure. In: International Joint Conference on Neural Networks, 2006. IJCNN’06, pp. 4919–4924. IEEE (2006)

  14. Liu, W., Pokharel, P.P., Principe, J.C.: Error entropy, correntropy and m-estimation. In: Proceedings of the 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, 2006, pp. 179–184. IEEE (2006)

  15. Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. Signal Process. IEEE Trans. 55(11), 5286–5298 (2007)

    Article  MathSciNet  Google Scholar 

  16. Mangasarian, O.L.: Pseudo-convex functions. J. Soc. Ind. Appl. Math. Ser. A Control 3(2), 281–290 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  17. Mangasarian, O.L., Mangasarian, O.L., Mangasarian, O.L.: Nonlinear Programming. Society for Industrial and Applied Mathematics, Philadelphia, PA (1994)

    Book  MATH  Google Scholar 

  18. Pardalos, P.M., Hansen, P.: Data Mining and Mathematical Programming, vol. 45. Amer Mathematical Society, Rhode Island, USA (2008)

  19. Principe, J.C.: Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives. Springer, Berlin (2010)

    Book  Google Scholar 

  20. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Risk tuning with generalized linear regression. Math. Oper. Res. 33(3), 712–729 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  21. Santamaría, I., Pokharel, P.P., Principe, J.C.: Generalized correlation function: definition, properties, and application to blind equalization. Signal Process. IEEE Trans. 54(6), 2187–2197 (2006)

    Article  Google Scholar 

  22. Singh, A., Principe, J.C.: Using correntropy as a cost function in linear adaptive filters. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009, pp. 2950–2955. IEEE (2009)

  23. Singh, A., Principe, J.C.: A loss function for classification based on a robust similarity metric. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2010)

  24. Syed, M.N., Principe, J.C., Pardalos, P.M.: Correntropy in data classification. In: Sorokin, A., Murphey, R., Thai, M.T., Pardalos, P.M. (eds.) Dynamics of Information Systems: Mathematical Foundations, pp. 81–117. Springer, Berlin (2012)

  25. Tukey, J.W.: A survey of sampling from contaminated distributions. Contrib. Probab. Stat. 2, 448–485 (1960)

    Google Scholar 

Download references

Acknowledgments

This research is partially supported by NSF.

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, 32606, USA

    Mujahid N. Syed

  2. Department of Industrial and Systems Engineering, and Biomedical Engineering, University of Florida, Gainesville, FL, 32606, USA

    Panos M. Pardalos

  3. Department of Electrical and Biomedical Engineering, University of Florida, Gainesville, FL, 32606, USA

    Jose C. Principe

Authors
  1. Mujahid N. Syed
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Panos M. Pardalos
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jose C. Principe
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mujahid N. Syed.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Syed, M.N., Pardalos, P.M. & Principe, J.C. On the optimization properties of the correntropic loss function in data analysis. Optim Lett 8, 823–839 (2014). https://doi.org/10.1007/s11590-013-0626-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-013-0626-5

Keywords