skip to main content
10.1145/3545839.3545842acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicomsConference Proceedingsconference-collections
research-article

Performance Improvement of Logistic Regression for Binary Classification by Gauss-Newton Method

Published: 10 September 2022 Publication History

Abstract

This paper proposes a new approach to optimizing cost function for binary logistic regression by the Gauss-Newton method. This method was applied to the backpropagation phase as a part of the training process to update the weighted coefficients. To show the performance of the approach, we used two data sets to train the logistic regression model for binary classification problems. Our experiment demonstrated that the proposed methods could perform better than gradient descent for both examples, as we expected. Furthermore, the performance of our approach is more advanced than the classical method, either in speed or accuracy.

References

[1]
M. J. Zaki and W. Meira Jr, Data Mining and Machine Learning: Fundamental Concepts and Algorithms. Cambridge University Press, 2020.
[2]
G. Strang, Linear algebra and learning from data. Wellesley-Cambridge Press Cambridge, 2019.
[3]
J. Chee and P. Toulis, “Convergence diagnostics for stochastic gradient descent with constant learning rate,” in International Conference on Artificial Intelligence and Statistics, 2018, pp. 1476–1485.
[4]
I. Kulikovskikh, S. Prokhorov, T. Lipić, T. Legović, and T. Šmuc, “BioGD: Bio-inspired robust gradient descent,” PLoS One, vol. 14, no. 7, p. e0219004, 2019.
[5]
L. Bottou, “Stochastic gradient descent tricks,” in Neural networks: Tricks of the trade, Springer, 2012, pp. 421–436.
[6]
R. W. Siregar and M. Ramli, “Analysis local convergence of gauss-newton method,” in IOP Conference Series: Materials Science and Engineering, 2018, vol. 300, no. 1, p. 12044.
[7]
K. Lange, “Newton's Method,” in Optimization, Springer, 2004, pp. 155–173.
[8]
P. Chen, “Hessian matrix vs. Gauss–Newton hessian matrix,” SIAM J. Numer. Anal., vol. 49, no. 4, pp. 1417–1435, 2011.
[9]
A. Botev, H. Ritter, and D. Barber, “Practical gauss newton optimisation for deep learning,” in International Conference on Machine Learning, 2017, pp. 557–565.
[10]
S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv Prepr. arXiv1609.04747, 2016.

Cited By

View all

Index Terms

  1. Performance Improvement of Logistic Regression for Binary Classification by Gauss-Newton Method

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICoMS '22: Proceedings of the 2022 5th International Conference on Mathematics and Statistics
    June 2022
    137 pages
    ISBN:9781450396233
    DOI:10.1145/3545839
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 September 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Gauss-Newton
    2. binary classification
    3. gradient descent
    4. logistic regression

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICoMS 2022

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 53
      Total Downloads
    • Downloads (Last 12 months)19
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 11 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media