Loading web-font TeX/Caligraphic/Regular
Variance-Reduced Shuffling Gradient Descent With Momentum for Finite-Sum Minimization | IEEE Journals & Magazine | IEEE Xplore

Variance-Reduced Shuffling Gradient Descent With Momentum for Finite-Sum Minimization


Abstract:

Finite-sum minimization is a fundamental optimization problem in signal processing and machine learning. This letter proposes a variance-reduced shuffling gradient descen...Show More

Abstract:

Finite-sum minimization is a fundamental optimization problem in signal processing and machine learning. This letter proposes a variance-reduced shuffling gradient descent with Nesterov’s momentum for smooth convex finite-sum optimization. We integrate an explicit variance reduction into the shuffling gradient descent to deal with the variance introduced by shuffling gradients. The proposed algorithm with a unified shuffling scheme converges at a rate of \mathcal {O} ( \frac {1}{T} ), where T is the number of epochs. The convergence rate independent of gradient variance is better than most existing shuffling gradient algorithms for convex optimization. Finally, numerical simulations demonstrate the convergence performance of the proposed algorithm.
Published in: IEEE Control Systems Letters ( Volume: 7)
Page(s): 1700 - 1705
Date of Publication: 22 May 2023
Electronic ISSN: 2475-1456

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.