Skip to main content
SpringerLink
Log in

Fractional-Based Stochastic Gradient Algorithms for Time-Delayed ARX Models

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In this study, two fractional-based stochastic gradient (FSG) algorithms for time-delayed auto-regressive exogenous (ARX) models are proposed. By combining momentum and adaptive methods, a momentum-based FSG and an adaptive-based FSG algorithms are developed. These two FSG algorithms have faster convergence rates when compared with the stochastic gradient algorithm. The mechanism of the convergence is proved in theory. Furthermore, two simulated examples are presented to illustrate the efficiency of the new proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability Statement

All data generated or analyzed during this study are included in this article.

References

  1. H. Akcay, P.P. Khargonekar, The least squares algorithm, parametric system identification and bounded noise. Automatica 29(6), 1535–1540 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  2. R. Almeida, A caputo fractional derivative of a function with respect to another function. Commun. Nonlinear Sci. Numer. Simul. 44, 460–481 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  3. Y. Bengio, P. Simard, P. Frasconi, Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural. Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  4. I.R. Birs, C.I. Muresan, S. Folea, O. Prodan, A comparison between integer and fractional order Pd\(\mu \) controllers for vibration suppression. Appl. Math. Comput. 1(1), 273–282 (2016)

    MathSciNet  MATH  Google Scholar 

  5. L. Bottou, Large-scale machine learning with stochastic gradient descent, in Proceedings of Compstat-2010, pp. 177–186 (2010)

  6. N.I. Chaudhary, S. Zubair, M.A.Z. Raja, A new computing approach for power signal modeling using fractional adaptive algorithms. ISA Trans. 68, 189–202 (2017)

    Article  Google Scholar 

  7. J. Chen, F. Ding, Y.J. Liu, Q.M. Zhu, Multi-step-length gradient iterative algorithm for equation-error type models. Syst. Control Lett. 115, 15–21 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  8. J. Chen, B. Huang, F. Ding, Y. Gu, Variational Bayesian approach for ARX systems with missing observations and varying time-delays. Automatica 94, 194–204 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. J. Chen, Y.J. Liu, Variational Bayesian-based iterative algorithm for ARX models with random missing outputs. Circuits Syst. Signal Process. 37(4), 1594–1608 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  10. J. Chen, Q.M. Zhu, M.F. Hu, L.X. Guo, P. Narayan, Improved gradient descent algorithms for time-delay rational state-space systems: intelligent search method and momentum method. Nonlinear Dyn. 101(1), 361–373 (2020)

    Article  Google Scholar 

  11. S.S. Cheng, Study on Fractional Order LMS Adaptive Filtering Algorithm (University of Science and Technology of China, Anhui, 2018)

    Google Scholar 

  12. F. Ding, H.B. Chen, L. Xu, J.Y. Dai, Q.S. Li, T. Hayat, A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation. J. Frankl. Inst. 355(8), 3737–3752 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. N. Doelman, The minimum of the time-delay wavefront error in adaptive optics. Mon. Notices R. Astron. Soc. 491(4), 4719–4723 (2019)

    Google Scholar 

  14. R.Q. Dong, Y. Zhang, A.G. Wu, Weighted hierarchical stochastic gradient identification algorithms for ARX models. Int. J. Syst. Sci. 52(2), 363–373 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  15. J.X. Feng, D. Lu, Stochastic gradient-based particle filtering method for ARX models with nonlinear communication output sub-model. Int. J. Model. Identif. Control 31(4), 331–336 (2019)

    Article  Google Scholar 

  16. M. Gan, Y. Guan, G.Y. Chen, C.P. Chen, Recursive variable projection algorithm for a class of separable nonlinear models. IEEE Trans. Neural Netw. Learn. Syst. 1–12 (2020). https://doi.org/10.1109/TNNLS.2020.3026482

  17. M. Gan, C.L.P. Chen, G.Y. Chen, L. Chen, On some separated algorithms for separable nonlinear squares problems. IEEE Trans. Cybern. 48(10), 2866–2874 (2018)

    Article  Google Scholar 

  18. P. Gao, Y. Gao, Quadrilateral interval Type-2 fuzzy regression analysis for data outlier detection. Math. Probl. Eng. 2019, 4914593 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  19. Y. Gao, J. Liu, Z. Wang, L. Wu, Interval type-2 FNN-based quantized tracking control for hypersonic flight vehicles with prescribed performance. IEEE Trans. Syst. Man Cybern. Syst. 51(3), 1–13 (2019)

    Google Scholar 

  20. J. Gmez, H. Pinedo, C. Uzctegui, The open mapping principle for partial actions of Polish groups. J. Math. Anal. Appl. 462(1), 337–346 (2018)

    Article  MathSciNet  Google Scholar 

  21. T.T. Hartley, C.F. Lorenzo, Fractional-order system identification based on continuous order-distributions. Signal Process. 83(11), 2287–2300 (2003)

    Article  MATH  Google Scholar 

  22. M.A. Henson, D.E. Seborg, Time delay compensation for nonlinear processes. Ind. Eng. Chem. Res. 33(6), 1493–1500 (1994)

    Article  Google Scholar 

  23. M. Jiao, D. Wang, Y. Yang, F. Liu, More intelligent and robust estimation of battery state-of-charge with an improved regularized extreme learning machine. Eng. Appl. Artif. Intell. 104, 104407 (2021)

    Article  Google Scholar 

  24. M. Jiao, D. Wang, J. Qiu, A GRU-RNN based momentum optimized algorithm for SOC estimation. J. Power Sources 459, 228051 (2020)

    Article  Google Scholar 

  25. Z.A. Khan, S. Zubair, H. Alquhayz, M. Azeem, A. Ditta, Design of momentum fractional stochastic gradient descent for recommender systems. IEEE Access 7, 179575–179590 (2019)

    Article  Google Scholar 

  26. A.M. Khan, R.K. Kumbhat, A. Chouhan, A. Alaria, Generalized fractional integral operators and M-series. J. Math. 2016, 1–10 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  27. M.H. Li, X.M. Liu, F. Ding, Filtering-based maximum likelihood gradient iterative estimation algorithm for bilinear systems with autoregressive moving average noise. Circuits Syst. Signal Process. 37(11), 5023–5048 (2018)

    Article  MathSciNet  Google Scholar 

  28. P. Ma, F. Ding, A. Alsaedi, T. Hayat, Decomposition-based gradient estimation algorithms for multivariate equation-error autoregressive systems using the multi-innovation theory. Circuits Syst. Signal Process. 37(5), 1846–1862 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  29. J.G. Milton, Time delays and the control of biological systems: an overview. IFAC-Pap. OnLine 48(12), 87–92 (2015)

    Article  Google Scholar 

  30. D. Needell, N. Srebro, R. Ward, Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm. Math. Program. 155(1), 549–573 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  31. K. Owolabi, Riemann–Liouville fractional derivative and application to model chaotic differential equations. Prog. Fract. Differ. Appl. 4, 99–110 (2018)

    Article  Google Scholar 

  32. X.Y. Peng, L. Li, F.Y. Wang, Accelerating minibatch stochastic gradient descent using typicality sampling. IEEE Trans. Neural Netw. Learn. Syst. 31(11), 4649–4659 (2020)

    Article  MathSciNet  Google Scholar 

  33. R. Scherer, S.L. Kalla, Y.F. Tang, J.F. Huang, The Grunwald–Letnikov method for fractional differential equations. Comput. Math. with Appl. 62(3), 902–917 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. J. Schoukens, L. Ljung, Nonlinear system identification: a user-oriented road map. IEEE Control Syst. Mag. 39(6), 28–99 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  35. M.D.L. Sen, About robust stability of Caputo linear fractional dynamic systems with time delays through fixed point theory. Fixed Point Theory Appl. 2011(1), 867932 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  36. M.D.L. Sen, N.S. Luo, Discretization and FIR filtering of continuous linear systems with internal and external point delays. Int. J. Control 60(6), 1223–1246 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  37. S.M. Shah, R. Samar, N.M. Khan, M.A.Z. Raja, Fractional-order adaptive signal processing strategies for active noise control systems. Nonlinear Dyn. 85(3), 1363–1376 (2016)

    Article  MathSciNet  Google Scholar 

  38. C. Song, Y.D. Wang, B.Q. Xu, Research on time delay of high-speed EMU network control system based on BP neural network. Technol. Innov. Appl. 9, 1–5 (2020)

    Google Scholar 

  39. Y. Tan, Z. He, B. Tian, A novel generalization of modified LMS algorithm to fractional order. IEEE Signal Process. Lett. 22(9), 1244–1248 (2015)

    Article  Google Scholar 

  40. L.J. Wan, F. Ding, Decomposition-and gradient-based iterative identification algorithms for multivariable systems using the multi-innovation theory. Circuits Syst. Signal Process. 38(7), 2971–2991 (2019)

    Article  Google Scholar 

  41. D.Q. Wang, Q.H. Fan, Y. Ma, An interactive maximum likelihood estimation method for multivariable Hammerstein systems. J. Frankl. Inst. 357, 12986–13005 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  42. D.Q. Wang, L.W. Li, Y. Ji, Y.R. Yan, Model recovery for Hammerstein systems using the auxiliary model based orthogonal matching pursuit method. Appl. Math. Model. 54, 537–550 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  43. C. Wang, K.C. Li, Aitken-based stochastic gradient algorithm for ARX models with time delay. Circuits Syst. Signal Process. 38(6), 2863–2876 (2019)

    Article  Google Scholar 

  44. J. Wang, W. Luo, J. Liu, L. Wu, Adaptive type-2 FNN-based dynamic sliding mode control of DC–DC boost converters. IEEE Trans. Syst. Man Cybern. Syst. 99, 1–12 (2019)

    Google Scholar 

  45. D.Q. Wang, F. Ding, Extended stochastic gradient identification algorithms for Hammerstein–Wiener ARMAX systems. Comput. Math. Appl. 56, 3157–3164 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  46. D.Q. Wang, S. Zhang, M. Gan, J.L. Qiu, A novel EM identification method for Hammerstein systems with missing output data. IEEE Trans. Ind. Inf. 16(4), 2500–2508 (2020)

    Article  Google Scholar 

  47. Y.H. Wei, M. Zhu, C. Peng, Y. Wang, Robust stability criteria for uncertain fractional order systems with time delay. Control Decis. 3, 511–516 (2014)

    Google Scholar 

  48. H.F. Xia, Y.Q. Yang, F. Ding, A. Alsaedi, T. Hayat, Maximum likelihood-based recursive least-squares estimation for multivariable systems using the data filtering technique. Int. J. Syst. Sci. 50(6), 1121–1135 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  49. H. Yan, X.F. Zhang, Adaptive fractional multi-scale edge-preserving decomposition and saliency detection fusion algorithm. ISA Trans. 107, 160–172 (2020)

    Article  Google Scholar 

  50. S. Zhang, D.Q. Wang, Y.R. Yan, Instrumental variable-based OMP identification algorithm for Hammerstein systems. Complexity 2018, 1–10 (2018)

    MATH  Google Scholar 

  51. S.H. Zhang, F. Zheng, X. Li, An attitude algorithm based on variable-step-size momentum gradient descent method. Electron. Opt. Control 27(9), 66–70 (2020)

    Google Scholar 

  52. Y.T. Zhao, Z.Y. Dan, H.F. Long, H.P. Liu, X.C. Hao, Dynamic soft measurement modeling method of T-LSSVR in systems with time delay. Acta Metrological Sinica 40(1), 146–152 (2019)

    Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to the editors and anonymous reviewers for their helpful comments and constructive suggestions regarding the revision of this paper.

Author information

Authors and Affiliations

  1. School of Science, Jiangnan University, Wuxi, 214122, People’s Republic of China

    Tianyang Xu, Jing Chen, Yan Pu & Liuxiao Guo

  2. School of Management and Engineering, Nanjing University, Nanjing, 210093, People’s Republic of China

    Tianyang Xu

Authors
  1. Tianyang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liuxiao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (No. 61973137), the Fundamental Research Funds for the Central Universities (No. JUSRP22016) and the Funds of the Science and Technology on Near-Surface Detection Laboratory (No. TCGZ2019A001)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, T., Chen, J., Pu, Y. et al. Fractional-Based Stochastic Gradient Algorithms for Time-Delayed ARX Models. Circuits Syst Signal Process 41, 1895–1912 (2022). https://doi.org/10.1007/s00034-021-01874-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-021-01874-8

Keywords

Search

Navigation