Skip to main content
Log in

Convergence analysis of the mixed-norm LMS and two versions for sparse system identification

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The previously proposed mixed-norm least-mean-square (MN-LMS) algorithm has shown outstanding performance compared to that of the conventional LMS algorithm. In this paper, the convergence analysis of the MN-LMS algorithm is derived. Based on that, two algorithms that exploit the sparsity of the system have been derived. The first algorithm is proposed by adding \(l_{1}\)-norm penalty to the cost function of the MN-LMS algorithms. This term enables us to attract the zero and/or near-to-zero filter coefficients to the zero value faster. However, when the system is near or exactly non-sparse, the algorithm almost fails. To overcome this limitation, we propose another algorithm that uses an approximation of \(l_{0}\)-norm penalty term in the cost function of the MN-LMS algorithm. This provides high performance even with completely non-sparse systems. The performances of the proposed algorithms are compared to those of the LMS and MN-LMS algorithms in an acoustic sparse system identification setting. The proposed algorithms provide significant performances compared to the other algorithms under different sparsities and signal-to-noise ratios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Haykin, S.: Adaptive Filter Theory, 4th edn. Prentice Hall, Upper Saddle River (2002)

    MATH  Google Scholar 

  2. Widrow, B., Stearns, S.D.: Adaptive Signal Processing. Prentice Hall, Upper Saddle River (1985)

    MATH  Google Scholar 

  3. Choi, Y. S.: A new subband adaptive filtering algorithm for sparse system identification with impulsive noise. J. Appl. Math. 2014, Article ID 704231, 7 (2014)

  4. Liu, J., Grant, S.L.: Proportionate adaptive filtering for block-sparse system identification. IEEE/ACM Trans. Audio Speech Lang. Process. 24(4), 623–630 (2016)

    Article  Google Scholar 

  5. Wei, Y., Zhang, Y., Wang, C., Zheng, Y.R.: A mixed (2, pp-like)-norm penalized least mean squares algorithm for block-sparse system identification. Circ. Syst. Signal Process. 37(10), 4683–4694 (2019)

    Article  Google Scholar 

  6. Rombouts, G.: Adaptive Filtering Algorithms for Acoustic Echo and Noise Cancellation. Thesis, Catholic University of Leuven (2003)

  7. Ni, J., Li, F.: Adaptive combination of subband adaptive filters for acoustic echo cancellation. IEEE Trans. Consum. Electron. 56(3), 1549–1555 (2010)

    Article  Google Scholar 

  8. Abutaleb, A.S.: An adaptive filter for noise cancelling. IEEE Trans. Circ. Syst. 35(10), 1201–1209 (1988)

    Article  MathSciNet  Google Scholar 

  9. Hamidia, M., Amrouche, A.: Improved variable step-size NLMS adaptive filtering algorithm for acoustic echo cancellation. Digit. Signal Process. 49, 44–55 (2016)

    Article  Google Scholar 

  10. Morgan, D.R., Craig, S.: Real-time adaptive linear prediction using the least mean square gradient algorithm. IEEE Trans. Acoust. Speech Signal Process. 24(6), 494–507 (1976)

    Article  Google Scholar 

  11. Mader, A., Puder, H., Schmidt, G.U.: Step-size control for acoustic echo cancellation filters-an overview. Signal Process. 80, 1697–1719 (2000)

    Article  Google Scholar 

  12. Tianhui, W., Liyuan, M., Yongjun, L. Yonggang, D.: An improved variable step size LMS adaptive spectral-line enhancement algorithm and its simulation. In: Proceedings of the International Conference on Pervasive Computing, Signal Processing and Applications (PCSPA 2010), pp. 727–730 (2010)

  13. Taheri, O., Vorobyov, S.A.: Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis. Signal Process. 104, 70–79 (2014)

    Article  Google Scholar 

  14. Su, G., Jin, J., Gu, Y., Wang, J.: Performance analysis of \(l_{0}\) norm constraint least mean square algorithm. IEEE Trans. Signal Process. 60(5), 2223–2235 (2012)

    Article  MathSciNet  Google Scholar 

  15. Chen, Y., Gu, Y., Hero, A.O.: Sparse LMS for system identification. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2009), pp. 3125–3128 (2009)

  16. Das, B.K., Chakraborty, M., Banerjee, S.: Adaptive identification of sparse systems with variable sparsity. In: IEEE International Symposium on Circuits and Systems (ISCAS2011), pp. 1267–1270 (2011)

  17. Wu, F.Y., Tong, F.: Gradient optimization p-norm-like constraint LMS algorithm for sparse system estimation. Signal Process. 93(4), 967–971 (2013)

    Article  Google Scholar 

  18. Jin, J., Qu, Q., Gu, Y.: Robust zero-point attraction least mean square algorithm on near sparse system identification. IET Signal Process. 7(3), 210–218 (2013)

    Article  MathSciNet  Google Scholar 

  19. Chambers, J., Avlonitis, A.: A robust mixed-norm adaptive filter algorithm. IEEE Signal Process. Lett. 4(2), 46–48 (1997)

    Article  Google Scholar 

  20. Boukis, C.,Mandic, D.P., Constantinides, A.G.: A generalized mixed norm stochastic gradient algorithm. In: 15th International Conference on Digital Signal Processing (ICDSP2007), pp. 27–30 (2007)

  21. Yu, Y., Zhao, H., de Lamare, R.C., Lu, L.: Sparsity-aware subband adaptive algorithms with adjustable penalties. Digit. Signal Process. 84, 93–106 (2019)

    Article  MathSciNet  Google Scholar 

  22. Gogineni, V.C., Mula, S.: Improved proportionate-type sparse adaptive filtering under maximum correntropy criterion in impulsive noise environments. Digit. Signal Process. 79, 190–198 (2018)

    Article  MathSciNet  Google Scholar 

  23. Delic, H., Hocanin, A.: Robust detection in DS/CDMA. IEEE Trans. Veh. Technol. 51(1), 155–170 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gülden Eleyan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eleyan, G., Salman, M.S. Convergence analysis of the mixed-norm LMS and two versions for sparse system identification. SIViP 14, 965–970 (2020). https://doi.org/10.1007/s11760-019-01628-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01628-9

Keywords

Navigation