Skip to main content
SpringerLink
Log in

Regularized TLS-EM for estimating missing data

  • Published:
Computational and Applied Mathematics Aims and scope Submit manuscript

Abstract

The expectation-maximization (EM) algorithm is a widely-used statistical computation method when dealing with incomplete-data problems. Estimating missing values is one of its common applications. Based on the regression method, the total least squares (TLS) are utilized as an alternative method of original least squares. The TLS method considers errors from both explanatory variables and response variables and performs better optimization results, in order to have better estimations. We also illuminate that the optimal function of total least squares (TLS) could be proved to be the spectral norm instead of the Frobenius norm, in certain conditions. Besides, the new algorithm will include \(L_1\) regularization for more sparsity so that the redundant explanatory variables can be recognized. Finally, we show the effect of the new algorithm, with a series of numerical examples demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

All datasets are publicly available.

References

  • Beck A, Ben-Tal A, Teboulle M (2006) Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J Matrix Anal Appl 28:425–445

    Article  MathSciNet  Google Scholar 

  • Buck SF (1960) A method of estimation of missing values in multivariate data suitable for use with an electronic computer. J R Stat Soc Ser B 22:302–306

    MathSciNet  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–22

    MathSciNet  Google Scholar 

  • Diao H, Wei Y, Xie P (2017) Small sample statistical condition estimation for the total least squares problem. Numer Algorithms 75:435–455

    Article  MathSciNet  Google Scholar 

  • Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32:407–499

    Article  MathSciNet  Google Scholar 

  • Fierro RD, Golub GH, Hansen PC, O’Leary DP (1997) Regularization by truncated total least squares. SIAM J Sci Comput 18:1223–1241

    Article  MathSciNet  Google Scholar 

  • Friedman JH, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33:1–22

    Article  PubMed  PubMed Central  Google Scholar 

  • Golub GH, Van Loan CF (1980) An analysis of the total least squares problem. SIAM J Numer Anal 17:883–893

    Article  MathSciNet  Google Scholar 

  • Golub GH, Van Loan CF (2013) Matrix computations, 4th edn. JHU Press, Baltimore

    Book  Google Scholar 

  • Golub GH, Hansen PC, O’Leary DP (1999) Tikhonov regularization and total least squares. SIAM J Matrix Anal Appl 21:185–194

    Article  MathSciNet  Google Scholar 

  • Han F, Wei Y (2022) TLS-EM algorithm of mixture density models for exponential families. J Comput Appl Math 403:113829

    Article  MathSciNet  Google Scholar 

  • Han F, Wei Y (2022) Tensor regularized total least squares methods with applications to image and video deblurring, arXiv preprint arXiv:1048550. arXiv:2211.00217

  • Han F, Miao Y, Sun Z, Wei Y (2023) T-ADAF: Adaptive data augmentation framework for image classification network based on tensor t-product operator. Neural Process Lett 55:10993–11016

    Article  Google Scholar 

  • Hastie T, Tibshirani R, Wainwright M (2015) Statistical learning with sparsity: the Lasso and generalizations. CRC Press, New York

    Book  Google Scholar 

  • Healy M, Westmacott M (1956) Missing values in experiments analysed on automatic computers. J R Stat Soc Ser C 5:203–206

    Google Scholar 

  • Lampe J, Voss H (2007) On a quadratic eigenproblem occurring in regularized total least squares. Comput Stat Data Anal 52:1090–1102

    Article  MathSciNet  Google Scholar 

  • Little RJA, Rubin DB (2019) Statistical analysis with missing data. Wiley, New York

    Google Scholar 

  • Liu Q, Jia Z, Wei Y (2022) Multidimensional total least squares problem with linear equality constraints. SIAM J Matrix Ana Appl 43:124–150

    Article  MathSciNet  Google Scholar 

  • Liu Q, Li C, Wei Y (2022) Condition numbers of multidimensional mixed least squares-total least squares problems. Appl Numer Math 178:52–68

    Article  MathSciNet  Google Scholar 

  • McLachlan GJ, Krishnan T (2007) The EM algorithm and extensions. Wiley, New York

    Google Scholar 

  • Meng L, Zheng B, Wei Y (2020) Condition numbers of the multidimensional total least squares problems having more than one solution. Numer Algorithms 84:887–908

    Article  MathSciNet  Google Scholar 

  • Pearson FK (1901) LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci 2:559–572

    Article  Google Scholar 

  • Pešta M (2008) Total least squares approach in regression methods. In: WDS 08 proceedings of contributed papers: part I–mathematics and computer sciences

  • Sima DM, Van Huffel S, Golub G (2004) Regularized total least squares based on quadratic eigenvalue problem solvers. BIT Numer Math 44:793–812

    Article  MathSciNet  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Stat Methodol 58:267–288

    MathSciNet  Google Scholar 

  • Van Huffel S, Vandewalle J (1991) The total least squares problem. Society for Industrial and Applied Mathematics, Philadelphia

    Book  Google Scholar 

  • Van Huffel S, Lemmerling P (2002) Total least squares and errors-in-variables modeling. Kluwer Academic Publishers, Dordrecht

    Book  Google Scholar 

  • Wu CFJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11:95–103

    Article  MathSciNet  Google Scholar 

  • Xie P, Xiang H, Wei Y (2017) A contribution to perturbation analysis for total least squares problems. Numer Algorithms 75:381–395

    Article  MathSciNet  Google Scholar 

  • Xu W, Qiao S, Wei Y (2008) A note on the scaled total least squares problem. Linear Algebra Appl 428:469–478

    Article  MathSciNet  Google Scholar 

  • Zheng B, Meng L, Wei Y (2017) Condition numbers of the multidimensional total least squares problem. SIAM J Matrix Anal Appl 38:924–948

    Article  MathSciNet  Google Scholar 

  • Zhou L, Lin L, Wei Y, Qiao S (2009) Perturbation analysis and condition numbers of scaled total least squares problems. Numer Algorithms 51:381–399

    Article  MathSciNet  ADS  Google Scholar 

  • Zhu H, Leus G, Giannakis GB (2010) Sparse regularized total least squares for sensing applications. In: 2010 IEEE 11th international workshop on signal processing advances in wireless communications (SPAWC)

  • Zhu H, Leus G, Giannakis GB (2011) Sparsity-cognizant total least-squares for perturbed compressive sampling. IEEE Trans Signal Process 59:2002–2016

    Article  MathSciNet  ADS  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the managing editor and three referees for their detailed comemnts. W. Fan is supported by the Ministry of Science and Technology of China under Grant G2023132005L. F. Han is supported by the National Natural Science Foundation of China under Grant 12271108 and Shanghai Municipal Science and Technology Commission under Grant 23WZ2501400. Y. Wei is supported by the National Natural Science Foundation of China under Grant 12271108 and Innovation Program of Shanghai Municipal Education Commission.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Fudan University, Shanghai, 200433, People’s Republic of China

    Wenkai Fan & Feiyang Han

  2. Shanghai Key Laboratory of Contemporary Applied Mathematics, School of Mathematical Sciences, Fudan University, Shanghai, 200433, People’s Republic of China

    Yimin Wei

Authors
  1. Wenkai Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feiyang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yimin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yimin Wei.

Ethics declarations

Conflict of interest.

The authors have not disclosed any competing interests.

Additional information

Communicated by Jinyun Yuan.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, W., Han, F. & Wei, Y. Regularized TLS-EM for estimating missing data. Comp. Appl. Math. 43, 43 (2024). https://doi.org/10.1007/s40314-023-02572-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40314-023-02572-8

Keywords

Mathematics Subject Classification

Search

Navigation