Abstract
In this paper, distributed regression estimation problem with incomplete data in a time-varying multi-agent network is investigated. Regression estimation is carried out based on local agent information with incomplete in the non-ignorable mechanism. By virtue of gradient-based design and adaptive filter, a distributed algorithm is proposed to deal with a regression estimation problem with incomplete data. With the help of convex analysis and stochastic approximation techniques, the exact convergence is obtained for the proposed algorithm with incomplete data and a jointly-connected multi-agent topology. Moreover, online regret analysis is also given for real-time learning. Then, simulations for the proposed algorithm are also given to demonstrate how it can solve the estimation problem in a distributed way, even when the network configuration is time-varying.
Similar content being viewed by others
References
Nedić A, Ozdaglar A. Distributed subgradient methods for multi-agent optimization. IEEE Trans Automat Control, 2009, 54: 48–61
Shi G, Johansson K. Robust consensus for continuous-time multiagent dynamics. SIAM J Control Optim, 2013, 51: 3673–3691
Zhang Y Q, Lou Y C, Hong Y G, et al. Distributed projection-based algorithms for source localization in wireless sensor networks. IEEE Trans Wirel Commun, 2015, 43: 3131–3142
Feng H, Jiang Z D, Hu B, et al. The incremental subgradient methods on distributed estimations in-network. Sci China Inf Sci, 2014, 57: 092103
Lou Y C, Hong Y G, Wang S Y. Distributed continuous-time approximate projection protocols for shortest distance optimization problems. Automatica, 2016, 69: 289–297
Yi P, Hong Y G, Liu F. Initialization-free distributed algorithms for optimal resource allocation with feasibility con-straints and application to economic dispatch of power systems. Automatica, 2016, 74: 259–269
Kokaram A C. On missing data treatment for degraded video and film archives: a survey and a new Bayesian approach. IEEE Trans Image Process, 2004, 13: 397–415
Molenberghs G, Kenward M G. Missing Data in Clinical Studies. New York: Wiley, 2007
Ibrahim J G, Chen M H, Lipsitz S R, et al. Missing data methods for generalized linear models: a comparative review. J Am Stat Assoc, 2005, 100: 332–346
Gholami M R, Jansson M, Strom E G, et al. Diffusion estimation over cooperative multi-agent networks with missing data. IEEE Trans Signal Inf Process Netw, 2016, 2: 276–289
Davey A, Savla J. Statistical Power Analysis with Missing Data: A Structural Equation Modeling Approach. Oxford, UK: Routledge Academic, 2009
Ram S S, Nedić A, Veeravalli V V. Distributed stochastic subgradient projection algorithms for convex optimization. J Optim Theory Appl, 2010, 147: 516–545
Graybill F, Iyer H K. Regression Analysis: Concepts and Applications. California: Duxbury Press Belmont, 1994
Feng Y, Sundaram S, Vishwanathan S V N, et al. Distributed autonomous online learning: regrets and intrinsic privacy-preserving properties. IEEE Trans Knowl Data Eng, 2013, 25: 2483–2493
Hazan E, Kale S. Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimiza-tion. J Mach Learn Res, 2014, 15: 2489–2512
Shamir O, Zhang T. Stochastic gradient descent for non-smooth optimization: convergence results and optimal aver-aging schemes. In: Proceedings of International Conference on Machine Learning, Edinburgh, 2012. 71–79
Towfic Z J, Chen J S, Sayed A H. On distributed online classification in the midst of concept drifts. Neurocomputing, 2013, 112: 138–152
Widrow B, Stearns S D. Adaptive Signal Processing. Cliffs: Prentice-Hall, 1985. 1–32
Sayed A H. Adaptation, learning, and optimization over networks. Found Trends Mach Learn, 2014, 7: 311–801
Sayed A H, Tu S Y, Chen J S, et al. Diffusion strategies for adaptation and learning. IEEE Signal Proc Mag, 2013, 30: 155–171
Polyak B T. Introduction to Optimization. New York: Optimization Software Inc., 1983. 2–8
Godsil C, Royle G. Algebraic Graph Theory. New York: Springer-Verlag, 2001. 1–18
Ferguson T S. A Course in Large Sample Theory. London: Chapman and Hall Ltd., 1996. 3–4
Durrett R. Probability Theory and Examples. Camberidge, UK: Camberidge Press, 2010. 328–347
Enders C K. Applied Missing Data Analysis. New York: The Guilford Press, 2010
Kushner H J, Yin G. Stochastic Approximation and Recursive Algorithms and Applications. New York: Springer-Verlag, 1997. 117–157
Widrow B, Mccool J, Larimore M G, et al. Stationary and nonstationary learning characteristics of the LMS adaptive filter. Proc IEEE, 1976, 64: 1151–1162
Yi P, Hong Y G. Stochastic sub-gradient algorithm for distributed optimization with random sleep scheme. Control Theory Technol, 2015, 13: 333–347
Larsen R J, Max M L. An Introduction to Mathematical Statistics and Its Applications. 4th ed. New York: Pearson, 2006. 221–280
Acknowledgements
This work was supported by National Key Research and Development Program of China (Grant No. 2016YFB0901902) and National Natural Science Foundation of China (Grant Nos. 61573344, 61333001, 61374168).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Y., Lin, P. & Hong, Y. Distributed regression estimation with incomplete data in multi-agent networks. Sci. China Inf. Sci. 61, 092202 (2018). https://doi.org/10.1007/s11432-016-9173-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-016-9173-8