Abstract
The design of decentralized learning algorithms is important in the fast-growing world in which data are distributed over participants with limited local computation resources and communication. In this direction, we propose an online algorithm minimizing non-convex loss functions aggregated from individual data/models distributed over a network. We provide the theoretical performance guarantee of our algorithm and demonstrate its utility on a real life smart building.
Supported by the Multidisciplinary Institute in Artificial Intelligence, Univ.Grenoble Alpes, France (ANR-19-P3IA-0003).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdel-Aziz, H., Koutsoukos, X.: Data-driven online learning and reachability analysis of stochastic hybrid systems for smart buildings. Cyber Phys. Syst. 5(1), 41–64 (2019)
Cai, M., Pipattanasomporn, M., Rahman, S.: Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Appl. Energy 236, 1078–1088 (2019)
Chen, L., Hassani, H., Karbasi, A.: Online continuous submodular maximization. In: Proceedings 21st International Conference on Artificial Intelligence and Statistics (AISTAT) (2018)
Chiliang, Z., Tao, H., Yingda, G., Zuochang, Y.: Accelerating convolutional neural networks with dynamic channel pruning. In: 2019 Data Compression Conference (DCC), p. 563 (2019)
Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)
Gupta, C., et al.: ProtoNN: Compressed and accurate kNN for resource-scarce devices. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1331–1340. PMLR (2017)
Gupta, S.K., Kar, K., Mishra, S., Wen, J.T.: Distributed consensus algorithms for collaborative temperature control in smart buildings. In: 2015 American Control Conference (ACC), pp. 5758–5763. IEEE (2015)
Hazan, E.: Introduction to online convex optimization. Found. Trends Optim. 2(3–4), 157–325 (2016)
He, L., Bian, A., Jaggi, M.: Cola: decentralized linear learning. In: Advances in Neural Information Processing Systems, pp. 4536–4546 (2018)
Hosseini, S., Chapman, A., Mesbahi, M.: Online distributed optimization via dual averaging. In: 52nd IEEE Conference on Decision and Control, pp. 1484–1489 (2013)
Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. In: Proceedings of the 30th International Conference on Machine Learning (2013)
Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., et al.: Advances and Open Problems in Federated Learning. Foundations and Trends® in Machine Learning 14(1), (2021)
Lian, X., Zhang, C., Zhang, H., Hsieh, C.J., Zhang, W., Liu, J.: Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In: Advances in Neural Information Processing Systems, pp. 5330–5340 (2017)
Lin, J., Rao, Y., Lu, J., Zhou, J.: Runtime neural pruning. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)
MacCarthy, M.: In defense of big data analytics. The Cambridge Handbook of Consumer Privacy, pp. 47–78 (2018)
McMahan, B., Ramage, D.: Collaborative machine learning without centralized training data. Google Research Blog 3 (2017)
Mitra, A., Thang, N.K., Nguyen, T.A., Trystram, D., Youssef, P.: Online Decentralized Frank-Wolfe: from theoretical bound to applications in smart-building (2022)
Nan, F., Wang, J., Saligrama, V.: Pruning random forests for prediction on a budget. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol, 29. Curran Associates, Inc. (2016)
Pipattanasomporn, M., et al.: Cu-bems, smart building electricity consumption and indoor environmental sensor datasets. Sci. Data 7, (2020)
Shotton, J., Sharp, T., Kohli, P., Nowozin, S., Winn, J., Criminisi, A.: Decision jungles: compact and rich models for classification. In: Burges, C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 26. Curran Associates, Inc. (2013)
Simons, T., Lee, D.J.: A review of binarized neural networks. Electronics 8(6), 661 (2019)
Wai, H., Lafond, J., Scaglione, A., Moulines, E.: Decentralized frank-wolfe algorithm for convex and nonconvex problems. IEEE Trans. Autom. Control 62(11), 5522–5537 (2017)
Wang, S., et al.: When edge meets learning: Adaptive control for resource-constrained distributed machine learning. In: IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, pp. 63–71 (2018)
Yan, F., Sundaram, S., Vishwanathan, S.V.N., Qi, Y.: Distributed autonomous online learning: regrets and intrinsic privacy-preserving properties. IEEE Trans. Knowl. Data Eng. 25(11), 2483–2493 (2013)
Zamora-Martinez, F., Romeu, P., Botella-Rocamora, P., Pardo, J.: On-line learning of indoor temperature forecasting models towards energy efficiency. Energy. Build. 83, 162–172 (2014)
Zhang, W., Zhao, P., Zhu, W., Hoi, S., Zhang, T.: Projection-free distributed online learning in networks. In: Proceedings of the 34th International Conference on Machine Learning, pp. 4054–4062 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mitra, A., Thang, N.K., Nguyen, TA., Trystram, D., Youssef, P. (2022). Online Decentralized Frank-Wolfe: From Theoretical Bound to Applications in Smart-Building. In: González-Vidal, A., Mohamed Abdelgawad, A., Sabir, E., Ziegler, S., Ladid, L. (eds) Internet of Things. GIoTS 2022. Lecture Notes in Computer Science, vol 13533. Springer, Cham. https://doi.org/10.1007/978-3-031-20936-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-20936-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20935-2
Online ISBN: 978-3-031-20936-9
eBook Packages: Computer ScienceComputer Science (R0)