Abstract
In practical pattern recognition, e.g., image classification or recognition, the problem of missing modality, i.e., new patterns never trained by a learner pop up, can cause a dramatic decrease on the recognition accuracy. Existing algorithms as few-shot learning (FSL) and zero-shot learning (ZSL) have not sufficiently used information from other users or clients. If patterns or knowledge from other sources can be utilized as much as possible, the damage of missing modality is expected to be reduced. Privacy protection must be considered when trying to fetch information from other users. Motivated by these, an enhanced federated learning with linear discriminant analysis (LDA) is developed here. The data of each user is regarded as a client, and the features of each client are first extracted with neural network-based classification. These features are uploaded to the central server and then aggregated with LDA as a central classification to possibly achieve all patterns’ features. The trained LDA is then downloaded to the client to fulfill the pattern recognition. The proposed algorithm is applied to an image classification, and the experimental results demonstrate its efficiency in dealing with pattern recognition with missing modality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jia, D., Wei, D., Socher, R., Li, L.J., Kai, L., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE Computer Vision & Pattern Recognition, pp. 248–255 (2009)
Abdel-Hamid, O., Mohamed, A.R., Jiang, H., Deng, L., Penn, G., Yu, D.: Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. (TASLP) 22(10), 1533–1545 (2014)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)
He, X., Liu, Q., Yang, Y.: MV-GNN: multi-view graph neural network for compression artifacts reduction. IEEE Trans. Image Process. 29, 6829–6840 (2020). https://doi.org/10.1109/TIP.2020.2994412
Wang, F.Y., Zhang, J.J., Zheng, X., Xiao, W., Yang, L.: Where does AlphaGo go: from church-turing thesis to AlphaGo thesis and beyond. Acta Automatica Sinica 3(2), 113–120 (2016)
Wang, W., Zheng, V.W., Yu, H., Miao, C.: A survey of zero-shot learning: settings, methods, and applications. ACM Trans. Intell. Syst. 10(2), 13.11–13.37 (2019)
Long, Y., Liu, L., Shao, L., Shen, F., Ding, G., Han, J.: From zero-shot learning to conventional supervised classification: unseen visual data synthesis, pp. 6165–6174. IEEE Computer Society (2017)
Ke, S.R., Hoang, T., Yong-Jin, L., Jenq-Neng, H., Jang-Hee, Y., Kyoung-Ho, C.: A review on video-based human activity recognition. Computers 2(2), 88–131 (2013)
Gan, C., Yang, Y., Zhu, L., Zhao, D., Zhuang, Y.: Recognizing an action using its name: a knowledge-based approach. Int. J. Comput. Vision 120(1), 61–77 (2016). https://doi.org/10.1007/s11263-016-0893-6
Tsai, R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (2003)
Berger, A.L.: A maximum entropy approach to natural language processing. Comput. Linguist. 22(1), 39–71 (1996)
Teller, V.: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Daniel Jurafsky and James H. Martin (University of Colorado, Boulder) Prentice Hall, Upper Saddle River. Prentice Hall ser’, Computational Linguistics 26(4), 638–641 (2000)
Satyanarayanan, M.: Pervasive computing: vision and challenges. IEEE Pers. Commun. 8(4), 10–17 (2002)
Mcmahan, H.B., Moore, E., Ramage, D., Hampson, S., Arcas, B.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2017)
Konen, J., Mcmahan, B., Ramage, D.: Federated Optimization: Distributed Optimization Beyond the Datacenter. Mathematics, pp. 1–5 (2015)
Brisimi, T.S., Chen, R., Mela, T., Olshevsky, A., Ioannis, C., Paschalidis, W.S.: Federated learning of predictive models from federated Electronic Health Records. Int. J. Med. Inform. 112, 59–67 (2018)
Hu, Y., Sun, X., Chen, Y., Lu, Z.: Model and feature aggregation based federated learning for multi-sensor time series trend following. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2019. LNCS, vol. 11506, pp. 233–246. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20521-8_20
Altae-Tran, H., Ramsundar, B., Pappu, A.S., Pande, V.: Low data drug discovery with one-shot learning. In: Book Low Data Drug Discovery with One-Shot Learning, pp. 263–274 (2016)
Kuznetsova, A., Hwang, S.J., Rosenhahn, B., Sigal, L.: Exploiting view-specific appearance similarities across classes for zero-shot pose prediction: a metric learning approach. In: Proceedings of Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 3523–3529 (2016)
Yoo, D., Fan, H., Boddeti, V.N., Kitani, K.M.: Efficient k-shot learning with regularized deep networks. In: Book Efficient K-Shot Learning with Regularized Deep Networks, arXiv:1710.02277 (2017)
Park, M., Kim, J., Kim, S., Liu, Y., Choi, S.: MxML: mixture of meta-learners for few-shot classification. In: Book MxML: Mixture of Meta-Learners for Few-Shot Classification, arXiv:1904.05658 (2019)
Acknowledgement
This work is partially supported by National Natural Science Foundation of China with Grant No. 61876184.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, X., Wang, X. (2021). LDA-Enhanced Federated Learning for Image Classification with Missing Modality. In: Zhang, H., Yang, Z., Zhang, Z., Wu, Z., Hao, T. (eds) Neural Computing for Advanced Applications. NCAA 2021. Communications in Computer and Information Science, vol 1449. Springer, Singapore. https://doi.org/10.1007/978-981-16-5188-5_17
Download citation
DOI: https://doi.org/10.1007/978-981-16-5188-5_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5187-8
Online ISBN: 978-981-16-5188-5
eBook Packages: Computer ScienceComputer Science (R0)