Abstract
Data missing is a common problem in multi-modal fusion, and existing incomplete multi-modal methods usually only consider the case of two modalities and ignore the semantic information of samples during data recovery. In this paper, we propose dictionary-induced manifold incomplete multi-modal latent space representation, which reconstructs missing views with dictionary to assist consensus representation and captures the local manifold structure with reverse graph regularization. Specifically, we adopt dictionary learning to recover missing data with linear combinations of available samples for latent space alignment, and Laplacian matrix is utilized to embed the structural information of the high-dimensional space into the low-dimensional manifold latent space for optimizing the common representation. The proposed method can not only deal with multi-modal data fusion task, but also recovering missing data by effectively mining the structural information among different modalities. Experimental results demonstrate that our method performs better than other incomplete multi-modal fusion methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhu, Q., Yang, J., Wang, S., Zhang, D., Zhang, Z.: Multi-modal non-euclidean brain network analysis with community detection and convolutional autoencoder. IEEE Trans. Emerg. Topics Comput. Intell, 1–11 (2022)
Li, S.Y., Jiang, Y., Zhou, Z.H.: Partial multi-view clustering. In: Proceedings of the AAAI Conference On Artificial Intelligence, vol. 28 (2014)
Yin, Q., Wu, S., Wang, L.: Unified subspace learning for incomplete and unlabeled multi-view data. Pattern Recogn. 67, 313–327 (2017)
Wang, Q., Lian, H., Sun, G., Gao, Q., Jiao, L.: icmsc: Incomplete cross-modal subspace clustering. IEEE Trans. Image Process. 30, 305–317 (2021)
He, J., Han, X.: Efficient skyline computation on massive incomplete data. Data Sci. Eng. 7(2), 102–119 (2022)
Pan, Y., Liu, M., Lian, C., Xia, Y., Shen, D.: Disease-image specific generative adversarial network for brain disease diagnosis with incomplete multi-modal neuroimages. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 137–145. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_16
Hu, D., et al.: Disentangled-multimodal adversarial autoencoder: Application to infant age prediction with incomplete multimodal neuroimages. IEEE Trans. Med. Imaging 39(12), 4137–4149 (2020)
Enders, C.K.: Applied missing data analysis. Guilford press (2010)
Wen, J., Zhang, Z., Xu, Y., Zhang, B., Fei, L., Liu, H.: Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In: Proceedings of the AAAI Conference On Artificial Intelligence, vol. 33, pp. 5393–5400 (2019)
Zhang, D., Wang, Y., Zhou, L., Yuan, H., Shen, D., Initiative, A.D.N., et al.: Multimodal classification of alzheimer’s disease and mild cognitive impairment. Neuroimage 55(3), 856–867 (2011)
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (Nos. 61732006, 62076129, 61501230, 62136004 and 61876082), National Science and Technology Major Project (No. 018ZX10201002), and the National Key R &D Program of China (Grant Nos.: 2018YFC2001600, 2018YFC2001602).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, B., Ye, H., Zhang, Z., Zhang, D., Zhu, Q. (2023). Dictionary-Induced Manifold Learning for Incomplete Multi-modal Fusion. In: Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., Amagasa, T. (eds) Web and Big Data. APWeb-WAIM 2022. Lecture Notes in Computer Science, vol 13422. Springer, Cham. https://doi.org/10.1007/978-3-031-25198-6_41
Download citation
DOI: https://doi.org/10.1007/978-3-031-25198-6_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25197-9
Online ISBN: 978-3-031-25198-6
eBook Packages: Computer ScienceComputer Science (R0)