skip to main content
research-article

Incomplete Multiview Clustering via Semidiscrete Optimal Transport for Multimedia Data Mining in IoT

Published: 08 March 2024 Publication History

Abstract

With the wide deployment of the Internet of Things (IoT), large volumes of incomplete multiview data that violates data integrity is generated by various applications, which inevitably produces negative impacts on the quality of service of IoT systems. Incomplete multiview clustering (IMC), as an essential technique of data processing, has the potential for mining patterns of incomplete IoT data. However, previous methods utilize notion-strong distances that can only measure differences between distributions at the overlap of data manifolds in fusing complementary information of data for pattern mining. They may suffer from biased estimation and information loss in capturing intrinsic structures of incomplete multiview data. To address these challenges, a semidiscrete multiview optimal transport (SD-MOT) is defined for IMC, which utilizes distances with weak notions to capture intrinsic structures of incomplete multiview data. Specifically, IMC is recast as an equivalent optimal transport between continuous incomplete multiview data and discrete clustering centroids, to avoid the strict assumption on overlap between manifolds in pattern mining. Then, SD-MOT is instantiated as a deep incomplete contrastive clustering network to remedy biased estimation and information loss on intrinsic structures of incomplete multiview data. Afterwards, a variational solution to SD-MOT is derived to effectively train the network parameters for pattern mining. Finally, extensive experiments on four representative incomplete multiview datasets verify the superiority of SD-MOT in comparison with nine baseline methods.

References

[1]
Deepak Adhikari, Wei Jiang, Jinyu Zhan, Zhiyuan He, Danda B. Rawat, Uwe Aickelin, and Hadi Akbarzade Khorshidi. 2023. A comprehensive survey on imputation of missing data in Internet of Things. ACM Comput. Surv. 55, 7 (2023), 133:1–133:38.
[2]
Kashif Ahmad and Nicola Conci. 2019. How deep features have improved event recognition in multimedia: A survey. ACM Trans. Multim. Comput. Commun. Appl. 15, 2 (2019), 39:1–39:27.
[3]
Mohammed Ali Al-Garadi, Amr Mohamed, Abdulla Khalid Al-Ali, Xiaojiang Du, Ihsan Ali, and Mohsen Guizani. 2020. A survey of machine and deep learning methods for Internet of Things (IoT) security. IEEE Commun. Surv. Tutorials 22, 3 (2020), 1646–1685.
[4]
Xiao Cai, Hua Wang, Heng Huang, and Chris H. Q. Ding. 2012. Joint stage recognition and anatomical annotation of drosophila gene expression patterns. Bioinform. 28, 12 (2012), 16–24.
[5]
Jing Gao, Meng Liu, Peng Li, Jianing Zhang, and Zhikui Chen. 2023. Deep multiview adaptive clustering with semantic invariance. IEEE Trans. Neural Networks Learn. Syst. (2023), 1–14.
[6]
Mohamed Abdel Hameed, M. Hassaballah, Mosa E. Hosney, and Abdullah Alqahtani. 2022. An AI-enabled internet of things based autism care system for improving cognitive ability of children with autism spectrum disorders. Comput. Intell. Neurosci. 2022 (2022), 2247675:1–12.
[7]
Menglei Hu and Songcan Chen. 2018. Doubly aligned incomplete multi-view clustering. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). 2262–2268.
[8]
Menglei Hu and Songcan Chen. 2019. One-pass incomplete multi-view clustering. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19). AAAI Press, 3838–3845.
[9]
Xu Ji, Andrea Vedaldi, and João F. Henriques. 2019. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE, 9864–9873.
[10]
Yu-Gang Jiang, Guangnan Ye, Shih-Fu Chang, Daniel P. W. Ellis, and Alexander C. Loui. 2011. Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In Proceedings of the 1st International Conference on Multimedia Retrieval (ICMR’11). ACM, 29.
[11]
Lusi Li, Zhiqiang Wan, and Haibo He. 2023. Incomplete multi-view clustering with joint partition and graph learning. IEEE Trans. Knowl. Data Eng. 35, 1 (2023), 589–602.
[12]
Miaomiao Li, Jingyuan Xia, Huiying Xu, Qing Liao, Xinzhong Zhu, and Xinwang Liu. 2023. Localized incomplete multiple kernel k-means with matrix-induced regularization. IEEE Trans. Cybern. 53, 6 (2023), 3479–3492.
[13]
Peng Li, Asif Ali Laghari, Mamoon Rashid, Jing Gao, Thippa Reddy Gadekallu, Abdul Rehman Javed, and Shoulin Yin. 2023. A deep multimodal adversarial cycle-consistent network for smart enterprise system. IEEE Trans. Ind. Informatics 19, 1 (2023), 693–702.
[14]
Shao-Yuan Li, Yuan Jiang, and Zhi-Hua Zhou. 2014. Partial multi-view clustering. In Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI’14). AAAI Press, 1968–1974.
[15]
Xingfeng Li, Quansen Sun, Zhenwen Ren, and Yinghui Sun. 2022. Dynamic incomplete multi-view imputing and clustering. In Proceedings of the 30th ACM International Conference on Multimedia. ACM, 3412–3420.
[16]
Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. 2021. COMPLETER: Incomplete multi-view clustering via contrastive prediction. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). Computer Vision Foundation/IEEE, virtual, 11174–11183.
[17]
Ulf Lindqvist and Peter G. Neumann. 2017. The future of the Internet of Things. Commun. ACM 60, 2 (2017), 26–30.
[18]
Cheng Liu, Si Wu, Rui Li, Dazhi Jiang, and Hau-San Wong. 2023. Self-supervised graph completion for incomplete multi-view clustering. IEEE Trans. Knowl. Data Eng. (2023), 1–14.
[19]
Xinwang Liu. 2021. Incomplete multiple kernel alignment maximization for clustering. IEEE Trans. Pattern Anal. Mach. Intell. (2021).
[20]
Xinwang Liu, Miaomiao Li, Chang Tang, Jingyuan Xia, Jian Xiong, Li Liu, Marius Kloft, and En Zhu. 2021. Efficient and effective regularized incomplete multi-view clustering. IEEE Trans. Pattern Anal. Mach. Intell. 43, 8 (2021), 2634–2646.
[21]
Xinwang Liu, Xinzhong Zhu, Miaomiao Li, Chang Tang, En Zhu, Jianping Yin, and Wen Gao. 2019. Efficient and effective incomplete multi-view clustering. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19). AAAI Press, 4392–4399.
[22]
Xinwang Liu, Xinzhong Zhu, Miaomiao Li, Lei Wang, En Zhu, Tongliang Liu, Marius Kloft, Dinggang Shen, Jianping Yin, and Wen Gao. 2020. Multiple kernel k-means with incomplete kernels. IEEE Trans. Pattern Anal. Mach. Intell. 42, 5 (2020), 1191–1204.
[23]
Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. 2019. COMIC: Multi-view clustering without parameter selection. In Proceedings of the 36th International Conference on Machine Learning (ICML’19), Vol. 97. PMLR, 5092–5101.
[24]
Sara Salim, Nour Moustafa, Benjamin P. Turnbull, and Imran Razzak. 2022. Perturbation-enabled deep federated learning for preserving Internet of Things-based social networks. ACM Trans. Multim. Comput. Commun. Appl. 18, 2s (2022), 120:1–120:19.
[25]
Huayi Tang and Yong Liu. 2022. Deep safe incomplete multi-view clustering: Theorem and algorithm. In Proceedings of the 39th International Conference on Machine Learning (ICML’22). PMLR, 21090–21110.
[26]
Rachakonda Venkatesh and N. Malarvizhi. 2023. IOT security and machine learning algorithms: Survey and analysis. In Proceedings of the 7th International Conference on Trends in Electronics and Informatics (ICOEI’23). 444–451.
[27]
Qianqian Wang, Huanhuan Lian, Gan Sun, Quanxue Gao, and Licheng Jiao. 2021. iCmSC: Incomplete cross-modal subspace clustering. IEEE Trans. Image Process. 30 (2021), 305–317.
[28]
Senhong Wang, Jiangzhong Cao, Fangyuan Lei, Jianjian Jiang, Qingyun Dai, and Bingo Wing-Kuen Ling. 2023. Multiple kernel-based anchor graph coupled low-rank tensor learning for incomplete multi-view clustering. Appl. Intell. 53, 4 (2023), 3687–3712.
[29]
Yang Wang. 2021. Survey on deep multi-modal data analytics: Collaboration, rivalry, and fusion. ACM Trans. Multim. Comput. Commun. Appl. 17, 1s (2021), 10:1–10:25.
[30]
Jie Wen, Zheng Zhang, Yong Xu, Bob Zhang, Lunke Fei, and Hong Liu. 2019. Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19). AAAI Press, 5393–5400.
[31]
Jie Wen, Zheng Zhang, Yong Xu, Bob Zhang, Lunke Fei, and Guo-Sen Xie. 2020. CDIMC-net: Cognitive deep incomplete multi-view clustering network. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20). ijcai.org, 3230–3236.
[32]
Jie Xu, Chao Li, Yazhou Ren, Liang Peng, Yujie Mo, Xiaoshuang Shi, and Xiaofeng Zhu. 2022. Deep incomplete multi-view clustering via mining cluster complementarity. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI’22). AAAI Press, Virtual Event, 8761–8769.
[33]
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. 2022. Multi-level feature learning for contrastive multi-view clustering. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22). IEEE, 16030–16039.
[34]
Nan Xu, Yanqing Guo, Xin Zheng, Qianyu Wang, and Xiangyang Luo. 2018. Partial multi-view subspace clustering. In Proceedings of the 26th ACM Multimedia Conference on Multimedia. ACM, 1794–1801.
[35]
Mouxing Yang, Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. 2023. Robust multi-view clustering with incomplete information. IEEE Trans. Pattern Anal. Mach. Intell. 45, 1 (2023), 1055–1069.
[36]
Mengmeng Yang, Ivan Tjuawinata, and Kwok-Yan Lam. 2022. K-means clustering with local d\(_x\)-privacy for privacy-preserving data analysis. IEEE Trans. Inf. Forensics Secur. 17 (2022), 2524–2537.
[37]
Mengmeng Yang, Ivan Tjuawinata, Kwok-Yan Lam, Jun Zhao, and Lin Sun. 2022. Secure hot path crowdsourcing with local differential privacy under fog computing architecture. IEEE Trans. Serv. Comput. 15, 4 (2022), 2188–2201.
[38]
Wei Zhang, Zhaohong Deng, Kup-Sze Choi, and Shitong Wang. 2023. End-to-end incomplete multiview fuzzy clustering with adaptive missing view imputation and cooperative learning. IEEE Trans. Fuzzy Syst. 31, 5 (2023), 1445–1459.
[39]
Yi Zhang, Xinwang Liu, Siwei Wang, Jiyuan Liu, Sisi Dai, and En Zhu. 2021. One-stage incomplete multi-view clustering via late fusion. In Proceedings of the 29th ACM International Conference on Multimedia. ACM, 2717–2725.
[40]
Handong Zhao, Hongfu Liu, and Yun Fu. 2016. Incomplete multi-modal visual data grouping. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI’16). ijcai.org, 2392–2398.
[41]
Ji Zhao and Deyu Meng. 2015. FastMMD: Ensemble of circular discrepancy for efficient two-sample test. Neural Comput. 27, 6 (2015), 1345–1372.
[42]
Liang Zhao, Zhikui Chen, Laurence T. Yang, M. Jamal Deen, and Z. Jane Wang. 2019. Deep semantic mapping for heterogeneous multimedia transfer learning using co-occurrence data. ACM Trans. Multim. Comput. Commun. Appl. 15, 1s (2019), 9:1–9:21.

Cited By

View all
  • (2025)Multilingual pretrained based multi-feature fusion model for English text classificationComputer Science and Information Systems10.2298/CSIS240630004Z22:1(133-152)Online publication date: 2025
  • (2025)BACVC: Bi-adaptive and cross-view consistency for incomplete multi-view subspace clusteringAlexandria Engineering Journal10.1016/j.aej.2025.01.089119(623-633)Online publication date: Apr-2025
  • (2024)An IoT-enabled device for remotely monitoring and controlling solar photovoltaic systemsICST Transactions on Scalable Information Systems10.4108/eetsis.561211Online publication date: 4-Sep-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 20, Issue 6
June 2024
715 pages
EISSN:1551-6865
DOI:10.1145/3613638
  • Editor:
  • Abdulmotaleb El Saddik
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2024
Online AM: 26 September 2023
Accepted: 15 September 2023
Revised: 31 July 2023
Received: 25 May 2023
Published in TOMM Volume 20, Issue 6

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Incomplete multiview data
  2. data integrity
  3. internet of things
  4. optimal transport

Qualifiers

  • Research-article

Funding Sources

  • National Natural Science Foundation of China
  • Liaoning Provincial Science Public Welfare Research Fund-Soft Science Research Program
  • “Peak Climbing Plan” Project of Dalian Municipal Central Hospital

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)325
  • Downloads (Last 6 weeks)47
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Multilingual pretrained based multi-feature fusion model for English text classificationComputer Science and Information Systems10.2298/CSIS240630004Z22:1(133-152)Online publication date: 2025
  • (2025)BACVC: Bi-adaptive and cross-view consistency for incomplete multi-view subspace clusteringAlexandria Engineering Journal10.1016/j.aej.2025.01.089119(623-633)Online publication date: Apr-2025
  • (2024)An IoT-enabled device for remotely monitoring and controlling solar photovoltaic systemsICST Transactions on Scalable Information Systems10.4108/eetsis.561211Online publication date: 4-Sep-2024
  • (2024)Introduction to the Special Issue on Integrity of Multimedia and Multimodal Data in Internet of ThingsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364304020:6(1-4)Online publication date: 8-Mar-2024
  • (2024)Multiview Fully Interpretable TSK Fuzzy Classifier Enhanced by Multiview Accompanying GMMsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.344538932:11(6288-6302)Online publication date: Nov-2024
  • (2024)A Bilevel Periodically Interactive Evolutionary Algorithm for Personalized Service Customization in Wireless-Powered Cooperative MECIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33866228:6(4090-4105)Online publication date: Dec-2024
  • (2024)Artificial Intelligence, Internet of things and 6G methodologies in the context of Vehicular Ad-hoc Networks (VANETs): SurveyICT Express10.1016/j.icte.2024.05.008Online publication date: May-2024
  • (2024)Internet of multimedia things (IoMT): A reviewThe Review of Socionetwork Strategies10.1007/s12626-024-00175-1Online publication date: 10-Oct-2024
  • (2024)Enhancing Multi-modal Contrastive Learning via Optimal Transport-Based Consistent Modality AlignmentPattern Recognition and Computer Vision10.1007/978-981-97-8795-1_11(157-171)Online publication date: 3-Nov-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media