Skip to main content
SpringerLink
Log in

Co-clustering based classification of multi-view data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-view learning is an attractive area of research where data is represented using multiple views, each containing some useful information. Many multi-view learning algorithms use a unified objective function that needs to be optimized simultaneously to achieve a common goal, using a similarity or distance measure, such as the Euclidean or Cosine metric. Having a unified objective function, however, does not allow independent learning from the different views which can later be integrated to improve the overall learning experience. In this paper, we explore the principles of multi-view learning, i.e., complementary and consensus principle, to propose a new multi-view classification algorithm that simultaneously allows for both independent and unified learning. The complementary principle says that each source of data contains some information that is missing in other views while the consensus principle aims to maximize the agreement across the multiple views. Our proposed technique treats the data both individually as well as a part of a multi-view data. Within a view, we exploit a supervised co-clustering-based measure that independently learns similarity between instances of that view. Across views, these similarlity values are then shared using transfer-learning. The learning process involves both co-clustering and multi-view learning within a supervised learning framework. The test data is predicted independently on each view and a concensus based prediction across the views is used for the final label. Individually, each of supervised learning, co-clustering, and multi-view learning has been used in the literature, each with its own set of advantages. To the best of our knowledge, the current study is the premier attempt to combine these concepts for the classification task. Results show that the proposed approach significantly outperforms other recent and state-of-the-art algorithms on several sparse and high-dimensional datasets using the accuracy and normalized mutual information criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ioannidis A, Chasanis V, Likas A (2016) Weighted multi-view key-frame extraction. Pattern Recogn Lett 72:52–61

    Article  Google Scholar 

  2. Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: recent progress and new challenges. Inf Fusion 38:43–54

    Article  Google Scholar 

  3. Sun S (2013) A survey of multi-view machine learning. Neural Comput Appl 23(7–8):2031–2038

    Article  Google Scholar 

  4. Hussain SF, Bashir S (2016) Co-clustering of multi-view datasets. Knowl Inf Syst 47(3):545–570

    Article  Google Scholar 

  5. Blum A, Mitchell T (1998) “Combining labeled and unlabeled data with co-training,” In: Proceedings of the eleventh annual conference on Computational learning theory, pp. 92–100

  6. Nigam K, Ghani R (2000) “Analyzing the effectiveness and applicability of co-training,” In: Proceedings of the ninth international conference on Information and knowledge management, pp. 86–93

  7. Muslea I, Minton S, Knoblock CA (2006) Active learning with multiple views. J Artif Intell Res 27:203–233

    Article  MathSciNet  Google Scholar 

  8. Wang W, Zhou Z-H (2007) “Analyzing co-training style algorithms,” In: European conference on machine learning, pp. 454–465

  9. Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72

    MathSciNet  MATH  Google Scholar 

  10. Sonnenburg S, Rätsch G, Schäfer C, Schölkopf B (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531–1565

    MathSciNet  MATH  Google Scholar 

  11. Hu M, Chen Y, Kwok JT-Y (2009) Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw 20(5):827–839

    Article  Google Scholar 

  12. Wang X, Liu X, Japkowicz N, Matwin S (2014) “Ensemble of multiple kernel SVM classifiers,” In: Canadian Conference on Artificial Intelligence, pp. 239–250

  13. Niu W, Xia K, Zu B, Bai J (2017) Efficient multiple kernel learning algorithms using low-rank representation. Comput Intell Neurosci 2017

  14. Varma M, Babu BR (2009) “More generality in efficient multiple kernel learning,” In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1065–1072

  15. Farquhar J, Hardoon D, Meng H, Shawe-Taylor JS, Szedmak S (2006) “Two view learning: SVM-2K, theory and practice,” In: Advances in neural information processing systems, pp. 355–362

  16. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(1):43–76

    Article  Google Scholar 

  17. Hussain SF, Pervez A, Hussain M (2020) Co-clustering optimization using artificial bee Colony (ABC) algorithm. Appl Soft Comput 97:106725

    Article  Google Scholar 

  18. Huang S, Xu Z, Tsang IW, Kang Z (2020) Auto-weighted multi-view co-clustering with bipartite graphs. Inf Sci 512:18–30

    Article  MathSciNet  Google Scholar 

  19. Hussain SF (2011) "Bi-clustering gene expression data using co-similarity." In: International conference on advanced data mining and applications, pp. 190–200. Springer: Berlin

  20. Hussain SF, Bisson G (2010) "Text categorization using word similarities based on higher order co-occurrences." In: Proceedings of the 2010 SIAM International Conference on Data Mining, pp. 1–12. Society for Industrial and Applied Mathematics

  21. Hussain SF (2019) A novel robust kernel for classifying high-dimensional data using support vector machines. Expert Syst Appl 131:116–131

    Article  Google Scholar 

  22. Hotelling H (1992) “Relations between two sets of variates,” In: Breakthroughs in statistics. Springer, pp. 162–190

  23. Akaho S (2006) “A kernel method for canonical correlation analysis,” ArXiv Prepr Cs0609071

  24. Gönen M, Alpaydin E (2008) “Localized multiple kernel learning,” In: Proceedings of the 25th international conference on Machine learning, pp. 352–359

  25. Rupnik J, Shawe-Taylor J (2010) “Multi-view canonical correlation analysis,” In: Conference on Data Mining and Data Warehouses (SiKDD 2010), pp. 1–4

  26. Wang W, Arora R, Livescu K, Bilmes JA (2015) “Unsupervised learning of acoustic features via deep canonical correlation analysis,” In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4590–4594

  27. Hussain SF, Khan K, Jillani R (2021) Weighted multi-view co-clustering (WMVCC) for sparse data. Appl Intell:1–19. https://doi.org/10.1007/s10489-021-02405-3

  28. Zien A, Ong CS (2007) Multiclass multiple kernel learning,” In: Proceedings of the 24th international conference on Machine learning, pp. 1191–1198

  29. Cortes C, Mohri M, Rostamizadeh A (2009) Learning non-linear combinations of kernels. In Advances in Neural Information Processing Systems 22:2010

  30. Houthuys L, Langone R, Suykens JA (2018) Multi-view least squares support vector machines classification. Neurocomputing 282:78–88

    Article  Google Scholar 

  31. You X, Xu J, Yuan W, Jing X-Y, Tao D, Zhang T (2019) Multi-view common component discriminant analysis for cross-view classification. Pattern Recogn 92:37–51

    Article  Google Scholar 

  32. Yang M, Deng C, Nie F (2019) Adaptive-weighting discriminative regression for multi-view classification. Pattern Recogn 88:236–245

    Article  Google Scholar 

  33. Tao H, Hou C, Yi D, Zhu J (2018) Multiview classification with cohesion and diversity. IEEE Trans Cybern

  34. Hardoon DR, Shawe-Taylor J (2011) Sparse canonical correlation analysis. Mach Learn 83(3):331–353

    Article  MathSciNet  Google Scholar 

  35. Xu Z, Jin R, Yang H, King I, Lyu MR (2010) Simple and efficient multiple kernel learning by group lasso. ICML, 2010

  36. Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953–997

    MathSciNet  MATH  Google Scholar 

  37. Han Y, Yang Y, Li X, Liu Q, Ma Y (2018) Matrix-regularized multiple kernel learning via $(r,∼ p) $ norms. IEEE Trans Neural Netw Learn Syst 29(10):4997–5007

    Article  MathSciNet  Google Scholar 

  38. Sun T, Chen S, Yang J, Shi P (2008) “A novel method of combined feature extraction for recognition,” In: 2008 Eighth IEEE International Conference on Data Mining, pp. 1043–1048

  39. Diethe T, Hardoon DR, Shawe-Taylor J (2008) Multiview fisher discriminant analysis

  40. Sharma A, Kumar A, Daume H, Jacobs DW (2012) “Generalized multiview analysis: A discriminative latent space,” In: 2012 IEEE conference on computer vision and pattern recognition, pp. 2160–2167

  41. Kan M, Shan S, Zhang H, Lao S, Chen X (2015) Multi-view discriminant analysis. IEEE Trans Pattern Anal Mach Intell 38(1):188–194

    Article  Google Scholar 

  42. Xu C, Tao D, Xu C (2015) Multi-view intact space learning. IEEE Trans Pattern Anal Mach Intell 37(12):2531–2544

    Article  Google Scholar 

  43. Ding Z, Fu Y (2016) “Robust multi-view subspace learning through dual low-rank decompositions,” In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1

  44. Bisson G, Hussain F (2008) "Chi-Sim: a new similarity measure for the co-clustering task." In 7th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 211–217

Download references

Acknowledgements

Mohsin Khan would like to thank the Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi, Pakistan for providing him a fully funded scholarship to pursue the MS degree under its GA-1 scheme.

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Machine Learning and Data Science (MDS) Lab, G.I.K Institute, Topi, KPK, 23460, Pakistan

    Syed Fawad Hussain & Mohsin Khan

  2. Faculty of Computer Science and Engineering, G.I.K. Institute, Topi, KPK, 23460, Pakistan

    Syed Fawad Hussain

  3. Riphah School of Computing and Innovation (RSCI), Riphah International University Lahore, Lahore, Pakistan

    Mohsin Khan

  4. Department of Computer Science, Bahria University Islamabad Campus, Islamabad, 44000, Pakistan

    Imran Siddiqi

Authors
  1. Syed Fawad Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohsin Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Imran Siddiqi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Syed Fawad Hussain.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hussain, S.F., Khan, M. & Siddiqi, I. Co-clustering based classification of multi-view data. Appl Intell 52, 14756–14772 (2022). https://doi.org/10.1007/s10489-021-03087-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03087-7

Keywords

Search

Navigation