Abstract
The Restricted Boltzmann machine (RBM) has been proven to be a powerful tool in many specific applications, such as representational learning, document modeling, and many other learning tasks. However, the extensions of the RBM are rarely used in the field of multi-view learning. In this paper, we present a new RBM model based on canonical correlation analysis, named as the correlation RBM, for multi-view learning. The correlation RBM computes multiple representations by regularizing the marginal likelihood function with the consistency among representations from different views. In addition, the multimodal deep model can obtain a unified representation that fuses multiple representations together. Therefore, we stack the correlation RBM to create the correlation deep belief network (DBN), and then propose the multimodal correlation DBN for learning multi-view data representations. Contrasting with existing multi-view classification methods, such as multi-view Gaussian process with posterior consistency (MvGP) and consensus and complementarity based maximum entropy discrimination (MED-2C), the correlation RBM and the multimodal correlation DBN have achieved satisfactory results on two-class and multi-class classification datasets. Experimental results show that correlation RBM and the multimodal correlation DBN are effective learning algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Zhang N, Ding S, Zhang J, Xue Y (2018) An overview on restricted boltzmann machines. Neurocomputing 275:1186–1199
Courville A, Desjardins G, Bergstra J, Bengio Y (2014) The spike-and-slab RBM and extensions to discrete and sparse data distributions. IEEE Trans Pattern Anal Mach Intell 36(9):1874–1887
Mittelman R, Kuipers B, Savarese S, Lee H (2014) Structured recurrent temporal restricted boltzmann machines. In: International Conference on Machine Learning, pp. 1647–1655
Zhang N, Ding S, Zhang J, Xue Y (2017) Research on point-wise gated deep networks. Appl Soft Comput 52:1210–1221
Nguyen TD, Tran T, Phung D, Venkatesh S (2016) Graph-induced restricted Boltzmann machines for document modeling. Inf Sci 328:60–75
Amer MR, Shields T, Siddiquie B, Tamrakar A, Divakaran A, Chai S (2018) Deep multimodal fusion: A hybrid approach. Int J Comput Vis 126(2–4):440–456
Basu S, Karki M, Ganguly S, DiBiano R, Mukhopadhyay S, Gayaka S, Kannan R, Nemani R (2017) Learning sparse feature representations using probabilistic quadtrees and deep belief nets. Neural Process Lett 45(3):855–867
Salakhutdinov RR, Hinton GE (2009) Deep boltzmann machines. In: International Conference on Artificial Intelligence and Statistics, pp. 448–455
Kang Y, Choi S (2011) Restricted deep belief networks for multi-view learning. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 130–145
Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: Recent progress and new challenges. Information Fusion 38:43–54
Zhang Y, Yang Y, Lia T, Fujita H (2019) A multitask multiview clustering algorithm in heterogeneous situations based on LLE and LE. Knowl-Based Syst 163:776–786
Wang H, Yang Y, Liu B, Fujita H (2019) A study of graph-based system for multi-view clustering. Knowl-Based Syst 163:1009–1019
Liu Q, Sun S (2017) Multi-view regularized Gaussian processes. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 655–667
Chao G, Sun S (2016) Consensus and complementarity based maximum entropy discrimination for multi-view classification. Inf Sci 367:296–310
Andrew G, Arora R, Bilmes J, Livescu K (2013) Deep canonical correlation analysis. In: International Conference on Machine Learning, pp. 1247–1255
Ravanbakhsh S, Póczos B, Schneider J, Schuurmans D, Greiner R (2016) Stochastic neural networks with monotonic activation functions. In: International Conference on Artificial Intelligence and Statistics, pp. 809–818
Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8):1711–1800
Li CL, Ravanbakhsh S, Poczos B (2016) Annealing Gaussian into ReLU: a new sampling strategy for leaky-ReLU RBM. arXiv preprint arXiv:1611.03879
Ding S, Zhang X, An Y, Xue Y (2017) Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification. Pattern Recogn 67:32–46
Mangasarian OL, Street WN, Wolberg WH (1995) Breast cancer diagnosis and prognosis via linear programming. Oper Res 43(4):570–577
Arabasadi Z, Alizadehsani R, Roshanzamir M, Moosaei H, Yarifard AA (2017) Computer aided decision making for heart disease detection using hybrid neural network-Genetic algorithm. Comput Methods Prog Biomed 141:19–26
Güvenir HA, Demiröz G, Ilter N (1998) Learning differential diagnosis of erythemato-squamous diseases using voting feature intervals. Artif Intell Med 13(3):147–165
Johnson B, Tateishi R, Xie Z (2012) Using geographically-weighted variables for image classification. Remote Sensing Letters 3(6):491–499
Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov RR (2014) Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res 15:1929–1958
Zhang J, Ding S, Zhang N, Xue Y (2016) Weight uncertainty in boltzmann machine. Cogn Comput 8(6):1064–1073
Acknowledgements
This work is supported by the Fundamental Research Funds for the Central Universities (No.2017XKZD03).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, N., Ding, S., Liao, H. et al. Multimodal correlation deep belief networks for multi-view classification. Appl Intell 49, 1925–1936 (2019). https://doi.org/10.1007/s10489-018-1379-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-018-1379-8