Abstract
Learning a classifier when only knowing about the features and marginal distribution of class labels in each of the data groups is both theoretically interesting and practically useful. Specifically, we are interested in the case where the ratio of the number of data instances to the number of classes is large. For this problem, we show that the performance of a previously proposed discriminative classifier will deteriorate quickly as the ratio grows. In contrast, we formulate a density estimation framework to learn a generative classifier by RBM in this scenario with guaranteed performance under mild assumption.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Stolpe, M., Morik, K.: Learning from Label Proportions by Optimizing Cluster Model Selection. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS (LNAI), vol. 6913, pp. 349–364. Springer, Heidelberg (2011)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy Layer-Wise Training of Deep Networks. In: NIPS, pp. 153–160 (2007)
Kueck, H., de Freitas, N.: Learning about individuals from group statistics. In: Uncertainty in Artif (UAI), pp. 332–339. AUAI Press, Arlington (2005)
Wedin, P.A.: Perturbation theory for pseudo-inverses. BIT Numerical Mathematics 13(2) (1973)
Rueping, S.: SVM Classifier Estimation from Group Probabilities. In: Proceedings of the 27th Annual International Conference on Machine Learning, ICML 2010 (2010)
Quadrianto, N., Smola, A., Caetano, T., Le, Q.: Estimating labels from label proportions. JMLR 10, 2349–2374 (2009)
Carreira-Perpinan, M., Hinton, G.: On contrastive divergence learning. In: 10th Int. Workshop on Artificial Intelligence and Statistics, AISTATS 2005 (2005)
Hinton, G.: A Practical Guide to Training Restricted Boltzmann Machines. UTML TR, 2010-003 (August 2010)
Hinton, G., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Nerual Computation 18, 1527–1554 (2006)
Salakhutdinov, R., Murray, I.: On the Quantitative Analysis of Deep Belief Networks. In: ICML (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fan, K., Zhang, H., Zang, Y., Wang, L. (2013). Estimation Based on RBM from Label Proportions in Large Group Case. In: Yang, J., Fang, F., Sun, C. (eds) Intelligent Science and Intelligent Data Engineering. IScIDE 2012. Lecture Notes in Computer Science, vol 7751. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36669-7_76
Download citation
DOI: https://doi.org/10.1007/978-3-642-36669-7_76
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36668-0
Online ISBN: 978-3-642-36669-7
eBook Packages: Computer ScienceComputer Science (R0)