Skip to main content
SpringerLink
Log in

A Maximum Entropy Approach for Collaborative Filtering

  • Published:
Journal of VLSI signal processing systems for signal, image and video technology Aims and scope Submit manuscript

Abstract

Collaborative filtering (CF) involves predicting the preferences of a user for a set of items given partial knowledge of the user's preferences for other items, while leveraging a database of profiles for other users. CF has applications e.g. in predicting Web sites a person will visit and in recommending products. Fundamentally, CF is a pattern recognition task, but a formidable one, often involving a huge feature space, a large data set, and many missing features. Even more daunting is the fact that a CF inference engine must be capable of predicting any (user-selected) items, given any available set of partial knowledge on the user's other preferences. In other words, the model must be designed to solve any of a huge (combinatoric) set of possible inference tasks. CF techniques include memory-based, classification-based, and statistical modelling approaches. Among these, modelling approaches scale best with large data sets and are the most adept at handling missing features. The disadvantage of these methods lies in the statistical assumptions (e.g. feature independence), which may be unjustified. To address this shortcoming we propose a new model-based CF method, based on the maximum entropy principle. For the MS Web application, the new method is demonstrated to outperform a number of CF approaches, including naive Bayes and latent variable (cluster) models, support vector machines (SVMs), and the (Pearson) correlation method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. B. Arwar, G. Karypis, J. Konstan, and J. Riedl, “Analysis of Recommendation Algorithms for E-commerce,” in Proc. of the 2nd ACM Conf. on Electronic Commerce, 2000.

  2. A. Berger, S. Della Pietra, and V. Della Pietra, “A Maximum Entropy Approach to Natural Language Processing,” Computational Linguistics, vol. 22, 1996, pp. 39-68.

    Google Scholar 

  3. J. Breese, D. Heckerman, and C. Kadie, “Empirical Analysis of Predictive Algorithms for Collaborative Filterin,” in Proc. of the Fourteenth Conf. on Uncertainty in Art. Intell., 1998.

  4. A.P. Dempster, N.M. Laird, and D.B. Rubin, “Maximum-Likelihood from Incomplete Data via the EM Algorithm,” Journal of the Roy. Stat. Soc., Ser. B, vol. 39, 1977, pp. 1-38.

    MathSciNet  MATH  Google Scholar 

  5. D. Fisher, K. Hildrum, J. Hong, M. Newman, M. Thomas, and R. Vuduc, “Swami: A Framework for Collaborative Filtering Algorithm Development and Evaluation,” in Proc. of the 23rd Annual Int. ACM SIGIR Conf. on Research and Development in Info. Retrieval, ACM Press, 2000, pp. 366-368.

  6. T. Hoffman, “Probabilistic Latent Semantic Indexing,” in Proc. of the 22nd Annual Int. ACM Conf. on Research and Development in Info. Retrieval, 1999.

  7. T. Hoffman and J. Puzicha, “Latent Class Models for Collaborative Filtering,” in Proc. of the IJCAI '99, 1999.

  8. E.T. Jaynes. “Information Theory and Statistical Mechanics,” in Papers on Probability, Statistics and Statistical Physics, R.D. Rosenkrantz (Ed.), Dordrecht, The Netherlands: Kluwer Academic Publishers, 1989 (Reprint of the original 1957 papers in Physical Review).

    Google Scholar 

  9. T. Joachims, Advances in Kernel Methods-Support Vector Learning, MIT Press, 1999.

  10. D.J. Miller and L. Yan, “Approximate Maximum Entropy Joint Feature Inference Consistent with Arbitrary Lower-Order Probability Constraints: Application to Statistical Classification,” Neural Computation, vol. 12, 2000, pp. 2175-2207.

    Article  Google Scholar 

  11. K. Nigam, J. Lafferty, and A. McCallum, “Using Maximum Entropy for Text Classification,” in Proc. of the IJCAI'99 Workshop on Machine Learning for Info. Filtering, 1999.

  12. S. Della Pietra, V. Della Pietra, and J. Lafferty, “Inducing Features of Random Fields,” IEEE Trans. on Patt. Anal. and Mach. Intell., vol. 19, 1996, pp. 380-393.

    Article  Google Scholar 

  13. L. Yan and D.J. Miller, “General Statistical Inference for Discrete and Mixed Spaces by an Approximate Application of the Maximum Entropy Principle,” IEEE Trans. on Neural Networks, vol. 11, 2000, pp. 558-573.

    Article  Google Scholar 

  14. L. Yan and D.J. Miller, “Approximate Maximum Entropy Learning for Classification: Comparison with Other Methods,” in Proc. of the 2001 IEEE Int. Workshop on Neural Networks for Sig. Proc., 2001, pp. 243-252.

  15. Q. Zhu, “On the Minimum Probability of Error of Classification with Incomplete Patterns,” Pattern Recognition, vol. 23, 1990, pp. 1281-1290

    Article  MathSciNet  Google Scholar 

  16. S.C. Zhu, Y.N. Wu, and D. Mumford, “Minimax Entropy Principle and its Application to Texture Modeling,” Neural Computation, vol. 9, 1997, pp. 1627-1660.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, The Pennsylvania State University, Rm 227-C EEW, University Park, PA, 16802-2701, USA

    John Browning & David J. Miller

Authors
  1. John Browning
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David J. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Browning, J., Miller, D.J. A Maximum Entropy Approach for Collaborative Filtering. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 37, 199–209 (2004). https://doi.org/10.1023/B:VLSI.0000027485.11890.15

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VLSI.0000027485.11890.15

Search

Navigation