Abstract
Maximum Margin Matrix Factorization is one of the very popular techniques of collaborative filtering. The discrete valued rating matrix with a small portion of known ratings is factorized into two latent factors and the unknown ratings are estimated by the resulting product of the factors. The factorization is achieved by optimizing a loss function and the optimization is carried out by gradient descent or its variants. It is observed that any of these algorithms yields near-global optimizing point irrespective of the initial seed point. In this paper, we propose to combine swarm-like search with gradient descent search. Our algorithm starts from multiple initial points and uses gradient information and swarm-search as the search progresses. We show that by this process we get an efficient search scheme to get near optimal point for maximum margin matrix factorization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The MAE that gradient based MMMF could reach in fixed number of iterations.
References
Borowska, B., Nadolski, S.: Particle swarm optimization: the gradient correction (2009)
Devi, V.S., Rao, K.V., Pujari, A.K., Padmanabhan, V.: Collaborative filtering by pso-based mmmf. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 5–8, October 2014
Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, vol. 1, pp. 39–43 (1995)
Figueiredo, E.M., Ludermir, T.B.: Effect of the PSO topologies on the performance of the PSO-ELM. In: Neural Networks Brazilian Symposium (SBRN), pp. 178–183. IEEE, October 2012
Kennedy, Y.S.J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann, San Francisco (2001)
Kennedy, J.: Small world and mega-minds: effects of neighborhood topologies onparticle swarm performance. In: Congress on Evolutionary Computation, pp. 1931–1938 (1999)
Kennedy, J.: Bare bones particle swarms. In: IEEE Swarm Intelligence Symposium, pp. 80–87 (2003)
Kennedy, J., Mendes, R.: Population structure and particle swarm performance. In: Proceedings IEEE Congress on Evolutionary Computation, vol. 2, pp. 1671–1676 (2002)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. IEEE Comput. 42(8), 30–37 (2009)
Lane, J., Engelbrecht, A., Gain, J.:Particle swarm optimization with spatially meaningful neighbours. In: Swarm Intelligence Symposium, SIS 2008, pp. 1–8. IEEE (2008)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Millie, P., Thangaraj, R., Abraham, A.: Particle swarm optimization: performance tuning and empirical analysis. Found. Comput. Intell. 3, 101–128 (2009)
Nati, N.S., Jaakkola, T.: Weighted low-rank approximations. In: 20th International Conference on Machine Learning, pp. 720–727. AAAI Press (2003)
Noel, M.M., Jannett, T.C.: Simulation of a new hybrid particle swarm optimization algorithm. In: Proceedings of the Thirty-Sixth Southeastern Symposium on IEEE System Theory, pp. 150–153 (2004)
Pant, M., Radha, T.. Singh, V.: A simple diversity guided particle swarm optimization. In: Proceedings of IEEE Congress Evolutionary Computation, pp. 3294–3299 (2007)
Peer, E.S., van den Bergh, F., Engelbrecht, A.P.: Using neighbourhoods with the guaranteed convergence PSO. In: Proceedings of the 2003 IEEE Swarm Intelligence Symposium, SIS 2003, pp. 235–242. IEEE (2003)
Ni, J.D.Q.: A new logistic dynamic particle swarm optimization algorithm based on random topology. Sci. World J. (2013)
Radha, T., Pant, M., Abraham, A.: A new diversity guided particle swarm optimization with mutation. In: Nature and Biologically Inspired, Computing, pp. 294–299 (2009)
Rennie, J.D., Srebro, N.: Fast maximum margin matrix factorization for collaborative prediction. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 713–719. ACM (2005)
Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Incremental singular value decomposition algorithms for highly scalable recommender systems. In: Fifth International Conference on Computer and Information Science, pp. 27–28. Citeseer (2002)
Szabo, D.B.: A study of gradient based particle swarm optimisers. Master’s thesis, Faculty of Engineering, Built Environment and In- formation Technology University of Pretoria, Pretoria, South Africa (2010)
Tanweer, M., Suresh, S., Sundararajan, N.: Self regulating particle swarm optimization algorithm. Inf. Sci. 294, 182–202 (2015)
Toscano-Pulido, G., Reyes-Medina, A.J., RamÃrez-Torres, J.G.: A statistical study of the effects of neighborhood topologies in particle swarm optimization. In: Madani, K., Correia, A.D., Rosa, A., Filipe, J. (eds.) Computational Intelligence. SCI, vol. 343, pp. 179–192. Springer, Heidelberg (2011)
Vesterstrom, J.S., Riget, J., Krink, T.: Division of labor in particle swarm optimisation. In: WCCI, pp. 1570–1575. IEEE (2002)
Yao, J., Han, D.: Improved barebones particle swarm optimization with neighborhood search and its application on ship design. Math. Probl. Eng., 1–12 (2013)
Zhang, R., Zhang, W., Zhang, X.: A new hybrid gradient-based particle swarm optimization algorithm and its applications to control of polarization mode dispersion compensation in optical fiber communication systems. In: International Joint Conference on Computational Sciences and Optimization, CSO 2009, vol. 2, pp. 1031–1033. IEEE (2009)
Acknowledgements
Part of this work is carried out at Central University of Rajasthan. Authors acknowledge Central University of Rajasthan for providing facilities.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Salman, K.H., Pujari, A.K., Kumar, V., Veeramachaneni, S.D. (2016). Combining Swarm with Gradient Search for Maximum Margin Matrix Factorization. In: Booth, R., Zhang, ML. (eds) PRICAI 2016: Trends in Artificial Intelligence. PRICAI 2016. Lecture Notes in Computer Science(), vol 9810. Springer, Cham. https://doi.org/10.1007/978-3-319-42911-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-42911-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42910-6
Online ISBN: 978-3-319-42911-3
eBook Packages: Computer ScienceComputer Science (R0)