Skip to main content
SpringerLink
Log in

GPU-accelerated parallel algorithms for linear rankSVM

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Linear rankSVM is one of the widely used methods for learning to rank. The existing methods such as trust region Newton method (TRON) with order-statistic tree cannot perform efficiently when any one of them is exploited to deal with the large-scale data sets. It is observed that training the linear rankSVM with L2-loss requires a large amount of data-intensive computing, and therefore is feasible to make use of the great computational power of GPU to improve the training speed. In this paper, we propose two efficient parallel algorithms (named PSWX-TRON and PSY-TRON) to train the linear rankSVM with L2-loss on the GPU architecture. The proposed parallel algorithms can work efficiently on GPU if the training set is divided in terms of different queries. The experimental results show that compared with the existing methods, the proposed algorithms not only can obtain the impressive training speeds, but also can perform well in prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.cse.chalmers.se/research/group/dcs/gpuqsortdcs.html.

  2. https://gist.github.com/mre/1392067.

  3. http://www.csie.ntu.edu.tw/~cjlin/liblinear/liblinear-1.96.tar.gz.

  4. http://research.microsoft.com/en-us/um/beijing/projects/letor/.

  5. http://www.csie.ntu.edu.tw/~cjlin/papers/ranksvm/ranksvml2_exp-1.3.tgz.

  6. http://staff.cs.utu.fi/~aatapa/software/RankSVM/.

  7. http://olivier.chapelle.cc/primal/.

  8. http://nvlabs.github.io/cub/index.html.

References

  1. Airola A, Pahikkala T, Salakoski T (2011) Training linear ranking svms in linearithmic time using red-black trees. Pattern Recogn Lett 32(9):1328–1336

    Article  Google Scholar 

  2. Burges C, Shaked T, Renshaw E, Lazier A, Deeds M, Hamilton N, Hullender G (2005) Learning to rank using gradient descent. In: Proceedings of the 22nd international conference on Machine learning, ACM, pp 89–96

  3. Cederman D, Tsigas P (2008) A practical quicksort algorithm for graphics processors. In: Proceedings of the 16th Annual European Symposium on Algorithms (ESA 2008), Springer, pp 246–258

  4. Chapelle O, Chang Y (2011) Yahoo! learning to rank challenge overview. J Mach Learn Res Proc Track 14:1–24

    Google Scholar 

  5. Chapelle O, Keerthi SS (2010) Efficient algorithms for ranking with svms. Inf Retr 13(3):201–215

    Article  Google Scholar 

  6. Conn AR, Gould NI, Toint PL (2000) Trust region methods. SIAM, philadelphia

  7. Cooper WS, Gey FC, Dabney DP (1992) Probabilistic retrieval based on staged logistic regression. In: Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, ACM, pp 198–210

  8. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  9. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9:1871–1874

    MATH  Google Scholar 

  10. Freund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969

    MathSciNet  Google Scholar 

  11. Fuhr N (1989) Optimum polynomial retrieval functions based on the probability ranking principle. ACM Trans Inf Syst (TOIS) 7(3):183–204

    Article  Google Scholar 

  12. Govindaraju NK, Raghuvanshi N, Henson M, Manocha D (2005) A cache-efficient sorting algorithm for database and data mining computations using graphics processors. University of North Carolina, Technical Report. http://gamma.cs.unc.edu/SORT/gpusort.pdf

  13. Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of ir techniques. ACM Trans Inf Syst (TOIS) 20(4):422–446

    Article  Google Scholar 

  14. Jin J, Lin X (2014) Efficient parallel algorithms for linear rankSVM on gpu. In: Proceedings of the 11 th IFIP WG 10.3 international conference on Network and parallel computing, Springer, pp 181–194

  15. Joachims T (2002) Optimizing search engines using clickthrough data. In: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, pp 133–142

  16. Joachims T, Finley T, Yu CNJ (2009) Cutting-plane training of structural svms. Mach Learn 77(1): 27–59

    Article  MATH  Google Scholar 

  17. Kuo TM, Lee CP, Lin CJ (2014) Large-scale kernel rankSVM. In: Proceedings of the 2014 society for industrial and applied mathematics (SIAM) international conference on data mining, SIMA, pp 812–820

  18. Lee CP, Lin CJ (2014) Large-scale linear ranksvm. Neural Comput pp 26(4):781–817

  19. Lin CJ, Weng RC, Keerthi SS (2008) Trust region newton method for logistic regression. J Mach Learn Res 9:627–650

    MathSciNet  MATH  Google Scholar 

  20. Merrill D, Grimshaw A (2011) High performance and scalable radix sorting: a case study of implementing dynamic parallelism for gpu computing. Parallel Process Lett 21(02):245–272

    Article  MathSciNet  Google Scholar 

  21. NVIDA CUDA (2007) Compute unified device architecture programming guide. NVIDIA, Santa Clara, CA

  22. NVIDA CUDA (2007) Cublas library programming guide, 1st edn. NVIDA Corporation. http://docs.nvidia.com/cuda/cublas

  23. NVIDA CUDA (2008) CUDA programming guide 2.1. NVIDIA Corporation. http://www.umiacs.umd.edu/~ramani/cmsc828e_gpusci/CUDA21.pdf

  24. Owens JD, Houston M, Luebke D, Green S, Stone JE, Phillips JC (2008) Gpu computing. Proc IEEE 96(5):879–899

    Article  Google Scholar 

  25. Richardson M, Prakash A, Brill E (2006) Beyond pagerank: machine learning for static ranking. In: Proceedings of the 15th international conference on World Wide Web, ACM, pp 707–715

  26. Sculley D (2009) Large scale learning to rank. In: Proceedings of the neural information processing systems 2009 workshop on advances in ranking, pp 1–6

  27. Steinkraus D, Buck I, Simard P (2005) Using gpus for machine learning algorithms. In: Document analysis and recognition, proceedings of 8th international conference on, IEEE, pp 1115–1120

  28. Valizadegan H, Jin R, Zhang R, Mao J (2009) Learning to rank by optimizing ndcg measure. In: Advances in neural information processing systems 22 (NIPS 2009), Curran Associates Inc., pp 1833–1891

  29. Wu Q, Burges CJ, Svore KM, Gao J (2008) Ranking, boosting, and model adaptation. Microsoft Research Technical Report, MSR-TR-2008-109. http://research.microsoft.com/pubs/69536/tr-2008-109.pdf

  30. Yu H, Kim Y, Hwang S (2009) Rv-svm: an efficient method for learning ranking svm. In: Proceedings of the 13 th Pacific-Asia knowledge discovery and data mining conference (PAKDD 2009), Springer, pp 426–438

Download references

Acknowledgments

This research is supported in part by the National Natural Science Foundation of China under Grants No. 61472454, and the Project of Department of Education of Guangdong Province under Grants No. 2013KJCX0128. The authors thank the anonymous reviewers for their useful comments and suggestions.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Sun Yat-sen University, Guangzhou, 510275, China

    Jing Jin & Xiaola Lin

  2. School of Information Management, Sun Yat-sen University, Guangzhou, 510275, China

    Xianggao Cai

  3. Computer Engineering Technical College, Guangdong Institute of Science and Technology, Zhuhai, 519090, China

    Guoming Lai

Authors
  1. Jing Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianggao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoming Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaola Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaola Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, J., Cai, X., Lai, G. et al. GPU-accelerated parallel algorithms for linear rankSVM. J Supercomput 71, 4141–4171 (2015). https://doi.org/10.1007/s11227-015-1509-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-015-1509-6

Keywords

Search

Navigation