Skip to main content
SpringerLink
Log in

Large-scale linear nonparallel SVMs

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Large-scale problems have been a very active topic in machine learning area. In the time of big data, it is a challenge and meaningful work to solve such problems. Standard SVM can make linear classification on large-scale problems effectively, with acceptable training time and excellent prediction accuracy. However, nonparallel SVM (NPSVM) and ramp loss nonparallel SVM (RNPSVM) are proposed with better performance than SVM on benchmark datasets. It is motivated to introduce NPSVMs into the area of large-scale issues. In this paper, we propose large-scale linear NPSVMs, solved by the alternating direction method of multipliers (ADMM), to handle large-scale classification problems. ADMM breaks large problems into smaller pieces, avoiding solving intractable problems and leading to higher training speed. The primal problems of NPSVM are convex and differentiable, and they can be managed directly by ADMM. But the objective functions of RNPSVM, composed of convex ones and concave ones, should first be processed by CCCP algorithm and transformed as a series of convex programs. Then, we apply ADMM to solve these programs in every iteration. Experiments of NPSVMs on large-scale problems verify that the algorithms can classify large-scale tasks effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Akbani R, Kwek S, Japkowicz N (2004) Applying support vector machines to imbalanced datasets. In: Machine learning: ECML 2004, Springer, Berlin, p 39–50

  • Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  • Bhaskar BN, Tang G, Recht B (2013) Atomic norm denoising with applications to line spectral estimation. IEEE Trans Signal Process 61(23):5987–5999

    Article  MathSciNet  Google Scholar 

  • Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and trends\(\textregistered \) in. Mach Learn 3(1):1–122

    MATH  Google Scholar 

  • Bradley PS, Mangasarian OL (1998) Feature selection via concave minimization and support vector machines. ICML 98:82–90

    Google Scholar 

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Deng J, Berg AC, Li K, Fei-Fei L (2010) What does classifying more than 10,000 image categories tell us? In: Computer vision–ECCV 2010, Springer, Berlin, pp 71–84

  • Deng N, Tian Y (2004) New method in data mining: support vector machines. Science Press, Beijing

    Google Scholar 

  • Deng N, Tian Y, Zhang C (2012) Support vector machines: optimization based theory, algorithms, and extensions. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9:1871–1874

    MATH  Google Scholar 

  • Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach learn 46(1–3):389–422

    Article  MATH  Google Scholar 

  • Jayadeva RK, C S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  • Kasiviswanathan SP, Melville P, Banerjee A, Sindhwani V (2011) Emerging topic detection using dictionary learning. In: Proceedings of the 20th ACM international conference on Information and knowledge management, ACM, pp 745–754

  • Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recognit Lett 29(13):1842–1848

    Article  Google Scholar 

  • Li W, Zhao R, Wang X (2012) Human reidentification with transferred metric learning. In: ACCV (1), pp 31–44

  • Liu D, Shi Y, Tian Y (2015) Ramp loss nonparallel support vector machine for pattern classification. Knowl-Based Syst 85:224–233

    Article  Google Scholar 

  • Ma J, Saul LK, Savage S, Voelker GM (2009) Identifying suspicious urls: an application of large-scale online learning. In: Proceedings of the 26th annual international conference on machine learning, ACM, pp 681–688

  • Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, Association for Computational Linguistics, pp 142–150

  • Naik GR, Kumar DK et al (2010) Twin SVM for gesture classification using the surface electromyogram. IEEE Trans Inf Technol Biomed 14(2):301–308

    Article  Google Scholar 

  • Qi Z, Tian Y, Shi Y (2012) Twin support vector machine with universum data. Neural Netw 36:112–119

    Article  MATH  Google Scholar 

  • Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316

    Article  MATH  Google Scholar 

  • Qi Z, Tian Y, Shi Y (2014) A nonparallel support vector machine for a classification problem with universum learning. J Comput Appl Math 263:288–298

    Article  MathSciNet  MATH  Google Scholar 

  • Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  • Tan J, Zhang C, Deng N (2010) Cancer related gene identification via p-norm support vector machine. In: The 4th international conference on computational systems biology, vol 1, pp 101–108

  • Tian Y, Ping Y (2014) Large-scale linear nonparallel support vector machine solver. Neural Netw 50:166–174

    Article  MATH  Google Scholar 

  • Tian Y, Ju X, Qi Z, Shi Y (2014a) Improved twin support vector machine. Sci China Math 57(2):417–432

    Article  MathSciNet  MATH  Google Scholar 

  • Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014b) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Article  Google Scholar 

  • Tian Y, Zhang Q, Liu D (2014c) \(\nu \)-nonparallel support vector machine for pattern classification. Neural Comput Appl 25(5):1007–1020

    Article  Google Scholar 

  • Tian Y, Ju X, Shi Y (2016) A divide-and-combine method for large scale nonparallel support vector machines. Neural Netw 75:12–21

    Article  Google Scholar 

  • Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  • Vapnik V (2000) The nature of statistical learning theory. Springer, Berlin

    Book  MATH  Google Scholar 

  • Weston J, Watkins C et al (1999) Support vector machines for multi-class pattern recognition. ESANN 99:219–224

    Google Scholar 

  • Zhang HH, Ahn J, Lin X, Park C (2006) Gene selection using support vector machines with non-convex penalty. Bioinformatics 22(1):88–95

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by grants from National Natural Science Foundation of China (Nos. 61472390, 11271361, 71331005, 11226089 and 91546201), Major International (Regional) Joint Research Project (No. 71110107026) and the Beijing Natural Science Foundation (No. 1162005).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China

    Dalian Liu & Yong Shi

  2. Department of Basic Course Teaching, Beijing Union University, Beijing, 100101, China

    Dalian Liu

  3. School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

    Dewei Li

  4. Research Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing, 100190, China

    Dewei Li, Yong Shi & Yingjie Tian

  5. Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing, 100190, China

    Yong Shi & Yingjie Tian

  6. College of Information Science and Technology, University of Nebraska at Omaha, Omaha, NE, 68182, USA

    Yong Shi

Authors
  1. Dalian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dewei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingjie Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Shi.

Ethics declarations

Conflict of interest

We declare that we have no conflicts of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Li, D., Shi, Y. et al. Large-scale linear nonparallel SVMs. Soft Comput 22, 1945–1957 (2018). https://doi.org/10.1007/s00500-016-2455-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2455-9

Keywords

Search

Navigation