Skip to main content
SpringerLink
Log in

Knowledge transfer in SVM and neural networks

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

The paper considers general machine learning models, where knowledge transfer is positioned as the main method to improve their convergence properties. Previous research was focused on mechanisms of knowledge transfer in the context of SVM framework; the paper shows that this mechanism is applicable to neural network framework as well. The paper describes several general approaches for knowledge transfer in both SVM and ANN frameworks and illustrates algorithmic implementations and performance of one of these approaches for several synthetic examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brachman, R., Levesque, H.: Knowledge Representation and Reasoning. Morgan Kaufmann, San Francisco (2004)

    MATH  Google Scholar 

  2. Fouad, S., Tino, P., Raychaudhury, S., Schneider, P.: Incorporating privileged information through metric learning. IEEE Transactions on Neural Networks and Learning Systems 24, 1086–1098 (2013)

    Article  Google Scholar 

  3. Demuth, H., Beale, M.: Neural Network Toolbox for Use with MATLAB. Mathworks, Inc., Natick, MA (2000)

    Google Scholar 

  4. Hagan, M., Demuth, H., Beale, M.: Neural Network Design. PWS Publishing Co, Boston, MA (1996)

    Google Scholar 

  5. Heaton, J.: Introduction to Neural Networks for Java. Heaton Research Inc (2008)

  6. Ilin, R., Streltsov, S., Izmailov, R.: Learning with privileged information for improved target classification. International Journal of Monitoring and Surveillance Technologies Research 2(3), 5–66 (2014)

    Article  Google Scholar 

  7. Kolmogorov, A.: Mathematics as a Profession. Moscow, Nauka (1988). (in Russian)

    MATH  Google Scholar 

  8. LeCun, Y., Bottou, L., Orr, G., Müller, K.: Efficient BackProp. In: Orr, G., Müller, K. (eds.) Neural Networks: Tricks of the Trade, pp 5–50. Springer, Heidelberg (1998)

  9. Lichman, M.: UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California School of Information and Computer Science (2013)

  10. Pechyony, D., Izmailov, R., Vashist, A., Vapnik, V.: SMO-style algorithms for learning using privileged information. In: 2010 International Conference on Data Mining, pp 235–241 (2010)

  11. Ribeiro, B., Silva, C., Chen, N., Vieirac, A., Carvalho das Nevesd, J.: Enhanced default risk models with SVM + . Expert Syst. Appl. 39, 10140–10152 (2012)

    Article  Google Scholar 

  12. Sharmanska, V., Lampert, C.: Learning to rank using privileged information. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp 825–832 (2013)

  13. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    Book  MATH  Google Scholar 

  14. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  15. Vapnik, V.: Estimation of Dependencies Based on Empirical Data, 2nd edn. Springer, New York (2006)

    Google Scholar 

  16. Vapnik, V., Vashist, A.: A new learning paradigm: learning using privileged information. Neural Netw. 22, 546–557 (2009)

    Article  MATH  Google Scholar 

  17. Vapnik, V., Izmailov, R.: Learning with intelligent teacher: similarity control and knowledge transfer. In: Statistical Learning and Data Sciences. LNAI 9047, pp 3–32. Springer, London (2015)

  18. Vapnik, V., Izmailov, R.: Learning using privileged information: similarity control and knowledge transfer. J. Mach. Learn. Res. 16, 2023–2049 (2015)

    MathSciNet  MATH  Google Scholar 

  19. Yang, H., Patras, I.: Privileged information-based conditional regression forest for facial feature detection. In: 2013 IEEE International Conference on Automatic Face and Gesture Recognition, pp 1–6 (2013)

Download references

Author information

Authors and Affiliations

  1. Columbia University, New York, NY, USA

    Vladimir Vapnik

  2. AI Research Lab, Facebook, New York, NY, USA

    Vladimir Vapnik

  3. Vencore Labs, Basking Ridge, NJ, USA

    Rauf Izmailov

Authors
  1. Vladimir Vapnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rauf Izmailov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rauf Izmailov.

Additional information

This material is based upon work partially supported by AFRL and DARPA under contract FA8750-14-C-0008 and the work partially supported by AFRL under contract FA9550-15-1-0502. Any opinions, findings and / or conclusions in this material are those of the authors and do not necessarily reflect the views of AFRL and DARPA.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vapnik, V., Izmailov, R. Knowledge transfer in SVM and neural networks. Ann Math Artif Intell 81, 3–19 (2017). https://doi.org/10.1007/s10472-017-9538-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-017-9538-x

Keywords

Mathematics Subject Classification (2010)

Search

Navigation