Skip to main content
Springer Nature Link
Log in

Linear combination of densities and its direct estimation framework with applications

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, typical learning task including data condensation, binary classification, identification of the independence between random variables and conditional density estimation is described from a unified perspective of a linear combination of densities, and accordingly a direct estimation framework based on a linear combination of Gaussian components (i.e., Gaussian basis functions) under integrated square error criterion is proposed to solve these learning tasks. The proposed direct estimation framework has three advantages. Firstly, different from most of the existing state-of-the-art methods in which estimating each component’s density in this linear combination of densities and then combining them linearly are required, it can directly estimate the linear combination of densities as a whole, and it has at least comparable to or even better approximation accuracy than the existing density estimation methods. Secondly, the time complexity of the proposed direct estimation framework is O(l 3) in which l is the number of Gaussian components in this framework which are generally viewed as the Gaussian distributions of the clusters in a dataset, and hence l is generally much less than the size of the dataset, so it is very suitable for large datasets. Thirdly, this proposed framework can be typically used to develop alternative approaches to classification, data condensation, identification of the independence between random variables, conditional density estimation and the similarity identification between multiple source domains and a target domain. Our preliminary results about experiments on several typical applications indicate the power of the proposed direct estimation framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Deng ZH, Chung FL, Wang ST (2008) FRSDE: fast reduced set density estimator using minimal enclosing ball. Pattern Recognit 41(4):1363–1372

    Article  MATH  Google Scholar 

  2. Tao JW, Chung FL, Wang ST (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45(11):3962–3984

    Article  MATH  Google Scholar 

  3. Wang J, Wang ST, Deng ZH, Chung FL (2012) Double indices induced FCM clustering and its integration with fuzzy subspace clustering. In: Proceedings of 2012 FUZZ-IEEE pp 1–8

  4. Deng ZH, Chung FL, Wang ST (2011) Clustering-inverse: a generalized model for pattern-based time series segmentation. J Intell Learn Syst Appl 3(1):26–36

    Google Scholar 

  5. He C, Girolami M (2004) Novelty detection employing an L2 optimal nonparametric density estimator. Pattern Recognit Lett 25(12):1389–1397

    Article  Google Scholar 

  6. Wang XM, Chung FL, Wang ST (2011) Theoretical analysis for solution of support vector data description. Neural Netw 24(4):360–369

    Article  MathSciNet  MATH  Google Scholar 

  7. Kim J, Scott C (2010) L2 kernel classification. IEEE Trans Pattern Anal Mach Intell 32(10):1822–1831

    Article  Google Scholar 

  8. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  9. Girolami M, He C (2003) Probability density estimation from optimally condensed data samples. IEEE Trans Pattern Anal Mach Intell 25(10):1253–1264

    Article  Google Scholar 

  10. Ray D, Majumder DD, Das A (2012) Noise reduction and image enhancement of MRI using adaptive multiscale data condensation. In: Proceedings of 2012 1st international conference on recent advances in information technology (RAIT) pp 107–113

  11. Angiulli F (2007) Condensed nearest neighbor data domain description. IEEE Trans Pattern Anal Mach Intell 29(10):1746–1758

    Article  Google Scholar 

  12. T Suzuki, M Sugiyama, J Sese, T Kanamori (2008) Approximating mutual information by maximum likelihood density ratio estimation. In JMLR workshop and conference proceedings, new challenges for feature selection in data mining and knowledge discovery, vol 4, pp 5–20

  13. Seth S, Principe C (2009) Estimation of density ratio and its application to design a measure of dependence. In: Proceedings of 2009 IEEE international workshop on machine learning for signal processing, pp 1–6

  14. Jos M, Barrios G et al (2005) Regression analysis and dependence. Metrika 61(1):73–87

    Article  MathSciNet  MATH  Google Scholar 

  15. Shen Z, Xie SQ, Pan CY (1979) Probability theory and mathematical statistics. Higher Education Press, Beijing

    Google Scholar 

  16. Zhuang FZ, Luo P, Xiong H et al (2010) Cross2domain learning from multiple sources: a consensus regularization perspective. IEEE Trans Knowl Data Eng 22(12):1664–1678

    Article  Google Scholar 

  17. Bollegala D, Weir D, Carroll J (2011) Using multiple sources to construct a sentiment sensitive thesaurus for cross domain sentiment classification. In: Proceedings of the 49th annual meeting of the ACL: human language technologies HLT 2011, vol 1, pp 132–141

  18. Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33:1065–1076

    Article  MathSciNet  MATH  Google Scholar 

  19. Fan RE, Chen PH, Lin CJ (2005) Working set selection using second order information for training support vector machines. J Mach Learn Res 6:1889–1918

    MathSciNet  MATH  Google Scholar 

  20. Fukumizu K, Gretton A et al (2008) Kernel measures of conditional dependence. Advances in neural information processing systems. MIT Press, Cambridge, pp 489–496

    Google Scholar 

  21. Takeuchi I, Nomura K, Kanamori T (2009) Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression. Neural Comput 21(2):533–559

    Article  MathSciNet  MATH  Google Scholar 

  22. Jones MC, Marron JS, Sheather SJ (1996) A brief survey of bandwidth selection for density estimation. J Am Stat Assoc 91(433):401–407

    Article  MathSciNet  MATH  Google Scholar 

  23. Raykar VC, Duraiswami R (2006) Fast optimal bandwidth selection for kernel density estimation. In: Proceedings of 6th SIAM international conference on data mining, pp 524–528

  24. Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, London

    Book  MATH  Google Scholar 

  25. Yen SJ, Wu YC, Yang JC, Lee YS, Lee CJ, Liu JJ (2013) A support vector machine-based context-ranking model for question answering. Inf Sci 224:77–87

    Article  Google Scholar 

  26. Liu X, Pan S, Hao Z, Lin Z (2014) Graph-based semi-supervised learning by mixed label propagation with a soft constraint. Inf Sci 277:327–337

    Article  MathSciNet  Google Scholar 

  27. Li HX, Yang JL, Zhang G, Fan B (2013) Probabilistic support vector machines for classification of noise affected data. Inf Sci 221:60–71

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Hong Kong Polytechnic University under Grant G-UA68, by the National Natural Science Foundation of China under Grants 61272210 and by the Natural Science Foundation of Jiangsu Province under Grants BK2011003, BK2011417, BK20130161, JiangSu 333 expert engineering grant (BRA2011142), and 2013 Postgraduate Student’s Creative Research Fund of Jiangsu Province.

Author information

Authors and Affiliations

  1. School of Digital Media, Jiangnan University, Wuxi, Jiangsu, China

    Min Xu & Shitong Wang

  2. Center of Smart Health, School of Nursing, Hong Kong Polytechnic University, Hong Kong, China

    Guanjin Wang

  3. Department of Computing, Hong Kong Polytechnic University, Hong Kong, China

    Fu-lai Chung & Shitong Wang

  4. School of Internet of Things Technology, Wuxi Institute of Technology, Wuxi, Jiangsu, China

    Min Xu

Authors
  1. Min Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Guanjin Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Fu-lai Chung
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Shitong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Shitong Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, M., Wang, G., Chung, Fl. et al. Linear combination of densities and its direct estimation framework with applications. Neural Comput & Applic 27, 1477–1495 (2016). https://doi.org/10.1007/s00521-015-1947-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-1947-3

Keywords