Skip to main content
SpringerLink
Log in

Fast Estimation of Multidimensional Regression Functions by the Parzen Kernel-Based Method

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1791))

Included in the following conference series:

  • 739 Accesses

Abstract

Various methods for estimation of unknown functions from the set of noisy measurements are applicable to a wide variety of problems. Among them the non–parametric algorithms based on the Parzen kernel are commonly used. Our method is basically developed for multidimensional case. The two-dimensional version of the method is thoroughly explained and analysed. The proposed algorithm is an effective and efficient solution significantly improving computational speed. Computational complexity and speed of convergence of the algorithm are also studied. Some applications for solving real problems with our algorithms are presented. Our approach is applicable to multidimensional regression function estimation as well as to estimation of derivatives of functions. It is worth noticing that the presented algorithms have already been used successfully in various image processing applications, achieving significant accelerations of calculations.

Supported by the program of the Polish Minister of Science and Higher Education under the name “Regional Initiative of Excellence” in the years 2019–2023 project number 020/RID/2018/19 the amount of financing 12,000,000 PLN. The work of the second Author was performed at Westpomeranian University of Technology, while on sabbatical leave from Concordia University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andrzejewski, W., Gramacki, A., Gramacki, J., et al.: Graphics processing units in acceleration of bandwidth selection for kernel density estimation. Int. J. Appl. Math. Comput. Sci. 23(4), 869 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  2. Antoniadis, A., Grégoire, G., Vial, P.: Random design wavelet curve smoothing. Stat. Probab. Lett. 35(3), 225–232 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  3. Box, G.E., Cox, D.R.: An analysis of transformations. J. R. Stat. Soc. Ser. B (Methodological) 26(2), 211–243 (1964)

    MATH  Google Scholar 

  4. Cohen, A., Daubechies, I., Vial, P.: Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmonic Anal. (1993)

    Google Scholar 

  5. Eubank, R.L.: Nonparametric Regression and Spline Smoothing, 2nd edn. Marcel Dekker, New York (1999)

    Book  MATH  Google Scholar 

  6. Fan, J., Marron, J.S.: Fast implementations of nonparametric curve estimators. J. Comput. Graph. Stat. 3(1), 35–56 (1994)

    Google Scholar 

  7. Gałkowski, T.: Kernel estimation of regression functions in the boundary regions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013. LNCS (LNAI), vol. 7895, pp. 158–166. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38610-7_15

    Chapter  Google Scholar 

  8. Gałkowski, T., Krzyżak, A.: Edge curve estimation by the nonparametric parzen kernel method. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. CCIS, vol. 1332, pp. 377–385. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63820-7_43

    Chapter  Google Scholar 

  9. Gałkowski, T., Krzyżak, A.: A new approach to detection of abrupt changes in black-and-white images. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12416, pp. 3–18. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61534-5_1

    Chapter  Google Scholar 

  10. Gałkowski, T., Krzyżak, A., Filutowicz, Z.: A new approach to detection of changes in multidimensional patterns. J. Artif. Intell. Soft Comput. Res. 10, 125–136 (2020)

    Article  Google Scholar 

  11. Gałkowski, T., Krzyżak, A., Patora-Wysocka, Z., Filutowicz, Z., Wang, L.: A new approach to detection of changes in multidimensional patterns. part 2. J. Artif. Intell. Soft. Comput. Res. 11, 217–227 (2021)

    Google Scholar 

  12. Gałkowski, T., Pawlak, M.: Nonparametric estimation of edge values of regression functions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2016. LNCS (LNAI), vol. 9693, pp. 49–59. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39384-1_5

    Chapter  Google Scholar 

  13. Gałkowski, T., Rutkowski, L.: Nonparametric recovery of multivariate functions with applications to system identification. Proc. IEEE 73(5), 942–943 (1985)

    Article  Google Scholar 

  14. Gałkowski, T., Rutkowski, L.: Nonparametric fitting of multivariate functions. IEEE Trans. Autom. Control 31(8), 785–787 (1986). https://doi.org/10.1109/TAC.1986.1104399

    Article  MATH  Google Scholar 

  15. Gasser, T., Müller, H.-G.: Kernel estimation of regression functions. In: Gasser, T., Rosenblatt, M. (eds.) Smoothing Techniques for Curve Estimation. LNM, vol. 757, pp. 23–68. Springer, Heidelberg (1979). https://doi.org/10.1007/BFb0098489

    Chapter  Google Scholar 

  16. Gramacki, A., Gramacki, J.: Fft-based fast computation of multivariate kernel density estimators with unconstrained bandwidth matrices. J. Comput. Graph. Stat. 26(2), 459–462 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Greengard, L.: Fast algorithms for classical physics. Science 265(5174), 909–914 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  18. Greengard, L., Strain, J.: The fast gauss transform. SIAM J. Sci. Stat. Comput. 12(1), 79–94 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  19. Grycuk, R., Gałkowski, T., Rutkowski, L., Scherer, R.: A novel method for solar image retrieval based on the parzen kernel estimate of the function derivative and convolutional autoencoder. In: International Joint Conference on Neural Networks IJCNN, 18–23 July 2022, Padova, Italy, pp. 1–7 (2022)

    Google Scholar 

  20. Hardle, W., Marron, J.S.: Optimal bandwidth selection in nonparametric regression function estimation. Ann. Stat. 1465–1481 (1985)

    Google Scholar 

  21. Härdle, W., Scott, D.: Smoothing in low and high dimensions by weighted averaging using rounded points. Comput. Stat. 7, 97–128 (1992)

    MATH  Google Scholar 

  22. Holmström, L.: The accuracy and the computational complexity of a multivariate binned kernel density estimator. J. Multivariate Anal. 72(2), 264–309 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  23. Müller, H.G.: Empirical bandwidth choice for nonparametric kernel regression by means of pilot estimators. Stat. Decisions 2, 193–206 (1985)

    MathSciNet  Google Scholar 

  24. Müller, H.G.: Smooth optimum kernel estimators near endpoints. Biometrika 78(3), 521–530 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  25. Nadaraya, E.A.: On estimating regression. Theor. Probab. Appl. 9(1), 141–142 (1964)

    Article  MATH  Google Scholar 

  26. NVIDIA, C.: Nvidia cuda programming guide (2012)

    Google Scholar 

  27. NVIDIA, C.: Nvidia’s next generation cuda compute architecture: Kepler gk110 (2013)

    Google Scholar 

  28. Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33(3), 1065–1076 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  29. Raykar, V.C., Duraiswami, R., Zhao, L.H.: Fast computation of kernel estimators. J. Comput. Graph. Stat. 19(1), 205–220 (2010)

    Article  MathSciNet  Google Scholar 

  30. Rosenblatt, M.: Conditional probability density and regression estimates. Multivariate Anal. II(25), 25–31 (1969)

    Google Scholar 

  31. Scott, D.W.: Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley, Hoboken (2015)

    Google Scholar 

  32. Silverman, B.W.: Algorithm as 176: kernel density estimation using the fast fourier transform. J. R. Stat. Soc. Ser. C (Appl. Stat.) 31(1), 93–99 (1982)

    Google Scholar 

  33. Stanton, J.M.: Galton, pearson, and the peas: a brief history of linear regression for statistics instructors. J. Stat. Educ. 9(3) (2001)

    Google Scholar 

  34. Stone, C.J.: An asymptotically optimal window selection rule for kernel density estimates. Ann. Stat. 12, 1285–1297 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wand, M.P., Jones, M.C.: Kernel Smoothing. Chapman and Hall, London (1995)

    Book  MATH  Google Scholar 

  36. Wand, M.: Fast computation of multivariate kernel estimators. J. Comput. Graph. Stat. 3(4), 433–445 (1994)

    MathSciNet  Google Scholar 

  37. Watson, G.S.: Smooth regression analysis. Sankhyā. Indian J. Stat. Ser. A 26, 359–372 (1964)

    MathSciNet  MATH  Google Scholar 

  38. Yang, C., Duraiswami, R., Gumerov, N.A., Davis, L.: Improved fast gauss transform and efficient kernel density estimation. In: Computer Vision, IEEE International Conference on, vol. 2, pp. 464–464. Technical Report CS-TR-4495, University of Maryland, College Park, MD. (2003)

    Google Scholar 

  39. Zhang, S., Karunamuni, R.J.: Deconvolution boundary kernel method in nonparametric density estimation. J. Stat. Plan. Infer. 139(7), 2269–2283 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Czȩstochowa University of Technology, Armii Krajowej 36, Czȩstochowa, Poland

    Tomasz Gałkowski

  2. Department of Computer Science and Software Engineering, Concordia University, Quebec, Montreal, H3G 1M8, Canada

    Adam Krzyżak

  3. Department of Electrical Engineering, Westpomeranian University of Technology, Sikorskiego 37, 70-313, Szczecin, Poland

    Adam Krzyżak

Authors
  1. Tomasz Gałkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam Krzyżak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz Gałkowski .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gałkowski, T., Krzyżak, A. (2023). Fast Estimation of Multidimensional Regression Functions by the Parzen Kernel-Based Method. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1791. Springer, Singapore. https://doi.org/10.1007/978-981-99-1639-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1639-9_21

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1638-2

  • Online ISBN: 978-981-99-1639-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation