Original software publicationGaussian kernel smooth regression with topology learning neural networks and Python implementation
Introduction
The topology learning neural networks of Growing Neural Gas (GNG) and Self-Organizing Incremental Neural Networks (SOINN) are further developments of the famous Self-Organizing Map (SOM). Unfortunately, among the mainstream machine learning softwares such as Sklearn [1], there is no such implementations. In addition, there has not been an publicly available implementation of the improved SOINN [2].
Semi-supervised learning is an active research area. There are great efforts in classification algorithm researches, but for semi-supervised regression, there is not enough attention.
The main contributions of the work are as follows.
- 1.
A novel semi-supervised regression framework called Semi-Supervised Learning Gaussian Kernel Smoother (SSL-GKS) is proposed.
- 2.
Kernel density estimation based on GNG and SOINN is implemented.
- 3.
The proposed framework can be used in combine with any clustering methods for semi-supervised regression.
Section snippets
Problems and background
From statistical point of view, the regression learning task is equivalent to modeling the joint distribution of explanatory and response variables. According to kernel density estimation (KDE) [3], joint distribution of explanatory variables X and response variables Y can be represented by weights of clustering centers where and index i are used for iteration of all cluster centers, is the distribution of clustering centers, and s is the smooth parameter. The learning task
Software architecture
The software is composed of 5 parts. (1) ‘utils.py’: Supporting utilities for csv file reading and Python dict operations. (2) ‘isoxnn2.py’ and ‘gng2.py’: GNG and SOINN algorithms. (3) ‘ui_isoinn.py’ and ‘ui_gng.py’) Programming interfaces for GNG and SOINN. (4) ‘gks.py’: SSL-GKS implementation (5) ‘reg_inn.py’ and ‘reg_gng.py’: Regression programming interfaces.
Software functionalities
Main functionalities are implemented by 4 Python classes listed below.
- 1.
class pygks.gks.GKS: By implementing Eq. (6), weights of
Empirical results
There are two sets of experiments on 6 datasets. First, the typical SSL setting is employed, where comparison results are on varying labeled datasets with labeling percentages growing. Second, we move to a real application, namely the traffic flow prediction. The datasets are downloaded from the Caltrans Performance measurement Systems (PEMS) database [8]. 6 days of data beginning from Nov. 24th, 2014 are chosen as training datasets, and the data from Nov. 30th, 2014 as the testing datasets.
Illustrative example
The following example is to show how to use the software.
Acknowledgments
This work was supported in part by the National Natural Science Foundations of China (Nos. 61272061 and 61301148).
Zhiyang Xiang received M.E. degree on computer science from Northwest A & F University, China. He is currently pursuing a Ph.D. degree in Hunan University, China. His research interests include neural networks algorithms and applications in information security.
References (18)
- et al.
A Gaussian mixture framework for incremental nonparametric regression with topology learning neural networks
Neurocomputing
(2016) Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods
Int. J. Electr. Power Energy Syst.
(2014)- et al.
Spectral regularization algorithms for learning large incomplete matrices
J. Mach. Learn. Res.
(2010) - et al.
Scikit-learn: machine learning in Python
J. Mach. Learn. Res.
(2011) - et al.
An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network
Neural Comput. Appl.
(2011) Multivariate Density Estimation: Theory, Practice, and Visualization
(2015)- et al.
Bayesian self-organising map for Gaussian mixtures
IEEE Proc. Vis., Image Signal Process.
(2001) - et al.
Density estimation method based on self-organizing incremental neural networks and error estimation
(2013) Density estimation for statistics and data analysis
(1986)
Cited by (8)
Short-term traffic volume prediction by ensemble learning in concept drifting environments
2019, Knowledge-Based SystemsCitation Excerpt :There are a number of new incremental learning methods that have not been used in traffic volume prediction that are worthy of our attention. In [25,26], the authors construct an incremental regression framework that can be used to model the linear and nonlinear relationships between the response vector and the explanatory variable. Unfortunately, there would be flaws if this framework is applied directly to traffic volume prediction because of the inability to handle non-stationarity.
Controlling Melody Structures in Automatic Game Soundtrack Compositions with Adversarial Learning Guided Gaussian Mixture Models
2021, IEEE Transactions on GamesSemi-parametric training of autoencoders with Gaussian kernel smoothed topology learning neural networks
2020, Neural Computing and ApplicationsStay of interest: A dynamic spatiotemporal stay behavior perception method for private car users
2019, Proceedings - 21st IEEE International Conference on High Performance Computing and Communications, 17th IEEE International Conference on Smart City and 5th IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2019Nonparametric kernel smoother on topology learning neural networks for incremental and ensemble regression
2019, Neural Computing and Applications
Zhiyang Xiang received M.E. degree on computer science from Northwest A & F University, China. He is currently pursuing a Ph.D. degree in Hunan University, China. His research interests include neural networks algorithms and applications in information security.
Zhu Xiao received M.E. and Ph.D. degrees on signal processing both from Xidan University, China. He is now an associate professor and Ph.D. supervisor with College of Computer Science and Electronics Engineering, Hunan University, China. His primary research interests include wireless communications. His research interests also include pattern recognition algorithms.
Dong Wang received M.E. and Ph.D. degrees on computer science from Hunan University, China. He is a Ph.D. director and a director to overseas graduate students in College of Computer Science and Electronics Engineering, Hunan University. His main research interests are computer networks and vehicular multimedia networks.
Jianhua Xiao received B.E. from Jiangxi Normal University, China. She is currently pursing a Ph.D. degree in Hunan University, China. Her research interests include pattern recognition and machine learning.