Elsevier

Neurocomputing

Volume 260, 18 October 2017, Pages 1-4
Neurocomputing

Original software publication
Gaussian kernel smooth regression with topology learning neural networks and Python implementation

https://doi.org/10.1016/j.neucom.2017.01.051Get rights and content

Abstract

Topology learning neural networks such as Growing Neural Gas (GNG) and Self-Organizing Incremental Neural Network (SOINN) are online clustering methods. With GNG and SOINN implemented as basic learners, this software completes two machine learning tasks, namely density estimation and regression. A kernel density estimation framework is implemented to transform the topology learning neural networks into density estimation methods. Besides, a kernel smoother to implement supervised and semi-supervised regression is devised. Moreover, the implemented frameworks can be used to transform other clustering methods into density estimation, supervised regression and semi-supervised regression.

Introduction

The topology learning neural networks of Growing Neural Gas (GNG) and Self-Organizing Incremental Neural Networks (SOINN) are further developments of the famous Self-Organizing Map (SOM). Unfortunately, among the mainstream machine learning softwares such as Sklearn [1], there is no such implementations. In addition, there has not been an publicly available implementation of the improved SOINN [2].

Semi-supervised learning is an active research area. There are great efforts in classification algorithm researches, but for semi-supervised regression, there is not enough attention.

The main contributions of the work are as follows.

  • 1.

    A novel semi-supervised regression framework called Semi-Supervised Learning Gaussian Kernel Smoother (SSL-GKS) is proposed.

  • 2.

    Kernel density estimation based on GNG and SOINN is implemented.

  • 3.

    The proposed framework can be used in combine with any clustering methods for semi-supervised regression.

Section snippets

Problems and background

From statistical point of view, the regression learning task is equivalent to modeling the joint distribution of explanatory and response variables. According to kernel density estimation (KDE) [3], joint distribution of explanatory variables X and response variables Y can be represented by weights of clustering centers W={Wi}, where WiRd and index i are used for iteration of all cluster centers, c={ci} is the distribution of clustering centers, and s is the smooth parameter. The learning task

Software architecture

The software is composed of 5 parts. (1) ‘utils.py’: Supporting utilities for csv file reading and Python dict operations. (2) ‘isoxnn2.py’ and ‘gng2.py’: GNG and SOINN algorithms. (3) ‘ui_isoinn.py’ and ‘ui_gng.py’) Programming interfaces for GNG and SOINN. (4) ‘gks.py’: SSL-GKS implementation (5) ‘reg_inn.py’ and ‘reg_gng.py’: Regression programming interfaces.

Software functionalities

Main functionalities are implemented by 4 Python classes listed below.

  • 1.

    class pygks.gks.GKS: By implementing Eq. (6), weights of

Empirical results

There are two sets of experiments on 6 datasets. First, the typical SSL setting is employed, where comparison results are on varying labeled datasets with labeling percentages growing. Second, we move to a real application, namely the traffic flow prediction. The datasets are downloaded from the Caltrans Performance measurement Systems (PEMS) database [8]. 6 days of data beginning from Nov. 24th, 2014 are chosen as training datasets, and the data from Nov. 30th, 2014 as the testing datasets.

Illustrative example

The following example is to show how to use the software.

Acknowledgments

This work was supported in part by the National Natural Science Foundations of China (Nos. 61272061 and 61301148).

Zhiyang Xiang received M.E. degree on computer science from Northwest A & F University, China. He is currently pursuing a Ph.D. degree in Hunan University, China. His research interests include neural networks algorithms and applications in information security.

References (18)

There are more references available in the full text version of this article.

Cited by (8)

View all citing articles on Scopus

Zhiyang Xiang received M.E. degree on computer science from Northwest A & F University, China. He is currently pursuing a Ph.D. degree in Hunan University, China. His research interests include neural networks algorithms and applications in information security.

Zhu Xiao received M.E. and Ph.D. degrees on signal processing both from Xidan University, China. He is now an associate professor and Ph.D. supervisor with College of Computer Science and Electronics Engineering, Hunan University, China. His primary research interests include wireless communications. His research interests also include pattern recognition algorithms.

Dong Wang received M.E. and Ph.D. degrees on computer science from Hunan University, China. He is a Ph.D. director and a director to overseas graduate students in College of Computer Science and Electronics Engineering, Hunan University. His main research interests are computer networks and vehicular multimedia networks.

Jianhua Xiao received B.E. from Jiangxi Normal University, China. She is currently pursing a Ph.D. degree in Hunan University, China. Her research interests include pattern recognition and machine learning.

View full text