Improving Feature Extraction Performance of Greedy Network-Growing Algorithm

Abstract

In this paper, we propose a new network-growing method to extract explicit features in complex input patterns. In [1], we have so far proposed a new type of network-growing algorithm called greedy network-growing algorithm and used the sigmoidal activation function for competitive unit outputs. However, the method with the sigmoidal activation function is introduced to be not so sensitive to input patterns. Thus, we have observed that in some cases final representations obtained by the method do not necessarily describe faithfully input patterns. To remedy this shortcoming, we employ the inverse of distance between input patterns and connection weights for competitive unit outputs. As the distance is smaller, competitive units are more strongly activated. Thus, winning units tend to represent input patterns more faithfully than the previous method with the sigmoidal activation function.