Feature detectors by autoencoders: Decomposition of input patterns into atomic features by neural networks

Abstract

In this paper, we propose a feature detector for the neural network. Our feature detector aims to decompose input patterns into minimum constituents or atomic features. Atomic features are classified into features, common to all the input patterns and features, specific to each pattern. Thus, our feature detector is mainly composed of a common feature detector, distinctive feature detectors. The other two components are an information maximizer and an error minimizer. The distinctive feature detector is realized by the information maximizer, which increases the information, specific to each pattern as much as possible. The error minimizer is a device to minimize the difference between targets and outputs, that is, a usual neural network. We applied our feature detector to two problems: detection of vertical and horizontal bars and the phonological feature detection. In both cases, experimental results confirmed that distinctive features could clearly be extracted and that the common feature detector could extract features, as close as possible to the common features.