Abstract
Advances in unsupervised learning have allowed the efficient learning of feature representations from large sets of unlabeled data. This paper evaluates visual features learned through unsupervised learning, specifically comparing biasing methods using Gaussian filters on a single-layer network. Using the restricted Boltzmann machine, features emerging through training on image data are compared by classification performance on standard datasets. When Gaussian filters are convolved with adjacent hidden layer activations from a single example during training, topographies emerge where adjacent features become tuned to slightly varying stimuli. When Gaussian filters are applied to the visible nodes, images become blurrier; training on these images leads to less localized features being learned. The networks are trained and tested on the CIFAR-10, STL-10, COIL-100, and MNIST datasets. It is found that the induction of topography or simple image blurring during training produce better features as evidenced by the consistent and notable increase in classification results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Coates, A., Lee, H., Ng, A.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of International Conference on Artificial Intelligence and Statistics, pp. 215–223 (2011)
Goh, H., Kusmierz, L., Lim, J.-H., Thome, N., Cord, M.: Learning invariant color features with sparse topographic restricted Boltzmann machines. In: Proceedings of IEEE Conference on Image Processing, pp. 1241–1244 (2011)
Hyvärinen, A., Hoyer, P., Inki, M.: Topographic independent component analysis. Neural Comput. 13, 1527–1558 (2001)
Kavukcuoglu, K., Ranzato, M., Fergus, R., LeCun, Y.: Learning invariant features through topographic filter maps. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1605–1612 (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1904–1916 (2015)
Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of International Conference on Document Analysis and Recognition, pp. 958–962 (2003)
Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., Ng, A.: Building high-level features with large scale unsupervised learning. In: Proceedings of International Conference on Machine Learning, pp. 81–88 (2012)
Sermanet, P., Kavukcuoglu, K., Chintala, S., LeCun, Y.: Pedestrian detection with unsupervised multi-stage feature learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3626–3633 (2013)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report, University of Toronto (2009)
Nayar, S., Nene, S.A., Murase, H.: Real-time 100 object recognition system. IEEE Trans. Pattern Anal. Mach. Intell. 18, 1186–1198 (1996)
Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)
Lee, H., Ekanadham, C., Ng, A.: Sparse deep belief net model for visual area V2. In: Proceedings of Advances in Neural Information Processing Systems, pp. 873–880 (2008)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Mobahi, H., Collobert, R., Weston, J.: Deep learning from temporal coherence in video. In: Proceedings of International Conference on Machine Learning, pp. 737–744 (2009)
Larochelle, H., Bengio, S.: Classification using discriminative restricted Boltzmann machines. In: Proceedings of International Conference on Machine Learning, pp. 536–543 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Yogeswaran, A., Payeur, P. (2016). Improving Visual Feature Representations by Biasing Restricted Boltzmann Machines with Gaussian Filters. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2016. Lecture Notes in Computer Science(), vol 10072. Springer, Cham. https://doi.org/10.1007/978-3-319-50835-1_74
Download citation
DOI: https://doi.org/10.1007/978-3-319-50835-1_74
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50834-4
Online ISBN: 978-3-319-50835-1
eBook Packages: Computer ScienceComputer Science (R0)