Abstract
The goal of this paper is to train a model based on the relation between two instances that represent the same unknown class. This task is inspired by the Symbol Grounding Problem and the association learning between modalities in infants. We propose a novel model called Classless Association that has two parallel Multilayer Perceptrons (MLPs) with a EM-training rule. Moreover, the training relies on matching the output vectors of the MLPs against a statistical distribution as alternative loss function because of the unlabeled data. In addition, the output classification of one network is used as target of the other network, and vice versa for learning the agreement between both unlabeled sample. We generate four classless datasets based on MNIST, where the input is two different instances of the same digit. Furthermore, our classless association model is evaluated against two scenarios: totally supervised and totally unsupervised. In the first scenario, our model reaches a good performance in terms of accuracy and the classless constraint. In the second scenario, our model reaches better results against two clustering algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asano, M., Imai, M., Kita, S., Kitajo, K., Okada, H., Thierry, G.: Sound symbolism scaffolds language development in preverbal infants. Cortex 63, 196–205 (2015)
Balaban, M.T., Waxman, S.R.: Do words facilitate object categorization in 9-month-old infants? J. Exp. Child Psychol. 64(1), 3–26 (1997)
Casey, R.G.: Text OCR by Solving a Cryptogram. International Business Machines Incorporated, Thomas J. Watson Research Center, New York (1986)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39(1), 1–38 (1977)
Gershkoff-Stowe, L., Smith, L.B.: Shape and the first hundred nouns. Child Dev. 75(4), 1098–114 (2004)
Harnad, S.: The symbol grounding problem. Phys. D 42(1), 335–346 (1990)
Hsu, Y.C., Kira, Z.: Neural network-based clustering using pairwise constraints. arXiv preprint arXiv:1511.06321 (2015)
Khan, I., Saffari, A., Bischof, H.: Tvgraz: multi-modal learning of object categories by combining textual and visual features. In: AAPR Workshop, pp. 213–224 (2009)
Knight, K., Nair, A., Rathod, N., Yamada, K.: Unsupervised analysis for decipherment problems. In: Proceedings of the COLING/ACL on Main Conference Poster Sessions, pp. 499–506. Association for Computational Linguistics (2006)
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: ACM International Conference on Multimedia, pp. 251–260 (2010)
Raue, F., Palacio, S., Breuel, T.M., Byeon, W., Dengel, A., Liwicki, M.: Symbolic association using parallel multilayer perceptron. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9887, pp. 347–354. Springer, Cham (2016). doi:10.1007/978-3-319-44781-0_41
Raue, F., Palacio, S., Dengel, A., Liwicki, M.: Classless association using neural networks. In: ICLR 2017 (2017, submitted). https://openreview.net/forum?id=ryh_8f9lg¬eId=ryh_8f9lg
Sutskever, I., Jozefowicz, R., Gregor, K., Rezende, D., Lillicrap, T., Vinyals, O.: Towards principled unsupervised learning. arXiv preprint arXiv:1511.06440 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Raue, F., Palacio, S., Dengel, A., Liwicki, M. (2017). Classless Association Using Neural Networks. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10614. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-68612-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68611-0
Online ISBN: 978-3-319-68612-7
eBook Packages: Computer ScienceComputer Science (R0)