Abstract
Current natural language processing analysis is mainly based on two different kinds of representation: structured data or word embeddings (WE). Modern applications also develop some kind of processing after based on these latter representations. Several works choose to structure data by building WE-based semantic trees that hold the maximum amount of semantic information. Many different approaches have been explores, but only a few comparisons have been performed. In this work we developed a compatible tuple base representation for Stanford dependency trees that allows us to compared two different ways of constructing tuples. Our measures mainly comprise tree reconstruction error, mean error over batches of given trees and performance on training stage.
We thank the support of Instituto Politécnico Nacional, SIP-IPN, COFAA-IPN. Partially funded by CONACyT (Language Technologies Thematic Network projects 260178, 271622).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)
Erhan, D., Bengio, Y., Courville, A., Manzagol, P.A., Vincent, P., Bengio, S.: Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11(Feb), 625–660 (2010)
Gardent, C., Kallmeyer, L.: Semantic construction in feature-based TAG. In: Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics, vol. 1, pp. 123–130. Association for Computational Linguistics (2003)
Huang, E.: Paraphrase detection using recursive autoencoder (2011). http://nlp.stanford.edu/courses/cs224n/2011/reports/ehhuang.pdf
Kallmeyer, L.: Using an enriched tag derivation structure as basis for semantics. In: Proceedings of TAG+ 6 Workshop, pp. 127–136. Citeseer (2002)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Prechelt, L.: Early stopping - but when? In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, pp. 55–69. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49430-8_3
Pulman, S.G.: Basic parsing techniques: an introductory survey. In: Encyclopedia of Linguistics (1991)
Rush, A.M., Reichart, R., Collins, M., Globerson, A.: Improved parsing and POS tagging using inter-sentence consistency constraints. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1434–1444. Association for Computational Linguistics (2012)
Schnabel, T., Labutov, I., Mimno, D., Joachims, T.: Evaluation methods for unsupervised word embeddings. In: Proceedings of the Empirical Methods in Natural Language Processing (2015)
Sidorov, G., Velasquez, F., Stamatatos, E., Gelbukh, A., Chanona-Hernández, L.: Syntactic n-grams as machine learning features for natural language processing. Expert. Syst. Appl. 41(3), 853–860 (2014)
Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151–161. Association for Computational Linguistics (2011)
Swersky, K., Buchman, D., Freitas, N.D., Marlin, B.M., et al.: On autoencoders and score matching for energy based models. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011), pp. 1201–1208 (2011)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(Dec), 3371–3408 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Calvo, H., Rivera-Camacho, R., Barrón-Fernndez, R. (2018). Semantic Loss in Autoencoder Tree Reconstruction Based on Different Tuple-Based Algorithms. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2018. Lecture Notes in Computer Science(), vol 11047. Springer, Cham. https://doi.org/10.1007/978-3-030-01132-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-01132-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01131-4
Online ISBN: 978-3-030-01132-1
eBook Packages: Computer ScienceComputer Science (R0)