Achref Benarab
ABSTRACT
In this paper, we present a low-dimensional vector representation method for the concepts and instances of an ontology. The main idea is to transform the ontological entities into digestible data for machine learning and deep learning algorithms that only use digital inputs. The generated vectors will represent the semantics contained in the source ontology. We use the semantic relationships connecting the concepts as a landmark to train expert neural networks using the noise contrastive estimation technique to project them into a vector space specific to this relationship with weightings dependent on their frequency. The resulting vectors are then combined and fed into an autoencoder to generate a denser representation. The generated representation vectors can be used to find the semantically similar ontology entities, allowing creating a semantic network automatically. Thus, semantically similar ontology entities will have relatively close corresponding vector representations in the projection space.
- Thomas R Gruber. A translation approach to portable ontology specifications. Knowledge acquisition, 5(2):199--220, 1993. Google Scholar
Digital Library
- Nicola Guarino. Some organizing principles for a unified top-level ontology. In AAAI Spring Symposium on Ontological Engineering, pages 57--63. AAAI Press Menlo Park, 1997.Google Scholar
- ASUCION Gomez-Perez. Developpement recents en matiere de conception, de maintenance et d'utilisation des ontologies. Terminologies nouvelles, 19:9--20, 1999.Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.Google Scholar
- Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746--751, 2013.Google Scholar
- Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137--1155, 2003. Google ScholarDigital Library
- Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543, 2014.Google ScholarCross Ref
- Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association, 2010.Google ScholarCross Ref
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119, 2013. Google ScholarDigital Library
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701--710. ACM, 2014. Google ScholarDigital Library
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pages 1067--1077. International World Wide Web Conferences Steering Committee, 2015. Google ScholarDigital Library
- Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855--864. ACM, 2016. Google ScholarDigital Library
- Petar Ristoski, Jessica Rosati, Tommaso Di Noia, Renato De Leone, and Heiko Paulheim. Rdf2vec: Rdf graph embeddings and their applications. Semantic Web, (Preprint):1--32, 2018.Google Scholar
- Andriy Mnih and Koray Kavukcuoglu. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in neural information processing systems, pages 2265--2273, 2013. Google ScholarDigital Library
- Chris Dyer. Notes on noise contrastive estimation and negative sampling. arXiv preprint arXiv:1410.8251, 2014.Google Scholar
- Michael Gutmann and Aapo Hyvarinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 297--304, 2010.Google Scholar
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org. Google ScholarDigital Library
- Patricia L Whetzel, Natalya F Noy, Nigam H Shah, Paul R Alexander, Csongor Nyulas, Tania Tudorache, and Mark A Musen. Bioportal: enhanced functionality via new web services from the national center for biomedical ontology to access and use ontologies in software applications. Nucleic acids research, 39(suppl 2):W541--W545, 2011.Google Scholar
- Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9 (Nov):2579--2605, 2008.Google Scholar
Index Terms
- An Ontology Embedding Approach Based on Multiple Neural Networks
Recommendations
Multiple neural networks for a long term time series forecast
The artificial neural network (ANN) methodology has been used in various time series prediction applications. However, the accuracy of a neural network model may be seriously compromised when it is used recursively for making long-term multi-step ...
Self-distribution binary neural networks
AbstractIn this work, we study network binarization (i.e., binary neural networks, BNNs), which is one of the most promising techniques in network compression for convolutional neural networks (CNNs). Although prior work has introduced many binarization ...
Knowledge entity learning and representation for ontology matching based on deep neural networks
We study the task of ontology matching that is used mainly for solving the semantic heterogeneity problems, which concentrates on finding semantically related entities between different ontologies. Many previous works exploit the character-level or ...
Comments