Skip to main content
SpringerLink
Log in

Automatic ontology construction from text: a review from shallow to deep learning trend

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

The explosive growth of textual data on the web coupled with the increase on demand for ontologies to promote the semantic web, have made the automatic ontology construction from the text a very promising research area. Ontology learning (OL) from text is a process that aims to (semi-) automatically extract and represent the knowledge from text in machine-readable form. Ontology is considered one of the main cornerstones of representing the knowledge in a more meaningful way on the semantic web. Usage of ontologies has proven to be beneficial and efficient in different applications (e.g. information retrieval, information extraction, and question answering). Nevertheless, manually construction of ontologies is time-consuming as well extremely laborious and costly process. In recent years, many approaches and systems that try to automate the construction of ontologies have been developed. This paper reviews various approaches, systems, and challenges of automatic ontology construction from the text. In addition, it also discusses ways the ontology construction process could be enhanced in the future by presenting techniques from shallow learning to deep learning (DL).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Abney S (1997) Part-of-speech tagging and partial parsing. In: Young S, Bloothooft G (eds) Corpus-based methods in language and speech processing, vol. 2. Springer, Netherlands, pp 118–136

    Chapter  Google Scholar 

  • Albukhitan S, Helmy T, Alnazer A (2017) Arabic ontology learning using deep learning. Paper presented at the Proceedings of the international conference on web intelligence, Leipzig, Germany

  • Arel I, Rose DC, Karnowski TP (2010) Deep machine learning—a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell Mag 5:13–18. https://doi.org/10.1109/mci.2010.938364

    Article  Google Scholar 

  • Arguello Casteleiro M, Maseda Fernandez D, Demetriou G, Warren R, Fernandez-Prieto MJ, Des Diz J, Nenadic G, Keane J, Robert S (2017) A case study on sepsis using PubMed and deep learning for ontology learning. In: Informatics for health: connected citizen-led wellness and population health, vol 235. pp 516–520. https://doi.org/10.3233/978-1-61499-753-5-516

  • Baldi P (2012) Autoencoders, unsupervised learning, and deep architectures. Paper presented at the proceedings of ICML workshop on unsupervised and transfer learning, Bellevue, Washington, USA

  • Basegmez E (2014) The next generation neural networks: deep learning and spiking neural networks. In: Advanced seminar in technical university of Munich, Munchen, 2014, pp 1–40

  • Bengio Y, LeCun Y (2007) Scaling learning algorithms towards AI. In: Large-scale kernel machines, p 34

  • Boytcheva S (2002) Overview of inductive logic programming (ILP) systems. Cybern Inf Technol 1:27–36

    Google Scholar 

  • Budanitsky A (1999) Lexical semantic relatedness and its application in natural language processing. University of Toronto

  • Buitelaar P, Cimiano P, Magnini B (2005) Ontology learning from text: an overview. In: Ontology learning from text: methods, evaluation and applications, 123:3–12

  • Chen X-W, Lin X (2014) Big data deep learning: challenges and perspectives. IEEE Access 2:514–525

    Article  Google Scholar 

  • Chen Y, Li W, Liu Y, Zheng D, Zhao T (2010) Exploring deep belief network for chinese relation extraction. In: Proceedings of the joint conference on Chinese language processing (CLP’10), pp 28–29

  • Chicco D, Sadowski P, Baldi P (2014) Deep autoencoder neural networks for gene ontology annotation predictions. Paper presented at the Proceedings of the 5th ACM conference on bioinformatics, computational biology, and health informatics, Newport Beach, California

  • Cimiano P, Völker J (2005) Text2Onto. In: Montoyo A, Muńoz R, Métais E (eds) Natural language processing and information systems. Proceedings of 10th international conference on applications of natural language to information systems, NLDB 2005, Alicante, Spain, June 15–17, 2005. Springer, Berlin, pp 227–238. https://doi.org/10.1007/11428817_21

  • Cimiano P, Hotho A, Staab S (2005) Learning concept hierarchies from text corpora using formal concept analysis. J Artif Intell Res 24:305–339

    Article  Google Scholar 

  • Cohen WW (2005) Stacked sequential learning. Carnegie-Mellon Univ Pittsburgh PA School of Computer Science

  • Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on machine learning. ACM, pp 160–167

  • Deng L (2012) Three classes of deep learning architectures and their applications: a tutorial survey APSIPA transactions on signal and information processing

  • Deng L (2014) A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on signal and information processing 3:1–29. https://doi.org/10.1017/atsip.2013.9

  • Deng L, Yu D (2011) Deep convex net: A scalable architecture for speech pattern classification. In: Annual conference of the international speech communication association, 2011. Interspeech, pp 2285–2288

  • Deng L, Yu D (2014) Deep learning. Signal Process 7:3–4

    Google Scholar 

  • Drymonas E, Zervanou K, Petrakis EGM (2010) Unsupervised ontology acquisition from plain texts: the OntoGain system. In: Natural language processing and information systems. Springer, Berlin, pp 277–287

  • El-Kilany A, Tazi NE, Ezzat E (2017) Building relation extraction templates via unsupervised learning. Paper presented at the Proceedings of the 21st international database engineering and applications symposium, Bristol, United Kingdom

  • Etzioni O, Banko M, Soderland S, Weld DS (2008) Open information extraction from the web. Commun ACM 51:68–74. https://doi.org/10.1145/1409360.1409378

    Article  Google Scholar 

  • Faure D, Nédellec C (1998) Asium: Learning subcategorization frames and restrictions of selection. In: Kodratoff Y (ed) Text mining workshop, 10th European conference on machine learning (ECML 98), Chemnitz, Germany.

  • Faure D, Poibeau T (2000a) First experiments of using semantic knowledge learned by ASIUM for information extraction task using INTEX. In: Ontology learning ECAI-2000 workshop, pp 7–12

  • Faure D, Poibeau T (2000b) First experiments of using semantic knowledge learned by ASIUM for information extraction task using INTEX. In: Proceedings of the ECAI workshop on ontology learning

  • Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. In: Iberoamerican congress on pattern recognition. Springer, pp 14–36

  • Fleischhacker D, Völker J (2011) Inductive learning of disjointness axioms. In: On the move to meaningful internet systems: OTM 2011. Springer, Berlin, pp 680–697

  • Gamallo P, Gonzalez M, Agustini A, Lopes G, De Lima VS (2002) Mapping syntactic dependencies onto semantic relations. In: Proceedings of the ECAI workshop on machine learning and natural language processing for ontology engineering, pp 15–22

  • Gillani Andleeb S (2015) From text mining to knowledge mining: An integrated framework of concept extraction and categorization for domain ontology. Doctoral dissertation, Budapesti Corvinus Egyetem

  • Gómez-Pérez A, Manzano-Macho D (2003) A survey of ontology learning methods and techniques. Deliverable 1.5, Onto Web consortium

  • Grefenstette E, Blunsom P, de Freitas N, Hermann KM (2014) A deep architecture for semantic parsing. arXiv preprint arXiv:14047296

  • Hahn U, Marko KG (2002) Ontology and lexicon evolution by text understanding. In: Proceedings of the ECAI 2002 workshop on machine learning and natural language processing for ontology engineering (OLT 2002), Lyon, France

  • Hahn U, Romacker M (2001) The SYNDIKATE text knowledge base generator. Paper presented at the Proceedings of the first international conference on Human language technology research, San Diego

  • Hassan A, Mahmood A (2018) Convolutional recurrent deep learning model for sentence classification. IEEE Access 6:13949–13957. https://doi.org/10.1109/ACCESS.2018.2814818

    Article  Google Scholar 

  • Herrera RG (2014) Knowledge management systems based on ontology learning. Universidad de Granada, Granada

    Google Scholar 

  • Hinton GE (2009) Deep belief networks. Scholarpedia 4:5947

    Article  Google Scholar 

  • Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507

    Article  MathSciNet  Google Scholar 

  • Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554

    Article  MathSciNet  Google Scholar 

  • Hu S, Zuo Y, Wang L, Liu P (2016) A review about building hidden layer methods of deep learning. J Adv Inf Technol 7:13–22. https://doi.org/10.12720/jait.7.1.13-22

    Article  Google Scholar 

  • Huang FJ, Boureau Y-L, LeCun Y (2007) Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  • Jiang X, Tan A-H (2005) Mining ontological knowledge from domain-specific text documents. In: Fifth IEEE international conference on data mining. IEEE

  • Jiang X, Tan AH (2010) CRCTOL: a semantic-based domain ontology learning system. J Am Soc Inf Sci Technol 61:150–168

    Article  Google Scholar 

  • Kietz J-U, Maedche A, Volz R (2000) A method for semi-automatic ontology acquisition from a corporate intranet. In: EKAW-2000 workshop “Ontologies and Text”, Juan-Les-Pins, France

  • Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:14085882

  • Klein D, Manning CD (2003) Accurate unlexicalized parsing. Paper presented at the Proceedings of the 41st Annual meeting on association for computational linguistics—volume 1, Sapporo, Japan

  • Kuang Z, Yu J, Li Z, Zhang B, Fan J (2018) Integrating multi-level deep learning and concept ontology for large-scale visual recognition. Pattern Recogn 78:198–214. https://doi.org/10.1016/j.patcog.2018.01.027

    Article  Google Scholar 

  • Le QV (2015) A tutorial on deep learning part 2: autoencoders, convolutional neural networks and recurrent neural networks. Google Brain 1–20

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436

    Article  Google Scholar 

  • Lin D, Pantel P (2001) DIRT- discovery of inference rules from text. Paper presented at the proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, San Francisco, California

  • Liu Q, Xu K, Zhang L, Wang H, Yu Y, Pan Y (2008) Catriple: extracting triples from wikipedia categories. In: Asian semantic web conference. Springer, pp 330–344

  • Maedche A, Staab S (2000) Discovering conceptual relations from text. In: Horn W (ed) Proceedings of the 14th European conference on artificial intelligence, Berlin, Germany. ECAI'00. IOS Press, pp 321–325

  • Maedche A, Volz R (2001) The text-to-onto ontology extraction and maintenance environment. In: Proceedings of the ICDM-workshop on integrating data mining and knowledge management, San Jose, California

  • Maimon O, Browarnik A (2015) Ontology learning from text: why the ontology learning layer cake is not viable. Int J Signs Semiot Syst 4:1–14. https://doi.org/10.4018/ijsss.2015070101

    Article  Google Scholar 

  • Mathews KA, Kumar PS (2017) Extracting ontological knowledge from textual descriptions through grammar-based transformation. Paper presented at the proceedings of the knowledge capture conference, Austin, TX

  • Mishra S, Jain S (2015) A study of various approaches and tools on ontology. In: 2015 IEEE international conference on computational intelligence and communication technology (CICT), pp 57–61. https://doi.org/10.1109/cict.2015.43

  • Mo D (2012) A survey on deep learning: one small step toward AI Dept Computer Science, Univ of New Mexico, USA

  • Morin E (1999) Automatic acquisition of semantic relations between terms from technical corpora. In: Proceedings of the fifth international congress on terminology and knowledge engineering-TKE’99

  • Mudhsh BAD, Al-Takhayinh AH, Al-Dala’ien OA (2015) Immediate constituent analysis (ICA). Int J Sci Res Publ 5(6)

  • Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1

    Article  Google Scholar 

  • Nakaya N, Kurematsu M, Yamaguchi T (2002) A domain ontology development environment using a MRD and text corpus. In: Proceedings of the fifth joint conference on knowledge-based software engineering frontiers in artificial intelligence and applications, pp 242–251

  • Neelakantan AR (2017) Knowledge representation and reasoning with deep neural networks. Doctoral Dissertation, University of Massachusetts, Amherst

  • Nivre J (2004) Incrementality in deterministic dependency parsing. Paper presented at the proceedings of the workshop on incremental parsing: bringing engineering and cognition together, Barcelona, Spain

  • Oliveira A, Pereira FC, Cardoso A (2001) Automatic reading and learning from text. In: Proceedings of the international symposium on artificial intelligence (ISAI)

  • Park J, Cho W, Rho S (2010) Evaluating ontology extraction tools using a comprehensive evaluation framework. Data Knowl Eng 69:1043–1061. https://doi.org/10.1016/j.datak.2010.07.002

    Article  Google Scholar 

  • Pereira FC, Oliveira A, Cardoso A (2000) Extracting concept maps with clouds. In: Proceedings of the Argentine symposium of artificial intelligence (ASAI)

  • Petrucci G, Ghidini C, Rospocher M (2016) Ontology learning in the deep. In: Knowledge engineering and knowledge management. Springer, Cham, pp 480-495

  • Sabou M, Wroe C, Goble C, Mishne G (2005) Learning domain ontologies for Web service descriptions: an experiment in bioinformatics. Paper presented at the Proceedings of the 14th international conference on World Wide Web, Chiba, Japan

  • Salakhutdinov R, Hinton G (2007) Semantic hashing. RBM 500:500

    Google Scholar 

  • Sánchez D, Moreno A (2008) Learning non-taxonomic relationships from web documents for domain ontology construction. Data Knowl Eng 64:600–623

    Article  Google Scholar 

  • Sarikaya R, Hinton GE, Deoras A (2014) Application of deep belief networks for natural language understanding. IEEE/ACM Trans Audio Speech Lang Process 22:778–784

    Article  Google Scholar 

  • Sekiuchi R, Aoki C, Kurematsu M, Yamaguchi T (1998) DODDLE: a domain ontology rapid development environment. In: Lee H-Y, Motoda H (eds) PRICAI’98: topics in artificial intelligence. Proceedings of 5th Pacific Rim international conference on artificial intelligence Singapore. Springer, Berlin, pp 194–204. https://doi.org/10.1007/bfb0095269

  • Shamsfard M, Barforoush AA (2003) The state of the art in ontology learning: a framework for comparison. Knowl Eng Rev 18:293–316

    Article  Google Scholar 

  • Shamsfard M, Barforoush AA (2004) Learning ontologies from natural language texts. Int J Hum Comput Stud 60:17–63

    Article  Google Scholar 

  • Sombatsrisomboon R, Matsuo Y, Ishizuka M (2003) Acquisition of hypernyms and hyponyms from the WWW. In: Proceedings of the 2nd international workshop on active mining

  • Specia L, Motta E (2006) A hybrid approach for relation extraction aimed at the semantic web. In: International conference on flexible query answering systems. Springer, pp 564–576

  • Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. Paper presented at the proceedings of the 16th international conference on World Wide Web, Banff, Alberta, Canada

  • Völker J, Hitzler P, Cimiano P (2007) Acquisition of OWL DL axioms from lexical resources. In: The semantic web: research and applications. Springer, Berlin, pp 670–685

  • Wang H (2015) Semantic deep learning. University of Oregon, Oregon

    Google Scholar 

  • Wang J, Liu J, Kong L (2018) Ontology construction based on deep learning. In: Advances in computer science and ubiquitous computing. Springer, Singapore, pp 505–510

  • Wong W, Liu W, Bennamoun M (2007) Tree-Traversing Ant Algorithm for term clustering based on featureless similarities. Data Min Knowl Disc 15:349–381. https://doi.org/10.1007/s10618-007-0073-y

    Article  MathSciNet  MATH  Google Scholar 

  • Wong W, Liu W, Bennamoun M (2012) Ontology learning from text: a look back and into the future. ACM Comput Surv (CSUR) 44:20

    Article  Google Scholar 

  • Zelle JM, Mooney RJ (1993) Learning semantic grammars with constructive inductive logic programming. In: AAAI, pp 817–822

  • Zhang J, Liu J, Wang X (2016) Simultaneous entities and relationship extraction from unstructured text. Int J Database Theory Appl 9:151–160

    Article  Google Scholar 

  • Zhong B, Liu J, Du Y, Liaozheng Y, Pu J (2016) Extracting attributes of named entity from unstructured text with deep belief network. Int J Database Theory Appl 9:187–196

    Article  Google Scholar 

  • Zhou L (2007) Ontology learning: state of the art and open issues. Inf Technol Manag 8:241–252

    Article  Google Scholar 

  • Zouaq A (2011) An overview of shallow and deep natural language processing for ontology learning. In: Wong W, Liu W, Bennamoun M (eds) Ontology learning and knowledge discovery using the web: challenges and recent advances, vol 2. Information Science Reference (IGI Global), USA, pp 16–37

  • Zouaq A, Gasevic D, Hatala M (2011b) Towards open ontology learning and filtering. Inf Syst 36:1064–1081. https://doi.org/10.1016/j.is.2011.03.005

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Sciences, Universiti Sains Malaysia, 11800, Gelugor, Pulau Pinang, Malaysia

    Fatima N. Al-Aswadi, Huah Yong Chan & Keng Hoon Gan

  2. Faculty of Computer Sciences and Engineering, Hodeidah University, Hodeidah, Yemen

    Fatima N. Al-Aswadi

Authors
  1. Fatima N. Al-Aswadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huah Yong Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keng Hoon Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatima N. Al-Aswadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Aswadi, F.N., Chan, H.Y. & Gan, K.H. Automatic ontology construction from text: a review from shallow to deep learning trend. Artif Intell Rev 53, 3901–3928 (2020). https://doi.org/10.1007/s10462-019-09782-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09782-9

Keywords

Search

Navigation