Skip to main content
SpringerLink
Log in
Book cover

International Conference on Database and Expert Systems Applications

DEXA 2016: Database and Expert Systems Applications pp 431–445Cite as

Ontology-Based Deep Restricted Boltzmann Machine

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9827))

Abstract

Deep neural networks are known for their capabilities for automatic feature learning from data. For this reason, previous research has tended to interpret deep learning techniques as data-driven methods, while few advances have been made from knowledge-driven perspectives. We propose to design a semantic rich deep learning model from a knowledge driven perspective, by introducing formal semantics into deep learning process. We propose ontology-based deep restricted Boltzmann machine (OB-DRBM), in which we use ontology to guide architecture design of deep restricted Boltzmann machines (DRBM), as well as to assist in their training and validation processes. Our model learns a set of related semantic-rich data representations from both formal semantics and data distribution. Representations in this set correspond to concepts at various semantic levels in a domain ontology. We show that our model leads to an improved performance, when compared with conventional deep learning models in classification tasks.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  2. Le, H.S., Oparin, I., Allauzen, A., Gauvain, J., Yvon, F.: Structured output layer neural network language models for speech recognition. IEEE Trans. Audio Speech Lang. Process. 21(1), 197–206 (2013)

    Article  Google Scholar 

  3. Mikolov, T., Deoras, A., Kombrink, S., Burget, L., Cernockỳ, J.: Empirical evaluation and combination of advanced language modeling techniques. In: Annual Conference of the International Speech Communication Association, pp. 605–608 (2011)

    Google Scholar 

  4. Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep Boltzmann machines. In: Advances in Neural Information Processing Systems, pp. 2222–2230 (2012)

    Google Scholar 

  5. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  6. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Proceedings of the 13th European Conference on Computer Vision, pp. 818–833 (2014)

    Google Scholar 

  7. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  8. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2013). arXiv preprint arXiv:1312.6199

  9. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)

    Google Scholar 

  10. Dou, D., Wang, H., Liu, H.: Semantic data mining: a survey of ontology-based approaches. In: IEEE International Conference on Semantic Computing, pp. 244–251 (2015)

    Google Scholar 

  11. Balcan, N., Blum, A., Mansour, Y.: Exploiting ontology structures and unlabeled data for learning. In: Proceedings of the 30th International Conference on Machine Learning, pp. 1112–1120 (2013)

    Google Scholar 

  12. Salakhutdinov, R., Hinton, G.E.: Deep Boltzmann machines. In: International Conference on Artificial Intelligence and Statistics, pp. 448–455 (2009)

    Google Scholar 

  13. Kolb, B., Whishaw, I.Q.: Fundamentals of Human Neuropsychology. Macmillan, London (2009)

    Google Scholar 

  14. Rosch, E., Mervis, C.B., Gray, W.D., Johnson, D.M., Boyes-Braem, P.: Basic objects in natural categories. Cogn. Psychol. 8(3), 382–439 (1976)

    Article  Google Scholar 

  15. Socher, R., Lin, C.C., Manning, C., Ng, A.Y.: Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on Machine Learning, pp. 129–136 (2011)

    Google Scholar 

  16. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Pearson Education, Upper Saddle River (2003)

    MATH  Google Scholar 

  17. Consortium, T.G.O.: Creating the gene ontology resource: design and implementation. Genome Res. 11(8), 1425–1433 (2001)

    Article  Google Scholar 

  18. Lindberg, D., Humphries, B., McCray, A.: The unified medical language system. Methods Inf. Med. 32(4), 281–291 (1993)

    Google Scholar 

  19. NCBO: The National Center for Biomedical Ontology. http://www.bioontology.org/

  20. Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 301–306 (2011)

    Google Scholar 

  21. Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)

    Google Scholar 

  22. Gruber, T.R.: A translation approach to portable ontology specifications. Knowl. Acquis. 5(2), 199–220 (1993)

    Article  Google Scholar 

  23. Wimalasuriya, D.C., Dou, D.: Components for information extraction: ontology-based information extractors and generic platforms. In: Proceedings of the 19th ACM Conference on Information and Knowledge Management, pp. 9–18 (2010)

    Google Scholar 

  24. Dentler, K., Cornet, R., Ten Teije, A., De Keizer, N.: Comparison of reasoners for large ontologies in the OWL 2 EL profile. Seman. Web 2(2), 71–87 (2011)

    Google Scholar 

  25. Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: Pellet: a practical OWL-DL reasoner. Web Seman. 5(2), 51–53 (2007)

    Article  Google Scholar 

  26. Motik, B., Shearer, R., Horrocks, I.: Hypertableau reasoning for description logics. J. Artifi. Intell. Res. 36, 165–228 (2009)

    MathSciNet  MATH  Google Scholar 

  27. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  28. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013). arXiv preprint arXiv:1301.3781

  29. Lang, K.: Newsweeder: learning to filter netnews. In: Proceedings of the International Conference on Machine Learning, pp. 331–339 (1995)

    Google Scholar 

  30. AIMLAB: Ontologies. http://aimlab-server.cs.uoregon.edu/ontologies

  31. Pang, B., Lee, L.: Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the Annual Meeting on Association for Computational Linguistics, pp. 115–124 (2005)

    Google Scholar 

  32. Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, p. 1642 (2013)

    Google Scholar 

Download references

Acknowledgment

This work is supported by the NIH grant R01GM103309. We acknowledge Ellen Klowden for her contributions to the manuscript. We also thank anonymous reviewers for their constructive comments, which helped improve the paper.

Author information

Authors and Affiliations

  1. Computer and Information Science, University of Oregon, Eugene, USA

    Hao Wang, Dejing Dou & Daniel Lowd

Authors
  1. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dejing Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Lowd
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wang .

Editor information

Editors and Affiliations

  1. Clausthal University of Technology , Clausthal-Zellerfeld, Germany

    Sven Hartmann

  2. Victoria University of Wellington , Wellington, New Zealand

    Hui Ma

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Wang, H., Dou, D., Lowd, D. (2016). Ontology-Based Deep Restricted Boltzmann Machine. In: Hartmann, S., Ma, H. (eds) Database and Expert Systems Applications. DEXA 2016. Lecture Notes in Computer Science(), vol 9827. Springer, Cham. https://doi.org/10.1007/978-3-319-44403-1_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-44403-1_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44402-4

  • Online ISBN: 978-3-319-44403-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation