Lexing Xie
ABSTRACT
Knowledge acquisition, representation, and reasoning has been one of the long-standing challenges in artificial intelligence and related application areas. Only in the past few years, massive amounts of structured and semi-structured data that directly or indirectly encode human knowledge be- came widely available, turning the knowledge representation problems into a computational grand challenge with feasible solutions in sight. The research and development on knowledge bases is becoming a lively fusion area among web in- formation extraction, machine learning, databases and information retrieval, with knowledge over images and multimedia emerging as another new frontier of representation and acquisition. This tutorial aims to present a gentle overview of knowledge bases on text and multimedia, including representation, acquisition, and inference. In particular, the 2015 edition of the tutorial will include recent progress from several active research communities: web, natural language processing, and computer vision and multimedia.
- S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh. Vqa: Visual question answering. arXiv preprint arXiv:1505.00468, 2015.Google Scholar
- J. Borge-Holthoefer and A. Arenas. Semantic networks: Structure and dynamics. Entropy, 12(5):1264--1302, 2010.Google ScholarCross Ref
- A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschka Jr, and T. M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, 2010.Google ScholarDigital Library
- X. Chen and C. Lawrence Zitnick. Mind's eye: A recurrent visual representation for image caption generation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.Google ScholarCross Ref
- X. Chen, A. Shrivastava, and A. Gupta. Neil: Extracting visual knowledge from web data. ICCV, 2013. Google ScholarDigital Library
- J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, June 2015.Google ScholarCross Ref
- X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang. Knowledge vault. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '14, pages 601--610, 2014. Google ScholarDigital Library
- O. Etzioni, M. Banko, S. Soderland, and D. S. Weld. Open information extraction from the web. Communications of the ACM, 51(12):68--74, 2008. Google ScholarDigital Library
- H. Fang, S. Gupta, F. Iandola, R. K. Srivastava, L. Deng, P. Dollar, J. Gao, X. He, M. Mitchell, J. C. Platt, C. Lawrence Zitnick, and G. Zweig. From captions to visual concepts and back. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.Google ScholarCross Ref
- H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu. Are you talking to a machine? dataset and methods for multilingual image question answering. arXiv preprint arXiv:1505.05612, 2015.Google Scholar
- T. L. Griffiths, M. Steyvers, and A. Firl. Google and the mind: Predicting fluency with pagerank. Psychological Science, 18(12):1069--1076, 2007.Google ScholarCross Ref
- M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, pages 853--899, 2013. Google ScholarDigital Library
- G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating simple image descriptions. In CVPR, pages 1601--1608. IEEE, 2011. Google ScholarDigital Library
- D. B. Lenat. Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11):33--38, 1995. Google ScholarDigital Library
- G. Li, Z. Ming, H. Li, and T.-S. Chua. Video reference: question answering on youtube. MM '09, pages 773--776, 2009. Google ScholarDigital Library
- H. Liu and P. Singh. ConceptNet -- a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211--226, 2004. Google ScholarDigital Library
- G. Miller. WordNet: a lexical database for English. Communications of the ACM, 38(11):39--41, 1995. Google ScholarDigital Library
- M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich. A review of relational machine learning for knowledge graphs: From multi-relational link prediction to automated knowledge graph construction. arXiv preprint arXiv:1503.00759, 2015.Google Scholar
- P. Perona. Vision of a visipedia. Proceedings of the IEEE, 98(8):1526--1534, 2010.Google ScholarCross Ref
- M. Ren, R. Kiros, and R. Zemel. Image question answering: A visual semantic embedding model and a new dataset. arXiv preprint arXiv:1505.02074, 2015.Google Scholar
- S. Riedel, L. Yao, A. Mccallum, and B. M. Marlin. Relation Extraction with Matrix Factorization and Universal Schemas. In HLT-NAACL '13, 2013.Google Scholar
- F. M. Suchanek, G. Kasneci, and G. Weikum. Yago: A Core of Semantic Knowledge. In WWW '07, 2007. Google ScholarDigital Library
- O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.Google ScholarCross Ref
- W. Wu, H. Li, H. Wang, and K. Q. Zhu. Probase: a probabilistic taxonomy for text understanding. SIGMOD, 2012. Google ScholarDigital Library
- L. Xie and X. He. Picture tags and world knowledge: Learning tag relations from visual semantic sources. In ACM Multimedia, October 2013. Google ScholarDigital Library
Index Terms
- Learning Knowledge Bases for Multimedia in 2015
Recommendations
Learning Knowledge Bases for Text and Multimedia
MM '14: Proceedings of the 22nd ACM international conference on MultimediaKnowledge acquisition, representation, and reasoning has been one of the long-standing challenges in artificial intelligence and related application areas. Only in the past few years, massive amounts of structured and semi-structured data that directly ...
Learning and Knowledge Management: Learning as an Integrative Role for Knowledge Creation
ICICM '13: Proceedings of the 2013 International Conference on Informatics and Creative MultimediaKnowledge management plays an important role to increase company performance. Although widely recognized importance of knowledge, there is relatively little understanding about an integrative role of learning in knowledge creation. In this paper, we ...
Revising (multi-) media learning principles by applying a differentiated knowledge concept
This paper reports on a study investigating the effect of single-media and multimedia presentations on the resulting knowledge. First, this study investigated the stability of established multimedia learning principles by measuring acquired knowledge in ...
Comments