skip to main content
10.1145/3399871.3399894acmotherconferencesArticle/Chapter ViewAbstractPublication PagesasseConference Proceedingsconference-collections
research-article

Cultural Heritage Content Management System by Deep Learning

Published: 03 July 2020 Publication History

Abstract

This research aims to develop a cultural heritage information management system with deep neural network. The cultural heritage information in case of Thai's architecture was used. The main contribution of this research was to develop an algorithm for retrieving information from image for telling story inside that image. The interesting information inside the image will be retrieved to present to the interested people who is interested in its contents. The development consists of telling stories from image. The appearance of the shape inside image can be used to distinct characteristics of image for example the era, architecture and style of image. The architecture was created including the story of the archaeological site through the learning of machine learning and image processing. The experimental results for a cultural heritage information management system with deep neural network was analyze by using the classification results of the proposed algorithms to classify era and architecture of the tested image. To test the performance of the purposed algorithms, images from the well-known historical area in Thailand were used which are image dataset in Phra Nakhon Si Ayutta province, Sukhothai province and Bangkok. The confusion matrix of the proposed algorithms gives the accuracy 80.67%, 79.35%and 82.47% in Ayutthaya era, Sukhothai era and Rattanakosin era. Results show that the proposed technique can efficiently find the correct descriptions compared to using the traditional method.

References

[1]
Andrej Karpathy, Li Fei-Fei. 2017. Deep Visual-Semantic Alignments for Generating Image Descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 39, Issue: 4, April 1 2017, pp. 664--676.
[2]
R. Socher, A. Karpathy, Q. V. Le, C. D. Manning, and A. Y. Ng. 2014. Grounded Compositional Semantics for finding and describing images with sentences. Transactions of the Association for Computational Linguistics, 2, 2014, pp. 207--218.
[3]
W. Zaremba, I. Sutskever, and O. Vinyals. 2014. Recurrent neural network regularization. preprint arXiv:1409.2329, 2014.
[4]
P. Young, A. Lai, M. Hodosh, and J. Hockenmaier. 2014. From image descriptions to Visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2 (2014).
[5]
M. Hodosh, P. Young, and J. Hockenmaier. 2013. Framing image description as a ranking task: data, models and evaluation metrics. Journal of Artificial Intelligence Research
[6]
Hao Su, Fan Wang, Li Yi, Leonidas J. Guibas. 2014. 3D-Assisted Image Feature Synthesis for Novel Views of an Object. CoRR abs/1412.0003.
[7]
Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, David Forsyth. 2010. Every Picture Tells a Story: Generating Sentences From Images. Computer Vision-ECCV 2010 Lecture Notes in Computer Science, Volume 6314, 2010, pp 15--29.
[8]
L. Deng and D. Yu. 2013. Deep Learning: Methods and Applications Foundations and Trends R in Signal Processing, vol. 7, nos. 3-4, pp. 197--387.
[9]
Bengio, Y.; Courville, A.; Vincent, P. 2013. Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(8): 1798--1828.
[10]
Bello-Orgaz G, Jung JJ, Camacho D. 2016. Social big data: Recent achievements and new challenges. Inf Fusion 28:45--592.
[11]
Chareonla C. 1981. Buddhist Arts of Thailand. Master Thesis, Magadh University, India.
[12]
Farhadi A, Hejrati SMM, Sadeghi MA, Young P, Rashtchian C., Hockenmaier J, Forsyth DA. 2010. Every picture tells a story: Generating sentences from images. In Proceedings of the 11th European conference on computer vision (ECCV 2010), Heraklion, Crete, Greece, September 5-11, Lecture Notes in Computer Science, vol 6314, Springer, pp 15--29.
[13]
Hodosh M, Young P, Hockenmaier J. 2013. Framing image description as a ranking task: Data, models and evaluation metrics. J Artif Intell Res 47(1):853--8995.
[14]
Jung JJ. 2015. Big bibliographic data analytics by random walk model. Mobile Networks and Applications 20(4):533--5376.
[15]
Jung JJ. 2016. Exploiting geotagged resources for spatial clustering on social network services. Concurr Comput: Pract Experience, 28(4):1356--13677.
[16]
Karpathy A, Li F. 2015. Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the 2015 IEEE conference on computer vision and pattern recognition, VPR2015, Boston, MA, USA, June 7-12, 2015, pp 3128--3137.
[17]
Lee O-J, Jung JE. 2017. Sequence clustering-based automated rule generation for adaptive complex event processing. Futur Gener Comput Syst 66:100--1099.
[18]
Lowe DG. 1999. Object recognition from local scale-invariant features. In: Proceedings of the international conference on com-puter vision, Kerkyra, Corfu, Greece, September 20-25. IEEE Computer Society, pp 1150--1157.
[19]
Lowe DG. 2004. Distinctive image features from scale-invariant key points. Int J Comput Vis 60(2):91--110.
[20]
Nguyen DT, Jung JE. 2017. Real-time event detection for online behavioral analysis of big social data. Futur Gener Comput Syst, 66:137--14512.
[21]
Socher R, Karpathy A, Le QV, Manning CD, Ng AY. 2014. Grounded compositional semantics for finding and describing images with sentences. Trans Assoc Comput Linguist, 2:207--21813.
[22]
Su H, Wang F, Yi L, Guibas LJ. 2014. 3d-assisted image feature synthesis for novel views of an object. CoRR arXiv: 1412000314.
[23]
Young P, Lai A, Hodosh M, Hockenmaier J. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Trans Assoc Comput Linguist 2:67--7815.
[24]
Zaremba W, Sutskever I, Vinyals O. 2014. Recurrent neural network regularization. CoRR arXiv:14092329
[25]
Fei-Fei Li. 2018. "CS231n Convolutional Neural Networks for Visual Recognition". cs231n.github.io. Retrieved 2018-12-13.

Index Terms

  1. Cultural Heritage Content Management System by Deep Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ASSE '20: Proceedings of the 2020 Asia Service Sciences and Software Engineering Conference
    May 2020
    163 pages
    ISBN:9781450377102
    DOI:10.1145/3399871
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    • Nanyang Technological University

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 July 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Cultural heritage information management system
    2. artificial intelligence
    3. deep neural network
    4. image processing
    5. machine learning

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ASSE '20

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 83
      Total Downloads
    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media