research-article

Annotating traditional Chinese paintings for immersive virtual exhibition

Authors:

Wen GaoAuthors Info & Claims

Journal on Computing and Cultural Heritage (JOCCH), Volume 5, Issue 2

Article No.: 6, Pages 1 - 12

https://doi.org/10.1145/2307723.2307725

Published: 06 August 2012 Publication History

Abstract

We propose a new method of annotating a masterpiece of traditional Chinese painting with voice dubbings and environmental sounds. The painting was created with moving focus drawing technique without rigorous perspective. A novel algorithm is proposed to infer the 3D space of the painting according to its layout and embed the audio annotations. For exhibition, the masterpiece is scanned into a high-resolution gigapixel image for presenting the drawing details, and we develop an interactive multimedia system with a panning and zooming interface to enable smooth navigation on the giant painting and exploring the historical culture. During the navigation, the system estimates the 3D position of the user's viewpoint from his/her actions, and subsequently synthesizes a realistic stereo audio field according to the viewer's orientation and distance from the annotations in the 3D space. The proposed system provides an immersive user experience by rendering a visual-audio consistent perception.

References

[1]

Carrozzino, M., Evangelista, C., and Bergamasco, M. 2009. The immersive time-machine: A virtual exploration of the history of livorno. In Proceedings of the 3rd ISPRS International Workshop (3D-ARCH'09). 1--5.

[2]

ChinaOnlineMuseum. 2011. Painting gallery of zhang zeduan. http://www.chinaonlinemuseum.com/gallery-zhang-zeduan.php.

[3]

Chu, N. and Tai, C. 2001. Animating Chinese landscape paintings and panorama using multi-perspective modeling. In Proceedings of Computer Graphics International. 107--112.

Digital Library

[4]

Crystal. 2010. Who Animate the Chinese Symphonic Picture Riverside Scene at Qingming Festival. Ming Bao Publishers, Hongkong.

[5]

Degener, P. and Klein, R. 2009. A variational approach for automatic generation of panoramic maps. ACM Trans. Graph. 28, 28, 1, Article 2.

Digital Library

[6]

Fong, W. C. 2003. Why Chinese painting is history. Art Bull. 85, 258--280.

[7]

Foni, A. E., Papagiannakis, G., and Magnenat-Thalmann, N. 2010. A taxonomy of visualization strategies for cultural heritage applications. ACM J. Comput. Cultu. Herit. 3, 1, Article 1.

Digital Library

[8]

Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2009. Reconstructing building interiors from images. In Proceedings of International Conference on Computer Vision. 80--87.

[9]

Hansen, V. 1996. The beijing qingming scroll and its significance for the study of Chinese history. J. Sung-Yuan Studies.

[10]

Horry, Y., Anjyo, K. I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. 225--232.

Digital Library

[11]

Jaimes, A. and Sebe, N. 2007. Multimodal human computer interaction: A survey. Comput. Visi. Image Understand. 108, 1-2, 116--134.

Digital Library

[12]

Koller, D., Frischer, B., and Humphreys, G. 2009. Research challenges for digital archives of 3d cultural heritage models. ACM J. Comput. Cult. Heri. 2, 3, Article 7.

Digital Library

[13]

Kopf, J., Uyttendaele, M., Deussen, O., and Cohen, M. F. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. 26, 3, Article 93.

Digital Library

[14]

Lin, J.-Y., Chen, Y.-Y., Ko, J.-C., and et al. 2009. I-m-Tube: An interactive multi-resolution tubular display. In Proceedings of the 17th ACM International Conference on Multimedia. 253--260.

Digital Library

[15]

Luan, Q., Drucker, S., Kopf, J., Xu, Y. Q., and Cohen, M. 2008. Annotating gigapixel images. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology. 33--36.

Digital Library

[16]

Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. MIT Press.

[17]

Roderick, W. 1965. Chang Tse-Tuan's Ch'ing-Ming Shang-Ho t'u. Doctoral dissertation, Princiton University.

[18]

Rowley, G. 1947. Principles of Chinese Painting. Princeton University Press.

[19]

SeoulAliveGallery. 2008. Talking to mona lisa. http://wn.com/Alive_Gallery, Seoul.

[20]

Szeliski, R. 2011. Computer Vision: Algorithms and Applications. Springer.

[21]

The Palace Museum (Beijing). http://www.dpm.org.cn/index1024768.html.

[22]

Vallance, S. and Calder, P. 2002. Multi-Perspective images for visualization. In Proceedings of the Pan-Sydney Area Workshop on Visual Information Processing. 1253--1260.

Digital Library

[23]

Willats, J. 1997. Art and Representation: New Principles in the Analysis of Pictures. Princeton Academic Press.

[24]

Witmer, B. G. and Singer, M. J. 1998. Measuring presence in virtual environments: A presence questionaire. Presence 7, 225--240.

Digital Library

[25]

Yang, X., Barnhart, R., Nie, C., Cahill, J., Lang, S., and Wu, H. 1997. Three Thousand Years of Chinese Painting (The Culture & Civilization of China). Yale University Press.

[26]

Yu, J. and McMillan, L. 2005. Multiperspective projection and collineation. In Proceedings of the International Conference on Computer Vision. 580--587.

Digital Library

[27]

Zhu, Y., Li, C., and Shen, I.-F. 2004. A new style of ancient culture: Animated Chinese Dunhuang murals. In ACM SIGGRAPH 04 Sketches. 130.

Digital Library

Cited By

Cheng YHuang MYang JSun W(2024)2.5D Immersive Representation of Chinese Claborate-Style Painting in Virtual Reality2024 10th International Conference on Virtual Reality (ICVR)10.1109/ICVR62393.2024.10868699(161-166)Online publication date: 24-Jul-2024
https://doi.org/10.1109/ICVR62393.2024.10868699
Peng YYang SLiao S(2024)An Immersive Ceramic Display and Personalized Decoration Method Based on WebVR2024 4th International Conference on Artificial Intelligence, Virtual Reality and Visualization10.1109/AIVRV63595.2024.10859552(28-33)Online publication date: 1-Nov-2024
https://doi.org/10.1109/AIVRV63595.2024.10859552
Chen XGe XWu QWang X(2024)Actively Viewing the Audio-Visual Media: An Eye-Controlled Application for Experience and Learning in Traditional Chinese Painting ExhibitionsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2371691(1-29)Online publication date: 23-Jul-2024
https://doi.org/10.1080/10447318.2024.2371691
Show More Cited By

Index Terms

Annotating traditional Chinese paintings for immersive virtual exhibition
1. Applied computing
  1. Arts and humanities
  2. Document management and text processing
    1. Document capture
      1. Graphics recognition and interpretation
2. Computing methodologies
  1. Computer graphics

Recommendations

Reconstructing Traditional Chinese Paintings with Immersive Virtual Reality
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

This paper describes the design of a virtual reality learning environment that reconstructs a traditional Chinese painting, Spring Morning in the Han Palace, using a head-mounted platform. The issues of art styles, enhanced interpretation and interaction ...
Immersive experience of Chinese poems and paintings

Chinese painting incorporated with poems is famous for its distinct characteristics and unspeakable moods. Ink lines are used to depict scenes in paintings while concise words are used to describe scenes and events in poems, making for a beautiful yet ...
The Cognitive Study of Immersive Experience in Science and Art Exhibition
Augmented Cognition
Abstract
In terms of the history of design development, human engineering emerged after World War II in the 1950s, and continued to the rise of personal computers in the 1980s. Human-oriented interface design and human-oriented interactive interfaces are ...

Comments

Information & Contributors

Information

Published In

cover image Journal on Computing and Cultural Heritage

Journal on Computing and Cultural Heritage Volume 5, Issue 2

July 2012

44 pages

ISSN:1556-4673

EISSN:1556-4711

DOI:10.1145/2307723

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2012

Accepted: 01 October 2011

Revised: 01 July 2011

Received: 01 October 2010

Published in JOCCH Volume 5, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
852
Total Downloads

Downloads (Last 12 months)64
Downloads (Last 6 weeks)9

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cheng YHuang MYang JSun W(2024)2.5D Immersive Representation of Chinese Claborate-Style Painting in Virtual Reality2024 10th International Conference on Virtual Reality (ICVR)10.1109/ICVR62393.2024.10868699(161-166)Online publication date: 24-Jul-2024
https://doi.org/10.1109/ICVR62393.2024.10868699
Peng YYang SLiao S(2024)An Immersive Ceramic Display and Personalized Decoration Method Based on WebVR2024 4th International Conference on Artificial Intelligence, Virtual Reality and Visualization10.1109/AIVRV63595.2024.10859552(28-33)Online publication date: 1-Nov-2024
https://doi.org/10.1109/AIVRV63595.2024.10859552
Chen XGe XWu QWang X(2024)Actively Viewing the Audio-Visual Media: An Eye-Controlled Application for Experience and Learning in Traditional Chinese Painting ExhibitionsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2371691(1-29)Online publication date: 23-Jul-2024
https://doi.org/10.1080/10447318.2024.2371691
Zhang WZhang JWong KWang YFeng YWang LChen W(2024)Computational Approaches for Traditional Chinese Painting: From the “Six Principles of Painting” PerspectiveJournal of Computer Science and Technology10.1007/s11390-024-3408-x39:2(269-285)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s11390-024-3408-x
Zhang SWang ZZhou YCui HFu SWang ZFan M(2023)OdorV-Art: An Initial Exploration of An Olfactory Intervention for Appreciating Style Information of Artworks in Virtual MuseumProceedings of the 16th International Symposium on Visual Information Communication and Interaction10.1145/3615522.3615544(1-8)Online publication date: 22-Sep-2023
https://dl.acm.org/doi/10.1145/3615522.3615544
Li JZheng ZChai YSu SWei XShi HXin X(2023)DianTea:Designing and Evaluating an Immersive Virtual Reality Game to Enhance Youth Tea Culture LearningProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3608707(1-8)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3608707
Cheng YHuang MSun W(2023)VR-Based Line Drawing Methods in Chinese Painting2023 9th International Conference on Virtual Reality (ICVR)10.1109/ICVR57957.2023.10169222(604-610)Online publication date: 12-May-2023
https://doi.org/10.1109/ICVR57957.2023.10169222
Chen XChen ZXiao LZhou M(2022)A Novel Sentiment Analysis Model of Museum User Experience Evaluation Data Based on Unbalanced Data Analysis TechnologyComputational Intelligence and Neuroscience10.1155/2022/20966342022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/2096634
Lin JShe YLin LChen SChen JLiu HXu XShen YLi J(2022)Out of theater: Interactive Mixed-reality Performance for Intangible Culture Heritage Glove PuppetryProceedings of the Tenth International Symposium of Chinese CHI10.1145/3565698.3565782(181-189)Online publication date: 22-Oct-2022
https://dl.acm.org/doi/10.1145/3565698.3565782
Ma WWang QShi DLiu SCheng CShi QLi QLi TXu YMorishima SItoh YShiratori TYue YLindeman R(2018)Walking into ancient paintings with virtual candlesProceedings of the 24th ACM Symposium on Virtual Reality Software and Technology10.1145/3281505.3283378(1-2)Online publication date: 28-Nov-2018
https://dl.acm.org/doi/10.1145/3281505.3283378
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents