skip to main content
research-article

Annotating traditional Chinese paintings for immersive virtual exhibition

Published: 06 August 2012 Publication History

Abstract

We propose a new method of annotating a masterpiece of traditional Chinese painting with voice dubbings and environmental sounds. The painting was created with moving focus drawing technique without rigorous perspective. A novel algorithm is proposed to infer the 3D space of the painting according to its layout and embed the audio annotations. For exhibition, the masterpiece is scanned into a high-resolution gigapixel image for presenting the drawing details, and we develop an interactive multimedia system with a panning and zooming interface to enable smooth navigation on the giant painting and exploring the historical culture. During the navigation, the system estimates the 3D position of the user's viewpoint from his/her actions, and subsequently synthesizes a realistic stereo audio field according to the viewer's orientation and distance from the annotations in the 3D space. The proposed system provides an immersive user experience by rendering a visual-audio consistent perception.

References

[1]
Carrozzino, M., Evangelista, C., and Bergamasco, M. 2009. The immersive time-machine: A virtual exploration of the history of livorno. In Proceedings of the 3rd ISPRS International Workshop (3D-ARCH'09). 1--5.
[2]
ChinaOnlineMuseum. 2011. Painting gallery of zhang zeduan. http://www.chinaonlinemuseum.com/gallery-zhang-zeduan.php.
[3]
Chu, N. and Tai, C. 2001. Animating Chinese landscape paintings and panorama using multi-perspective modeling. In Proceedings of Computer Graphics International. 107--112.
[4]
Crystal. 2010. Who Animate the Chinese Symphonic Picture Riverside Scene at Qingming Festival. Ming Bao Publishers, Hongkong.
[5]
Degener, P. and Klein, R. 2009. A variational approach for automatic generation of panoramic maps. ACM Trans. Graph. 28, 28, 1, Article 2.
[6]
Fong, W. C. 2003. Why Chinese painting is history. Art Bull. 85, 258--280.
[7]
Foni, A. E., Papagiannakis, G., and Magnenat-Thalmann, N. 2010. A taxonomy of visualization strategies for cultural heritage applications. ACM J. Comput. Cultu. Herit. 3, 1, Article 1.
[8]
Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2009. Reconstructing building interiors from images. In Proceedings of International Conference on Computer Vision. 80--87.
[9]
Hansen, V. 1996. The beijing qingming scroll and its significance for the study of Chinese history. J. Sung-Yuan Studies.
[10]
Horry, Y., Anjyo, K. I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. 225--232.
[11]
Jaimes, A. and Sebe, N. 2007. Multimodal human computer interaction: A survey. Comput. Visi. Image Understand. 108, 1-2, 116--134.
[12]
Koller, D., Frischer, B., and Humphreys, G. 2009. Research challenges for digital archives of 3d cultural heritage models. ACM J. Comput. Cult. Heri. 2, 3, Article 7.
[13]
Kopf, J., Uyttendaele, M., Deussen, O., and Cohen, M. F. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. 26, 3, Article 93.
[14]
Lin, J.-Y., Chen, Y.-Y., Ko, J.-C., and et al. 2009. I-m-Tube: An interactive multi-resolution tubular display. In Proceedings of the 17th ACM International Conference on Multimedia. 253--260.
[15]
Luan, Q., Drucker, S., Kopf, J., Xu, Y. Q., and Cohen, M. 2008. Annotating gigapixel images. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology. 33--36.
[16]
Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. MIT Press.
[17]
Roderick, W. 1965. Chang Tse-Tuan's Ch'ing-Ming Shang-Ho t'u. Doctoral dissertation, Princiton University.
[18]
Rowley, G. 1947. Principles of Chinese Painting. Princeton University Press.
[19]
SeoulAliveGallery. 2008. Talking to mona lisa. http://wn.com/Alive_Gallery, Seoul.
[20]
Szeliski, R. 2011. Computer Vision: Algorithms and Applications. Springer.
[21]
The Palace Museum (Beijing). http://www.dpm.org.cn/index1024768.html.
[22]
Vallance, S. and Calder, P. 2002. Multi-Perspective images for visualization. In Proceedings of the Pan-Sydney Area Workshop on Visual Information Processing. 1253--1260.
[23]
Willats, J. 1997. Art and Representation: New Principles in the Analysis of Pictures. Princeton Academic Press.
[24]
Witmer, B. G. and Singer, M. J. 1998. Measuring presence in virtual environments: A presence questionaire. Presence 7, 225--240.
[25]
Yang, X., Barnhart, R., Nie, C., Cahill, J., Lang, S., and Wu, H. 1997. Three Thousand Years of Chinese Painting (The Culture & Civilization of China). Yale University Press.
[26]
Yu, J. and McMillan, L. 2005. Multiperspective projection and collineation. In Proceedings of the International Conference on Computer Vision. 580--587.
[27]
Zhu, Y., Li, C., and Shen, I.-F. 2004. A new style of ancient culture: Animated Chinese Dunhuang murals. In ACM SIGGRAPH 04 Sketches. 130.

Cited By

View all
  • (2024)2.5D Immersive Representation of Chinese Claborate-Style Painting in Virtual Reality2024 10th International Conference on Virtual Reality (ICVR)10.1109/ICVR62393.2024.10868699(161-166)Online publication date: 24-Jul-2024
  • (2024)An Immersive Ceramic Display and Personalized Decoration Method Based on WebVR2024 4th International Conference on Artificial Intelligence, Virtual Reality and Visualization10.1109/AIVRV63595.2024.10859552(28-33)Online publication date: 1-Nov-2024
  • (2024)Actively Viewing the Audio-Visual Media: An Eye-Controlled Application for Experience and Learning in Traditional Chinese Painting ExhibitionsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2371691(1-29)Online publication date: 23-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Journal on Computing and Cultural Heritage
Journal on Computing and Cultural Heritage   Volume 5, Issue 2
July 2012
44 pages
ISSN:1556-4673
EISSN:1556-4711
DOI:10.1145/2307723
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2012
Accepted: 01 October 2011
Revised: 01 July 2011
Received: 01 October 2010
Published in JOCCH Volume 5, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Chinese paintings
  2. immersive experience
  3. moving focus
  4. virtual exhibition

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)64
  • Downloads (Last 6 weeks)9
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)2.5D Immersive Representation of Chinese Claborate-Style Painting in Virtual Reality2024 10th International Conference on Virtual Reality (ICVR)10.1109/ICVR62393.2024.10868699(161-166)Online publication date: 24-Jul-2024
  • (2024)An Immersive Ceramic Display and Personalized Decoration Method Based on WebVR2024 4th International Conference on Artificial Intelligence, Virtual Reality and Visualization10.1109/AIVRV63595.2024.10859552(28-33)Online publication date: 1-Nov-2024
  • (2024)Actively Viewing the Audio-Visual Media: An Eye-Controlled Application for Experience and Learning in Traditional Chinese Painting ExhibitionsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2371691(1-29)Online publication date: 23-Jul-2024
  • (2024)Computational Approaches for Traditional Chinese Painting: From the “Six Principles of Painting” PerspectiveJournal of Computer Science and Technology10.1007/s11390-024-3408-x39:2(269-285)Online publication date: 1-Mar-2024
  • (2023)OdorV-Art: An Initial Exploration of An Olfactory Intervention for Appreciating Style Information of Artworks in Virtual MuseumProceedings of the 16th International Symposium on Visual Information Communication and Interaction10.1145/3615522.3615544(1-8)Online publication date: 22-Sep-2023
  • (2023)DianTea:Designing and Evaluating an Immersive Virtual Reality Game to Enhance Youth Tea Culture LearningProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3608707(1-8)Online publication date: 26-Sep-2023
  • (2023)VR-Based Line Drawing Methods in Chinese Painting2023 9th International Conference on Virtual Reality (ICVR)10.1109/ICVR57957.2023.10169222(604-610)Online publication date: 12-May-2023
  • (2022)A Novel Sentiment Analysis Model of Museum User Experience Evaluation Data Based on Unbalanced Data Analysis TechnologyComputational Intelligence and Neuroscience10.1155/2022/20966342022Online publication date: 1-Jan-2022
  • (2022)Out of theater: Interactive Mixed-reality Performance for Intangible Culture Heritage Glove PuppetryProceedings of the Tenth International Symposium of Chinese CHI10.1145/3565698.3565782(181-189)Online publication date: 22-Oct-2022
  • (2018)Walking into ancient paintings with virtual candlesProceedings of the 24th ACM Symposium on Virtual Reality Software and Technology10.1145/3281505.3283378(1-2)Online publication date: 28-Nov-2018
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media