Keywords

1 Introduction

Recently, cultural heritage-related policies around the world have been converted from a paradigm of preservation of cultural heritage to one of services application. In the case of the European Union, through the “Europeana” project [1], it has built a web portal that can provide easy access to digital resources of European museums, libraries, archives, and audiovisual collections. This lets normal users find, share, use, and be inspired by the rich cultural and academic heritage of Europe.

On the domestic side, several agencies have tried to utilize a wide range of heritage-related content educationally, productively, and conservatively, with the goal of openness of cultural heritage and openness to users. Also, affiliated and related organizations such as the Cultural Heritage Administration [2] are promoting the “cultural heritage hub-bank” project [3] to digitize information of cultural heritage, as well as the Culture and Information Integration project.

The “K-Culture Time Machine” project uses a variety of cultural technologies to structure diverse cultural content from associated organizations and projects, including the “Cultural Heritage hub-bank,” and construct new cultural content connected to time and space, then develop a technique that provides the structured content to industries (culture, tourism, IT) and the public. To integrate heterogeneous dataset including aforementioned “Cultural Heritage hub-bank”, designing a new data model is a vital process for our project. As Europeana designed data model which aims to integrate and link several data set across cultural institutions of Europe, we also invented a new data model that encompasses a wide range of metadata for cultural institution in Korea. This approach and data model aim to provide a possibility of semantic link between heterogeneous dataset.

The project is developing an open smart platform that exploits the semantic context of cultural heritage contents over the level of raw data to provide customized services to users. This project also provides services with various visualization techniques (virtual reality, augmented reality, etc.) for cultural heritage by traveling through a variety of historical periods in its contents. Last, the project is developing image-based time-space content configuration techniques and a software framework that enables visualization with various devices (Fig. 1).

Fig. 1.
figure 1

Overall procedure of the K-Culture time machine project.

Full size image

We perform a validation and feedback process of research and development on the implemented prototype service “Journey of a Korean world cultural heritage.” The target service is a major cultural heritage that effectively couples tangible and intangible cultural heritage by considering the verification of content fusion techniques and the accuracy of the time-division layer of virtual space-time traveling (Fig. 2).

Fig. 2.
figure 2

Possible examples of the implemented prototype service.

Full size image

2 Enabling Technology

We aim to provide new content that could interest the younger generation accustomed to smart devices by utilizing time-spatial data and visualization techniques (AR, VR, etc.). For this purpose, we research development information management framework, spatio-temporal analysis framework, content fusion framework, visualization framework, and context-aware framework. For developing these frameworks, integrated data model and metadata standard for various cultural institutions are core element. Then we develop authoring tools, mobile SDK, and a mesh-up service that allows space-telling service (Fig. 3).

Fig. 3.
figure 3

Block diagram of the K-Culture time machine project.

Full size image

2.1 Integrating Technologies to Access Multidimensional Cultural Content

Technology for generating a data structure and integrating heterogeneous metadata: The cooperation of the relevant agencies is required to collect data because it is difficult to gather data individually. Also, because of the variety of metadata of the cultural content agency, it is difficult to maintain the consistency of the linked data model for spatiotemporal information integration.

Thus, in this project we design a standard data model for the integration of the heterogeneous cultural heritage database. The purpose of standardization is not to switch the structure of the data to a metadata holding individual structure but each metadata are to be mapped to a general-purpose metadata model. We need to benchmark the site of a similar domestic and international service, CIDOC-CRM [4] and Europeana portal to provide space-time fusion content and analyze the metadata for the content that institutions hold. We also examine cases suitable for the content acquisition system and obtain advice on designing the metadata from domain experts.

Research on technology for extracting causal relationships and correlations for time, space, subject, and field of content: If the content data model is not proper, the correlation/causality extraction techniques cannot produce a good result. Therefore, by performing the data model design and verification step repeatedly, we can determine a suitable design direction. To do this, we benchmark the existing algorithms to extract cause/correlation and design a data model that can easily extract correlation.

Studying a technique for creating a new space-time relationship in accordance with the relevance of the content: For some improper results of the space-time relationship generated by the algorithm, the user action of the service level is required. In this case, administrators can view and edit the results using the editing tools. They are able to edit the generated space-time relationship and also create a new space-time relationship. The editing tools are designed to allow for future scalability.

2.2 AR/VR-Based Visualization Technique of Space-Time Cultural Content

Automated 3D image feature map generation technique using image capture and a wearable display of mobile equipment: Operation processing speed can be lowered to the level of the mobile and wearable devices. By adopting the sensor information of the mobile device to the computer vision and image processing algorithms, the algorithm can be optimized and faster. We develop image capturing, 3D point reconstruction, and matching feature points. The technique of generating a feature descriptor and matching [5] it should run in the low-performance computing environment for a user using a mobile or wearable device.

Mobile devices based on augmented and virtual reality techniques to visualize the spatiotemporal cultural content: It is difficult to visualize many different types of cultural content information in a single rendering layer. Thus, visualization technique should consider the following aspects: quality of visualized content, compatibility and reliability of visualization equipment, and ease of information access [6]. Based on this, there is a need for a study on automatic layout design and a structured representation of effective visualization of culture content. To achieve this, we design and develop a module for the cultural content data process considering static/dynamic aspects, web-based visualization systems using HTML5 standard [7], user gaze information acquisition for wearable display devices, and a three-dimensional feature-map-based culture content layout method.

Head-mounted wearable-device-based augmented/virtual-reality-based space-time cultural content visualization technology: With expansion of head-mounted devices in the PC environment, calculation processing speed will be lower depending on the computing power level. Optimization and simplification of image processing and computer vision algorithms to solve this problem are required. In addition, because the 3D enhanced graphics performance also varies depending on the processing speed, there is a need for polygon rendering optimization studies. In addition, an animated and interactive K-Culture time machine is an important part of the user’s understanding of a historical fact.

2.3 Participation/Share-Enabled Open Cultural Content Platform

Multidimensional query processing techniques for combined time and space content: Content search performance may be degraded due to the ambiguity of the multidimensional query, and the processing time may also be delayed by a large amount of data. To address this, rather than a simple keyword-based search, a semantic-based search technique is necessary to improve search performance. While taking advantage of an open-source search engine for multi-level data access, it is necessary to reduce the processing time through the optimized search algorithm research and development so that users (viewers) can experience content related to the cultural heritage of the various forms they wish to see in the heritage (video, sound, 3D augmented reality content).

Open service platform technology supports heterogeneous cultural content and links, convergence: A participation-sharing platform must be able to guarantee personal security. In order to solve this problem, personal information requests should be minimized and basic personal information should be stored encrypted.

With reference to public data API, we design Open API and user-specific service scenarios and develop an appropriate participation/sharing platform for each scenario. An authoring tool for linking and convergence of heterogeneous cultural content is essential to its development, and corresponding common UI components increase its reusability. This also aims to develop user-participatory service models to enable a wide range of users, including cultural commentators, who are free to develop augmented-reality-based story-telling [8] and share the content of the story rather than consuming one-way content from a content delivery service provider [9].

2.4 Prototype Service to Spread the Space and Temporal Culture Content

2D/3D map-based cultural content and spatial information fusion: It is not a simple process to access POI information in an open map. The development of multi-layer rendering techniques will be integrated. In addition, there is a need for a variety of representations in accordance with the story (map). Development of the open map-based UI may define various types of representation, such as moveable story-maps, depending on the time, distributed-space-based story, meaning-based visualization, etc.

Mobile and web-based AR/VR pilot service development: Compatibility with mobile OSs and devices is addressed using a function that is not dependent on the hardware (standard method). A problem may occur when processing data that are not appropriate for the standard time division precision supported by the services provided, which can be designed in consideration of the UI to choose the precision according to the data. Technically, the mobile app is developed as a hybrid method to avoid overlapping between platforms. Web development uses a standardized approach that does not depend on the particular browser.

Technology to create a space-time combined culture content creation and distribution package: The content and distribution package generated could have difficulty performing the actual service. Content should be generated by taking into account the type of service and distribution package (web-based, mobile-system-based, and wearable-system-based). Also, content-creation and package-generation functions should be built into the editing tool. In addition, the metadata format of the space-temporal content can also be standardized, as can the data format (visualization) for loading the information needed for visualization.

3 Prototype Development

We developed a prototype application of the K-Culture Time Machine, an in-situated authoring and viewing system, for augmented reality-based tour guidance that helps a user to easily create and experience an augmented reality scene of a cultural heritage. In the authoring step, a user takes a picture of the object of interest and searches digital content related to the cultural heritage on the semantic web. These kinds of authorized augmented reality scenes are saved into a virtual map based on the GPS based position. In the future, many other users can also modify or complement the authorized augmented scene based on the virtual map. In the viewing step, visitors can trigger the augmented scene if they follow and close their viewing device on the observation viewpoint, which was previously generated by authors in the authoring step. Then, visitors can experience the augmented scene of the cultural heritage by downloading authorized content with ARML (Augmented Reality Markup Language) format [10]. The application is possible through three-dimensional computer vision technology, and its precision is very high compared to the approximate location based information visualization utilizing existing GPS sensor (Fig. 4).

Fig. 4.
figure 4

Authoring and experience of the K-Culture time machine.

Full size image

4 Conclusion

This study presents a basis for providing a space-time-associated possibility of integrated heritage. The material of the different parts of cultural heritage has the potential to be utilized in a variety of ways, including digital visualization. Beyond the traditional closed and administrative cultural heritage information architecture, an integrated and organically managed heritage information structure will contribute to the activation of cultural heritage.

The user-context-aware framework is designed on the basis of the standard protocol to provide the appropriate service by identifying the individual service user’s physical, environmental, and resource context. These features are expected to recommend meaningful content from much unconstructed content, provide guidance, and enable personalized space-telling.