Technical SectionCombining traditional and indirect augmented reality for indoor crowded environments. A case study on the Casa Batlló museum☆
Graphical abstract
Introduction
Museums and historic buildings offer a rich representation of local and national culture and history in a physical form, but are rarely well understood by the general public if additional information is not provided. In other words, such buildings visited without information about their context might become meaningless and thus invisible. For this reason, the need to develop new methodologies for cultural heritage learning and art appreciation is always present, and this is increasingly supported by digital technologies. Augmented Reality (AR) technology has been successfully integrated in recent years in museums, heritage sites, science exhibitions, and areas visited by tourists. Some of these sites include AR applications as part of the exhibition [1], [2], [3], [4], while others use the technology as a guide system [5], [6], [7], [8] making use of mobile devices. Mobile AR applications have been proposed in recent years, though their use is usually limited due to the special characteristics of museums, namely the fact that artificial markers cannot be placed and that crowded, small rooms can usually be found. Additionally, traditional AR may demand high computational capabilities if complex scenes and/or virtual objects are being processed in real time, a condition that cannot be supported by many current smartphones. The goal of this paper is to enable augmented experiences for visitors to the Casa Batlló museum (Barcelona, Spain) by replacing conventional audio-guides by novel mobile AR guides that can be properly integrated into museum’s visits, dealing with both the restrictions of the museum and the limitations of AR technology. To this end, a novel approach is shown which combines traditional and indirect AR. While we use standard approaches to deal with the traditional AR part, in order to deal with the indirect AR part we introduce the concept of augmented cubes that are generated with an authoring tool. Indirect AR was initially proposed for outdoor environments [9] and, to the best of our knowledge, the use of this technology in the museum context has not been explored by the research community so far. Additionally, we present our findings of real visitors testing the application. According to [10], these kinds of non-laboratory tests are usually not available in the cultural heritage sector. Therefore, the outcomes reported here may be relevant for this sector. Casa Batlló (Barcelona, Spain) is an outstanding modernist architectural work by the famous architect Antoni Gaudí and it was declared a World Heritage Site in 2005 by UNESCO (Fig. 1). The building receives an average of 3,000 visitors per day due to its great architectural value, and the fact that each of its rooms has an unconventional design, closely related to nature. Currently, the museum rooms are empty, as the old furniture has been removed. In recent years, different technological tools have been used by the museum to increase art appreciation (e.g. an audio-guide); however, a highly interactive, rich multimedia tool has not been available.
By means of technologically augmenting the museum with a mobile-based application, we intend to introduce a greatly improved experience for tourists visiting Casa Batlló allowing them to see both the current and the original state of the building, including its old furniture, as well as adding other visual effects allowing them to move freely throughout the museum. We use traditional AR to augment some specific assets, as well as a physical model of the museum which is located in one of its rooms, while indirect AR is used to augment the museum’s rooms after the off-line virtualization of the real environment. While the solution given to augment specific assets and the museum’s physical model is similar to other approaches, the solution given to augment the rooms is novel and also deals with more technical challenges, which are tackled with the use of an authoring tool.
In order to virtualize the real environment using the indirect AR approach we have proceeded as follows: the synthetic world is built using virtual cubes (basic shapes), which are then augmented with information of the real world (images) and synthetic information (3D objects, animations and videos), and thus transformed into augmented cubes. To create the augmented cubes, we have developed an extension of environmental texture mapping to create extended cube maps, where real and static virtual information (e.g. complex furniture 3D models) are merged, including depth data, so other dynamic virtual contents (e.g. video effects and animations) can be combined in real time during the visualization. The integration of all this information is rather complex, and requires advanced computational techniques since we have to deal with different kinds of contents, formats, visualization effects, etc., and thus we have designed an authoring tool to manage all the required processes. Additionally, each of the augmented scenes is accompanied with audio (through headphones) that provides visitors with a brief history of the room and the augmented elements. The main advantages of such a solution compared to traditional AR are:
- •
A customizable field of view (FOV), which is not restricted by the mobile device’s camera optics.
- •
There is no need to add artificial markers on the walls, as user positioning is not solved with image tracking. Additionally, sunlight or strong illumination conditions do not affect our solution.
- •
The number of people in the room is not important, as occlusions are not produced in our augmented environment.
- •
The static rich multimedia content is merged off-line and transformed (with the authoring tool) into data which is computationally suited for real time management in mobile devices without loss of content. This means that virtual objects can be defined with an unlimited number of vertices in order to reach the desired level of detail.
- •
The dynamic rich multimedia content, rendered in real time in the mobile application, is perfectly merged with the static content using our extension of cube maps.
Our application offers a smart solution that solves both the environmental restrictions of museums (e.g. crowded spaces) and the computational constrains of the technology. This application is currently being used by Casa Batlló’s visitors, though the characteristics of our solution makes it of general use, and thus it can be extended to other museums or indoor environments, hence being of general use.
The rest of the paper is organized as follows. In Section 2 we introduce related work regarding both traditional and indirect AR, as well as analysing the suitability of these technologies to our case study, i.e. the interior of a museum. In Section 3 we explain our solution. Firstly, we detail the augmentation of the museum’s rooms by means of indirect AR. We introduce the concept of augmented cubes and the detailed process to build and render them in real time with smartphones. We also explain our authoring tool and how we deal with the contents of the augmented cubes, namely the real scenario, the static virtual models and the dynamic virtual contents. Secondly, we detail the solution given to augment some assets of the museum by means of traditional AR. In Section 4, we report the results of some usability tests performed with a total of 122 museum visitors. We further provide some qualitative outcomes regarding visitors’ attitude and behaviours. Finally, in Section 5 conclusions are given, which highlight the benefits of our proposed mobile AR guide.
Section snippets
Traditional augmented reality
In recent years mobile AR-based applications have been successfully integrated in museums, heritage sites, science exhibitions and similar educational or tourist related environments, appearing as a promising technology to increase appreciation of art, attract new visitors, increase the visitors motivation and promote participation and enrolment, among others [5], [8], [11]. For instance, a mobile AR guide was designed and evaluated in the Museum of Fine Arts in Renner (France), which consisted
Description of the AR mobile application
In Fig. 3, the workflow for the generation of the augmented scenes in our application is depicted, for both traditional and indirect AR. The application automatically switches between one or the other kind of augmentation, depending on user location, which is retrieved by using Bluetooth technology. The data generation and procedures involved in the workflow are explained in detail in the following sub-sections.
Retrieving user experience
In this section we report our findings from user testing of our application. The tests were carried out in April 2013, and correspond to the evaluation of our application in its first version, where the questions depicted in Table 1 were used for the tests. We also show some qualitative findings related to usage patterns and visitors behaviour, as reported on different occasions by Casa Batlló, after using the first version of our application. Today, the application presents high quality
Conclusions
Mobile AR technology can provide many benefits to enhance visitor’s experience in museums, though traditional AR is not always straightforward to use in such places due to several environment restrictions and technological limitations. Here, we have proposed a new mobile-based AR application by combining the concepts of traditional and indirect AR. Traditional AR is used in those special cases where typical museum environmental restrictions are not met, while indirect AR is used otherwise. The
Acknowledgements
This project has been supported by Casa Batlló. Thanks are due to other researchers of ARTEC participating at some parts of the project. All the images of Casa Batlló are property of Casa Batlló S.L.U.
References (27)
- et al.
Photogrammetry and augmented reality for cultural heritage applications
Photogramm Record
(2009) - et al.
Mobile augmented reality for cultural heritage
(2012) - PTC-Inc. Vuforia. 2017....
- et al.
The virtual showcase as a new platform for augmented reality digital storytelling
Proceedings of the workshop on virtual environments 2003. EGVE ’03
(2003) - et al.
AR-immersive cinema at the Aula Natura visitors center
IEEE MultiMed
(2010) - et al.
Multiuser augmented reality system for indoor exhibitions
(2011) - et al.
Ontology-based user modeling in an augmented audio reality system for museums
User Model User-Adapt Interact
(2005) - et al.
Bridging the gap between the digital and the physical: design and evaluation of a mobile augmented reality guide for the museum visit
Proceedings of the 3rd international conference on digital interactive media in entertainment and arts DIMEA ’08
(2008) - et al.
Development of mobile AR Tour application for the national palace museum of Korea
(2011) - et al.
Visitar: a mobile application for tourism using AR
Proceedings of SIGGRAPH Asia 2013 symposium on mobile graphics and interactive applications. SA ’13
(2013)
Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum
Comput Educ
Indirect augmented reality
Comput Graph
Archeovirtual 2011: an evaluation approach to virtual museums
Proceedings of 18th international conference on virtual systems and multimedia
Cited by (0)
- ☆
This article was recommended for publication by Mark Billinghurst.