Elsevier

Computers & Graphics

Volume 69, December 2017, Pages 92-103
Computers & Graphics

Technical Section
Combining traditional and indirect augmented reality for indoor crowded environments. A case study on the Casa Batlló museum

https://doi.org/10.1016/j.cag.2017.09.001Get rights and content

Highlights

  • We address the problems of AR in museums: crowded rooms and markers not allowed.

  • We present a novel solution that combines traditional and indirect AR.

  • Blending static and dynamic content in real time results in high quality graphics.

  • Our solution is currently being used by the museums visitors, ca. 3000 per day.

Abstract

Mobile audio-guides are currently being used in museums to enhance the visitors’ experience. Over the past few years, some Augmented Reality (AR) solutions have been explored in this context, making it possible to provide augmented visual and sonic stimuli through the use of smartphones. However, the special conditions of museums (e.g. artificial markers cannot be used, small, crowded rooms, etc.) and the limited computational capacity of smartphones mean that there are important restrictions in the use of this technology. In this paper we introduce a novel mobile augmented guide for the Casa Batlló museum (Barcelona, Spain) which is based on a combination of traditional and indirect AR. Through the proposed solution, we further explore the use of indirect AR for the interior of buildings as, at present, this technology is usually used for outdoor environments. The developed application is currently being used by Casa Batlló visitors. Retrieval of user experience shows some of the benefits of the proposed solution and opens the door for other similar solutions in museums.

Introduction

Museums and historic buildings offer a rich representation of local and national culture and history in a physical form, but are rarely well understood by the general public if additional information is not provided. In other words, such buildings visited without information about their context might become meaningless and thus invisible. For this reason, the need to develop new methodologies for cultural heritage learning and art appreciation is always present, and this is increasingly supported by digital technologies. Augmented Reality (AR) technology has been successfully integrated in recent years in museums, heritage sites, science exhibitions, and areas visited by tourists. Some of these sites include AR applications as part of the exhibition [1], [2], [3], [4], while others use the technology as a guide system [5], [6], [7], [8] making use of mobile devices. Mobile AR applications have been proposed in recent years, though their use is usually limited due to the special characteristics of museums, namely the fact that artificial markers cannot be placed and that crowded, small rooms can usually be found. Additionally, traditional AR may demand high computational capabilities if complex scenes and/or virtual objects are being processed in real time, a condition that cannot be supported by many current smartphones. The goal of this paper is to enable augmented experiences for visitors to the Casa Batlló museum (Barcelona, Spain) by replacing conventional audio-guides by novel mobile AR guides that can be properly integrated into museum’s visits, dealing with both the restrictions of the museum and the limitations of AR technology. To this end, a novel approach is shown which combines traditional and indirect AR. While we use standard approaches to deal with the traditional AR part, in order to deal with the indirect AR part we introduce the concept of augmented cubes that are generated with an authoring tool. Indirect AR was initially proposed for outdoor environments [9] and, to the best of our knowledge, the use of this technology in the museum context has not been explored by the research community so far. Additionally, we present our findings of real visitors testing the application. According to [10], these kinds of non-laboratory tests are usually not available in the cultural heritage sector. Therefore, the outcomes reported here may be relevant for this sector. Casa Batlló (Barcelona, Spain) is an outstanding modernist architectural work by the famous architect Antoni Gaudí and it was declared a World Heritage Site in 2005 by UNESCO (Fig. 1). The building receives an average of 3,000 visitors per day due to its great architectural value, and the fact that each of its rooms has an unconventional design, closely related to nature. Currently, the museum rooms are empty, as the old furniture has been removed. In recent years, different technological tools have been used by the museum to increase art appreciation (e.g. an audio-guide); however, a highly interactive, rich multimedia tool has not been available.

By means of technologically augmenting the museum with a mobile-based application, we intend to introduce a greatly improved experience for tourists visiting Casa Batlló allowing them to see both the current and the original state of the building, including its old furniture, as well as adding other visual effects allowing them to move freely throughout the museum. We use traditional AR to augment some specific assets, as well as a physical model of the museum which is located in one of its rooms, while indirect AR is used to augment the museum’s rooms after the off-line virtualization of the real environment. While the solution given to augment specific assets and the museum’s physical model is similar to other approaches, the solution given to augment the rooms is novel and also deals with more technical challenges, which are tackled with the use of an authoring tool.

In order to virtualize the real environment using the indirect AR approach we have proceeded as follows: the synthetic world is built using virtual cubes (basic shapes), which are then augmented with information of the real world (images) and synthetic information (3D objects, animations and videos), and thus transformed into augmented cubes. To create the augmented cubes, we have developed an extension of environmental texture mapping to create extended cube maps, where real and static virtual information (e.g. complex furniture 3D models) are merged, including depth data, so other dynamic virtual contents (e.g. video effects and animations) can be combined in real time during the visualization. The integration of all this information is rather complex, and requires advanced computational techniques since we have to deal with different kinds of contents, formats, visualization effects, etc., and thus we have designed an authoring tool to manage all the required processes. Additionally, each of the augmented scenes is accompanied with audio (through headphones) that provides visitors with a brief history of the room and the augmented elements. The main advantages of such a solution compared to traditional AR are:

  • A customizable field of view (FOV), which is not restricted by the mobile device’s camera optics.

  • There is no need to add artificial markers on the walls, as user positioning is not solved with image tracking. Additionally, sunlight or strong illumination conditions do not affect our solution.

  • The number of people in the room is not important, as occlusions are not produced in our augmented environment.

  • The static rich multimedia content is merged off-line and transformed (with the authoring tool) into data which is computationally suited for real time management in mobile devices without loss of content. This means that virtual objects can be defined with an unlimited number of vertices in order to reach the desired level of detail.

  • The dynamic rich multimedia content, rendered in real time in the mobile application, is perfectly merged with the static content using our extension of cube maps.

Our application offers a smart solution that solves both the environmental restrictions of museums (e.g. crowded spaces) and the computational constrains of the technology. This application is currently being used by Casa Batlló’s visitors, though the characteristics of our solution makes it of general use, and thus it can be extended to other museums or indoor environments, hence being of general use.

The rest of the paper is organized as follows. In Section 2 we introduce related work regarding both traditional and indirect AR, as well as analysing the suitability of these technologies to our case study, i.e. the interior of a museum. In Section 3 we explain our solution. Firstly, we detail the augmentation of the museum’s rooms by means of indirect AR. We introduce the concept of augmented cubes and the detailed process to build and render them in real time with smartphones. We also explain our authoring tool and how we deal with the contents of the augmented cubes, namely the real scenario, the static virtual models and the dynamic virtual contents. Secondly, we detail the solution given to augment some assets of the museum by means of traditional AR. In Section 4, we report the results of some usability tests performed with a total of 122 museum visitors. We further provide some qualitative outcomes regarding visitors’ attitude and behaviours. Finally, in Section 5 conclusions are given, which highlight the benefits of our proposed mobile AR guide.

Section snippets

Traditional augmented reality

In recent years mobile AR-based applications have been successfully integrated in museums, heritage sites, science exhibitions and similar educational or tourist related environments, appearing as a promising technology to increase appreciation of art, attract new visitors, increase the visitors motivation and promote participation and enrolment, among others [5], [8], [11]. For instance, a mobile AR guide was designed and evaluated in the Museum of Fine Arts in Renner (France), which consisted

Description of the AR mobile application

In Fig. 3, the workflow for the generation of the augmented scenes in our application is depicted, for both traditional and indirect AR. The application automatically switches between one or the other kind of augmentation, depending on user location, which is retrieved by using Bluetooth technology. The data generation and procedures involved in the workflow are explained in detail in the following sub-sections.

Retrieving user experience

In this section we report our findings from user testing of our application. The tests were carried out in April 2013, and correspond to the evaluation of our application in its first version, where the questions depicted in Table 1 were used for the tests. We also show some qualitative findings related to usage patterns and visitors behaviour, as reported on different occasions by Casa Batlló, after using the first version of our application. Today, the application presents high quality

Conclusions

Mobile AR technology can provide many benefits to enhance visitor’s experience in museums, though traditional AR is not always straightforward to use in such places due to several environment restrictions and technological limitations. Here, we have proposed a new mobile-based AR application by combining the concepts of traditional and indirect AR. Traditional AR is used in those special cases where typical museum environmental restrictions are not met, while indirect AR is used otherwise. The

Acknowledgements

This project has been supported by Casa Batlló. Thanks are due to other researchers of ARTEC participating at some parts of the project. All the images of Casa Batlló are property of Casa Batlló S.L.U.

References (27)

  • C. Portalés et al.

    Photogrammetry and augmented reality for cultural heritage applications

    Photogramm Record

    (2009)
  • A. Angelopoulou et al.

    Mobile augmented reality for cultural heritage

    (2012)
  • PTC-Inc. Vuforia. 2017....
  • O. Bimber et al.

    The virtual showcase as a new platform for augmented reality digital storytelling

    Proceedings of the workshop on virtual environments 2003. EGVE ’03

    (2003)
  • C. Portalés et al.

    AR-immersive cinema at the Aula Natura visitors center

    IEEE MultiMed

    (2010)
  • J. Gimeno et al.

    Multiuser augmented reality system for indoor exhibitions

    (2011)
  • M. Hatala et al.

    Ontology-based user modeling in an augmented audio reality system for museums

    User Model User-Adapt Interact

    (2005)
  • A. Damala et al.

    Bridging the gap between the digital and the physical: design and evaluation of a mobile augmented reality guide for the museum visit

    Proceedings of the 3rd international conference on digital interactive media in entertainment and arts DIMEA ’08

    (2008)
  • J.-B. Kim et al.

    Development of mobile AR Tour application for the national palace museum of Korea

    (2011)
  • G. Michele et al.

    Visitar: a mobile application for tourism using AR

    Proceedings of SIGGRAPH Asia 2013 symposium on mobile graphics and interactive applications. SA ’13

    (2013)
  • K.-E. Chang et al.

    Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum

    Comput Educ

    (2014)
  • J. Wither et al.

    Indirect augmented reality

    Comput Graph

    (2011)
  • S. Pescarin et al.

    Archeovirtual 2011: an evaluation approach to virtual museums

    Proceedings of 18th international conference on virtual systems and multimedia

    (2012)
  • Cited by (0)

    This article was recommended for publication by Mark Billinghurst.

    View full text