1. Introduction
Museums, as audience-centered institutions playing a range of educational and social roles, are now more than ever aware of the importance of delivering equality, diversity, and inclusion.
4 However, the museum’s unique environment presents challenges to visually impaired visitors in accessing information prior to a visit. First, contemporary museums have distinctive architecture, internal design, and “inter-floor structures” (that is, stairs and walkways connecting the floors, see an example in Figure
1d) as a part of their exhibitions.
16 Blind visitors struggle with comprehending intricate “multidimensional information,” including the building’s shape, inter-floor structures, exhibit names, descriptions, sizes, and locations. Second, while some museums offer a straightforward layout with predetermined routes, most of them feature open arrangements of exhibits with potentially unclear routes.
5 In such museums, visitors typically explore and select exhibits based on their personal interests. By encouraging such “free explorations,” these museums effectively trigger a sighted visitor’s curiosity, but conventional posted information may cause blind visitors access difficulties and orientation frustrations.
Accessible maps are the means for visually impaired visitors to learn about a site. Tactile maps are often available in public spaces and institutions to help the user build a mental map before going to a new place.
21 Since the effectiveness and understandability of a tactile map largely depend on the user’s tactile skills and abilities,
17 three-dimensional (3D) maps with volumetric symbols and audio-tactile labels have been developed for ease of understanding and allowing autonomous map exploration. The current 3D-printed audio-tactile maps show thrilling possibilities, but limitations persist. These maps usually present a simple one-floor layout, which is insufficient to support a structural mental map of a multidimensional museum.
Due to the fact that a museum contains a large amount of multidimensional information, and is not a frequently visited place, blind visitors might feel it’s particularly challenging to obtain information, orient themselves, and build a mental map. To bridge the gap between museums and blind visitors in terms of information access, and to investigate the suitable format of an accessible and inclusive museum map, the following research questions emerge:
•
RQ1. How can we make the vast amount of needed information (for example, architecture and interior structures, exhibits, facilities, locations, and route-finding) accessible and understandable on a museum map?
•
RQ2. Is building a mental map possible and significant in the museum context?
Prior research developed single-floor plans
10,15 or reproduced external structures,
13 rarely addressing complex multi-floor settings like museums. Using a participatory and user-centered approach, we designed BentoMuseum, a 3D and layered museum map with audio-tactile interactions, to support blind users in obtaining information and understanding the 3D attractions through tactile explorations (Figure
1).
a The system contains two main elements: the 3D and layered floors (Figure
1a), which can be either interlocked to allow vertical exploration between floors (Figure
1c) or separated to support horizontal exploration of a single floor; interactive touchpoints on the floor that allow audio feedback by touch (Figure
1b). When one floor is placed on a touch screen, information and tactile navigation with audio support can be triggered by tapping. Our innovation lies in the novel design of stackable 3D floors on a touch screen, which includes 3D and 2D attributes for providing comprehensive information that encompasses external and internal structures, exhibits, facilities, and simulated navigation by tracing paths and intersections.
We invited 12 participants with severe vision impairment to be museum tour designers and instructed them to use the system as part of an authentic museum tour. They were encouraged to explore extensively, obtain information, select exhibits of personal interest, and construct mental maps. Participants expressed their map exploration styles and elaborated on their needs for information access. Our results suggest: (1) Using the system, the participants were able to actively obtain information that links shape, location, and content. Consequently, they were able to choose exhibits of interest and build a rough mental map. (2) Touching inter-floor structures motivated blind users to explore the museum map. Along with the navigation, it supported them in building a 3D mental map. (3) Building a rough mental map beforehand was beneficial for the subsequent visit. It provided orientation, enhanced the sense of safety and confidence about not getting lost, and led to a positive and inclusive museum experience.
3. Participatory System Design
The design concept is implemented in a science museum, Miraikan – The National Museum of Emerging Science and Innovation,
b which has a distinctive structure and symbolic interior attractions. It is a seven-floor building with a large-area atrium (with the 2nd, 4th, and 6th floors mainly atrium space) and structural attractions such as a series of escalators that directly connect all the floors (Figure
3a), a walkway called Oval Bridge that goes around a “globe-like” display named the Geo-Cosmos (Figure
1d), and a Dome Theater with half of it inside the building and the other half extended into the exterior (Figure
3b). It also lacks maps that can be perceived by touch.
We employed a participatory and user-driven methodology to design a map adapted to the museum. The design sessions include seven interviews with the blind designer (once in prototype 1, three times in prototype 2, and three times in preparing for the final design), one event that involved twenty blind museum visitors and three staff members, and one group meeting with those staff members.
3.1 Motivation: “What if I can open the model and get more information about the floors?”
One of the designers, P0, is a blind adult female, as well as being an interaction designer and researcher. After being presented a 3D model of the museum, she expressed the need to understand the interior:
“I have heard about the symbolic globe-like display and the Oval Bridge around it. But it’s so hard to imagine them just through descriptions. I wish I could open the model and touch them.”c This was the initial attribute of the map we hoped to investigate: a 3D model that contains internal structures.
3.2 Prototype 1: Feedback of a Realistic Model
An initial map design took the form of a realistic 3D print of the museum floor (Figure
2a). We sliced the 3D model into floors and encapsulated the detailed information such as the walls and tables. A tactile map resembling the 3D map’s layout was developed and printed on swell paper for comparison. During a two-day event called
Inclusion Week, two maps along with other 3D prints were explored in the wild by 20 blind visitors, for 5 to 10 minutes each person. From their comments, we learned the following needs to satisfy in developing an understandable map:
•
Content: Simplified and categorized forms were needed. Users highly praised the understandable form of the Oval Bridge on the 3D map but also pointed out that the detailed depictions of exhibits were not digestible.
•
Tactile exploration: A relatively smooth surface without acute edges was preferred. Small and pointy objects (that is, walls and tables) on the 3D map hindered hand scanning.
•
Explanations: Automated audio-tactile interactions were desired. Both maps were not understandable unless the museum staff gave explanations.
The two maps were then tested by P0 during a 30-minute interview. Further requirements were confirmed based on her knowledge of the museum and expertise in design:
•
Facilities: In addition to exhibits, basic map elements such as restrooms, elevators, and escalators also needed to be included.
•
Orientation and navigation: The map should support identifying the entrance, main route, and how to move around. These elements support the development of a “mental map,” which is crucial for blind people.
The feedback highlighted that the 3D map effectively conveyed structural characteristics, while the tactile map preserved scanning, aligning with prior findings.
12 This motivated our focus on 3D maps, while harnessing the strengths of both 3D volumetric and 2D relief attributes.
3.3 Prototype 2: 3D Floors and Audio-tactile Interactions
Contents.
Based on the feedback, we categorized the museum’s multidimensional information into three types of information, and we provided design criteria for each of them:
•
Structural attractions include inter-floor structures and symbolic spatial structures (Figure
1c,
1d,
3a,
3b). Our design choices include: (1) Simplifying structures into primary forms with understandable relative scales. For example, the parallel escalators and stairs were made simpler into one slope with textures (Figure
3a). (2) Simplifying prominent walls into
tall and
wide cuboids. (3) Embedding magnets to support easy stacking and lining up of floors, which has proved to be effective in developing 3D objects for the blind.
6 Floors can be partially stacked to simulate how to walk between them (Figure
3a) or fully stacked to show a facade (Figure
2c).
•
The
exhibits included booths, wall-divided spaces, and artifacts placed in open spaces. Our design choice was to simplify them into outlines that were proportional to the real space they took (Figure
3c). This design supports clear separation, differentiation, and scanning. The outlined shape was hollowed to enable audio-tactile interaction (described later).
•
The
facilities in the museum were summarized into eight frequent items. We represented them using volumetric symbols (Figure
3d), with design guidelines from previous work.
9,13 For those facilities that take a large space (for example, lobby and restaurant), their outlines were hollowed out to show the area and enable touch interaction.
To support orientation and navigation on the map, we further defined paths and intersections to indicate how the user can travel.
•
The
path indicates a route on open ground. According to the actual layout and flow, we defined a main path and sub-paths as routes that connect each exhibit’s entrance to the main path. All of the paths are represented by
-wide embossed lines (Figure
2b).
•
The
intersection is represented as a
hollowed square located at the crossing of the paths, which is distinguishably smaller than the exhibit areas (Figure
2b).
Interactions. To automate the explanations with different levels of detail, we implemented audio-tactile labels using capacitive sensing on a touch screen. A 12.9-inch iPad Pro was used as a platform to sense touch. When a floor is placed on it, a touch can be sensed directly on the hollowed exhibits. On a structural attraction with a geometric shape (for example, Geo-Cosmos in Figure
1c), the audio-tactile label was implemented by redirecting touch from the screen to the 3D surface using conductive ink, following touch screen redirection technical practices.
19 A
wide tube was cut out in the geometric shape, filled with the conductive ink, and had its top and bottom painted with conductive ink. We also pasted a
wide circular tactile sticker at the center to indicate the touchpoint (see an example in Figure
1c). To tactually distinguish hollowed interactable areas from the atrium, we attached a paper with textures on the back of the floor (Figure
2b). An app that processes touch and provides voiceover information was developed in Unity. We adopted a double tap as the recognized touch from the previous work
2 to prevent accidental triggering during the exploration. Two modes were developed to serve the needs of free exploration and route-finding:
•
In the Exploration mode, double-tapping a touchable area triggers the audio explanations.
•
In the Navigation mode, the user double-taps two exhibits to select the destination and the start. A route with a start, destination, and a number of intersections in-between is generated. Next, The user is instructed to move a finger to the start, which is the location in the exhibit intersected by the path. Once the user moves there (without any tapping), she is directed to trace the path to the next location of the route until she reaches the destination.
The final floors are approximately
stacked, and
tall
thick each floor (Figure
2c), which was at a 1/400 scale of the actual museum. This is the largest size that can fit onto an iPad to support audio-tactile exploration. It is designed in Autodesk Fusion 360, and printed with Formlabs Form 3L SLA 3D printer, using Clear Resin material.
3.4 Final Design: Content and Customization
We then conducted a 1.5-hour group meeting with three museum staff members, who are not only proficient as museum guides but also experienced in guiding blind users. We decided to include the following contents:
•
The audio guide for the 3D structure or the exhibit, which speaks at one of two levels of detail in turn when tapped. The first level contains name, keyword (for example, universe, earth, life), and accessibility info (for example, “Over there, you can touch a 3D model of the rocket engine.”) The second level contains a 15-second description about it.
•
The audio guide for an intersection, which speaks the surrounding information when tapped (for example, “This intersection is connected to an earth-type exhibit on the top and a universe-type exhibit on the bottom. Eight exhibits are on the left. Five are on the right.”)
The following updates were made to enable user customization:
•
Three physical buttons (stop the voiceover, modify the speed, and change Exploration/Navigation mode) were developed. They were clipped onto the iPad and can be triggered using a double-tap (Figure
2b).
•
In the Navigation mode, the route explanation style can be switched between the turn-by-turn (default) and the north-up navigation.
In summary, all elements in our proposed map are as follows: (1) the 3D and layered floors, which include inter-floor structural attractions, the outlined exhibits, and facilities shown by volumetric symbols; (2) the audio-tactile interactions, which include the two-level audio guide of exhibits, an audio guide at the intersections, and navigation by tracing the paths and intersections.
4. User Study
We conducted a user study at the same science museum to investigate our research questions and evaluate the effectiveness of our proposed system. The staff who joined the final design process (Section
11) stressed that visitors came with different interests and expectations. A fixed task and a rigorous evaluation of the performance might discourage the participants, who are also important stakeholders. We came to agree that a tour design task should be flexible to reflect different user styles and support curiosity and autonomy, which are the museum’s important social roles. We included a tour after the map exploration to help visitors generate feedback towards a real museum visit. Each individual study took two to three hours in an order of tour design, conducting the tour, and post-tour interview.
4.1 Participants
We recruited 12 blind participants (male = 5, female = 7) with ages ranging from 24 to 71 years old (mean = 53.8, SD = 13.1). They were recruited via an e-newsletter for people with visual impairments, and compensated $75 plus travel expenses for their time. All of them were first-time visitors who held minimal preset knowledge about this museum where the study took place. Six participants were frequent museum visitors who visited other museums more than once a year (P2, P5, P7–P9, P11). Three visited other museums every two to three years (P1, P3, P4), and three rarely visited a museum (P6, P10, P12). All of them had experience with tactile materials, including tactile graphics (P2, P4–P12) and 3D models (P1–P11).
4.2 Task and Procedure
4.2.1 Structural Exploration
We first introduced the stacked BentoMuseum, its orientation, and the external structures. We then encouraged the participant to disassemble the floors to experience the “Bento Box” characteristics. The subsequent reassembly process incorporated touch exploration of inter-floor elements like escalators and the Oval Bridge. Finally, the participants were primed with a list of 10 icons (eight from Figure
3d, plus the escalator and wall). This phase took 10 minutes.
4.2.2 Training Phase
A training phase was conducted to familiarize participants with the audio-tactile interactions. The 1st floor map in Exploration mode was presented on the touch screen. Steps included: (1) double-tap one exhibit to hear its name; (2) double-tap for further details; (3) double-tap an intersection for surrounding information; (4) explore to find the guest room; (5) adjust voiceover speed using speed button; (6) stop audio using stop button; (7) switch to Navigation mode using navigation button; (8) set special exhibit zone as goal, guest room as start, and then trace the route following the audio guide. This phase also took about 10 minutes.
4.2.3 Tour Design Task
A loosely structured tour design task was conducted. The individual participant was asked to imagine the following real-world scenario: The system is placed at the entrance of the museum, and they are using this system to select the exhibits of interest and design a unique tour for themselves. With the help of the staff, they can place any floor on the touch screen. From a total of 28 exhibits, they were asked to select 6 (equivalent to a two-hour tour) and to try to build a mental map with routes connecting the spots within 45 minutes. Considering the real-world scenario, they were also free to take notes. During the task, a researcher was taking notes of the selected spots for later evaluation. We video and audio recorded the session and saved app log data for later analysis.
When time was up or the participant was finished, they were asked to orally explain the (1) name and (2) orientation and location of each spot. Based on their explanations after the task and during the tour, we determined which level of mental map they possessed from among five defined levels:
Level 1: Hardly remember any exhibits they chose.
Level 2: Remember some of the exhibits they chose.
Level 3: Remember all of the exhibits they chose.
Level 4: Remember all of the exhibits they chose and which floor each exhibit is on.
Level 5: Remember all of the exhibits they chose and the location of each exhibit.
We also asked the participants to give a self-evaluation of what level of the mental map was needed.
4.2.4 Conducting the Tour
To validate the mental map in a real-world setting and gather insights, participants were invited to take a 15-minute walkthrough of their designed tours with a museum guide. Elaborated guidance of a chosen exhibit was included to provide a brief museum experience. Participants concentrated on traveling and validating their mental map, and they were allowed to fully explore the museum after the study.
4.2.5 Post-Tour Interview
A roughly 30-minute interview was conducted after the participant was settled back in the guest room. The interview included two forms: a seven-point Likert rating, from strongly disagree (
) to strongly agree (
), and free responses. Four sections composed the interview: (1) rating the overall experience of using the system (Q1–Q3 in Figure
4); (2) rating the overall system usability (Q4–Q6 in Figure
5); (3) rating the 3D floors and audio-tactile interactions, which were further divided into eight specific elements in terms of A. understandability and B. usefulness (Q7.A–Q14.B in Figure
6); (4) free responses about using the map prior to the visit, the strengths and limitations of the system, applications, and other findings after the tour. For all of the ratings, we asked participants to consider or imagine accessing information by audio means, such as reading a homepage when preparing for a visit, as a baseline (
).