Keywords

1 Introduction

Our society is facing various fundamental social problems, including challenges to environmental sustainability and human well-being. Without altering human behavior, it will be essentially impossible to overcome these problems [7, 31]. Researchers have been working to develop persuasive technologies that guide human behavior through computing approaches [15, 24]. Recently, advanced wearable technologies have allowed us to change our vision and to see non-existing things and occurrences in real spaces or to replace some of the actual things that exist in these spaces [9, 28]. In particular, a light-weight head-mounted display can present an eye-level view on a display, but the view can be modified through virtual reality technologies, and thus the meaning of the real space can be changed. In our approach, user experience felt by refining the real space’s meaning through fictionality is alternate reality experience. The alternate reality experience is typically achieved through modifying our eye-sights, and makes our world interactive by influencing human attitude and behavior implicitly through virtual reality technologies.

Despite advances in this technology, its possibilities have not been well discussed. In particular, to encourage people to reflect deeply on how their current behavior affects their futures – and to get them to change undesirable behaviors – we need a better way to show them different possible futures [9, 22, 23]. In this paper, we conducted a participatory design workshop where participants developed persuasive services to overcome social problems. In these services, people wear head-mounted displays to show modified eye-level views that enhance the meaning of real spaces using fictional events. One important lesson learned is the importance of balancing fiction and reality when designing and representing persuasive and effective possible futures as consequences of current behaviors. Additionally, narratives showing why behavior changes are necessary and how behavior can be changed are both useful and essential for encouraging behavior changes.

This paper is organized as follows. In Sect. 2, we present related work to the proposed idea in this paper. Section 3 shows the enhancement of the meaning of the real space, and Sect. 4 shows an overview of Alternative Reality. In Sect. 5, we present how to conduct our participatory design workshop, and in Sect. 6, we describe some lessons learned from the workshop. Finally, Section concludes the paper.

2 Related Work

Altering human behavior through technology has been seen as a promising way to break undesirable daily habits, and some researchers have attempted to develop design frameworks for designing persuasive services [24].

Most recently, digital marketing practitioners have adopted an approach known as gamification. Using badges and leaderboards is a typical approach to achieving gamification. Several studies have discussed systematic gamification designs [3, 5]. In traditional gamification, a set of game mechanics is widely adopted for motivating human behavior; however, incorporating game mechanics into the real world is not easy. Thus, simple mechanics such as badges, leaderboards and points are typically used.

Guiding people’s attitudes and behaviors is an important design issue when trying to address various fundamental social issues such as sustainability, health and happiness [7, 31]. Enhancing the semiotic meaning of real space through incorporated fictionality is a powerful technique that can alter people’s attitudes and behaviors [20]. As shown in [29], procedural rhetoric is a promising theoretical foundation for increasing persuasiveness by making the enhanced real space meaningful.

There are several existing case studies that use augmented reality technologies to enhance the meaning of real space to influence people’s behavior. For example, in [27], the authors propose a service for implicitly influencing the satisfaction people experience while drinking a beverage and for controlling beverage consumption by creating a volume perception illusion using augmented reality technologies. The system proposed in [16] aims to create a method of modifying perceptions of satiety and controlling nutritional intake by changing the apparent size of food using augmented reality technologies.

Augmented reality techniques can be used to enhance existing artifacts [1, 18]. For example, [32] describes several augmented reality games that are enhanced versions of traditional physical games. Specifically, Augmented Go [10] demonstrates a promising approach to maintaining the advantages of the physicality of the board game while adding virtuality. Virtual Aquarium [15] shows a virtual fish tank in which the movement of the fish reflects a user’s tooth-brushing behavior. Enhanced TCG [28] enhances our real world by replacing a real-world component with a fictional component for changing the semiotic meaning of the real world.

There are several ways to incorporate fictionality into the real world [19]. One typical approach is to use live action role playing (LARP) [14] or alternative reality games (ARG) [12]. During LARP, players play fictional roles based on a pervasive role-playing concept [11, 13] and a game master to control the gap between fiction and reality. ARG adopts a concept named transmedia storytelling [20], using multiple media to incorporate fictional stories into the real world. These approaches are promising, but the approach requires a rigorous plan that requires a long time to reduce the gap between fiction and reality. Augmented reality and virtual reality technologies offer another possibility to incorporate fictionality into the real world. For example, in [16, 27, 28], by using head-mounted displays, a user immersively changes the meaning of the real world to alter his or her attitude and behavior. The magic circle is defined as the boundary between the real world and the virtual world [17]. If a user is not aware of a magic circle between the worlds, the user cannot notice that the virtual world is not real. Therefore, he/she feels that the virtual scenes actually happen in the real world. The most important issue in realizing immersion blurs the magic circle.

When attempting to solve serious social problems such as sustainability issues, health issues, and happiness [7, 31], guiding people’s attitudes and behaviors is an important design issue. Enhancing the semiotic meaning of the real world through incorporated virtuality is a powerful technique for altering people’s attitudes and behaviors [21]. As shown in [29], procedural rhetoric is a promising theoretical foundation to increase persuasiveness by making the enhanced real world meaningful.

In [4], Dunne and Raby use design to offer new forms of expression for complex and critical issues; these forms of expression are grounded in the most abstract, speculative and future-focused considerations. Critical questions about emerging technology in everyday situations have presented preferable futures as opposed to predicting the future. They call this design approach Speculative Design. The approach taken in Alternative Reality can be considered an example of Speculative Design because the aim is to investigate whether a user feels as though he/she is watching a future scene when the scene uses only components that exist in the real world.

Our approach is also similar to the Substitutional Reality (SR) system [30]. In the SR system, people’s reality is manipulated by allowing them to experience live scenes in which they are physically present and recorded scenes that were recorded and modified in advance without losing a user’s reality. Our approach is more general than the SR system because Alternative Reality can use CG images generated by VR techniques not only recorded images.

Several previous investigations used a 3D model composed from real scenes. For example, in [2], a user interacts with the 3D model of a building to learn routes inside the building. The user can learn the real routes in the real town in the virtual world.

3 Enhancing the Meaning of the Real Space

In this study, we focus on the following five lenses in discussing the enhancement of the meanings of real places, as shown in Fig. 1. Each lens becomes a design frame that can be used to consider how to enhance the meaning of real spaces, with the goal of influencing people’s behavior from the frame’s point of view.

Fig. 1.
figure 1

Enhancing a real space

Full size image

Living Things are typically humans or animals. Objects are typically static things that appear in real spaces such as bicycles in Fig. 1. Landscapes are the backgrounds in real spaces. In particular, we consider the meanings of sets of objects (like buildings in the street, as shown in Fig. 1). Occurrences are events that happen in real spaces, such as the street performances in the figure. In Fig. 1, a sign in the street is typical information. These five lenses were used in the workshop to structurally discuss how each service enhances the meaning of real spaces.

In existing approaches to enhancing real spaces, Objects or Landscapes are typically enhanced. Additionally, Information is explicitly superimposed on the real space to help people make better decisions. On the other hand, typical VR applications offer fictional Occurrences to create impressive environments, and people tend to remember them better than they remember real Occurrences. Additionally, for effective behavior changes, the presence of empathetic Living Things, such as friends, pets or beautiful flowers, is essential [15].

4 Alternative Reality

Alternative Reality makes it possible to connect the real world with the virtual world from a single temporal perspective [8]. The worlds can also be seamlessly integrated because the virtual world consists of real landscapes, objects and persons. This means that it may be possible to enhance the real world by presenting fictional occurrences along with real events, and thus people experience an enhanced hybrid world in the real world rather than in a fictional world (as in a movie). Incorporating fictionality into real space strongly influences human attitudes and behavior. Thus, this approach can be used to guide people towards a more desirable lifestyle.

In Alternative Reality, a user watches a sequence of scenes on an HMD. As shown in Fig. 2, the sequence consists of several scenes. Some scenes are captured from contemporary scenes in the real world (Real Scenes in Fig. 2). The scenes are recorded by a 360-degree camera and shown on the HMD in real-time. However, some scenes in the sequence are not real scenes; such scenes may actually be constructed through VR techniques and are fictional (Virtual Scenes in Fig. 2). Additionally, the virtual scenes may include several events that do not occur in the contemporary real world. Typically, these scenes are constructed using 3D models of real persons, objects and landscapes in advance, but some real persons who are not actually present may appear. One of the important requirements of Alternative Reality is that the user feels that these real and virtual scenes are continuous and, thus, is not aware of the boundary between the two scenes. Therefore, he/she feels that the virtual scenes are actually happening in the real world. The most important issue in achieving this immersion is blurring the boundary between fiction and reality.

Fig. 2.
figure 2

An example based on alternative reality

Full size image

The use of an HMD provides a better immersion experience by showing a video stream that captures the real world and replacing some real components with fictional components. Additionally, the interactivity with real space offers agency, and the user believes that he/she autonomously chooses his/her activities by him/herself.

One typical way of enhancing the meaning of a real space based on Alternative Reality presents possible futures in the present real space so as to guide people’s behavior. This may be effective because the possible futures clearly show the consequences of the user’s current behavior, and behavior changes are encouraged if the possible futures are not desirable. As shown in [8], we have developed two case studies, named Interactive Improbable Object (shown in Fig. 3) and Fictional Future (shown in Fig. 4) to demonstrate the feasibility of Alternative Reality.

Fig. 3.
figure 3

Some scenes in interactive improbable object

Full size image
Fig. 4.
figure 4

Some scenes in fictional future

Full size image

5 Experiences Conducting a Workshop

In the workshop, each group created one or more digitally enhanced speculative images, depicting possible aspects of services based on the Alternative Reality concept. These services aim to make people reflect deeply on how their current behavior affects their futures and to give them opportunities to change their undesirable behavior. We discussed an effective way of guiding human behavior by integrating fictionality into real space based on Alternative Reality. Incorporating fictionality into real space has the potential to guide our behavior by naturally presenting the goals and effects of our behavior as possible futures. We analyze and propose what types of services make fictionality effective and what pitfalls and benefits Alternative Reality offers when naturally integrating persuasive fictionality. These discussions enable us to extract useful insights in order to develop services that will use AR technologies to encourage better human lifestyles in the future.

In the workshop, five groups (with three people in each group) discussed design issues in the following two stages. In the first stage, each group suggested a couple of proposals for services that used the Alternative Reality concept to guide human behavior. This discussion allowed us to determine what types of services could use Alternative Reality effectively. Then, each group discussed alternative approaches to implementing the proposed services – without Alternative Reality – and presented the benefits and potential pitfalls of these alternative approaches. These discussions made clear the types of services that could use Alternative Reality effectively.

Most of the services proposed in the workshop that aimed to guide human behavior were based on a narrative, such as reminding someone of something, cleaning a room, or looking at one’s poor physical condition. For example, in the service based on looking at one’s poor physical condition, a user looks through an HMD and sees his/her body shape growing gradually fat if he/she continues bad eating habits. However, showing the future effect of a bad habit is limited in its power to persuade people; it is important to show people – through the narrative – how to change their habits and why they should do so [15]. In contrast, there are a few suggestions that do not use the narrative, such as instructing someone in the correct way to use a machine or changing the color of a food.

Services that include a narrative can naturally deliver messages that persuade people. For example, in typical movies, people are persuaded by narrators or actors simply by watching the movie [6]. Based on the five lenses described above, enhancing a real space with fictional occurrences is difficult to achieve through traditional AR technologies because most AR technologies focus on the enhancement of objects and landscapes in the real space, or just show information superimposed on the real space. On the other hand, incorporating a narrative into the real space is not difficult because fictional occurrences instigated by a friend (a living thing) and inserted in virtual scenes can be easily realized through Alternative Reality. Thus, in the human behavior change services proposed in the workshop, the use of Alternative Reality is more appropriate than the use of traditional AR technologies.

In the second stage of the workshop, each group selected one service from those suggested in the previous stage and discussed the design of the service in detail. We selected the following two services because they highlighted how to use a narrative as a central mechanism to persuade people to change their behavior.

The first service is a reminder service, as shown in Fig. 5. When a user is likely to leave an umbrella on a train or a bag in a cafe, his/her view inserts a virtual scene of a fictional occurrence where an authentic person reminds the user to remember the umbrella or bag before leaving instead of alarming him/her about not forgetting it. The user believes that the authentic person teaches him/her to be more careful about not forgetting the item. Using a narrative is essential to make the user behave more carefully because the narrative easily presents why behavior changes are necessary and how to change one’s behavior. In contrast, the user treats an alarm as a single occurrence and changes his/her behavior only temporally, rather than for the long term.

Fig. 5.
figure 5

An illustration of a reminder system to make a user more careful.

Full size image

The second service trains people to express opinions that they usually hesitate to discuss, as shown in Fig. 6. Using this service makes users more able to offer helpful opinions in a positive way and helps avoid misunderstandings. The narrative, which involves empathetic friends, lowers the hurdles to sharing opinions. When the user is not busy, a virtual scene shows an occurrence where a friend asks for his/her opinion on a person whom they met a little earlier. Practicing giving his/her opinion about someone he/she met earlier lowers the hurdle to sharing opinions of others, and the user’s habit – perhaps caused by his/her shyness – can be changed.

Fig. 6.
figure 6

An illustration of a training system to give opinions that are usually hesitated to say.

Full size image

6 Lessens Learned

The results of the workshop suggest that services using the Alternative Reality concept allow people to guide human behavior by, first, showing the fictional effects of behavior as possible ideal futures and, second, by presenting narratives indicating why behavior changes are necessary and how they are possible. Narratives can be easily incorporated through virtual occurrences that are one of the five lenses presented earlier in this paper. The five lenses are a useful set of tools for analyzing the reality of the enhanced real space. Each lens becomes a frame through which to analyze the reality of the enhanced elements in the real space one by one.

When using traditional AR technologies, if the enhanced real space does not offer exaggerated effects, people may not be aware of the enhanced meanings of the real space. If using Alternative Reality, as shown above, it is possible to incorporate fictional occurrences by inserting them in a virtual scene. This approach shows the merits of adopting Alternative Reality when designing services that aim to change human behavior. Fictional occurrences can easily include a narrative and naturally present persuasive messages via a friend’s voice. Although the approach is ambient, the effect of the voice is strong enough to make the user aware of the meanings of the possible futures, and the user’s agency is increased because communicating with a friend is interactive without compromising the immersion effect of the real space. This approach also increases the possibility of enhancing human intrinsic motivation, which is essential for long-term behavior changes [15].

We still need to investigate alternative ways of guiding human behavior. As shown above, reality is an important design resource for designing effective methods of behavior change, but it is a fact that a narrative alone may not be effective at persuading people. For example, people are sometimes stubborn and do not listen to their friends’ voices. The fictional effect, which makes people marvel, is also an important design resource for human behavior change [11]. Based on the insights discussed in design fiction, the suspension of disbelief is an essential design strategy for designing believable fictionality [26]. This strategy is different from traditional approaches to maintaining the reality of incorporated fictionality in real space. Investigating how to design believable fictionality that still makes users marvel is one important future direction in our research.

One potential pitfall is that the current approach may cause some miscommunication between the user and his/her friends because inserted occurrences with these friends are fictional. Thus, the friend may not know about the fictional occurrences. Consequently, asking about these fictional occurrences may cause some potential misunderstandings. Additionally, the friend’s personality should be consistent with that of the virtual friend who appears in the services. If not, the fictional occurrences may lose their sense of reality and reduce the effectiveness of behavior changes. Maintaining consistency between fiction and reality is also one of our important future directions.

7 Conclusion

The paper discussed how to enhance the meaning of real spaces in order to overcome social problems through behavior changes. We presented some lessons learned that we extracted from our experiences conducting a participatory design workshop where participants designed persuasive services that enhanced real space.