Keywords

1 Introduction

Mugs may be one of the most pervasive objects that accompany us in daily life. Their natural presence in various activities such as business meetings, friendly conversations, morning commutes and small-talks in hallways mark them as a potential actor in social encounters. From this point of view, we consider that mugs are social objects worth investigating for HCI to explore the interaction opportunities that computerized everyday objects could present for enriching existing social interactions [6]. Although there are some efforts in augmenting mugs, the focus is mostly on designing smart drinkwares that provide information about the liquid content and drinking-related activities [24, 25], or on considering them as non-interactive flat displays for triggering social interaction [12, 23]. However, there has not been sufficient research on what kind of affordances mugs could offer as tangible interfaces and how people might use them in face-to-face social settings.

As an attempt to shed light on these gaps, we turned to Turkish coffee fortune-telling ritual, a socio-cultural practice based on “reading” coffee ground shapes inside the cup and narrating this visual content to the other; thus, creating from an ordinary mug a display and interface for social interaction (Fig. 1). Our choice was motivated by the approach in HCI that considers investigating cultural phenomena and practices as fruitful resources to exploit existing mental models of individuals and explore new metaphors for tangible interaction [11, 20]. As this fortune-telling practice is based on deliberate physical interaction with a coffee cup (revolving, tilting, touching, pointing etc.) in a natural setting it allowed us to gather empirical data on actions performed with/on cups. Second, as the practice requires the person to examine the visual content inside a cup, that is the coffee ground shapes, it helped us explore potential interaction techniques for handheld cylindrical objects with an inner display.

Fig. 1.
figure 1

Coffee ground shapes inside a cup (h: 5.5 cm, r: 5.5 cm) that people “read” to tell fortune.

Full size image

Therefore, we first conducted coffee cup reading sessions with 18 fortune-tellers and analyzed their nonverbal behavior with the cup. The analysis yielded 11 characteristics that reflected how people used such a form factor with a visual content during social interaction. Following the analysis, we wanted to see whether these behaviors have the potential to be considered as valid and generalizable across cultures. Thus, we conducted a second study with 34 people (18 native Turkish and 16 native English speakers) that was designed on purpose to imitate the structure of coffee fortune-telling by concealing any direct cultural reference to it. Our aim was to observe whether people across cultures would still reflect the 11 characteristics in a setting where they were not exposed to the cultural dimension of the traditional practice. As a result, we observed a similarity between the groups in relation to the characteristics. Although this experiment alone would not be sufficient to prove that our findings are valid for all, it helped us to have an idea on the potential generalizability of the characteristics.

In this paper, we first provide a background on Turkish coffee fortune-telling and present the previous research on augmented mugs. We then report both of our studies in detail explaining the 11 characteristics we found, discussing the related interaction techniques we derived from each, and giving an account of the cross-cultural experiment. We conclude by contextualizing our findings by two potential scenarios in which an augmented mug is used as a tangible interface in a social interaction setting.

2 Background and Related Work

2.1 Overview of the Turkish Coffee Fortune-Telling

Turkish coffee drinking and fortune-telling sessions create an enjoyable, warm and sometimes humorous social environment. In other words, Turkish coffee fortune-telling transforms coffee from a simple drink into an object that mediates a friendly exchange of thoughts and emotions. As Turkish coffee is made by using a very fine powder, it leaves a considerable residue at the bottom of the cup. This coffee ground is believed to bear traces regarding the future of the drinker. Essentially, Turkish coffee fortune-telling ritual is based on “reading” this ground and telling what the person will face or what events will happen, how and when (Fig. 2). The fortune-teller uses the cup and saucer as a source of information to construct the fortune-telling discourse. Although, fortune-telling is generally associated with foreseeing the events to come, the practice is not limited to that. During the ritual fortune-tellers also refer to the past and present life of the person by discussing the state of various aspects such as emotions, friends, relationships, education, family etc. The perspective that fortune-tellers present is considered as a guidance which makes people reflect on their situation, behavior and actions. In other words, this phenomenon creates an opportunity for self-reflection and helps decision-making. Studies indicate that the fortune-telling is perceived as a method to seek social support and advice in different societies [1, 13, 14]. This supportive dimension of fortune-telling has also been examined from an interaction design point of view in terms of advice mediating handheld devices [4].

Fig. 2.
figure 2

Coffee cup turned upside down after drinking (1, 2, 3), reading the cup and saucer (4, 5)

Full size image

2.2 Augmented Mugs

Studies that consider coffee mug as an interactive object and examine it thoroughly are rare. The most notable one is The Media Cup developed by Gellersen et al. which is an augmented ceramic coffee mug prototype with diverse sensors placed on the bottom [3]. Through these sensors the mug could perceive the temperature of the liquid it contained, sense its motion and send these data to other devices in the environment. The researchers’ main interest here was to create a network of computerized devices that would communicate to each other. Therefore, they did not address comprehensively the social dimension that coffee culture presents. Although they tried to detect different positions and user gestures related to the use of the mug (stationary, moving, drinking out of it and fiddling with it), they did not examine or discuss the role of these in augmenting face-to-face social interactions.

Coffee mugs conceived as interactive objects to be used in social interaction are also rare. There are two related projects in this respect. One of them is the Paulig Muki, a coffee mug that allows people to share images with each other both remotely and in a co-located setting [23]. When people pour fresh coffee in their mug they see an image sent by a friend in the flat e-ink display on the outer surface of the mug. This project demonstrates that coffee mugs can introduce alternative ways of communication between people. Although it is a product that triggers social interaction and can be used in face-to-face settings, it does not sense how people handle the mug and benefit from the potential interaction techniques these gestures (drinking, holding, shaking the cup etc.) could offer to augment the cup and social interaction.

The other salient work is Mugshots, a coffee mug prototype with a small flat LCD display attached on the outer surface [12]. They were inspired from the theory of Goffman who suggested that social interactions are performative. By using the theater metaphor he explained that people perform differently in front stage and back stage, which correspond to public and private spaces. The aim of the researchers was to introduce the mug into the workplace to facilitate this switching between private and public performance. For instance, the user can see the picture on the flat display related to an incoming call from a relative in the personal office. When the user goes out on the corridor and meets with colleagues the picture on the display changes to an image predefined by the user for interactions in public. Although their study was exploring how coffee could be a social object and trigger conversations in face-to-face communication, they did not address how users handled the mug as an object and what kind of manipulative gestures they could perform with the mug during the conversation.

On the other hand, mug represents an unconventional form factor compared to flat surfaces as it has a cylindrical structure composed of curved surfaces. It is made of two circular adjacent areas: the inner and the outer surface. This way, it offers a potential space for cylindrical displays with both sides whose exploration could lead to novel interfaces and design challenges, however, we did not come across a study on how users could interact with these unconventional screens. The most relevant example of a seamlessly embedded curved screen is a mug designed by Intel that has LED lights integrated into the outer surface and displays basic images such as smileys, numbers or letters [22]. Although this mug makes it possible to use the outer surface of a cup for displaying information, it is not touch sensitive and it does not consider the inner surface as a space for interaction.

In summary, the literature review presented above indicates two research gaps: there is (a) no empirical data on how people tend to physically interact with mugs in a social setting and (b) lack of interaction techniques for handheld cylindrical objects that have a display on the inner surface. Our research aimed at addressing these gaps by conducting the studies reported in the following sections.

3 Study 1: Observing Fortune-Telling Rituals

3.1 Participants

Fortune-Tellers.

We recruited 18 undergraduate students who were knowledgeable in fortune-telling and have been practicing it in a social context. There were 15 females and 3 males (M age  = 22.4, SD = 2.1).

Listeners.

Two female university students, one 22 and the other 23 years old, were recruited to participate in the fortune-telling sessions. These participants were selected among female individuals who were undergraduate/graduate students and passionate about fortune-telling. Only one listener participated in each session. The purpose for recruiting two listeners was to schedule more sessions in a shorter time.

3.2 Procedure

The fortune-telling sessions were held at coffeehouses, or at cafes within the university campus. We chose coffeehouses as experiment space as they are popular places for fortune-telling activities. In order to obtain a consistency between each session we always conducted the study indoors and were attentive to maintain the same setting in each place: we selected calm environments, arranged participants to sit always at the table facing each other and placed the camera at the same angle and position. Each fortune-telling session was composed of a fortune-teller, a listener and the researcher. The fortune-teller and the listener were sitting at the table facing each other. The fortune-teller was telling the listener her fortune by reading the cup and saucer she used to drink the coffee on location. There was no limit to the duration of the session and the average duration of each session was around 10–15 min. Before each session the fortune-tellers were asked to complete an informed consent form and a questionnaire with demographic questions. The fortune-tellers were not given any instructions regarding gesture or object use. We used a cover research subject in the form to avoid participant bias and told the participants that our project aims to examine “Fortune-telling scenes in Cinema” and therefore compare real-life fortune-telling sessions with those depicted in films. We also planned a debriefing to send after the study.

The sessions started with the fortune-teller reading the cup of the listener. When the fortune-teller has finished with the cup s/he continued by reading the saucer. When the reading of the saucer was completed, the session ended. The sessions were video recorded with a smartphone camera by the researcher sitting at a distance from the table. A small tripod was also used in order to have a stable image, consistency in camera position and angle, and give freedom to the researcher to observe and take notes during the sessions. In some small locations we employed wide-angle lens to be able to maintain the same view. The camera was positioned to frame the hands, arms and faces of both participants to record the body movements.

3.3 Coding

After data was collected, all speech in the video recordings was transcribed into written format by native Turkish speakers. 2 researchers coded six participants (30% of the data), discussed conflicts and then the rest of the coding was completed based on agreed terms. We assumed both qualitative and quantitative approach in our coding. After watching all the videos we identified two main categories of bodily engagement with the cup during interaction: manipulative gestures and positioning of the cup. For the first category, we coded how fortune-tellers handled the coffee cup and what kind of actions they performed on and with it. For the second, we coded how the participants approached the cup and where they most engaged with it in the space around them. Regarding these categories, we aimed to capture the following instances during the coding: manipulative gestures performed on and with the cup; touch and mid-air gestures involving both surfaces of the cup; areas used for positioning the cup and frequency of contact with it; position of the cup according to body and the table.

Regarding the positioning areas the participants used, 4 different approaches emerged from the video recordings. We noticed that at some points during the sessions they were holding and manipulating the cup (1) on table; (2) at lower torso; (3) at upper torso; or (4) leaving it on the table and using only free-hand gestures. On table refers to manipulating the cup on the table surface. Lower torso refers to the area between the table surface and the chest while upper torso indicates the area from the chest to the eye-level. No contact refers to leaving the cup on the table, fully interrupting contact with it and starting to perform free-hand gestures. In determining these areas and calculating the spatial relationship between body, table and cup we consulted two sources [18, 19]. Beyond these points above we were attentive to search for interesting patterns or singular gestures that we could not predict prior to this study that could be a source of inspiration for the field. We thought that coding and analyzing these non-verbal behaviors could yield insights on how people bodily engage with coffee cups during social interaction and, this way, could inform us in designing interaction techniques for augmented mugs (Fig. 3).

Fig. 3.
figure 3

Still image from the first study. The listener on the left, the fortune-teller on the right.

Full size image

3.4 Results and Discussion

In the light of the coding described above, we obtained characteristics that could be a reference for user interactions with Handheld Cylindrical Object (HCO). We grouped our findings into the 3 following categories: Common Manipulative Gestures, Object-handling Styles and Idiosyncratic Behaviors. For each characteristic we first describe the underlying user behavior, then propose and discuss the potential interaction techniques for augmented mugs with inner display. We included Idiosyncratic Behaviors as we wanted to gather all kinds of user engagement with the cup and indicate as many interaction techniques as possible for designers to consider for such a form factor.

Common Manipulative Gestures.

Below we placed a table that shows the manipulative gestures performed by the fortune-tellers.

Revolving and Tilting.

These gestures were the most common manipulative gestures performed by the participants (Table 1). We intend by revolving, the hand gesture that turns the cup around itself. By tilting we intend the hand movements that change the inclination of the cup forward and backward in vertical axis. Revolving and tilting gesture performed together indicates that these two movements occur at the same time. The participants used this gesture as they were trying to see and browse through the visual content on the inner surface. Except for one participant, the most used gestures were revolving, tilting, or revolving and tilting together in every participant (Fig. 4).

Table 1. Common manipulative gestures performed on/with the cup in the first study conducted with 18 participants.
Full size table
Fig. 4.
figure 4

Revolving and tilting gestures were used to examine the visual content inside the cup

Full size image

While revolving alone was used to browse contents inside the cup, tilting seemed also to provide different viewing angles of the inner surface. Beyond these, people also made slight gestures that moved the cup around such as downwards, upwards, towards self and away from self. Therefore, a Handheld Cylindrical Object (HCO) should be able to respond to these changes in angle and rotation. Let us think of a curved screen embedded into the inner surface of a mug. In terms of interaction, revolving the mug clockwise and counterclockwise can be used for basic navigation on this inner display (i.e., going to the next page). The speed of revolving gesture might be used to jump from one visual element to the other on the screen such as navigating among thumbnail photos. In this regard, tilting the HCO might correspond to vertical scrolling action. Furthermore, revolving and tilting gestures might also be used to create gradual changes in media elements on the inner display such as adjusting the visual content (i.e., changing the brightness/contrast of an image). Designers can exploit these gestures not only for manipulating the media inside the mug but also to control the surrounding devices such as TV screens [7].

Tilting the Cup to Share the Visual Content with the Listener.

Almost half of the participants tilted the cup towards the listener and pointed to the exact spot where they recognized a visual element and made sure the listener was able to see it (Fig. 5). Such instances of sharing the information were observed in 8 out of 18 participants (Table 1).

Fig. 5.
figure 5

The cup was tilted towards the listener to share the visual content

Full size image

Therefore, HCO should provide the possibility for data sharing with the conversation partner. This gesture demonstrates that the inner surface of the cup is considered as a space not only for examining content but for sharing as well. In terms of interaction, the mug can be interpreted as an interface that allows the user to switch between private and public content. It is up to the user to expose what s/he sees inside the mug. In this respect, it also shows that a coffee mug does not only remain a personal artifact but might become a shared one during social interaction. For instance, while the mug is tilted towards the conversation partner to show an image, the user can manipulate the image by moving a finger on the outer surface of the mug. Designers can explore the potential of content sharing on devices with such a form factor by employing in-air or touch gestures in the interaction process.

Touching.

Touching the outer surface of the cup was also a common gesture (Table 1). There were cases where this action was a singular and slight touch, as if poking or rubbing the cup. In other cases, participants continuously played with the outer surface or engaged in various actions (Fig. 6a). For example, we observed that one participant tapped on the outer surface while she was trying to remember what she was going to say. She made the same action 3 times during her session. In this case, she uses the outer surface of the cup to communicate her frustration to the conversation partner.

Fig. 6.
figure 6

Gentle and light touches were applied on the outer and inner surface of the cup

Full size image

We also observed contact with the inner surface of the cup. One participant believed she saw an image which signaled undesirable events so she touched the inner surface to manipulate and erase the contents, therefore preventing those negative outcomes (Fig. 6b). This is intriguing in the sense that the participant got into contact with the inner surface with the intention of altering the information. It is also notable that an action which might not be socially acceptable, such as touching inside the mug, could be perceived as normal in the context created by an augmented mug. Therefore, the both surface of an HCO should be able to detect and respond to these types of contacts. In terms of interaction, touch gestures on the inner and outer surface of the mug can be used to select and manipulate content elements. Touching certain points on the mug can trigger actions such as sending messages, activating a process, responding to an incoming call etc. The fact that the user can see the inner and outer surface at the same time can create an interconnected touch space on the mug by providing a handheld double-sided display. For instance, while the user moves a finger along the outer surface, a visual element in the corresponding path on the inner surface can be triggered by this touch and start moving. Designer can benefit from this simultaneous touch interaction on both surfaces to explore novel interaction techniques for augmented mugs.

Pointing.

Pointing was another common gesture among the participants. We found that 14 of them indicated the inner or outer surface of the cup during at least one instance of the conversation (Fig. 7).

Fig. 7.
figure 7

Pointing gestures indicated inner and outer surface of the cup

Full size image

In pointing, participants indicated either a specific visual element inside the cup, or they pointed at the general direction of the cup to refer to a concept they were talking about, as if the referent was actually in the cup. For instance, a participant said “… but this is a man…” without actually showing the part that contains this specific image but instead indicating the cup in general. In this case, pointing is done mainly for the participants themselves as an expressive gesture and there is not an intention of signifying the pinpoint location of visual element. However, their gesture helps to highlight the referent in the discourse. In terms of interaction, this demonstrates that people do not only tend to use touch gestures but also hover above and around the mug. This behavior opens up many possibilities for designers. Therefore, HCO should be able to detect this kind of in-air movement of a single finger. Around-device interactions have been discussed in HCI in terms of extending the interaction space around mobile devices to include also mid-air gestures as an input method [15]. While these studies focus on smartphones with flat screens, devices with unconventional handheld displays, such as spherical ones, and cylindrical ones that have both inner and outer display, have not yet been explored. As we observed in our study, participants perceived the whole cup as the space where the elements of their narrative live and indicated these during the talk. These expressive gestures can be used to pull visuals from inside and send them to the outer display. They can also trigger various forms of media (i.e., visual, auditory, haptic) on the surrounding devices, or even smart clothes.

Object-Handling Styles.

Below we show how the participants engaged with the cup in terms of contact and space.

Position of the Cup According to the Table Surface.

The cup is mostly held in the space between the table surface and 35 cm above it. The highest point the cup was held was the eye level, which is 60 cm above the table surface (Fig. 8a). Table 2 lists the positioning areas in which the gestures were performed. Therefore, HCO should be able to recognize the gestures performed in this height range.

Fig. 8.
figure 8

(a) Approx. distance of the cup from the table surface; (b) approx. distance of the cup from the body;

Full size image
Table 2. Indicates how frequently the cup was handled by the participants and in which positioning areas.
Full size table

Position of the Cup According to the Body.

The cup is mostly held approximately 30 cm in front of the body. While showing the contents to the listener, the participants extended their arms at most to approximately 60 cm (Fig. 8b). Therefore, HCO should be able to recognize the gestures performed in this range. The distances given in this characteristic, and the previous one, could also guide us in determining the visual content properties on the inner and outer surface of the mug in terms of visibility and readability (i.e., text size, image resolution etc.).

Contact with the Cup.

In 86% of all gestures performed, the cup was held in hand (Table 2) (Fig. 9). Remaining 14% gestures were performed on table, although at least one hand was still holding the cup. Therefore, HCO should be designed in a way that its default status will be handheld. However, the interaction while the mug is left or handled on the table should not be ignored.

Fig. 9.
figure 9

Most of the gestures were performed while the cup was held in hand

Full size image

Holding the Cup in a Straight Axis.

Our observations revealed that the cup was mostly held in a straight axis, where the bottom was parallel to the table surface, although backwards and forwards tilting were also common (Fig. 10). Therefore, HCO should be able to recognize its different inclination states. The changes in inclination might seem problematic for designers in creating stable content on a mug. However, it might also be an opportunity in designing for fluid content [21] that user can control by revolving and tilting the mug (such as controlling a drop on the inner surface).

Fig. 10.
figure 10

Backwards indicates tilting the cup away from self. Forwards indicates tilting the cup towards self.

Full size image

Interrupting Contact with the Cup.

We noticed that participants performed most of the manipulative gestures while examining the visual contents. They carefully surveyed the cup, making many revolving and tilting actions, and once they began to speak, they left the cup on the table (Fig. 11b). Most slight touch gestures occurred during speech to indicate the contents, while angle or position changing motion gestures were mainly performed during silent examination periods. While the participants were holding the cup in air with both hands during this examination, one of them let go of the cup before they started speaking. In this case, one hand was holding the cup and the other was performing communicative gestures, which are used to express thoughts and intentions such as pointing at an object or hand movements describing a concept or thing [17] (Fig. 11a). In either way, contact with the cup was highest during examination and lowest during the conversation which is a commentary of the visual input.

Fig. 11.
figure 11

The cup was used to regulate the talk.

Full size image

Therefore, HCO should not only detect contact with the surface but also expressive hand movements performed around itself. As the mug accompanies the person while speaking, the gestures around the mug can be recognized as commands. For example, adjusting contact could be used to mark certain points in the talk, as leaving one hand indicates the start of speech.

Idiosyncratic Behaviors.

Below we share the singular qualities of the participants’ actions during fortune-telling.

Moving the Body Around the Cup Left on the Table.

2 participants left the cup on the table and had no contact with it. However, they moved their bodies around it to be able to see the inner contents instead of moving the cup itself (Fig. 12). They leaned forward or changed sitting position to get better visual frames. There were only 4 instances where the cup was left on the table without any contact however there was a gesture aiming at it. Therefore, HCO should be able to detect body movements around itself. The inclination of the body, head gestures and eye movements can provide information to the mug in terms of facial expression and gaze. The content inside the mug can change according to the proximity of the user’s body.

Fig. 12.
figure 12

Body moving around the cup

Full size image

Using Long and Continuous Manipulative Gestures.

We noticed that few participants continuously played with the cup in their hands and these manipulative gestures lasted much longer than other gestures (Fig. 13). Therefore, HCO should be able to detect a succession of movements as one action alone. Although a rare behavior, designers can explore the potential effect of long duration movements and continuous contact with the object for navigation purposes (e.g., browsing visual content slowly and in detail, zooming in a piece of data etc.).

Fig. 13.
figure 13

Long and continuous manipulative gestures

Full size image

4 Study 2: Cross-Cultural Behavior Experiment

Our second study consists of a cross-cultural experiment to see whether the characteristics found in the first study tend to be culture-specific or generalizable.

4.1 Experimental Design

By conducting this study, we wanted to observe if people would still reflect the 11 characteristics found in the previous study when they were not exposed to the cultural dimension of Turkish coffee fortune-telling ritual. The traditional practice is based on two main components: examining the coffee ground on the inner surface of the cup and constructing a narrative on someone’s life based on this content. In designing the experiment we imitated this underlying structure, but did not mention the cultural name and roles associated with it, that is, fortune-telling and fortune-tellers. Instead, we defined this narrative structure as a personal storytelling experience. Thus, we entitled the experiment as “Storytelling in Interpersonal Communication” and described it as a study on the role of storytelling between students in the campus. Complementary to this experimental setup, we included a non-Turkish population by recruiting native English speakers alongside native Turkish ones as narrators. These participants were asked to compose and tell a story, by looking at the symbols placed inside a mug, in which the protagonist was the listener.

In addition to the modifications above, we also changed the material components of the traditional practice. As the coffee cup is associated with fortune-telling practice in Turkey, we preferred to use a coffee mug, which is bigger in dimension, yet still manageable by single-hand, and usually not used for drinking Turkish coffee. It also presents a larger surface area that can be used for displaying visual content. To create the visual content on the inner surface, we used icons as symbols instead of abstract visuals resembling coffee ground, as otherwise this could have been perceived as a reference to the fortune-telling practice. First, we created a set of 16 icons from the website FlaticonFootnote 1. In doing this, we looked for providing visual consistency within the set to avoid creating bias so we randomly selected the icons among black/white ones only, and those with similar visual qualities (Fig. 14). Then, we arranged these icons in two basic layouts, ordered and unordered, inside two identical mugs. The first layout was composed of grouped icons in columns, distributed in an ordered manner that contained four icons. The second layout was composed of ungrouped icons distributed irregularly around the inner surface of the mug. The motivation behind using these two basic layouts was to gather data on the possible effects of content arrangement on the narrators. Therefore, we printed two copies of the icons as 12 × 12 mm stickers and we placed each set on the inner surface of the mugs (Fig. 15).

Fig. 14.
figure 14

The set of 16 icons we placed inside each mug

Full size image
Fig. 15.
figure 15

Mugs (h: 8.5 cm, r: 7.5 cm) with two different layouts (grouped and ungrouped icons)

Full size image

Although we asked the participants just to look at the content inside the mug, we also placed some icons, selected and printed in the same manner, on the outer surface to break the potential association of the experiment setting with fortune-telling practice. In addition, we preferred calling the icons as symbols in order not to make the participants perceive the mug as a digital device.

4.2 Participants

Narrators.

In total we had 34 participants as narrators. For the group of native Turkish speakers we recruited 18 people (M age  = 20.45, SD = 2.54) among the students at our university. For the group of native English speakers we recruited 16 people (M age  = 22.2, SD = 3.69) among the exchange students in the campus. The participants were awarded course credits and bookshop vouchers for their participation.

Listeners.

Two female university students, one 20 and the other 21 years old, were recruited to participate in the sessions. Both were native Turkish speakers and had a good command of English. Only one listener participated in each session. They were awarded money for their participation.

4.3 Procedure

The experiment was held at the coffeehouse in the university campus to maintain the same environment as in the first study. The narrator, the listener and two researchers were present in the sessions. The narrator and the listener were sitting in front of each other at the table. One researcher recorded the sessions and the other gave the instructions to the participants. The sessions were videotaped with a digital camera. We used a tripod to have a stable image and consistent frame throughout the sessions. The camera was positioned to frame the hands, arms and faces of both participants to record the body movements, as in the previous study.

The sessions were held in two sections as there were two different mugs. Before each session the participants were asked to complete an informed consent form. In the first section we explained the procedure and asked them to compose a story to tell to the listener by examining the symbols on the inner surface of the mug given to them. We reminded the narrators to examine only the symbols placed inside the mugs and tell a story in which the listener would be the protagonist, as in fortune-telling sessions. We also told them that they were free to select, compose and interpret the symbols in any way they would like to. The listener was expected to follow the stories by providing basic verbal and nonverbal cues (e.g., saying yes-no, nodding) and ask questions whenever she was curious to know more about the details. Each story was expected to last for approximately 5 min. When the narrator was finished with the first mug, the second section started with the other mug following the same procedure in which we asked them to compose a different story. In order to prevent the primacy effect we changed the mug order in each experiment session. For example, if a session started with the mug of grouped icons, the following session with the next participant started with the mug of ungrouped icons. At the end of the sessions the narrators completed a survey with demographic information and questions related to the storytelling experience with the mug.

4.4 Results and Discussion

We coded the video recordings of the sessions according to the categories we used in our first study. We excluded one participant as he did not follow the procedure and created stories by using the icons on the outer surface of the mugs. Our analysis showed that the participants from both groups demonstrated considerably similar behavior in terms of the manipulative gestures with and on the mug (Table 2). Although the dimensions of the objects used in both studies were different, people tended to interact with the cup and the mug in similar ways. As in the first study, the most common manipulative gestures used by the participants were revolving and tilting and only revolving, which were mostly used to browse the icons inside the mug. Also tilting was a common gesture as a way to examine the inner surface. Touching and pointing, although performed less, were used to contact the mug with fingers and indicate the content (Table 3).

Table 3. Common manipulative gestures performed on/with the cup in the second study conducted with 33 participants.
Full size table

The motivation for conducting a cross-cultural study was not that of comparing different cultural groups but rather designing a setting that allowed us to observe if people across cultures would demonstrate similar behaviors in interacting with coffee mugs. The main finding of this follow-up experiment is that people, regardless of their cultural experience, tended to perform the same actions with and on a coffee mug in a social context. Their behavior reflected 10 characteristics out of 11, which indicates that the characteristics might not be culture-specific. However, further studies are required to evaluate this. We noticed that one characteristic, tilting, was not used in this study for sharing the visual content with the listener, as it was the case with the coffee cup in the fortune-telling sessions. The ambiguous nature of the shapes created by coffee ground might have caused the fortune-tellers to share what they saw in the cup with the listener in the process of assigning a meaning to them. Instead, the icons in the mug were reflecting concrete images such as airplane, heart and thunder, and ready to be used as components for the stories without much need for interpretation. It seems that ambiguity created a collaborative space for making meaning together from the vague components of the visual content [9].

Regarding the role of the layout of the symbols, the survey at the end of the experiment showed that the grid-like, ordered layout on the inner cylindrical surface made it easier to compose stories as it provided a guide for navigating the visual content, and thus selecting and narrating the story components. Designers can explore this generative and supportive potential of a grid-based navigation in handheld cylindrical objects for storytelling purposes.

5 Potential Scenarios

The discussions around the results of our studies did not only demonstrate that the fortune-telling practice provided a natural and rich resource for identifying user behavior characteristics with coffee cups but also suggested two scenarios for the following contexts in which augmented mugs can be employed: (a) exploring lifelogs and (b) advising encounters.

5.1 Exploring Lifelogs

Elisa owns an augmented coffee mug that is connected to the apps and devices she uses for lifelogging and it receives the data they collect. This augmented mug presents Elisa’s personal data by an interface on the inner display that covers the whole cylindrical surface inside. The outer surface of the mug is touch sensitive and used for interacting with the inner display. Elisa meets her date Erik today. She is so excited and thinks that she can bring her mug as an ice-breaker. After they have finished with their coffee, Elisa hands over her mug to Erik and asks him to tell what he sees through her personal data in the mug. Erik tries to understand what this data, represented by an abstract visualization for privacy and playfulness that are also present in coffee fortune-telling, could correspond to and how it could be part of a narration. Accompanied by Elisa he browses through the data and composes stories by combining different pieces to extend the conversation (Fig. 16).

Fig. 16.
figure 16

Generating new or unexpected connections through continuous and exploratory navigation offered by the mug.

Full size image

This scenario illustrates how the coffee mug, a familiar component of social interactions, can be used as a tangible interface for sharing and examining personal data, and transforming it into narratives [5, 10]. In line with previous research, we argue that a way to understand the personal data and discover new connections might be changing our tendency from goal-directed information seeking towards an exploratory and serendipitous one [8]. In this regard, the coffee mug, a handheld cylindrical object, appears as a meaningful interface to extend and re-purpose the lifelog material for generating new connections and perspectives as its circular form factor embodies a continuous, random and exploratory navigation of the content.

5.2 Advising Encounters

Peter owns an augmented coffee mug that listens to the conversations around itself and provide information on the inner display regarding the content. The texts and visuals shown inside are visible only to Peter as the form factor of the mug prevents the other person to see the inner surface. One of his students, Mark, comes to see him to talk about his class project. During the talk he mentions a television show that the professor never heard of. As Peter do not want to interrupt his student’s discourse, and appear ignorant, he triggers a search on the internet by touching the mug (Fig. 17a). While Mark is explaining his project, Peter is getting information on the show by looking at the results on the display (Fig. 17b). If Peter wants he can share the content with Mark on the outer display of the mug (Fig. 17c). As the search is saved, Peter can read more about the subject on computer after the meeting.

Fig. 17.
figure 17

The mug provides textual and visual information instantly

Full size image

This scenario illustrates how an augmented coffee mug might be helpful for advisors’ need for instant information during a meeting without revealing themselves as its form factor provides a private display space [16]. Sometimes advisors might not want to expose their search activity as they would not like to break the natural flow of the communication as well as it might compromise their knowledgeable image in front of the others [2]. We believe that an augmented mug might be perceived less intrusive than other digital devices in an interpersonal communication as the mug is associated with social contexts like coffee talks. This natural presence makes it a suitable object to be employed as a tangible interface for social interaction.

6 Conclusion

In this paper, we presented our empirical investigation of Turkish coffee fortune-telling ritual, a traditional practice, as an inspiration for exploring the potential of augmented mugs in social settings. Our aim was to examine how people engage physically with coffee cups during social interaction and how these behaviors can inform the design of augmented mugs. In this regard, we conducted two studies. First one was an observational study with 18 fortune-tellers. Our analysis yielded 11 characteristics regarding the use of coffee cups that we grouped in three categories: 4 Common Manipulative Gestures, 5 Object-positioning Styles and 2 Idiosyncratic Behaviors. In this regard, the analysis showed that during the examination of the content inside the cup, people hold it with single hand and mostly at chest level; they use mostly revolving and tilting gestures; touching the outer surface and pointing at the cup are also common. In addition, we noticed that people move their body around the cup as well as perform long and continuous gestures with it. Furthermore, we found that the cup is used for social interaction purposes, such as leaving it on table as a sign for the start of a speech. This shows that coffee mugs are also used in guiding one’s talk and as a support for marking certain points in communication. These characteristics demonstrated how people tend to physically interact with coffee cups and constituted an empirical ground that enabled us to discuss potential interaction techniques for augmented mugs with inner display.

Then, we wanted to observe if people would still reflect these 11 characteristics when they were not exposed to the cultural dimension of Turkish coffee fortune-telling ritual. Thus, we designed a follow-up cross-cultural study with 34 participants, native Turkish and English speakers, in which we imitated the structure of fortune-telling without including any direct reference to it. Our analysis showed that the participants reflected 10 characteristics out of 11, which indicates the possibility that almost all the findings from the first study might be generalizable; that is potentially not culture-specific. Regarding the composition of visual content, we found that people consider grid-based layout helpful as navigation structure for storytelling on the inner surface of the mug. This grid-based structure can be exploited in designing novel tangible interfaces for storytelling purposes with cylindrical form factor. Furthermore, as these results were obtained in a setting independent of the fortune-telling context, these characteristics can be exploited as part of a tangible interface for interpersonal communication in various contexts.

The outcomes of both studies helped us propose scenarios that describe an augmented mug as a tangible interface for two potential contexts: exploration of lifelogging data and need for instant information in advising encounters. We believe that our characteristics and scenario enable an informed discussion of potential contexts and novel interaction techniques for augmented mugs, and for handheld cylindrical objects in general. They also provide a base for making prototypes and conducting user studies for further exploration of interaction possibilities with such form factor in social settings.