1 Introduction

All over the world, obesity has become a serious public health problem that needs to be addressed [5]. Its incidence should be curbed because it increases the risk of many medical conditions, both physical and mental. It increases the risk of contracting diseases, which not only decreases quality of life but also increases the costs of national health coverage. Some countries have been facing the crisis of a deficit-ridden health insurance system. Thus, there is a compelling need for methods of reducing the incidence of obesity. To promote the treatment and prevention of illnesses caused by obesity, it is essential to establish healthy eating habits. One of the significant factors contributing to obesity is believed to be the excessive consumption of sugar-sweetened beverages. Such beverages are the main cause of excess energy intake in children. Striegel-Moore et al. revealed that the consumption of sugar-sweetened drinks had a significant association with body mass index, which may lead to a “high-risk of overweight” [17].

Numerous methods to promote physical activity and draw people’s attention to the amount of food that they are eating have been proposed to decrease rates of obesity [7, 8]. However, strong willpower and constant positivity are frequently required to increase people’s physical activities and improve their diets. Although those methods can lead to dietary improvements, they typically do not solve the problem of how to maintain the effort. These dietary improvement methods often require continuous effort on the part of the consumer to actually change their eating habits. Sustaining a highly conscious effort to adequately control the eating activity is difficult.

On the other hand, recent psychological and economic studies have revealed that the amount of food consumed is influenced by both the characteristics of the food itself and environmental factors during eating. For example, if a person decides to eat a certain quantity of soup, the size of the bowl acts as a contextual cue [19]. These contextual cues can be modified with augmented reality (AR). AR has great potential for enhancing and modifying individuals’ perception of real life experiences. Since human senses are highly interconnected, perception through one sensory system is changed by stimuli that are simultaneously received through other senses. This phenomenon is referred to as “cross-modality” [16]. As a result, retaining the impression from an eating experience can be modified significantly by changing stimuli through only one modality. For example, recent AR research has shown that changing appearance using AR changes individuals’ perception of smell [9] and taste [11, 12]. Moreover, some researchers have focused on changing environmental factors such as the apparent size of food as a cue to affect the perception of satiety. Narumi et al. have proposed the “augmented satiety” method for food-volume augmentation, using shape deformation processing in real time [10]. They showed that their system could change consumption volume from about −10 % to about 15 % by changing only the food’s apparent size without changing perceived fullness. Meanwhile, this system can only be applied to the case of solid food and cannot be applied to beverages. Thus, to undermine the effects of sugar-sweetened beverage on obesity, another technique that is applicable to liquid food and beverages is required.

Another psychological research study revealed that beverage consumption is influenced by the shape of containers such as glasses and cups. Raghubir et al. explain that this is caused by an illusion of volume perception such as in the case of the Fick illusion (vertical–horizontal illusion) [14]. Therefore, an illusion that affects volume perception, if used in the context of the volume of a beverage, can likely be utilized to change drinking behavior. In line with this, the authors implemented an “illusion cup” system (Fig. 1), which can overlay virtual cups of varied shapes onto a real cup, and confirmed that the apparent height of the cup is a vital factor in changing drinking behavior with regard to one sip [18]. However, previous research has not confirmed whether the system can change beverage consumption for long periods of time. Therefore, in this paper, we conducted an experiment that aimed to confirm the effect of the illusion cup system on beverage consumption over the course of one hour.

Fig. 1.
figure 1

Concept of our system for changing beverage consumption

Full size image

2 Beverage Consumption and Environmental Factors

As mentioned earlier, humans cannot accurately evaluate the volume or nutritional value of the food they consume. Hence, humans estimate consumption volume by using indirect cues such as stomach and bowel distension, elevated blood-glucose levels, and the apparent size of the food. These evaluations are not exact because some cues are evaluated relative to an individual’s surroundings.

Therefore, food and beverage consumption is affected by several external cues such as the apparent volume [10] and size of the cutlery [1, 6, 12, 16]. Some of these cues evoke visual illusions, which change individuals’ estimation of the food volume. Individuals’ consumption varies according to the perceived volume. Therefore, this change in volume estimation affects food and drink consumption. For example, in the context of beverage consumption, Attwood et al. showed that (1) participants were 60 % slower to consume an alcoholic beverage from a straight glass compared to a curved glass, (2) participants also misjudged the half-way point of a curved glass to a greater degree than they did in the case of a straight glass, and (3) there was a trend toward a positive association between the degree of error and total drinking time [1]. This effect, however, was not observed for non-alcoholic beverages. Attwood et al. explained that this effect was caused by misjudgments of the remaining amount. Moreover, when we drink from two kinds of cups with different shapes but the same capacity, we unconsciously consume more from the tall narrow glass than the short wide glass [14]. Raghubir et al. explained that this is caused by a difference in the perceived beverage volume owing to the vertical–horizontal illusion, in which people tend to focus on the vertical dimension of an object at the expense of the horizontal dimension. Because of this illusion, people perceive that a long narrow glass contains a larger volume of a beverage than does a short wide glass, despite the fact that the two glasses have the same capacity. Raghubir et al. also showed that the perceived volume of a beverage negatively affects the perceived consumption of the beverage. In short, people perceive that they have drunk more from a short wide glass than from a long narrow glass even though they have consumed the same volume from the two glasses. Based on these facts, it is assumed that people unconsciously try to maintain consistent satisfaction from drinking beverages and change their actual consumption when the apparent volume changes. Therefore, the apparent volume of beverages can influence drinking behavior. Girao also showed that the elongation of a glass positively influences the perceived volume of the beverage contained in it, while indirectly and inversely affecting the perceived beverage consumption [6].

As stated earlier, the perception of satiety is influenced by several factors. Controlling these contextual cues by evoking an illusion effect with human-computer interaction techniques may allow us to control (i.e., enhance or weaken) perception of satiety without conscious effort. This can be accomplished without changing the actual volume of the beverage. Thus, our previous study proposed a system that can elongate and contract the apparent height of a cup to implicitly control beverage intake by using the perceptual effects of illusion and AR technology [18]. We showed that this system could implicitly change the volume of one sip of a beverage from about −22 % to about 18 % by changing only the apparent height of the cup by ±30 %, using AR. This study examined only the effect of AR on one mouthful of beverage. Although it is assumed that the total consumption of a beverage is proportional to the amount consumed in one mouthful, the effect of the method on changing the total consumption should be confirmed. Thus, we decided to conduct an experiment that investigates the effect of the illusion cup system on beverage consumption over the course of one hour.

3 AR System to Deform the Apparent Shape of a Cup

People often estimate size and shape relatively; thus, volume perception is assessed according to the size of neighboring objects such as food, dishes, cutlery, and a person’s hand. Therefore, we needed to design an AR system that changes the shape of a cup consistently with its surroundings.

We constructed a system that can shorten/lengthen the apparent height of a cup, using an absolute AR setup. The system deforms only the appearance of the cup in the captured image in real time, using an image-based deformation technique. The equipment consists of a laptop, a video see-through head-mounted display (head-mounted display + webcam), a blue tablecloth, and a cup. The shape of the cup is almost cylindrical and it has a handle. Its height is 9.6 cm. The diameter of its top is 7.1 cm and that of its base is 6.5 cm. Its capacity is about 280 cm2. The cylindrical part of the cup is white and the handle is orange. We used this type of cup because its cylindrical shape is suitable for changing only its height. Although we used this cup for the experiments in this research, we believe our method can also be used with other types of containers such as glasses or plastic bottles. Users of this system have to wear only a video see-through head-mounted display and need to be drinking a beverage (Fig. 2). Figure 3 shows the flow of the image processing method for shortening/lengthening the height of the cup. Each step is discussed in greater detail in the subsequent sections.

Fig. 2.
figure 2

Augmented reality system that changes the apparent height of the cup

Full size image
Fig. 3.
figure 3

Image processing for shortening/lengthening the height of the cup.

Full size image

3.1 Extraction of the Cup and the User’s Hand in the Captured Image

First, the captured image was converted from RGB space into HSV space, and the relevant areas of the cup and user’s hand were extracted from the captured image based on the hue and saturation. The cup and liquid surface were extracted from one area of the image, and the user’s hand and the handle of the cup were extracted from a separate area; these groupings were chosen because of the similarity of the colors of the extracted items. Then, we calculated the centroid of the cup and the liquid surface area. We created a blue background using chroma keying, which facilitates color extraction, as shown in Fig. 2. A blue tablecloth is often used to cover a dining table, and is not an unusual item in an eating setup. Previous research [7] has also used this kind of setup. Thus, we considered it acceptable for an initial user study designed to test the effectiveness of the proposed system.

3.2 Changing the Height of the Cup

Next, we deformed the cup and the liquid surface area. We detected the edges of this area with a Sobel filter, and obtained the gradients of the cup by using the generalized Hough transform. Then, the cup was lengthened or shortened in the direction parallel to the gradients (Fig. 4). At this point, the deformation was performed without changing the centroid of the area. This is because the difference between center-of-gravity perception via haptic sensation and via vision may contribute to a feeling of strangeness and weaken the effect of our AR method on consumption volume. Some research has shown that when appearance is changed using AR, individuals’ haptic perception changes [2–4]. Indeed, Omosako et al. showed that changing apparent shape using AR affects center-of-gravity perception [13]. Therefore, we decided to keep the position of the centroid unchanged so that such an effect would be avoided.

Fig. 4.
figure 4

Changing the height of the cup without changing its centroid.

Full size image

Finally, we overlaid the shortened/lengthened cup, hand and handle images over the background image which we prepared in advance.

4 User Study on the Effect of Changing Cup Height on Beverage Consumption Over Long Periods of Time

Previous research [18] has clarified that our system, which involves using AR to change the height of a cup, can change drinking behavior with regard to one sip. The study confirmed that people drank about 18 % more with a cup whose appearance was lengthened by 30 %, and they drank about 22 % less with a cup whose appearance was shortened by 30 %. Although these results show that the system can change the pace of drinking, it remains unclear whether it can change beverage consumption for long periods. Humans may change their drinking behavior with a different pace of drinking. Therefore, using our AR system, we conducted a new user study that measures beverage volume consumed over the course of one hour in order to examine whether the method can affect beverage consumption for long periods.

4.1 Experimental Design

The study consisted of 15 male subjects. The subjects’ ages ranged from 20 to 23 years and the average age was 21.5 years. The subjects were screened to ensure that they were in good health, had no food allergies/restrictions, were not currently dieting for weight loss or trying to gain weight, were not depressed, and were not using any medication known to affect one’s appetite. The subjects were not informed of the actual purpose of the study.

The experiment used a within-subjects design. On three separate days, subjects came to our laboratory to drink juice. Although out of all existing beverages, carbonated drinks are the major cause of people becoming overweight, in this study, we used apple juice (Kokusan Tsugaru 100 % by Koshin Milk Products Co. Ltd.). This is because apple juice remains unchanged for a long period of time (and would therefore remain unchanged throughout the experiment), whereas carbonated drinks change in terms of taste (they lose their carbonation) in a short period of time.

During each testing session, the subjects were presented with a cup belonging to one of three conditions: Short condition (×0.70), Normal condition (×1.0), and Long condition (×1.3) (Fig. 5). To eliminate any effect of presentation order of apparent size, the presentation order was randomly assigned and balanced across subjects. The days on which the subjects were tested were separated by at least two days in order to prevent any satiation effects. We kept the temperature and humidity in the laboratory between 25[°C] and 28[°C] and 55[%] and 65[%], respectively, as these levels are believed to be comfortable levels of temperature and humidity in the summer.

Fig. 5.
figure 5

Appearance of the cup in the Short (left), Normal (center), and Long (right) conditions

Full size image

To prevent the subjects from knowing the true purpose of the study, we told them that the study’s purpose was to measure fatigue from wearing an HMD. If participants only wore an HMD and drank juice when they wanted, they would have tended to focus too much attention on the juice; this may have prevented us from measuring natural consumption. Therefore, it was necessary to give the subjects a task to perform. However, a visual burden would dilute the effect of our method. Consequently, we asked the subjects to sit and listen to radio news. The news they listened to had been broadcasted the previous night; thus, on each day of testing, the news the subjects heard was different.

When the remaining juice in the subjects’ cups had been reduced to less than half, we poured more juice in order to prevent the remaining quantity from affecting subjects’ consumption volume. We measured total consumption by measuring the weight of the cup just before pouring in more juice and before and after the trial. Moreover, we placed an electrical scale under the tablecloth so that the subjects could not see it (Fig. 6). We measured how many sips the subjects drank and how much juice was consumed in each sip. However, these values were subject to an unavoidable error of some grams because the measurements were made through the tablecloth.

Fig. 6.
figure 6

A user sitting and listening to radio news with the help of earphones

Full size image

To compare the effect of the cup’s appearance change in each condition, we measured “rate of change” (RC), which was defined as follows:

$$ {\text{Rate}}\,{\text{of}}\,{\text{Change = }}\frac{\begin{aligned} {\text{Consumption}}\,{\text{Volume}} \hfill \\ {\text{in}}\,\{ {\text{Long}},\,{\text{Short}}\} \,{\text{condition}} \hfill \\ \end{aligned} }{\begin{aligned} {\text{Consumption}}\,{\text{Volume}}\,\,{\text{in}}\,{\text{Normal}}\,{\text{condition}} \hfill \\ {\text{compared}}\,{\text{with}}\,{\text{that}}\,{\text{in}}\,\{ {\text{Long}},\,{\text{Short}}\} \,{\text{condition}} \hfill \\ \end{aligned} } $$

This formula for RC was also used in our previous study [18]. We hypothesized that the RC would be larger than 1.0 in the Long condition and smaller than 1.0 in the Short condition.

4.2 Procedure

The subjects were asked to keep their activity level on the three days prior to all three testing session days as similar as possible. Further, the subjects were instructed to not consume any food or drink for two hours before the experiment. The subjects kept a brief record of their activity patterns on the day before each testing session day and on each testing session day. Prior to each testing session, the subjects recorded their daily routine and we made sure that they had followed the prescribed protocol before each session, namely, at least two hours had passed since they eaten or drunk anything and their activity level had been similar to that in the days before the earlier testing sessions (e.g., no all-night work or strenuous exercise). If the subjects’ activity patterns differed from those of the day prior to the previous testing session(s), we postponed the testing session for at least one day.

We told the subjects that they could drink as much juice as they wanted, that they should hold the handle when they drank, that they should put the cup in the same place (which was actually on the electrical scale), and that they should not look around. Next, subjects were set up with the HMD and earphones. Then, we put the cup filled with juice in front of them. After the subjects had finished all the experiments, we asked them what they thought the difference among the three testing sessions was and what they thought the purpose of the experiment was. The subjects also answered a free-response questionnaire.

4.3 Results and Discussion

Average juice consumption and the standard error of the mean in each experimental condition were as follows: Short, 219.9 ± 35.5[g]; Normal, 242.9 ± 40.0[g]; and Long, 280.8 ± 35.4[g]. The average RC and its standard error were 0.94 ± 0.18 in the Short condition and 1.21 ± 0.13 in the Long condition.

We conducted Thompson’s rejection tests for each condition. As a result, one subject’s RC in the Short condition and another subject’s RC in the Long condition were rejected. In the case of the former, the subject’s physical condition ratings before the Short condition were largely different from his/her prior testing session ratings. Although we confirmed that the ratings did not have a difference of more than 30[mm], most of the ratings before the Short condition were still different by a little less than 30[mm] from those of the prior testing session. Thus, we considered that this subject’s data were not valid because of this difference in his physical condition.

In the case of the other subject whose data was not considered valid, the number of sips he took was 7 in the Short condition and 8 in the Normal condition, but only 1 in the Long condition. Moreover, he was exposed to the Long condition on the third day, which was the day he answered the free-response questionnaire. His response to one of the free-response questions was that the flavor of the juice caused bad feelings. This suggests that this subject’s feelings toward the juice became more negative and that he deliberately reduced consumption on the third day due to these negative feelings. Therefore, we excluded all the data of these two subjects from our analysis.

The other 13 subjects’ average juice consumption and the standard error of the mean in each experimental condition were as follows: Short, 206.0 ± 38.1[g]; Normal, 251.0 ± 45.8[g]; and Long, 283.7 ± 40.9[g]. The average RC and its standard error were 0.86 ± 0.083 in the Short condition and 1.25 ± 0.11 in the Long condition (Fig. 7). In other words, the subjects consumed 14 % more in the Long condition than in the Normal condition and 22 % less in the Short condition than in the Normal condition. We used paired t-tests with the Bonferroni-Holm correction. The tests revealed significant differences in the RC between the Long and Normal (p = 0.0496) and Long and Short conditions (p = 0.0242). It also showed a marginally significant difference in the RC between the Short and Normal conditions (p = 0.0595). The average and standard error of number of sips, and volume consumed in a sip were as follows: Short, 6.6 ± 1.1[times] and 34.8 ± 3.5[g]; Normal, 7.7 ± 1.2[times] and 38.6 ± 4.4[g]; and Long: 7.3 ± 1.0[times], 44.2 ± 4.5[g]. However, as mentioned earlier, these results contain error. We normalized these two results based on the results under the Normal condition and conducted z-tests on them; the tests showed no significance.

Fig. 7.
figure 7

Average RC in consumption volume in one hour and the SE in each height condition.

Full size image
Fig. 8.
figure 8

Average RC in a sip and the SE in each height condition (from [19]).

Full size image

Across all the conditions, no significant differences were found before the testing session with regard to ratings of hunger, thirst, nausea, tiredness, and prospective consumption (Fig. 9). Furthermore, there were also no significant differences across all the conditions with regard to these ratings after each testing session. These results indicated that our experimental design was valid. These results also showed that our illusory cup system maintains perceived satisfaction obtained from drinking even after the subjects have drunk the beverage. This is particularly compelling given that consumption is only influenced by the visual augmentation of the cup. Our results suggest that increasing the apparent length of a cup affects beverage intake and the perception of satisfaction; further, this effect holds true even when the beverage amount remains the same. Only two subjects noticed the difference in the height of the cup between the three days and none of them answered that they were conscious of their consumption volume. Therefore, we could confirm that the proposed AR system does not require the user’s effort or awareness.

Fig. 9.
figure 9

Ratings of physical conditions before and after the testing session in the experimental conditions (Average ± SE).

Full size image

Our results suggested that changing the apparent height of a cup can affect beverage consumption over the course of one hour. Figure 8 illustrates the average RC and its standard error in the previous study [18]. In that study, subjects consumed 18 % more in one sip in the Long condition than in the Normal condition and 22 % less in the Short condition than in the Normal condition. An independent t-test between the RC in the Short condition in the previous study and that in this study revealed no significant difference. The same test conducted between the RC in the Long condition in the previous study and that in this study also showed no significant difference. Therefore, it is assumed that our method has the same effect on beverage volume consumed in one sip and beverage volume consumed over the course of one hour, although the variance of consumption volume is larger because of the longer time span of measuring in the latter case.

5 Conclusion

In this paper, we aimed to confirm whether the illusion cup system, which involves changing the appearance of a cup using AR, can change beverage consumption over long periods. We conducted an experiment to investigate the effect of the illusion cup system on beverage consumption over the course of one hour. Our results showed that the system could change beverage consumption over the course of one hour from about −14 % to about 25 % by changing only the apparent height of the cup. By comparing these results and those of previous research [18], we showed that the total beverage consumption in one hour is proportional to the amount consumed in one mouthful, and the effect of the method on changing total consumption continues over the course of one hour.

The ability to control the subjects’ beverage consumption with minimal effort is a conspicuous advantage of our system. However, this study does have some limitations. First, the illusionary effect of the cup may decrease when we drink a beverage without looking at the cup. On the other hand, some research has indicated that the illusion effect of cutlery was enhanced while the subjects watched TV, since the effect worked in peripheral vision [20]. Thus, we need to assess the effect of our method when the subject is watching TV. Second, users of our method need to wear an HMD. As an alternative method, projection-based AR can be used for changing contextual cues. Recently, there have been many studies that have brought interactive surface techniques to dining tables [15]. By projecting patterns that evoke optical illusions onto cups with these techniques, we may be able to develop an augmented satiety method for beverages that does not require wearing any special apparatus.

While the current study is the first step toward designing an AR-based method that can help individuals regulate their beverage consumption, and while it no doubt has its limitations, we believe that the proposed system can help individuals more effortlessly control their beverage consumption. This should have significant effects on promoting nutritional health.