1 Introduction

Human-computer interaction has come a long way since the first computers: from punch cards in the first computers, through keyboard-only terminals and, later, mouse-based graphical interface, and on to various touch screens, voice control and gesture interfaces.

Human-computer interfaces have become more intuitive: if punched cards could be read only by experienced programmer, the touch screen concept is understandable for three years old kids.

Another aspect of interface development is the blurring of differences between virtual reality and the real world. The phrase “Augmented Reality” is used commonly and not only in science fiction books.

User interfaces have become much more accessible and similar to concrete reality. A human interacts with a computer in the same way as he does with other humans and gets intuitive, understandable feedback. Those technologies are not yet well-developed, but we can already see the direction in which user interfaces are developing: humans can speak to a computer and a computer can speak back; a human can see a virtual world (or augmented world) around him and move virtual items using gestures, he can get physical feedback from the computer – for example a device-costume can hit the user.

Virtual reality interfaces allow humans to sense the virtual world just as they sense the physical world. The only thing that prevents humans from complete immersion in a virtual world is the fact that the items that belong to the virtual world remain there and only there. A human can move them by more-or-less intuitive gestures from one virtual place to another, but it is impossible to move items from the virtual world into the real world. Virtual artifacts remain virtual, we can’t touch them, and they disappear when the virtual world is turned off.

We challenge this restriction. We define a tangible interface and blur the last border between real and virtual worlds. We allow the user to turn virtual objects into physical objects and vice versa. To use HCI terminology: the human-computer interaction now includes passing objects between human and computer.

2 Background

2.1 The Tangible Interface – Description

The “objects passing between human and computer” here should be understood literally.

In one direction, the user can drag a virtual (drawn) object out of the screen in the same way he can drag it on the top of a touch screen. In this case the item will “get out” of the screen. (a virtual, drawn item will disappear from the screen and a real, physical item will appear outside the screen) – the item will be realized.

Or, in the opposite direction, the user can throw an item into the screen. The real item will disappear (or get out of the user’s range) and a virtual item will be shown on the screen. The item will be virtualized.

To enable this activity, we build a new input-output device called Realizer/Virtualizer, that recognizes and consumes physical items in the case of virtualization and emits physical items in the case of realization.

2.2 Theoretical Background

The processing of the information that we receive from the physical world involves all the senses. Cognitive research shows that visual data is much more understandable than audio data for most people, but the tactile sensations are more effective when dealing with little children. They study their surrounding by touching things, and their ability to understand the meaning of virtual artifacts is limited. For this reason, we expect that the tangible interface will involve the children in the experience and increase their understanding, creating an enhanced user experience.

In the case of adults, the difference seems less dramatic, but still we expect the tangible interface to improve their attitude and the perception of the game.

2.3 Classification

The concept of items’ transportation between real and virtual worlds is new and this technology doesn’t fall into any known category.

At first one might be inclined to define it as “augmented reality”. Indeed, the system includes virtual and physical parts. However, the phrase “augmented reality” refers to mixed real and virtual visualization of the world. Here the technology is different; it includes real and virtual parts, but the real parts stay real and the virtual parts stay virtual. Both worlds coexist, and items can easily and intuitively be transported between them. Although the terminology of “augmented reality” is not limited to visual effects only, the usage of such terminology here misses the target. After all, this technology can be used together with augmented reality technology or without it.

In the taxonomy of Benford (2) (see Fig. 1) the place of tangible interface even is not uniquely defined: there is no place for an interface that is virtual and real at the same time.

Fig. 1.
figure 1

Benford’s taxonomy of augmented realities

Full size image

One might also attempt to associate this technology with the “internet of things”. IoT augments the capabilities of real world objects by connecting them to the Internet, but this does not create objects which transition from physical to virtual reality. Physical items are recognized by computers and affect their behavior, but the real stays real and the virtual stays virtual. Further, tangible user interface technology can be used without, with or in addition to IoT technology.

This technology describes a new human-computer interface type, that has not yet been the object of serious research. Most references to such technology are in science fiction books, not a source we can study productively.

3 Experiment Description

3.1 System and Setup

This paper describes the evaluation of a system with a tangible interface implemented as a computer game. The game was played on regular 2D screen (no use of virtual or augmented reality by VR glasses), no voice command or gesture interfaces were used. The system that was used in the experiment used standard interfaces along with the tangible interface described here.

The tangible items that were used in the experimental game were metal coins of 27.5 mm diameter, represented virtually as a gold coin image of 39 pixels width. An input-output device was used: each time a user ordered the removal of a coin out from the screen, the virtual embodiment of the coin disappeared, and the physical coin fell out of the coin machine. Each time a user threw a coin into the coin machine, the virtual coin appeared on the screen.

In the game, the player is a seller in a grocery shop, customers come and ask to buy products, the player gives them the products and the customer takes the products and pays. Figure  2 shows an example of a screenshot when the customer arrives.

Fig. 2.
figure 2

The experimental system – game screenshot

Full size image

When the supplier arrives, he offers products for sale. In this scenario, the player should decide if he buys products and how many (he can choose not to buy). If the player chooses to buy, he must pay for the product or products.

A control group played the same game with only one difference: earned coins were stored in a virtual wallet – a gold coin image and a number drawn near it, the common representation of virtual wallets in computer games.

The experiment measured the change in the perception of virtual world when using the tangible interface. Because children and adults perceive the worlds differently, the experiment was done on two different groups:

  1. 1.

    Adults (in their 20 s).

  2. 2.

    Children (ages 6–8).

3.2 Experiment Flow

As mentioned earlier, the experiment included two test groups: adults and children. In each part thirty participants used the tangible interface version and thirty used the virtual version (Figs. 3 and 4).

Fig. 3.
figure 3

The experiment system – tangible interface version

Full size image
Fig. 4.
figure 4

The experiment system – virtual version

Full size image

In this game the decision the player makes is how many products to buy from the supplier. There were seven events of supplier arrival during the game. Two parameters are considered in this paper: how many items the player decided to buy and how much time it took him to get the decision. The measurement of the bought quantity is made in percentage from the maximum available quantity (it varies from product to product and from one round of the game to another. 100% means the player bought the maximum quantity possible).

4 Results

4.1 Adults Group

The adult participants who played the game with the tangible interface spent less money and it took them more time to make their decisions, compared to the players in the game without the tangible interface. We can see that after a stabilization period, the virtual version’s participants bought about 60% of the maximum quantity and the augmented reality group bought about 45% (Fig. 5).

Fig. 5.
figure 5

Results of adults experiment – investment

Full size image

The decision time period was longer in the game with the tangible interface, as we can see in Fig. 6: about 5 s in the tangible interface version, compared to about 3.5 s while playing the virtual version.

Fig. 6.
figure 6

Results of adults experiment – decision time period

Full size image

Adults took less risk and their decisions are more considered in the game using the tangible interface.

4.2 Children Group

One sees immediately that the children’s results were totally different from those of the adults: The children that played the game with the tangible interface spent much more money than the children in the reference group. They bought about 70% of the possible quantity when the virtual group’s children bought less than 30% of it (Fig. 7). The decision period was similar in both cases.

Fig. 7.
figure 7

Results of children experiment – investment

Full size image

5 Conclusion

The results in the children’s group seem to be the very different from those in the adults group. However, we propose that the psychological reason for the behavior difference caused by using tangible interface within kids and adults is the same.

This experiment verifies well known psychological truths: the usage of the sense of touch increases user attention and involvement in an activity. If it is better to see once than hear a hundred times, then it’s better to touch once than see a hundred times.

We know from cognitive development theory that using the sense of touch is essential for little children, and their ability to deal with abstractions develops later in childhood. The experimental finding verifies this theory: the difference between the children’s groups was much more pronounced. The adults use abstract concepts during their daily life (using a credit card as opposed to cash, for example) and they do it successfully, in general. Children’s ability to use abstraction is limited – from their point of view the virtual game and the game with tangible interface are two different games.

Both adults and children understand better the scenario of the game while using physical coins. This understanding causes the adults to think more before taking decision and to take less risky decisions. The same understanding causes kids to understand the process of buying and to enjoy it. That’s why the children bought more while using physical coins. This understanding of the game causes both to adults and to children to do what they want to do: to manage the finances for adults and to enjoy the game for children.

Children are influenced by tangible interface drastically and tangible interface is much more attractive for children.

The game with tangible interface is perceived totally in different way.

6 Possible Implementation

This paper introduces the concept of the tangible interface and examines deeply the usage of tangible interface in children’s game. But the tangible interface is not limited to this area only. Other areas that seem to have benefit while using tangible interfaces are gambling platforms and real-task training platforms.