Keywords

1 Background

Computers are widely used in various situations throughout society and they are being introduced into the field of learning. In primary schools, for example, there are many classes where tablet-size computers are being used. TUI is attracting attention in the field of learning. With Tangible User Interface (TUI), a computer can be controlled by touching a physical object directly by hand. Therefore, the user can use a computer intuitively. Because of this advantage, TUI can be employed in the field of the learning.

However, Durfee [1] has pointed out that different results occur when a different interface is used even for the same task. This suggests that behavior and ways of thinking change depending on the interface used to solve a problem.

2 Purpose

In this paper, we focused on the effects of Tangible User Interface on human learning process and aims to investigate how the types of user interfaces causes the cognitive learning processes. By clarifying these characteristics, we believe that we will be able adopt what kind of user interface suit the task.

3 Related Works

Many researches have been done so far for education using TUI. TV globe [2], for example, allows children to learn the geography by connecting the physical globe to TV and Illuminating Clay [3] was developed to aim to understand the changes of the terrain. TUI is also used to learn computer programing. In Topobo [4], children can control the robot movement by combining the block. In these researches, however, the effects to be obtained for learning by adopting TUI are not investigated.

Only a few works have focused on the relationships between the types of user interface and learning and investigating the effects on learning according to the types of UI. Ito et al. [6] was compared TUI and GUI in learning process. In this research, they found that the subjects can enjoy the learning process with TUI while they feel painful with GUI in spite of the same tasks. However, they could not find the difference in their behavior and the results.

4 Experiment

We conducted the experiment to clarify the effect of TUI on human learning processes according to the types of user interfaces. For comparison, other interfaces are needed to clarify the characteristics of TUI. For this reason, we adopted GUI as a control group for comparison. Because GUI is used to control ordinary computers, we considered that it is suitable as a control group.

4.1 Learning Experiments Using a TUI

We chose logical circuit building as learning tasks that are easy to be implemented the tasks in the same appearance with GUI and TUI. The subjects are required to build a logical circuit watching a truth table by connecting some logic circuit. The three logic circuit used in this experiment were the “AND”, the “OR” and the “NOT”. In addition, the lead wire objects and the lamp objects, to connect among each gate and to confirm the output of the circuit respectively, were also prepared. In GUI condition, all operations for the logic circuit are “click” and “drag” on the icons with a mouse. In TUI, each logic gate is represented as a cube- shaped block which is graspable. The circuit can be created by arranging the blocks in 2-demensional place. The subjects asked to “think aloud”, and, their behavior and verbal protocol data were recorded by VCR.

4.2 Method

Each subject was asked to attend experiments the 6 times which were held once a week. Personality diagnostic test was conducted in first week to cancel the effect of the affinity between the personality and the UI types by being assigned the subjects to each condition based on the results of “the way of thinking”.

After the second round, the subjects asked solve the task of logic circuit. The task in second round is to fill the truth table from the connection diagram. The task after the third round is to build the connection diagram from the truth table.

5 Results and Discussion

5.1 Analysis of Behaviors and Verbal Protocols

We analyzed the subjects’ behavior and their verbal protocol data. There was no difference in the problem solving strategies between TUI and GUI in the early and medium terms. The difference between the conditions, however, were observed in the latter term.

Tables 1, 2 and 3 shows some parts of one subject’s behaviors and the verbal protocol data, which was typical in each conditions in each term. We classified their behaviors into three types of actions. First, “move object” is action that subject operate the objects. Second, “Operation check” is action that subject push the switch for checking the results of output.

Table 1. GUI subject A Protocol data
Full size table
Table 2. GUI subject A Protocol data
Full size table
Table 3. TUI subject D Protocol data
Full size table

Table 1 shows the data with GUI in the initial term. The table indicates that the subjects often read the instruction and watched the samples. This means, in this term, the subject did not understand how to operate the system and the structures of the tasks of the logic circuit, were making efforts to understand them and groped for the solution. From such a thing we call this term “knowledge acquisition stage”. In TUI condition, the subjects also stood on the similar stage in the initial term.

When the subject understood the method of operation of system and problem structures of making logic circuit, they moved to the next stage in the medium term, thinking in head and checking right by system. Table 2 shows the typical data with GUI in medium term. The utterance of “Do not light the lamp” can be considered that wrong results was obtained against his considerations. The subject with GUI in the this term tried to make whole solution of the diagram task and to guess the result in his mind, and the system were used only to check the results. We call this stage “logical thinking stage”. In TUI condition, the same stages were observed in the middle term.

In TUI condition, the subjects moved to the next stage in the latter term. In this stage, the subjects attempted to create the solution using the system though the system was used only to check the results in the logical thinking stage. Table 3 shows the verbal protocol data with TUI in the latter term. In this table, the subjects uttered “Just a moment. So Yes, yes, I made a mistake.” This utterance was appeared after the check of a local part of result among the whole solution, and it means they did not guess the result in his mind. Also, “Not ‘OR’, ‘AND’ should be correct” was appeared in moving the object and he tried to check the result as soon as he came up the idea of the local solution. In this stage, therefore, the subject with TUI in the latter term visualized a part of the connection diagram in their mind using the system and by repeating this process they tried to reach the solution. We defined this stage “visual thinking stage”.

On the other hand, the subjects with GUI did not move to the “visual thinking stage” and stayed the “logical thinking stage” even in the latter term. In GUI condition, the subjects moved the object after the deep consideration and checking the result after the guessing the result of whole solution. When their solution was wrong, they considered again in their mind.

These differences become more apparent when the task become more difficult. In difficult tasks, the subjects stayed consideration and did not cause any actions, while the subjects with TUI tried to find the solution by moving the objects without considering too deeply. In addition, TUI checked the result after guessing soon and when the result was incorrect they moved to the next operation soon. In this way, TUI is tend to allow to go into action as soon as an idea comes up.

5.2 Mode Classification Based on the COCOM

In order to verify why the difference of the strategies caused described in the previous section, we apply to the Contextual Control Model (COCOM) [7] in A Contextual Control Model.

In COCOM, the states of human thinking are classified using the four parameter,” number of goals”, ”plans available”, “event horizon”, and “execution mode”, into four control modes: “Scrambled control”, “Opportunistic control”, “Tactical control” and “Strategic control”. Figure 1 shows pattern diagram of COCOM.

Fig. 1.
figure 1

Pattern diagram of COCOM

Full size image

Here, we focus the “visual thinking stage” with TUI and the “logical thinking stage” of GUI and TUI to compare the control mode transition between the stages Tables 4 and 5 show the data which observed typically in each stage.

Table 4. GUI subject A logic thinking stage(COCOM)
Full size table
Table 5. TUI subject D visual thinking stage(COCOM)
Full size table

In “logic thinking stage”, the problem solving process was done mainly by the transfer between “tactical” and “opportunistic” modes. Table 4 shows data of logic thinking stage with GUI. No. 1 and No. 2 in this table show the initial state where this subject was considering which logic circuit should be used to achieve the goal and started the problem solving process in “tactical” mode. Logic circuit In No. 3, he failed to connect the gates using lead lines and rebuilt a new goal. Here, however, as the new goal was for very local solution, he was transferred to “opportunistic” mode. After repeating these processes, he changed his tactics to establish a new goal of grasping the current state and he moved back to “tactical” mode.

Table 5 shows the mode transfer states in the visual thinking stage with TUI. Similar to the logical thinking stage with GUI, the problem solving process were done mainly by the transfer between “tactical” and “opportunistic” modes. Looking at No. 1, 2, 3, however, this subjects moved to “opportunistic” mode, because he tried to find the goals to achieve by checking the lamp and the result as a workaround but failed to find the adequate goal and repeated the same process until No. 6. As the operation in No. 6 was accidentally correct, he could move back to the “tactical” mode.

Here, though the states of the subjects were sometimes classified into the “opportunistic” mode in both stage of “logical thinking” and “visual thinking”, their observed states were quite different between them. “Opportunistic” mode observed in the visual thinking stage is close to “scramble” mode. In COCOM, the mode is determined by majority of four parameters. While, the most cases where the subjects were determined in “opportunistic” mode in case of visual thinking stage, some parameters indicated scrambled mode, all parameter indicated opportunistic mode in case of the logical thinking stage.

This means the original COCOM model is not very appropriate to analyze the detail of the human learning process. COCOM is naturally a model to apply the plant operators who have already been well trained. So, such kind of operations to find what will happen by them has not been envisaged. And, it did not appear “scrambled” and “strategic” in this classification. Furthermore, when the subject did not smoothly find the goal, many of modes is determined “tactical”, because it is not possible to clarify the difference between “strategic”.

So, we need to improve COCOM to be adequate for the human learning tasks.

6 Improved Model

We made two improvements to a Contextual Control Model fit the analysis of human learning process.

First, new parameters, “action mass” and “intention for the goal”, to determine the control mode are added to a Contextual Control Model to identify “strategic control” mode from “tactical control” mode. “Intention for the goal” is to indicate the width of the range subjects sweep the path to the goal in thinking. “Action mass” is a parameter to indicate whether they have found the operation to achieve their current sub-goal.

The other improvement is to add a new control mode: “Explorative Control” mode. Though COCOM has five modes including “Explorative Control” originally, “Explorative Control” mode was eliminated from COCOM because plant operators are generally well trained and they seldom move to this mode. We considered that this mode is necessary to analyze the human learning process and decided to add this mode.

So, in our improved model, the five modes are defined as follows.

Figure 2 shows pattern diagram of improved model.

Fig. 2.
figure 2

Pattern diagram of improved model

Full size image

6.1 Mode Classification Based on T Suggested Model

Based on the improved model, we assumed more detail analysis of the difference between the stages. We show some typical part of the data in Tables 6 and 7.

Table 6. GUI subject B in logical thinking stage
Full size table
Table 7. TUI subject D in visual thinking stage
Full size table

In the logical thinking stage with GUI, the problem solving were proceeded mainly in only “strategic” mode and “tactical” mode. In Table 6 shows the data of the logical thinking stage with GUI. At No. 1, the subjects were in “strategic control” mode because, when the subject saw the problem, he could build the sub-goal and what kind of action is required. At No. 2, though the sub-goal was wrong, he could stay the same mode, because he could deal with the mistake. So, at No. 3 he can make a new sub-goal by moving to “tactical control” mode. Due to refine the sub-goal at No. 3, he moved to “strategic control” mode at No. 4 and tried to achieve it. At No. 5, he was able to see the path for the goal moving to “strategic control” mode and succeeded to solve the problem. In this way, most subjects tried to solve the problem by the transition between “Tactical Control” and “Strategic Control” in the logical thinking stage with GUI. Similar process was observed with TUI in solving easy problems. However, different strategy was used when the problems were difficult with TUI, though the same strategy were used with GUI in spite of the difficulty of the problems.

Table 7 shows visual thinking stage with TUI. With TUI in this stage, the subjects often move to “explorative control” mode and it was observed especially in solving difficult problems. At No. 1, he reset this thought and tried to make sub-goals sweeping the path to the goal in “Tactical Control”. At No. 2, he read the question to confirm the goal in “Tactical Control”. Here, he gave up to find the sub-goals toward the goal directly, moved to “explorative control” mode, and, and started moving logic circuit at No. 3, No. 4 tentatively. Due to moving to “explorative control” mode, he got a clue and went back to “Tactical Control” mode and could create a new sub-goal.

However, he did not divide the sub-goal into the series of actions and moved to “opportunistic control” mode. At No. 7, he started the deep thought about goal moving to “Tactical Control” mode. At No. 9, he was able to find some sub-goals in “Strategic Control” and succeeded to solve the problem. In this way, with TUI, the subjects frequently transfer between “tactical control” and “explorative control” modes to get clues by moving to “explorative control” mode. And when they find them, they return to “tactical control” mode.

In this way, the subjects of GUI solve the problem mainly in “strategic control”. Subject of TUI was distinctive how to solve in difficult problem. First, subject was in “Tactical Control”. Second, subject was in “Explorative Control”. Final, subject returned in “Tactical Control”. In other words, subject of TUI try to move logic circuit in difficult problem and explore the hint from try move. How to solve of GUI was not seen much. It is the characteristic that TUI solves the problem in “explorative”.

7 Conclusions

We investigated the types of user interfaces effect the human learning by using the improved cognitive model, and found that there are some differences in learning process between GUI and TUI. In GUI condition, many subjects started establishing sub-goals in the tactile control and strategic control modes and stay in them even after the learning is proceeded. In TUI condition, on the other hand, though the subjects use the same strategy as GUI in the initial term of learning, they found the other strategy and become to utilize it in the latte term of learning. In this strategy, when they face to the difficulties to establish sub-goals, they try to find clues by moving to the explorative mode and attempted to achieve the sub-goal by trial and error.

It can be said that TUI must be appropriate to learn ill-defined problems to require the emergence of creativity and develop new ideas, such as story making and brainstorming while GUI is appropriate for learning the well-defined problems, such as calculating and programming. In the near future, we should investigate it.

Furthermore, though we found the differences of learning process between with GUI and TUI, the reason why the differences are caused are not clarified. This is our other future work.

8 Application

We concluded that TUI make it can be said to easy to develop. In solving the problems which require of various ways of thinking because broad searches in the problem space are needed for such kind of problems. For example, we think that it may be applied for tasks such as storytelling.

In the future, we must clarity farther characteristics of TUI and GUI, and we believe it will contribute to the e-learning fields.