Keywords

1 Research Background and Purpose

Challenges are endlessly made to innovate product design. A keen attention is being paid to user’s experience using emotional elements as a driving force of innovation. Donald Norman in Interface classified the levels of interaction between users and a product into visceral, behavioral, and reflective level. In visceral level, human uses intuition in determining the experience of using a product. This stage has a very important impact on purchase, use, and sale of a product.

Auditory user interface (AUI) or experience is the most representative domain of the emotional elements with a product that can give a user differentiated experience. Apple’s Magic Mouse, for example, installed a very tiny speaker in a mouse in order to give its users a unique experience while scrolling it and it wasn’t tiny innovation at all. Providing information through auditory sense, AUI is a very important element because it renders esthetics, significance, and usefulness to its users, and unity and corporate identity to companies. The sound that agree well with the characteristics of a product and one that is good to hear and has esthetic appeal play a role of making users have a positive image on the product that creates such sounds.

Therefore, it is high time we needed a guidance and a system that can help design AUI from emotional point of view. If there is an AUI guideline to emotions that can be commonly applicable to products, refined user’s auditory experiences can be designed and provided based on it. In this respect, the present study has the purpose of developing an AUI guideline that can explain what sounds should be provided by a certain emotional concept of a product.

2 Related Work

The UI (User Interface) is classified into five types according to the viewpoint of human sense. This is representative classification according to the senses such as a physical UI (PUI) that can be physically touched, a graphical UI (GUI) that can visually recognize information, an auditory UI (AUI) based on sound information, an olfactory UI (OUI) based on olfactory information, and a gustatory UI (GUI) based on the gustation. The value of using AUI is highlighted to enhance emotional experiences for products and service innovation. The auditory experience provides aesthetic value to the user depending on the mood of the listening sound. In addition, appropriate feedback on operation has a great effect on usability and the auditory sense itself has a new functionality (Fig. 1).

Fig. 1.
figure 1

Classification of the Interfaces to the five senses

Full size image

The UI (User Interface) is classified into five types according to the viewpoint of human sense. This is representative classification according to the senses such as a physical UI (PUI) that can be physically touched, a graphical UI (GUI) that can visually recognize information.

A representative example of well-designed UX/ UI based on auditory experience is the Korean subway system. The Korean subway provides different sounds to passengers when getting on the inbound or outbound lane. For passengers who have to choose the subway in the same place, these different sounds can improve usability to distinguish information (Fig. 2).

Fig. 2.
figure 2

Scenes on getting on the subway in Korea

Full size image

An example of a good application of auditory experience to a product is Apple’s Magic Mouse. Apple’s Magic Mouse has a small built-in speaker that provides feedback sound information on scrolling when the user scrolls. Most users do not even know that a speaker is built in the mouse. However, through this, users can receive more reliable feedback to their behavior (Fig. 3).

Fig. 3.
figure 3

Apple’s magic mouse

Full size image

“Boot up sound” is also an important example of Apple’s sound experience. When an Apple computer boots up, a specific sound is output to provide feedback that it has been booted up. This sound is not only an indication of the status of the product but also a representative example of expressing the concept of the product and the identity of the company. In fact, Apple is gradually updating the boot up sound every time a new product is released, and users are taking it as a symbol of Apple (Fig. 4).

Fig. 4.
figure 4

Apple’s MAC OS boot up scene

Full size image

As shown in the examples above, the auditory experience seems to be a small element, but it has an important effect on product design. As the requirements for systematic AUI design for product design have increased, many researchers have conducted research on this.

Lee (2003) analyzed, as a guideline for designing information, that it is helpful for users to provide different pitch according to the depth of the information to recognize present location, the more the depth is deep, the higher pitch is used and the more depth is low, the more effective it is to use the lower pitch. Lee (2007) and Hwang (2015) also conducted a study, as a guide to the relation between emotions and function, on the ability to provide a higher level of satisfaction to users when a sound with different emotions is provided in accordance with the type of function. According to Lee’s study (2007) and Hwang (2015), users prefer the emotion sound close to a high arousal to functions related to on & ascending, and the emotion sound close to a low arousal to functions related to off & descending. In addition, they preferred positive emotions for functions related to inform and preferred negative emotions for functions related to errors. Park (2012) conducted a study that derived guidelines for a specific product such as a washing machine. A detailed sound of each function was extracted to study the tone, intensity, and height of the sound. Kim (2000) conducted a study on guidelines for auditory feedback to improve the usability of portable digital electronic devices in addition to washing machines.

When we look around the existing research, we can see that researches related to information design, functions, usability of washing machine and mobile phone are the main subjects. The purpose of this study is to provide emotional AUI guidelines that can be universally applicable to information devices from a different view point.

3 User Research

To develop a guideline that can be applied widely, this study carried out an experiment on users, focusing on ‘beep’ sounds, which are often used for a product. The number of sounds was set to be 4 and 4 melodic patterns were used: ascending, descending, and combining. As for musical chords that come with various sounds, they consisted of major, minor, diminished, and augmented chord. Based on this composition, a total of 21 sound samples were created and they were tested in 269 participants (Table 1).

Table 1. Users’ Profile
Full size table

In addition, Besides, Russell’s Circumflex Model based on 28 adjectives were employed to measure users’ emotional elements that respond to those sounds (Table 2).

Table 2. Classification of emotion by russell’s circumflex model
Full size table

4 Result

The emotional responses to the beep sounds showed that the participants experienced such emotion as ‘awakened’, ‘tense’, and ‘startled’ when they consisted of one or two sounds. It is understood that the short and sporadic sound was heard like an alarm to them. For a beep sound that consisted of 3 or more sounds, the respondents showed divided emotional responses by chord type. Major chords gave the participants such emotion as ‘satisfied’, ‘peaceful’, ‘happy’, and ‘pleased’, regardless of the number of sounds and melodic type while minor chords presented emotions related to ‘unlively’, ‘depressed’, ‘disheartened’, and ‘calm’. Diminished and augmented chords can be made with a tone of 3 or more sounds. To diminished scales or chords, they showed emotions related to ‘disheartened’, ‘calm’, ‘unlively’, and ‘tense’ while they experienced such emotions as ‘awakened’ and ‘tense’ as well as ‘tired’ and ‘unlively’ from augmented chords (Tables 3 and 4).

Table 3. Ranks of emotional experience frequency by sound
Full size table
Table 4. Ranks of emotional experience frequency comparing male and female
Full size table

5 Conclusion and Future Work

This study conducted evaluation on users’ emotions responding to sound type for the purpose of designing AUI. The results demonstrated that emotion is closely related to the number of sounds and chords. More specifically speaking, emotion responded differently to chord types (major, minor, diminished, augmented), so this finding can help making a guideline to sound selection suitable for product concept. In future, it is necessary to conduct a study to see the connection between each emotion and product functions agreeable to it as an advanced academic attempt.