1 Research Background and Purpose

Former 2D interfaces required numerous performances of users to implement certain tasks, such as reaching to see favorite picture takes 3 steps of tapping camera roll icon, back button to see categories of albums, and finally tapping favorite category icon. It also provided a limited span of feedback interactions, whether wobbling or becoming transparent, due to limited 2D space. To complement and solve these usability issues of performing several steps for designated actions and limitations of interactions that mobile space could give as a feedback, 3D interface, especially the cube interface, was chosen as an effective means of interaction for its simple 3D structure able to contain diverse functions. Furthermore, z-force or 3D touch technology, suggesting new ways of control with the possibility of recognizing different pressure given, has been introduced to provide users the experience of a wide scope of interactions and to reduce the depth on using mobile interface, reducing the gap between real and simulated physical objects displayed on screen by utilizing real world metaphor [1]. In an era where different mediums exist to enhance user’s experience in operating virtual space, it is necessary to design a new 3D interface with natural control methods mimicking real-world interactions of people also with providing varied feedback that could be conducted only in 3D mobile space (Fig. 1).

Fig. 1.
figure 1

Concept image for research

Full size image

2 Related Work

Studies are aimed to provide users with a diverse experience in operating mobile space and are divided into research on 3D interface exceeding the limitations of a two-dimensional space and new ways of controlling the interface such as 3D touch.

2.1 Research on 3D Interface

Studies on the 3D interface using the cube has been conducted, which often appeared as an interaction medium in previous research concerned with the 3D interface. A research utilized a virtual cube interface using its rotation for running applications exceeding the limitations of 2D GUI. The work used each side of the cube to map respective functions providing fun and allowing multitasking for users [2]. Another study suggested a cubic-like 3D user interface for multi-dimensional data browsing using an arrangement of 3D space and three axes to allow users to access data in necessary [3]. Also, Visualization technique for presenting heretical information accessible through CRT display was devised [4]. Z-agon, a palmtop sized mobile multi-display browser cube, operates by physical movements of the cube to change channels of the contents assigned to each side of the cube [5]. Wang et al. [6] proposed a representation method of Content-Aware Cube unwrapping to manufacture actual stereo shapes using a panorama cube. Moreover, a study on Cubee, a cubic 3D display for physics-based interaction, suggested an interactive cubic fish tank VR display, bringing a dynamic simulation into the participant’s physical space and creating a unique physical experience with computer-generated animation through a 3D display system [7] (Table 1).

Table 1. Different usage of cubic metaphor in previous studies
Full size table

2.2 Research and Application of New Ways of Controlling the Interface

Studies were conducted on novel ways of controlling the interface by recognizing the force pressed onto the z-axis; the nanta, application program shows different movements and splashing of water responding to dissimilar strengths given to the z-axis [8]. Heo et al. [9] suggested a novel force-enhanced touchpad with multi-level user interaction capable of tracking finger hover and measuring normal and shear forces together. This technology was applied and tested in video browsing, substituting the joystick in precisely supporting video control. Multimodal zoomable user interface using 3D touch interaction was studied using a sensor array to measure the distance, responding to the position and distance of the user’s finger relative to the screen [10]. Research to prove the usability of MTUIs (multi-touch user interfaces) for manipulating 3D objects on mobile devices was conducted, compared to that of tradition GUI buttons [11]. PUYO-CON enables input based on direct touch, force, and shape transformation on soft material for enhancing entertainment experiences for game players [12]. These studies showed the possibility of using and mimicking physical properties in controlling virtual mobile space (Table 2).

Table 2. Different suggestions of controlling the interface in previous studies
Full size table

While various studies giving users the opportunity to experience novel 3D interfaces and interactions have been conducted, there seem points of improvement, allowing users to feel a sense of 3-dimensional space by combining merit of cube metaphor with a new way of input method with spatial awareness. In other words, whether being just a 3-dimension cube interface or providing a new way of controlling the interface individually, an integrated interface following user’s needs for experiencing virtual 3D mobile space and using various applications in intuitive and convenient ways is needed. Therefore, z-force cubic interface suggests a cubic interface united with z-force touch interaction, which mimics real-life physical manipulation, providing users an experience of virtual 3-dimensional space.

3 System Design

Z-Force cubic interface focuses on providing a novel experience by using the cube as an interactive medium and applying z-force touch on the overall function of the UI to provide real-life like interaction and differentiated service for using mobile space. 3D cube featured with z-force touch allows users to intuitively experience a 3D interface.

3.1 Prototype Device

For concept development, a prototype was designed and installed in Android tablet and smartphone. The application consists of 15 simple 3D cubes on one phase, able to move along the z-axis for the new set of phase with long pressing, a second, which worked as a substitute of force pressing. Functions of zoom in and out, rotation of each cube, and running application assigned to the sides of cube located at the center were designed for the gesture development appropriate for the system (Fig. 2).

Fig. 2.
figure 2

Design of the application prototype

Full size image

3.2 Z-Force Touch Gesture

Z-force cubic interface, using z-force touch as an interactive medium, utilizes a highly applicable input method based on research defining z-force touch gestures and user’s preferences [13]. Several touch gestures were chosen, considering previous heuristic research with developed prototype, resulting in fixation of the following touch gesture in controlling the interface (Fig. 3 and Table 3).

Fig. 3.
figure 3

Heuristic research on the concept and its feasibility

Full size image
Table 3. Suggestion of touch gestures according to different functions
Full size table

3.3 Navigation

Z-force cubic interface provides a unique navigation system, giving an experience as if one is controlling an object in real life and offer two types of navigation system, controlling independent or groups of cube and manipulating axis of the interface.

Each cube is rotational along the x, y, and z-axis following the movement of the finger dragging. Users force click on an icon and are able to easily drag over to the desired shortcut of application, which is assigned on each side of the cube. Applications run by simply tapping on the designated icon. Sets of icons are pushed back and forth moving along the z-axis by force pressing the clear space between the icons as if objects from distance get closer with stretching out one’s hand away. Z-force cubic interface provides zoom in/out function, allowing users to adjust the number and the size of the icons seen on the screen from the maximum of 15 cubes to minimum of only one when zoomed (pinched) with force (Figs. 4, 5 and 6).

Fig. 4.
figure 4

Navigation system of z-force cubic interface (alteration of the icon’s location, z-axis control, running application)

Full size image
Fig. 5.
figure 5

Rotation of a single cube

Full size image
Fig. 6.
figure 6

Zoom in/out function of z-force cubic interface

Full size image

3.4 Manipulation of Application

Z-force touch gives the experience of controlling virtual space in manipulating applications with natural metaphors. Through swiping the icons along the x, y, and z-axis, users can get a preview of detailed functions of the application. For the message cube, along with the function of providing shortcuts on each side of the cube, such as creating new messages, frequent or favorite contacts, the user can get the simplified preview on the recent message just by rotating the cube. The z-force touch allows users to have additional control on a single cube such as flipping through photos. Cube on YouTube allows users to check the updated, trending, watching videos and allows users to search videos without the inconvenience of running the application.

Swiping with force pressing is applied on seeing lists of album or notes giving the feeling of as if one reaches and pushes farther to bring an object out in real life. Z-force touch interface also uses a 3D cube to see a 360-degree photo or video and allow users to send virtual icons such as a heart via text messaging (Figs. 7, 8 and 9).

Fig. 7.
figure 7

List view on swiping with force pressing

Full size image
Fig. 8.
figure 8

Sending VR icon to partner

Full size image
Fig. 9.
figure 9

Previewing video on watch

Full size image

3.5 Interactive Feedback

One of the unique functions of the z-force cubic interface is its alerts provided by rotating or resizing the icon itself, giving user intuitive feedback and fun. On incoming alerts, the cube automatically rotates to give an instinctive alert for missed calls, SNS alerts, and etc. For notification interaction, the cube notifies the users by shrinking and swelling, moving beyond the static feedback of current interfaces (Fig. 10).

Fig. 10.
figure 10

Preview function of the cube interface/rotation of cube informing missed calls/update notification through shrinking and swelling

Full size image

4 Discussion & Conclusion

In this paper, a novel means of experiencing mobile space, a z-force cubic interface was introduced. The concept of z-force cubic interface combines virtual 3D cubic interface with z-force touch interaction which takes after real-life natural movements and suggested different applicable functions and interaction on using the system. This research aims to open a new era for users to experience 3D mobile interface without additional equipment such as VR gadgets. With a customizable and dynamic interface, along with the various functions it provides, users will be capable of controlling diverse, intuitive, and virtual 3D mobile space with enhanced usability and fun.