Keywords

1 Introduction

State of the art in computer science allows users to use various software and applications for the purpose of language translation; however the availability of hardware for this usage is very limited. Apart from computer and smart phone applications only limited systems such as wearable devices in the form of glasses have been developed in the domain of language translation [1]. With the aim of combining hardware and software and as an application of our proposed AI model, in this work, we have equipped the smart phone translation APP with robotic platform by adding touch and voice sensors, improving the audio output quality via a speaker and designing wheels for the robot movement.

In the age of digital technology, smart phones became an essential tool for everyone. Moreover, several software and APPs have been developed for the purpose of language translation. These several APPs for translation enable us to benefit from basic translation between languages. Even though the accuracy is not perfect yet, but the current technology can help us to understand the basic meaning of statements in a different language. Our motivation in this research was to equip the available APPs on smart phones with embodiment and navigation facility in order to develop a mobile translator robot. We have developed artificial intelligent systems which can process the user input for smooth and correct functionality of the robot.

Our ultimate goal is that the translation robot can function fully autonomous and that’s why AI modules such as artificial endocrine system and state flow are employed in our system. In this way the robot would acts intelligently based on the situation during interaction.

In our research we have focused on the robotic interaction design and development and for the translation module. We decided to simply use available translation modules such as Google translator. Hence our contribution is to extend the available translator APPs into embodied robot with intelligent behavior. We believe that empowering the APPs with robot navigation and interaction capabilities could improve the quality of communication and furthermore makes it more convenient and efficient. Despite various APPs, translation application has been very limited in robotic systems. The wearable translation robot is basically intelligent glasses, which can automatically translate multiple languages in real-time [1]. Such kind of technology could be promoted with applications such as Google’s glass. Our aim was to change the wearable nature to a robot shape.

Various advanced robots are equipped with basic translation capabilities. However, that is still limited to expensive robots, which are currently in the research phase. Our goal was to make the robotic system low cost, functional and available for public use. We have considered various modes for the functionality of the translation robot. The main function is to have voice as the medium of communication. Furthermore, users can use text for data entry. Additionally, the robot can use the camera of the smart phone in order to scan, identify and translate any written text in the environment which is shown to the robot. This paper is extension of our work for this research [2, 9].

2 Background

2.1 Artificial Endocrine System

Homeostasis is the property of a system in which variables are regulated so that internal conditions remain stable and relatively constant [3]. The endocrine system include pituitary gland, thyroid gland, adrenal gland, gonads, insulin, parathyroid glands etc. They can secrete a variety of hormones. Hormone delivery to the desired effect with the blood cells, take to change a chemical change in the body, to coordinate physiological function. The basic flowchart of hormones functionality is presented in Fig. 1.

Fig. 1.
figure 1

Flowchart of the hormonal reactions functionality

Full size image

We have employed Artificial Endocrine System (AES) on top layer of our robot navigation system. Artificial endocrine system concept is inspired from the biological system which empowers the robot to behave smoothly like the way hormones help biological system to behave smoothly with collaboration with the emotional system. AES is the calculation of the biological role of the endocrine system containing the basic model and the endocrine system by biological principles, models inspired by the wisdom of generic methods [4]. With such system, we will be able to make a response to external stimuli, and has control of the system using artificial hormones.

AES system has been used in various robotics applications such as Lovotics [5]. Physiological unit of the Lovotics artificial intelligence employs artificial endocrine system consisting of artificial emotional and biological hormones. Artificial emotional hormones include Dopamine, Serotonin, Endorphin, and Oxytocin. For biological hormones Melatonin, Norepinephrine, Epinephrine, Origin, Ghrelin, and Leptin hormones are employed which modulate biological parameters such as blood glucose, body temperature and appetite [6]. By using the artificial endocrine system in the robot’s AI, the robot can operate smoothly in an unstable environment. Another advantage of AES system is to make it possible to express “slow” relation between causes and effects as it takes time for an artificial hormone to be effective. Such property generates smooth and realistic behaviors by the robot.

Another advantage of AES system is to make it possible to express “slow” relation between causes and effects (as presented in Fig. 3) as it takes time for an artificial hormone to be effective. Such property generates smooth and realistic behaviors by the robot.

2.2 Translation

We have considered various modes for the functionality of the translation robot. The main function is to have voice as the medium of communication. Furthermore, users can use text for data entry. Additionally, the robot can use the camera of the smart phone in order to scan, identify and translate any written text in the environment which is shown to the robot. In order to give an appropriate command, we have employed extra sensors such as touch sensor to incorporate with the translation APP. Such interaction is not limited to direct commands. We have developed Artificial Intelligent (AI) systems, which can process the user input for smooth and correct functionality of the robot. Our ultimate goal is that the translation robot can function fully autonomous, and that is why AI modules such as artificial endocrine system and state flow are employed in our system (Fig. 2).

Fig. 2.
figure 2

Software architecture of the system

Full size image

Considering the limited functionality of available translation software, we tried to keep our design simple and believable where the user does not expect perfect translation but enjoys the help of the robot of conveying the message in different language.

Data from touch and sound sensors are transmitted to the processor unit which includes three modules. The State Flow module handles state transitions of the robot. Artificial Endocrine System (AES) is highest level of AI for smooth behavior generation. Finally, the Control unit includes a PID controller for navigation of two motors, which are connected to wheel.

3 Interaction Channels

In the time that almost everyone has a smartphone, the use of telephone is beyond traditional way of merely audio communication. Our communication in enriched with various capability and possibilities were phones have been changed to media device with multimodalities. Language is still a barrier in international communications and smartphones are getting equipped with different tools to provide easier understanding. We see the change from traditional phones to smartphones and believe that future communication tools require robotics structure where output would be beyond simple display. We believe that robotics can change the way of current communication during interactions. In our developed translator robot three channels of communication are still via current smart phones.

3.1 Audio

Our robot is using two audio sensors, one is the one on the smart phone and the extra one is integrate in the robotic body. The microphone on the smart phone is directly employed for using the capabilities of the smart phone to use available APPs such as Google Translate. The other sound sensor is used beyond natural language processing (NLP) were sound parameters are input of robot AI system for controlling the behavior of the robot. In this case robot has two processors: One on the smart phone and the other on the microcontroller of the robot which run parallel software. The reason for separating these two channels was to use independent channel for two different tasks of NLP and control considering the heavy computation cost.

3.2 Video

Using the camera on the smartphone to scan the environment, interpret and understand the text translated into the other person, translation results will be presented on the screen as well. Such application is useful for example when user is in a restaurant and would like to read a menu in a foreign language. We also consider more development when robot can help the user to read various letters and translate with smart notification. For example a person who lives in another country can put all his utility bills in front of the robot and robot can send notification to pay bills in appropriate time.

3.3 Text

Using the screen and text entry capability of the smart phone, the users can also input the text by simply typing. In our user studies such functionality was especially useful when it was mistake in NLP and users wanted to enter a statement with exact words. These three modes are to facilitate the needs of users in different conditions to select the appropriate translation mode.

4 Robot

Our proposed robot design is beyond basic functionality of translation in this version of the robot. Current robot is especially useful in application like restaurants and coffee shops where the role of robot can combine translation and entertainment interaction. Our experiences in the lab where we have members from various countries were also pleasant when the robot became a medium between lab members as entertaining way of communication. One of our success achievements was to change the way people are disconnected these days by focusing on their own smart phones. By using the robot, smart phone is moved from hands of individuals to between people.

5 Method

The simple schematic of the proposed AI model is presented in Fig. 3. It is expressed that our proposed AES can relate causes and effects by taking into account the time factor.

Fig. 3.
figure 3

The schematic of the proposed cause-effect system

Full size image

For the purpose of implementation we have employed hydraulic system (Using physical modeling module of Simulink) in our AI engine in order to develop as system according to Fig. 4.

Fig. 4.
figure 4

The structure of the one unit of AES system

Full size image

When the system receives external signal information, which in our translator robot is touch and sound, the system behaves naturally to use that cause to define two artificial hormones. One relates to sound and one to touch. For example, if the input sound signal is active for long duration, it increases the level of artificial sound hormone which leads that robot navigates slower to hear all the conversation. We defined various commands for the touch sensors for changing mode. For example if touch sensor is activated multiple times, the level of relevant hormone increases, which makes the robot to change between modes frequently.

The output of AES is related to the State Flow module where several states are defined and managed in the robot. The detail of state flow is illustrated in Fig. 5. In the state flow module, we let the robot translate back and forth between two people. Based on our experiments, we adjusted the timing. The default value for changes of navigation between two people is 5 s. Users can commend the robot via voice or more directly via touch sensor. We explain the way of handling the robot for the user before experiment. In many cases, users also like to use the smartphone interface which can simply interrupt the robot movement using the touch sensor.

Fig. 5.
figure 5

Translate robot state flow

Full size image

6 Result

Figure 6 is the prototype of the robot hardware architecture including a touch sensor and a sound sensor which is responsible for receiving and transmitting signals to the central controller, and two outputs to control the motor, which makes the robot users can easily back and forth, Speak is to make use of who can more clearly hear the contents of the translation, but the top of the translation of the robot we have prepared an upright physical infrastructure to carry a Smartphone.

Fig. 6.
figure 6

The hardware structure of the translator robot

Full size image

As presented in Fig. 7 the robot prototype is covered to improve the believability of the robot. An example of changes in these two hormones is illustrated in Fig. 8 when we tested it for sinusoid input signals.

Fig. 7.
figure 7

Appearance of the robot

Full size image
Fig. 8.
figure 8

AES hydraulic translate simulation with touch sensor and sound sensor

Full size image

An instant of robot performance is illustrated in Fig. 9. In this experiment the robot is placed between two people where they can communicate in two different languages with the assistance of the translator robot.

Fig. 9.
figure 9

Translate robot between two users

Full size image

Another example of using the translator robot is presented in Fig. 10 where the user is employing the translator robot in a restaurant which the menu is in another language and the camera sensor is used to read the menu. Our experiment shows that such facility could be provided directly by the restaurant which engages the customer and facilitates the ordering process.

Fig. 10.
figure 10

One user is using the translator robot to read the menu in a foreign language

Full size image

Compare to using a human translator the robotic translator has several advantages. Apart from cost benefit, the robot is able to translate between several languages were human translators often only know couple of languages. Some users also expressed the positive issue of privacy when using the robot compare to having a human to translate. Users also found the robot entertaining during interaction. The translator robot acts beyond functionality when users enjoy having a smart robot navigating between them and assist them in conversation.

7 Conclusion

We have presented a robotic system which is used for the purpose of translation between different humans using various modes of interaction such as audio and text. The proposed system empowers available APPs with embodiment which enables the translation system to navigate in the environment and facilitate multilingual communication between users. We have tested this system in the country were English is not the main language and communication between foreigners is often troublesome because of the language barriers. Our robotic design also triggers positive interaction between users due to curiosity about the developed robot. This system can be used in various applications such as meetings, ordering foods in restaurants, shops, museums and tourist attractions. In future, we aim to further develop this robot with more extra performance and behavior such as adaptive movements. We also plan to perform formal user studies to improve the design and behavior of the robot.

Apart from the mobile robot platform for hardware we have also presented a comprehensive software structure for controlling the behavior of the robot. Apart from low level PID control system, the state flow architecture could manage the transition between various states and in highest level of software the artificial endocrine system manages the main behavior of the robot based on touch and audio sensory data.

In future we aim to focus on user experience analysis [7] and also we hope to be able to design a variety of different shapes and patterns of the shell, through the concept of user experience with mobile devices [8] collected for different user groups with different preferences.