Keywords

1 Introduction

The management of unmanned aerial vehicles is one of the most important problem, which should be solved for the widespread implementation of UAV systems. The flying touch networks use the UAVs as devices to collect data from land sensors and transfer it to the server for processing. Such networks can be used, for example, in industrial production or agricultural industry for monitoring the processes or the UAVs can be used as flying routers [1,2,3,4].

There are three types of management of the UAV: manual control, automated management, automatic control. In case of remote manual control, the pilot is guided by the image transferred from the camera. Such control can be exercised by means of technologies of augmented reality and the tactile Internet. Augmented reality (AR) represents addition of the surrounding physical world with virtual data with the help of special devices of display, for example, of glasses [5]. The main task of the tactile Internet is transfer information about impact on a remote subject and the return transfer of response of this subject to influence with the minimal delays [6].

The managing man has an opportunity to influence the UAV through the application of augmented reality turns of the head and a ducking, watching change of the image from the camera in the device of display of augmented reality.

Solving a problem of management of the UAV by means of applications of augmented reality, it is necessary to solve several subtasks:

  • to provide transfer data about turn of the head from the device of augmented reality on the quadcopter;

  • to provide transmission of a video image from the camera UAV on the AR device;

  • to estimate quality of service on the network;

  • to estimate quality of experience of the user in augmented reality at management of the UAV.

2 Features of Data Transfer

The pilot can control movements of the quadcopter or the camera mounted on it by means of turns and a ducking. As the suspension bracket of the camera provides rotation only on a vertical axis, for support of the maximum review it is necessary to turn the pilotless device. In the AR device, which is on the head of the pilot, display of a video stream from the camera is provided. Thanks to it, the pilot can understand location of the UAV and transfer information about desirable turn of the camera. For this purpose, high-quality information display from camera and a timely response of the UAV and camera to movements of the managing man should be provided.

During transmission of a video image, we can face a problem of noises in data transfer channel. Transfer of video information is influenced by such parameters of quality of service of a network as losses and time delays [6]. It influences on quality of experience and can lead failures in control of the UAV, because in case of control the pilot can possess information about location of the device proceeding from the arriving video information. In this regard, for research was selected the model of a network (Fig. 1), in which the device of augmented reality exchanges data with the quadcopter with use technology of wireless data transfer of Wi-Fi of IEEE 802.11n. This technology allows to transfer the large volume of information with throughput to 150 Mbps [7], as it is required for high-quality transmission of video information.

Fig. 1.
figure 1

Model of network for interaction between the AR device and UAV.

Within this paper, influence of quality of service of network on quality of perception of the user in the AR system at management of the UAV will be investigated.

3 Setting of Experiment

Purpose: A research of influence of quality of experience in systems of augmented reality on control of the UAV with use of the laboratory stand.

For this research the laboratory stand in laboratory of the Internet of Things of St. Petersburg State University of Telecommunications was developed [8, 9]. The hardware complex consists of the UAV (quadcopter 3D Robotics IRIS Plus) with the camera onboard. To capture video, an Intel Edison module connected to a Microsoft LifeCam HD-3000 camera is used, which allows to record HD video in 720p. Videostreaming is implemented by means of the FFServer library and using the RTP and RTSP protocols. Via the device of wireless access, the quadcopter is connected to AR glasses through Wi-Fi gateway. Epson Moverio BT-200 augmented reality glasses represent the transparent binocular video augmented reality glasses. They are equipped with such sensors as a gyroscope and the accelerometer, have an opportunity to interact through Wi-Fi access point. The AR glasses are equipped with the Android OS, which allowed us to use the LibVLC library and quickly develop an application for transmitting video from the quadcopter and managing its camera. The structural scheme of the stand is provided on Fig. 2.

Fig. 2.
figure 2

Structure of the experimental stand

For collection of the experimental data and entering of additional noises into a network, the additional server with installed NetDisturb software was implemented between augmented reality glasses and the quadcopter. NetDisturb allows to make different noises and distortions in a network between the quadcopter and points, such as time delays, losses, capacity reduction [10]. NetDisturb was installed on a PC with two Gigabit Ethernet network cards. Two Wi-Fi access points were connected to this PC. The first AP was on channel 1 and the second AP was on channel 11, acting as router and repeater, correspondingly. Also software based on the pcap.net library for interception and the analysis of a traffic was developed to estimate influence of quality of service of a network on quality of experience of augmented reality in AR glasses in case of control of the quadcopter. The experiment was carried out in a room large enough to allow the UAV fly freely up to 7 m away.

The software, created for determination of influence of quality of service on quality of experience in systems of augmented reality, intercepts a traffic of data between the quadcopter and AR glasses. On base of received data, exactly time, when the packet was received, and packet size calculation of Hurst exponent is made. Hurst exponent is the measure, used in time series analysis [11,12,13]. Parameter will decrease, if the time delay between couple of identical changes of values of a time series increases. In this case the coefficient reflects a level of self-similarity of a traffic:

  • while 0 < H < 0, 5 – a time series is not self-similar, antipersistent, more probable for changing of the direction of a deviation, high definitions of a deviation follow for low and conversely;

  • while H = 0, 5 – random time series, the following value don’t depend on the previous values;

  • while H > 0, 5 – time series is self-similar, persistent.

  • The self-similar traffic is a traffic which looks equally at any temporal scales. In other words, the probability that the traffic on the following step will deviate average value in the same direction, as on the previous step, is so high as Hurst exponent is closer to one.

For clear determination of Hurst exponent, interception of packages will be carried out during one minute. For 1 min time series we have to calculate dispersion – a deviation of a random value from its average value among.

$$ M\left[ {(R/S)_{t} } \right]\sim cn^{H} ,{\text{when n }} \to \infty \, $$
(1)

where (R/S)t — the normalized assessment of width of a range, c — a constant, n — the size of the researched unit of observations, H — Hurst exponent, i.e. a level of self-similarity of the researched process. For the given finite time series of X = X1, X2, …, Xn we will calculate average value of m:

$$ m = \frac{1}{n} \cdot \sum\limits_{i = 1}^{n} {X_{i} } $$
(2)

Then we will calculate a time series of deviations from average Y:

$$ Y_{t} = X_{t} - m,\,\,{\text{t}} = 1,2 \ldots ,{\text{n}} $$
(3)

Calculate a time series of deviations Z:

$$ Z_{t} = \sum\limits_{i = 1}^{t} {Y_{i} } ,\,\,{\text{t}} = 1,2 \ldots ,{\text{n}} $$
(4)

Calculate a range series R:

$$ {\text{R}}_{\text{t}} = \hbox{max} (Z1,Z2, \ldots ,Z{\text{t}}) - \hbox{min} (Z1,Z2, \ldots ,Z{\text{t}}),{\text{t = 1,2}} \ldots , {\text{n}} $$
(5)

Calculate a standard deviations series S:

$$ \sqrt {\frac{1}{t}\sum\limits_{i = 1}^{t} {(X_{i} - u)^{2} ,\,\,\,\,{\text{t}} = 1,2 \ldots ,{\text{n}}} } $$
(6)

where u — average definition of values of a row from X1 to Xt.

Calculate a normalized range series (R/S):

$$ (R/S)_{t} = {{R_{t} } \mathord{\left/ {\vphantom {{R_{t} } {S_{t} }}} \right. \kern-0pt} {S_{t} }},\,\,\,{\text{t}} = 1,2 \ldots {\text{n}} $$
(7)

To average a row (R/S)t on areas [X1, Xt] in case of different values t on an interval from 1 to n, one usually takes not all n points, but some enough set, which is evenly distributed on all interval from 1 to n.

The received averaged values and corresponding to them block lengths of averaging of t, need to be marked in logarithmic axes. The received points need to be approximated the linear function, constructed by means of the least-squares method. The slope (an inclination tangent of angle to the positive direction of an X-axis) the received line according to a formula (1) will be assessment of Hurst’s coefficient of an initial time series.

In this research Hurst exponent is used for determination of correlation between quality of service and quality of experience in AR systems in case of control of the UAV.

4 Description of Experiment

The examinee of augmented reality glasses, where information from the camera of the quadcopter is displayed, turns the head, thereby directing the camera, and subjectively estimates a response of the quadcopter on the done movements and change of the image in glasses. The experiment u be made several times, each time displays influence of different noises, such as

figure a

a time delay, losses and reduction of throughput. For correct subjective evaluation, the experiment without introduction of noises was taken for a standard, i.e. as close as possible to ideal conditions of data transfer.

5 Analysis of Results of Experiment

After experiment without introduction of noises, Hurst exponent was calculated. It was equal to 0, 61. It means that the traffic was self-similar and these results can be taken as a standard in case of further assessment of quality of experience, i.e. assessment of these results is 5. Further experiments with adding of a time delay were made. Results are provided in Table 1.

Table 1. Experiment with delays

From the table is seen, that in case of time delays less 100 ms, quality of experience of video remains at the satisfactory level, but there are a lot of difficulties in case of control with this delay. Because of delay the managing man doesn’t know when it is necessary to stop movement, i.e. control is made at random, commands are transferred with delay, the response returns with a time delay. In this case Hurst exponent truly defines correlation between quality of service and quality of experience in the AR systems – the greatest value complies to the highest assessment of the testee. Thus, in case of time delays higher than 150 ms it is impossible to exercise control of the UAV by means of the application of augmented reality.

The next experiments were made with adding of random losses in link between the quadcopter and the AR device. Results of this experiment are provided in Table 2.

Table 2. Experiment with losses

Proceeding from the received results, one may say, that losses significantly influence quality of experience. In addition to difficulties with control, there are also noises in recognition of the image on the device of augmented reality. Quality of experience of video promptly falls in case of increase in number of losses, it becomes difficult to sort the image, besides some timepoints “fall out” of video information, that exerts a negative impact on control of the quadcopter.

The last experiment was connected to throughput restriction, resulting in packet losses.

According to Table 3 it is possible to draw a conclusion that throughput restriction critically influences quality of experience of the user in systems of augmented reality. Also in case of capacity reduction, control of the quadcopter significantly becomes more difficult, because the managing man ceases to hold information about any movement of the aircraft. All data are transferred with big time delays, part of them is lost, in case of observation in the device of augmented reality begins the dizziness.

Table 3. Experiment with throughput restriction

6 Conclusion

For control of unmanned aerial vehicles, it is possible to use applications of augmented reality. Because control takes place in real time, it is necessary to impose certain requirements to quality of service of a network for support of high-quality and comfortable control of the UAV. In this paper, influence of quality of service of a network on quality of experience of the user in the device of augmented reality by transmission of a video image from the aircraft camera was considered. By the received results we will draw a conclusion that on a network with the minimal time delays to 100 ms it is possible to control the quadcopter by means of the application of augmented reality without loss of quality of experience and control.

Further, it is possible to continue a research in the area of influence of quality of service of a network on a quadcopter response (the tactile Internet) in case of control by means of the DR applications, and also to consider the AR applications as a method of management of other different robotic systems.