Keywords

1 Introduction

Tech United Eindhoven represents the Eindhoven University of Technology in the RoboCup competitions. The team started participating in the Middle Size League in 2006. In 2011 the service robot AMIGO was added to the team to participate in the RoboCup@Home league. In the Middle Size League competitions, the team has been playing the final for 11 years now, while achieving the first place four times: 2012, 2014, 2016 and 2018. Before RoboCup 2018, the Middle Size League team consists of 4 PhD’s, 1 PDEng, 8 MSc, 4 BSc, 5 former TU/e students, 3 TU/e staff members and one member not related to TU/e.

This paper describes the major scientific improvements of our soccer robots over the past year, and elaborates on some of the main improvements or developments in preparation of the RoboCup 2018 tournament. Additionally, in Sect. 3 some statistics concerning the past tournament will be presented. First in Sect. 2, an introduction on the hardware and software of our fifth generation soccer robot is given. The developments in design and control towards our sixth generation soccer robot, the eight-wheeled robot, are presented in Sect. 4. Improvements to the skills of our robots are presented in Sect. 5. Our progress on including concepts from artificial intelligence into the robot software are presented in Sect. 6. In Sect. 7, we give some insights into one of our main strengths: passing. Section 8 gives concluding remarks and presents our outlook on the coming years.

2 Robot Platform

Our robots have been named TURTLEs (acronym for Tech United RoboCup Team: Limited Edition). Currently, we are using our fifth generation TURTLE while we are developing the sixth generation, which is presented in Sect. 4. In this section we will however treat the fifth generation, which makes up the biggest part of our team. Subsect. 2.1 will treat the hardware of this platform, whereas Subsect. 2.2 will treat the software.

2.1 Hardware

Development of the TURTLEs started in 2005. Through tournaments and numerous demonstrations, these platforms have evolved into the fifth generation TURTLE, a very robust platform. For an outline of our robot design the reader is referred to the schematic representation published in the second section of our team description paper of 2014 [1]. In 2016, a redesign of the upper body of the robot was made to integrate Kinect V2 cameras and create a more robust frame for the omni-vision unit on top of the robot. This prevents the need for recalibration of mirror parameters when the top of the robot is hit by a ball. A detailed list of hardware specifications, along with CAD files of the base, upper-body, ball handling and shooting mechanism, has been published on a ROP wiki.Footnote 1

Fig. 1.
figure 1

(Photo: Bart van Overbeeke)

Fifth generation TURTLE robots, with on the left-handside the goalkeeper robot.

Full size image

2.2 Software

The software controlling the robots is divided into three main processes: Vision, Worldmodel and Motion. These processes communicate with each other through a real-time database (RTDB) designed by the CAMBADA team [2]. The vision process is responsible for environment perception using omni-vision images and provides the location of the ball, obstacles and the robot itself. The worldmodel combines the ball, obstacle and robot position information provided by vision with data acquired from other team members to get a unified representation of the world. The motion process is based on a layered software model. The highest level is strategy. Strategy defines actions which are executed by roles deployed on the TURTLEs. These actions consist of a limited set of basic skills such as shooting and dribbling, which require motion control of relevant actuators, the lowest level of the software. More detailed information on the software can be found in [3] or in the flow charts part of the qualification package.

3 RoboCup 2018 Statistics

Five teams participated in the Middle Size League tournament of RoboCup 2018, two teams from China, one team from Portugal and two teams from The Netherlands. A total of 34 matches have been played, of which Tech United played 14 matches. During those 14 matches, Tech United scored 88 goals, an average of over 6 goals per match. The semi-final resulted in a 10-0 score, the final match resulted in a 1-0 score.

By analysing the actions of the TURTLE’s, we found 240 attempts for a shot on goal, resulting in a success rate of approximately 37%. During the tournament 15 goals were scored by other teams in our goal. Our goalkeeper, even though always being positioned in the goal, drove 2008 m during all the matches, based on odometry data. The field players on average drove 13 km, with TURTLE 2 driving almost 20 km during the tournament. While driving, the TURTLEs managed to localize in almost 90% of the time, where TURTLE 4 managed to localize 98% of the total time. These numbers are lower than previous tournaments (usually 96%), this is due to the increase in field size. These statistics differ per TURTLE, even though the TURTLEs are similar in hardware and software, this can be due to role, calibration accuracy or total playing time.

4 Eight-Wheeled Platform

This section elaborates on the design of the eight-wheeled platform. Subsect. 4.1 will elaborate on some of the design features of the eight-wheeled platform. The challenges faced during the low-level motion control design are presented in Subsect. 4.2.

4.1 Design of the Eight-Wheeled Platform

The current platform is equipped with three omni-directional wheels rigidly connected to the base, achieving holonomicity which makes our platform potentially agile. In this configuration, however, not all the torque delivered by the motors is used in the desired movement. Moreover, high forward acceleration causes the front wheels to slip, removing the ability to apply torque from the motors to the field. These drawbacks form the main motivation for the development of the eight-wheeled platform, also presented in [3].

Fig. 2.
figure 2

The eight-wheeled platform with four suspended wheel combinations which are able to rotate around its center hinge.

Full size image

The challenge in designing a platform with four or more wheels is resolving the over-actuated system. The eight-wheeled platform, presented in Figs. 2(a) and (c), has three degrees-of-freedom and is five times over-actuated. To allow five internal movements, each of the wheel combinations is suspended with the rotation point below the ground and the back wheels are suspended over a hinging axle. In this way, the wheels are always in contact with the ground to transfer the torque from the motors to the ground.

4.2 Low-Level Control of the Eight-Wheeled Platform

The setup of the platform is graphically represented in Fig. 2(b). In this figure it can be seen that this platform consists of four sets each having two hub-drive wheels. Each pair of wheels can rotate around its suspension by actuating the corresponding wheels in opposite direction. As a result, strictly speaking the platform is non-holonomic, but due to the ability of each pair of wheels to rotate, in a relatively short time-intervals compared to the motion of the platform, a kind of semi-holonomicity is achieved.

Fig. 3.
figure 3

Low-level control architecture of the eight-wheeled platform.

Full size image

In order to manipulate the position x, y and orientation \(\phi \) of the center C of the platform, the control strategy of Fig. 3 has been designed. Based on the desired velocity of the platform, \(\dot{q}_r = [\dot{x} ~ \dot{y} ~ \dot{\phi }]^T\), both the reference velocity for each of the eight wheels \(v_{w,r} \in \mathbb {R}^{[8 \times 1]}\) and the desired pivot-angle \(\delta _r \in \mathbb {R}^{[4 \times 1]}\) can be determined in a feedforward fashion using the inverse kinematics of the platform. As three degrees of freedom are controlled using eight actuators, the system is over-actuated. Therefore, an error in the pivot \(\delta \) of each wheelpair leads to undesired internal forces and slip. As a result, if the pivot-error is not within reasonable bounds, the pivot-controller is prioritized over the platform controller, meaning that the wheels are re-oriented before the platform is actuated. In order to correct for this pivot-error, via a feedback controller, a compensation is added to the wheel velocities. The magnitude of this correction term is equal for both wheels in each wheelpair, but they have opposite direction. Finally, by measuring both the wheel velocities \(v_w \in \mathbb {R}^{[8 \times 1]}\) and the pivot angle \(\delta \in \mathbb {R}^{[4 \times 1]}\), the velocity of the platform can be determined using the forward kinematics of the system.

4.3 Results During RoboCup 2018

Unfortunately, we did not manage to employ the eight-wheeled platform during one of the games of RoboCup 2018. The low-level motion control did not meet the performance requirements for being able to play, resulting in a too low maximum velocity of the platform being. Right after the tournament work reconvened and promising results will ensure playing time for the eight-wheeled platform next year.

5 TURTLE Skills

This section focuses on two developments regarding the skills of the TURTLEs. Subsect. 5.1 focuses on improving ball state estimation (position and velocity). The focus of Subsect. 5.2 is on the implementation of the “human-alike dribble”, a dribble where the TURTLEs softly push the ball forward using the ball handling.

5.1 Improved Ball State Estimate

A correct ball position and velocity estimate is crucial for the TURTLEs. The performance of the present method is not satisfactory any longer in all situations. The current estimator buffers detections of the ball and fits this with a state trajectory in a least squares sense. In a highly dynamic environment, such as a MSL soccer field, the filter needs to adapt quickly to changing situations. A standard Extended Kalman filter would respond slow on a maneuvering ball depending on the process and measurement noise covariance matrices. To make sure the Kalman filter is able to adapt fast on a changing ball velocity, an Extended Kalman Filter with Inflatable Noise Variance (EKF with InNoVa) [4] is proposed.

Figure 4 presents a comparison between the response of the EKF and the EKF with InNoVa for a disturbance. One can observe from this comparison that the EKF with InNoVa converges to the actual velocity in x direction faster than the EKF. As similar performance is observed in other test cases, the proposed EKF with InNoVa will replace our present algorithm.

Fig. 4.
figure 4

A comparison between the EKF and EKF with InNoVa for a wallbounce, the ball (\(V_{0} = 1\,\mathrm{m/s}\)) bounces off a wall at \(t=2\,\mathrm{s}\), the wall is not included in the model.

Full size image

5.2 Human-Alike Dribble

Within the Middle Size League, robots have a confined dribble space defined as a 3 m radius around the point where the robot intercepted the ball. Currently, the TURTLEs shoot or pass to let go of the ball. However, significant strategical advantages could be gained by softly pushing the ball forward and regaining it again. Previously, our robots had to shoot or give a pass to let go of the ball, therefore a controlled push was implemented.

Before giving a controlled push, the robot has to be aligned and the ball handling levers need to be in a predefined position. In the \(70\,\mathrm{m/s}\) the wheels have contact with the ball, the wheels ramp up the speed of the ball to about \(0.5\,\mathrm{m/s}\) relative to the robot, to give a controlled push. Slip measurements are performed to determine the maximum acceleration before the ball handling wheels lose grip on the ball. Slip was found not to affect the velocity of the ball below \(1.5\,\mathrm{m/s}\) which is thus large enough. The proposed control strategy consists of the existing feedback controller combined with a feedforward controller. This control strategy has been found to yield sufficient accuracy for executing the human-dribble.

5.3 Results During RoboCup 2018

The human-alike dribble has not been integrated in the software during RoboCup 2018, manpower was distributed to other, higher priority, tasks.

6 Artificial Intelligence

We are exploring the possibilities of Artificial Intelligence for this league in two ways. Subsect. 6.1 will elaborate on using Artificial Intelligence (AI) for detailed analysis of the omni-vision images. Another approach, where AI is used to predict the next action of the opponent is presented in Subsect. 6.2.

6.1 Detailed Opponent Detection

In last years team description paper we reported on a detection method for opponent label detection using neural networks. Due to the new rule allowing robots to wear shirts, the presented approach was no longer practical. Therefore, we adopted the procedure: first we take pictures of every robot with a normal digital camera, the images are then distorted to resemble the images from the omni-vision system. A normal digital camera will be used to speed up the procedure of taking pictures from opponent robots and prevent us from having to use one of the TURTLEs for this time-consuming task. Every image undergoes additional augmentations in the form of rotations, scaling, color variations and distortions. Recognition of robots is now done on three levels. The first level classifies the robot’s team. The second level classifies the robot’s orientation in front, left, right or back. The third level is, then again, the number on the number plate.

In last years team description paper [3] we reported on a method for opponent label detection using neural networks. Due to the new rule allowing robots to wear shirts, the presented approach was no longer practical. Therefore, we adopted the following procedure: four pictures are taken for each robot from front, back, left and right with a standard camera and then distorted to resemble the robot images in the omni-directional camera image. This is done using an affine transform of the input picture of which the corner points are mapped onto a corresponding image of the robot in the omni-directional camera. Using rotations, color - and scale variations, the 16 input images per team for each match are augmented to 32.000 \(28\times 28\) color images. These form the input to a three-layer fully convolutional neural network, which is trained to classify single robots to team membership or orientation. The last convolutional layer is fed into a Global Average Pooling Layer, which is used to generate a Class Activation Map. We achieve a 95% accuracy on single robot team classification, against a validation set consisting of single robot images, lifted from omni-directional camera images, shot during test matches. The orientation classification is less reliable since it is depending on the color and shape of the shirts and the protruding ball handling unit, which is not always clearly visible.

This single robot network is then used on the entire omni-directional image by creating a class-activation map in which the highest activation points of each class indicate the position of a robot in the image. We are still working on solving the problem of recognizing our own robots in both classes as a result of the bottom leds.

The entire process of making pictures, augmenting them and training the neural network is completed in 30 min. Using omni-directional images from a robot directly would be better but involves getting both teams on the field, transferring these images, selecting individual robots from them and then train the network. This is not feasible during an actual competition, hence the described approach, which simplifies getting the data in the time available between matches.

To understand the performance of the recognition, a visualization of the feature kernels and activation layers, additionally allowing the fine tuning of the network hyper-parameters. At the moment of writing the first level (team) has a reliability better than \(95\%\), the second level achieves a performance around \(80\%\). Work on the third level did not yield any valuable results yet. Results of the first level have been included here. An omni-vision image as in Fig. 5(a) is input to the neural network, per team(color) a class activated map (CAM) is compiled which shows where certain features are present in the input image, see Figs. 5(b) and (c).

Fig. 5.
figure 5

Class activated maps for the detection of robot team, results for the cyan team and orange team are presented. (Color figure online)

Full size image

6.2 Opponent Action Prediction

Being able to predict the opponents action grants a strategical lead with respect to the opponent. To train a network capable of this, the world state information of previous tournaments will be used. The world state information is spatially represented as three 8-bit occupancy grid maps of \(28\times 40\) pixels, the information on opponents, peers and the ball is each stored in a different map. Temporal information is included in the value of the pixel, the longer ago an e.g. opponent arrived at a certain pixel, the lower the value. The three 8-bit occupancy grid maps can be represented as a single RGB image, as in Fig. 6, the ball, opponents and peers are represented by green, red and blue, respectively.

Currently, the achievable performance of an convolutional neural network is determined with an indicative experiment, by recognizing the refbox tasks from this occupancy map. The network was able to classify the correct refbox tasks with an accuracy of \(98.5\%\), a promising result. With this promising result, research will proceed to predicting the opponents action. The neural network operated offline and used data collected during the match as input.

Fig. 6.
figure 6

Occupancy grid maps for a specific time instant, from left to right: peer positions, opponent positions and ball position. (Color figure online)

Full size image

6.3 Results During RoboCup 2018

The results presented in this section are proof of concepts, these methods have not been integrated in the software before RoboCup 2018. After the tournament the focus shifted to the actual integration of these algorithms into the current software and into the current processing units.

7 Passing

Dynamic teamplay is one of the strengths of Tech United. When in possession of the ball, there are always two robots without the ball (possible pass receivers) trying to reach a position on the field that is optimal for receiving a pass, continuously trying to avoid opponents blocking any free line between them and the robot with the ball (the pass giver). These passes do not have to be direct, i.e. the ball can be shot into an open area in the field, where the pass receiver will try to dynamically intercept it. This poses additional difficulty for the opponent to block a pass. This section will explain how the robots decide where and who to pass to, and how this pass is handled.

7.1 Where and Who to Pass To?

Every robot has a set of cost functions that are evaluated on a grid of positions on the soccer field. For the possible pass receivers, these cost functions combine penalties for being too close to or behind opponents, penalties for being at a position that has a low scoring chance, penalties for being at a position that has no free line for a next pass, penalties for being at a certain illegal position on the field, and a driving cost. These penalties are weighted for two different optimization problems, one for receiving a pass and shooting at goal, and one for receiving a pass and passing to the next robot. E.g. the latter one has zero cost on the penalties for being at a position that has a low scoring chance. The point in the grid with the lowest combined cost, is the position on the field that has the highest probability for receiving a successful pass with corresponding consecutive action (shoot at goal or pass to the next robot). These positions and the corresponding costs are communicated to the pass giver. The pass giver then compares these costs and decides who to pass to. The pass target is the corresponding position. The number of the robot that will receive the pass, and the pass target are then communicated to the rest of the team. The receiving robot will decide for itself what to do next. This way of deciding who and where to pass to is new since RoboCup 2018. Before, the only reason to pass to a robot was that this pass receiving robot had a better chance at scoring. Using this new method, more passes back occur with the intention of passing to the next robot that has a good scoring position. Therefore, in general, more passes occur, letting the ball do the work and making it harder for the opponent to defend.

7.2 Pass Handler

When the robots have determined where and who to pass to, the so-called “pass handler” handles the execution of the pass. This is a synchronized state machine between the pass giver and the pass receiver, that has been fully redesigned for RoboCup 2018. The new pass handler has as little states as possible and has a cleaner way to synchronize states, leading to less synchronization issues and more successful passes. The pass giver calculates how long the pass receiver takes to drive (\(t_{receiver}\)) to the pass target (\(x_{pass}\)) and adjusts the passing speed (\(v_{pass}\)) accordingly:

$$\begin{aligned} v_{pass} = \dfrac{||x_{pass}-x_{0}||_{2}}{t_{receiver}+t_{margin}} \end{aligned}$$
(1)

Here, \(x_{0}\) is the current position of the pass giver and \(t_{margin}\) is some positive time margin to allow for inaccuracies in positioning of the receiver and the actual achieved shooting velocity of the pass giver. To avoid passing too hard and ensure that the ball does not bounce of the ball handling of the pass receiver, the passing speed is limited.

7.3 Result During RoboCup 2018

The new pass handler and improved decision making has resulted in dynamic teamplay during RoboCup 2018. Compared to previous years, there were more passes and a higher percentage of successful passes. By having multiple possible pass receivers and giving passes with the intention to directly pass to the next robot, the game became more dynamic and harder to defend for the opponent. This ultimately resulted in the World Title.

8 Conclusions

In this paper we have described the major scientific improvements of our soccer robots in preparation of the RoboCup 2018 tournament. Not all of the developments have actively contributed to the result, but the methods developed will be integrated in preparation of future tournaments. The sixth generation TURTLE is a robot with 8-wheels, designed for improved agility on the field. The ball state estimation of the TURTLEs has been improved by means of and EKF with InNoVa, achieving better performance for the different dynamic situations. As our current focus is on integrating neural networks into our software, the researches presented towards the application of AI in our software will soon be put to the test. One of our main strengths is passing, which is based on a synchronized state machine between pass giver and receiver. Altogether we hope our progress contributes to an even higher level of dynamic and scientifically challenging robot soccer. The latter, of course, while maintaining the attractiveness of our competition for a general audience. We are determined to create a new generation of TURTLEs with improved agility and ball handling. Meanwhile, our efforts in implementing a configurable strategy framework and applications of artificial intelligence in software will continue. In this way we hope to go with the top in Middle-size league for some more years and contribute to our goal in 2050!