Full length articleTorch: Strategy evolution in swarm robots using heterogeneous–homogeneous coevolution method☆
Introduction
Recently, the development of robot systems has brought significant changes to human society [1]. It is predicted that a number of robot systems will be applied in the entertainment industry [2], sports [3], and work [4] in the near future. In addition, many characteristics of robot systems inspire studies in the field of information and communication technologies [5], thereby promoting the development of industrial information integration engineering. Considering the development of science and technology, robot systems are expected to accomplish more complex tasks [6]. Whereas robot systems enjoy numerous advantages, a single robot incurs a high failure probability because of its internal complexity; it is difficult to repair after damage, expensive, etc. [7]. Consequently, swarm robotics systems (SRSs) have gradually become the research focus [8], [9]. The SRSs have strong robustness [10] and can emerge intelligent behaviours [11] that a single robot system does not have because of the individuals perform tasks autonomously. Currently, SRSs have been applied to applications, including object transportation [12], region coverage [13], and fire detection [14].
A swarm robot system consists of several individual robots with their strategies. A strategy is the corresponding relationship between the state and action and determines the robot’s ability to complete tasks. According to the different types of member robots, a swarm can be divided into homogeneous and heterogeneous swarms. A homogeneous swarm is defined as a robot swarm with the same hardware structure, control module, and behaviour strategy [15]. This indicates that the individual robots will make consistent decisions and behaviours in the same environment. A heterogeneous swarm is a robot swarm in which each individual has different strategies; therefore the individuals choose different behaviours in the same environment. In this study, we focus on the homogeneous swarm because it comes from the behavioural models of natural systems [16], and it is robust, invulnerable, and scalable compared to the heterogeneous swarm [17], [18]. It can also promote the study on swarm systems as part of the heterogeneous swarm [19].
Considering the development of robot technology and increase in people’s needs, the task scenarios of swarm robots are expected to be diversified and dynamic such as various task scenario maps [20] and dynamic task targets [21]. The robot systems for a single task will be difficult to be widely used because of their limited application scenarios. With the in-depth study, the single robot with the ability of online rule generation is designed in a mobile robot control application [22]. However, most of the current SRSs are designed for specific scenes, whereas making the swarm robots complete tasks accurately after switching scenes is impractical in most cases. Therefore, the SRSs are expected to evolve strategies autonomously to adapt to the various task scenarios. Being a key technology, swarm robots’ strategy evolution ability directly affects the function and universal applicability of an SRS in reality. In swarm robot applications, the problem of swarm autonomous strategy evolution has been widely studied nonetheless, there are still many challenges [23], [24].
- •
In general, the efficiency of strategy evolution in an SRS is low. This is because the evaluation of the behaviour strategy of an SRS is a very time-consuming process, and it usually needs to complete a task on the simulator to obtain the evaluation results of the behaviour strategy. In addition, to achieve a better behaviour strategy, it usually needs thousands of times of behaviour strategy evaluation to obtain a satisfactory result. This leads to repeated task execution and consumes lots of computing resources.
- •
Many strategy evolution methods depend on the control centre and require an overall understanding of the scenarios in the evolution process [25]. However, it is often difficult for an SRS to obtain the global environment’s information in advance in the actual task process. Therefore, the evolved behaviour strategies of swarm robots are difficult to apply to actual task scenarios.
- •
The existing design of swarm behaviour strategy usually adopts the strategy expression method based on the parametric equations. This indicates that the behaviour strategy of the swarm robots is weighted by the pre-set weights. The optimization of a behaviour strategy is only the weight-parameters’ optimization. Owing to the limitation of search space, the strategies based on the parametric equations are often difficult to apply to complex task scenarios.
To address the aforementioned challenges, we propose TORCH, i.e., heterogeneous–homogeneous swarm coevolution method, which aims to coordinate the strategy evolution and distributional characteristic of behaviour strategies. TORCH includes a swarm coevolution mechanism to accelerate the evolution process and a novel strategy expression method (behaviour expression tree) to interpret the strategy. It is worth noting that the proposed method uses only the local information obtained by environment perception and neighbour communication to evolve the strategy, without the need of accessing the global information. To verify the effectiveness and feasibility of our method, the flocking task scene of swarm robots is constructed. The flocking problem is a classic problem in which the swarm robots are expected to complete the position transfer at a fast speed and high density. In this task, each robot in the swarm is required to adopt the same strategy. Based on the TORCH, the swarm can independently evolve a structured strategy to adapt to the task scenarios through the feedback of the environment in executing the tasks.
The main contributions of this study are as follows:
- •
A heterogeneous–homogeneous swarm coevolution mechanism is devised to improve the efficiency of strategy evolution. The proposed method can simultaneously evaluate behaviour strategies ( is the scale of a swarm) in a single task execution. This greatly improves the evolutionary efficiency of swarm behaviour strategies.
- •
Based on the proposed TORCH, the swarm robots can optimize behaviour strategies by interacting with the environments. The strategy evolution of swarm robots using the TORCH method requires only the local information. Thus, the evolved strategies are more applicable to distributed actual task scenarios.
- •
A new flexible behaviour strategy search space for swarm robots is designed, improving the ability of swarm robots to perform complex tasks. Specifically, considering the behaviour strategy expression of the swarm robots, compared to the strategy expression method based on the parametric equations, the behaviour expression tree method has a more flexible behaviour strategy search space and can be applied to complex task scenarios.
The rest of this study is organized as follows: Section 2 reviews the related studies. The mechanism of heterogeneous–homogeneous coevolution is presented in Section 3. Section 4 introduces the structure and evolution method of the behaviour expression tree. Section 5 demonstrates the simulation results and conducts the performance analysis. Finally, the study is concluded with a discussion on future studies in Section 6.
Section snippets
Related work
Strategy evolution is an important research topic in the applications of swarm robots, and several studies have focused on enabling the swarm to evolve new strategies automatically, improving its intelligence. Two fundamental technologies are widely applied to achieve this goal: heuristic algorithm (HA) and reinforcement learning (RL) [26]. Heuristic algorithms for search problems can be considered as classical methods to solve this problem. Yu et al. [27] used a super hyper-heuristic algorithm
Heterogeneous–homogeneous swarm coevolution method
The swarm for strategy evolution is composed of several robots, and each robot chooses a behaviour strategy (that is, the robot’s gene). Because the standard robot swarm is supposed to be homogeneous [18], the goal of the evolution is to find the best strategy for a homogeneous swarm. When all the robots in a swarm execute the strategy, the swarm will obtain the highest evaluation and complete the task as expected. Many studies have focused on strategy evolution while requiring multiple
Robot control structure: behaviour expression tree
In the strategy evolution of swarm robots, each robot corresponds to a gene which is evolved to adapt to the environment. This gene is also the behavioural strategy for the robot which is evolvable. Inspired by the expression tree, we proposed the behaviour expression tree as the control structure to express the strategies of the robots in a swarm in the TORCH. The behaviour expression tree is a kind of hierarchical and structured expression of robot control using a tree. The advantages of the
Experimental results
In this section, four experiments are designed to verify the proposed TORCH. First, to verify that the proposed method is feasible, the TORCH effectively evolves a strategy for the flocking task. Thereafter, we apply the evolved strategy in a changed task scenario to verify the method’s adaptability to different task scenarios. Finally, comparative experiments are designed to verify the evolutionary efficiency improvement of the proposed method and performance enhancement of the evolved
Conclusion and future work
In this study, we propose an effective strategy evolution method, TORCH. The TORCH uses the heterogeneous–homogeneous swarm coevolution mechanism to improve the performance and convergence speed of the strategy evolution method. Furthermore, TORCH is based on a novel strategy expression method known as the behaviour expression tree which is an extension of the conventional expression tree to enhance the performance of the evolved strategy. The proposed TORCH only uses local information in the
CRediT authorship contribution statement
Meng Wu: Writing – original draft, Methodology, Investigation. Xiaomin Zhu: Writing – review & editing, Funding acquisition. Li Ma: Conceptualization, Validation. Ji Wang: Conceptualization, Supervision. Weidong Bao: Supervision, Project administration. Wenji Li: Writing – review & editing. Zhun Fan: Writing – review & editing, Project administration, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (58)
Industrial information integration-a literature review 2006–2015
J. Ind. Inf. Integr.
(2016)Surrogate-assisted evolutionary computation: recent advances and future challenges
Swarm Evol. Comput.
(2011)- et al.
Swarms of unmanned aerial vehicles - a survey
J. Ind. Inf. Integr.
(2019) - et al.
Distributed learning algorithms for swarm robotics
Neurocomputing
(2016) - et al.
A dynamic multiobjective evolutionary algorithm based on a dynamic evolutionary environment model
Swarm Evol. Comput.
(2019) - et al.
A cellular mechanism for multi-robot construction via evolutionary multi-objective optimization of a gene regulatory network
Biosystems
(2009) - et al.
Bio-inspired self-organising multi-robot pattern formation: a review
Robot. Auton. Syst.
(2017) - et al.
Evolutionary computing based mobile robot localization
Eng. Appl. Artif. Intell.
(2006) - et al.
Velocity controllers for a swarm of unmanned aerial vehicles
J. Ind. Inf. Integr.
(2021) - et al.
The grand challenges of science robotics
Science Robotics
(2018)
A study of layered learning strategies applied to individual behaviors in robot soccer
A distributed framework for programming the artificial intelligence of mobile robots in smart manufacturing systems
A survey on industrial information integration 2016–2019
J. Ind. Integr. Manag.
Cooperative robots to observe moving targets: review
IEEE Trans. Cybern.
Robust morphogenesis of robotic swarms [Application notes]
IEEE Comput. Intell. Mag.
Multi-robot learning in a cooperative observation task
Distrib. Auton. Robot. Syst.
Occlusion-based cooperative transport with a swarm of miniature mobile robots
IEEE Trans. Robot.
Robotics. programmable self-assembly in a thousand-robot swarm
Science
Mixed reality and remote sensing application of unmanned aerial vehicle in fire and smoke detection
J. Ind. Inf. Integr.
The swarm-bots project
Swarmanoid: a novel concept for the study of heterogeneous robotic swarms
IEEE Robotics Autom. Mag.
An analysis of emergent taxis in a wireless connected swarm of mobile robots
Swarm Robotics - A Formal Approach
Cooperative stigmergic navigation in a heterogeneous robotic swarm
Grouping-based adaptive spatial formation of swarm robots in a dynamic environment
Int. J. Adv. Robot. Syst.
Evolving hierarchical gene regulatory networks for morphogenetic pattern formation of swarm robots
An interpretable fuzzy system learned through online rule generation and multiobjective aco with a mobile robot control application
IEEE Trans. Cybern.
Analyzing collective behavior in evolutionary swarm robotic systems based on an ethological approach
Cited by (17)
Application of Efficient Mobile Robot Navigation through Machine Learning Technique
2024, AIP Conference ProceedingsInvolvement of Unmanned Aerial Vehicles and Swarm Intelligence in Future Modern Warfare: An Overview
2024, Studies in Computational IntelligenceKinematic Analysis Of Swarm Robots For Solar Panel Installation
2023, AIP Conference ProceedingsSwarm Robotics: A Survey from a Multi-Tasking Perspective
2023, ACM Computing SurveysEvolutionary Design of Cooperative Transport Behavior for a Heterogeneous Robotic Swarm
2023, Journal of Robotics and MechatronicsMRS Drone: A Modular Platform for Real-World Deployment of Aerial Multi-Robot Systems
2023, Journal of Intelligent and Robotic Systems: Theory and Applications
- ☆
This work was supported in part by the National Natural Science Foundation of China under Grants 61872378 by the Science and Technology Planning Project of Guangdong Province of China under grant (180917144960530, 2019A050519008, 2019A050520001), by the Project of Educational Commission of Guangdong Province of China under grant 2017KZDXM032, by the State Key Lab of Digital Manufacturing Equipment & Technology, China under grant DMETKF2019020, and by Hunan Provincial Innovation Foundation For Postgraduate, China under grant CX20200081.