Applying robotic frameworks in a simulated multi-agent contest

Abstract

In the Multi-Agent Programming Contest 2017 the TUBDAI team of the Technische Universität Berlin is using the complex multi-agent scenario to evaluate the application of two frameworks from the field (multi-)robot systems. In particular the task-level decision-making and planning framework ROS Hybrid Behaviour Planner (RHBP) is used to implement the execution and decision-making for single agents. The RHBP framework builds on top of the framework Robot Operating System (ROS) that is used to implement the communication and scenario specific coordination strategy of the agents. The united team for the official contest is formed by volunteering students from a project course and their supervisors.

Appendices

Appendix A: Team overview: short answers

A.1 Participants and their background

ᅟ:

What was your motivation to participate in the contest?

Members of our group have a long history in participating in this contest. The contest is used as a platform to apply our generic multi-agent frameworks to several multi-agent problems of the contest and to test the robustness and performance of our solutions. This year we wanted to evaluate how well our framework RHBP (ROS Hybrid Behaviour Planner), which allows to implement decision-making and planning in a agent-related fashion for multi-robot systems, performs on a larger scale (many agents, different roles) of the contest scenario.

ᅟ:

What is the history of your group? (course project, thesis, …)

Researchers of the DAI-Labour started to participate in the contest in 2007. Since then they have contributed to every edition of the contest and have won four of them using successive generations of the JIAC multi-agent framework. Our current team originates from the supervisors and two volunteering students from the “Application System Project” summer term 2017 course of the Technische Universität Berlin. The applied framework RHBP is developed in one Ph.D. thesis and several independent Bachelor and Master’s theses.

ᅟ:

What is your field of research? Which work therein is related?

Our field of research is multi-agent systems applied in the robotics domain.

A.2 The cold hard facts

ᅟ:

How much time did you invest in the contest (for programming, organising your group, other)?

Approximately 300 hours have been spent, this includes the time participating students and supervisors have invested during the project course as preparation for the contest.

ᅟ:

How many lines of code did you produce for your final agent team?

The entire contest specific implementation has 5056 lines of Python code. From the full amount 820 lines of code are related to the simulation server protocol implementation, the behaviour implementations and coordination has 4236 lines of code. Contest independent framework code is not counted.

ᅟ:

How many people were involved?

We have been four people in contest team.

• Christopher-Eyk Hrabia (PhD Student at Technische Universität Berlin)

• Patrick Marvin Lehmann (MSc Student at Technische Universität Berlin)

• Nabil Battjbuer (MSc Student at Technische Universität Berlin)

• Axel Hessler (Post-Doc at Technische Universität Berlin)

ᅟ:

When did you start working on your agents?

We started to work on the project, mostly infrastructure code like protocol handling, in the preparation for the related summer term course in the begin of April 2017. The coordination and behaviour implementations started in May 2017.

A.3 Strategies and details

ᅟ:

What is the main strategy of your agent team?

Our solution aims to robustly fulfil as many jobs (mission, priced, auction) as possible. We favour missions and auctions over priced jobs because not fulfilling mission jobs would result in a fine and auction job have the advantage of being exclusive for a team once the auction is won.

Furthermore, our solution tries to adapt and recover from unexpected situations, for instance not anymore available items, failed actions or jobs already fulfilled by another team. Due to the reason that we have started this year’s implementation from scratch, we do not aim to implement posting own jobs or putting special considerations on the opponent observation. Furthermore, we only use a limited set of possible agent actions, which is minimally required to fulfil the jobs. For instance, we do not make use of item transfer between agents, explicit storing or dumping. The only optional action we apply is gathering resources to reduce the costs of acquiring items.

ᅟ:

How does the team work together? (coordination, information sharing, …)

We divided the 28 agents into 4 groups, each group consisting of the same agent type compilation (number of groups was chosen to have at least one agent per role in a group). In each group one lead agent (the one with the smallest id) is responsible for rating and selecting jobs for its group, before it decomposes them into task for further coordination. All tasks of a job are rated by all agents of a group. Each single agent then applies on the task. This allows agents to decentralised determine the task assignment based on the lowest rate for each task. Each group is operating independently, except for the leaders sharing information about taken jobs with each other to avoid parallel operation on the same job. The lead agent of each group is also responsible for the bidding process to get the preferred auction jobs.

ᅟ:

What are critical components of your team?

The most problematic and critical part of our implementation was the assignment of tasks to individual agents. Here, we had to deal with several bugs and concurrency issues that eventually result in unsolvable jobs or deadlocks. Several constraints, originating in varying role capabilities, lead in major problems, too.

ᅟ:

Can your agents change their behaviour during runtime? If so, what triggers the changes?

The agents change their behaviour in respect to the higher-level coordination triggered by the lead agent of a group. All agents use the same behaviours. Specific behaviours are initiated depending on the currently assigned task or event, like detected failures.

ᅟ:

Did you have to make changes to the team during the contest?

We were constantly improving our implementation during the contest. The implementation was never free of bugs and for this reason we kept working and debugging on the code until the last match. Moreover, on the first day of the competition week our auction bidding implementation was not properly tested and functioning as expected and therefore we deactivated it after the first simulation.

ᅟ:

How do you organise your agents? Do you use e.g. hierarchies? Is your organisation implicit or explicit?

We use four groups and one group leader per group, see above for more details.

ᅟ:

Is most of your agents’ behaviour emergent on an individual or team level?

We apply a hybrid planning and decision making framework (RHBP - ROS Hybrid Behaviour Planner) for the execution of the agent behaviours. This results in very adaptive and reactive behaviour for individual agents, based on the current perception. But the deliberative part still tries to optimise/plan for shortest routes and resource management as charging. All in all this might lead to emergent behaviour on individual and team level.

ᅟ:

If your agents perform some planning, how many steps do they plan ahead?

The deliberative part of our frameworks allows the agent to plan its operation (on action-level) until the end of the currently assigned task. An agent task in our implementation is something like deliver, buy, gather, assemble or assist assemble an item. Thus the planning considers what to do and where to do it whilst maintaining and planning all required charging in order to achieve this in the most effective way. Moreover, we applied GraphHopper to directly provide further reasoned information (role specific distances) from the perception as input to the planning. Job decomposition implicitly provides the number of steps to be planned depending in the size and complexity of the job.

ᅟ:

If you have a perceive-think-act cycle, how is it synchronised with the server?

The decision-making and planning cycle, which we consider the “think”, is initially triggered by the “request action” (“perceive”) from the server for each agent. However, in our implementation it is possible that one decision-making and planning cycle does not directly lead to one simulation action (“act”) that can be communicated to the server. For this reason we repeat the decision-making and planning cycle until we get a decision that results in an action or as long as we are in the time limit of a simulation step. In case of failures or timeouts the agents reply with a recharge action.

A.4 Scenario specifics

ᅟ:

How do your agents decide which jobs to complete?

Mission jobs and won auction jobs are favoured and completed until they are finished or expired. The other jobs are selected based on their rating, which e.g. considers the amount of money required for the items and the distance to travel. Here, the agent groups try to remain busy by first trying to win auctions, and, if not, completing the best posted job currently available.

ᅟ:

Do you have different strategies for the different roles?

No.

ᅟ:

Do your agents form ad-hoc teams for each job?

No. We use fixed groups, as explained above. They are formed in the beginning of simulation after the “sim-start” message.

ᅟ:

What do your agents do when they do not pursue any job?

The agents charge their batteries when no job is available or feasible. We do not run additional operations, like resource discovery or item caching.

ᅟ:

How did you go about debugging your system?

In the beginning of the project we only started single agents or modules and directly stepped through the code using PyCharm. Later, the more complex decision-making and planning was debugged by applying the ROS logging capabilities and problem related log messages. Furthermore, we implemented a simple statistics module that allows us to collect measures from the execution of test simulations and matches.

ᅟ:

What were prominent questions you would have asked your system during development? (i.e. “why did you just do X?”)

• Why is agent XY idling (in our case recharging) at location XY and not pursuing its assigned task?

• Why was job XY cancelled?

A.5 And the moral of it is …

ᅟ:

What did you learn from participating in the contest?

Participating in such a contest always takes much more time then expected and intended. Positive is the possibility of testing/evaluating, applying and further developing own tools, concepts and frameworks. Negative is that you have to spend a lot of time in the implementation on very scenario specific problems (e.g. properly splitting the jobs into indivisible tasks), which cannot be reused somewhere else and results more or less in diligence work.

ᅟ:

What are the strong and weak points of your team?

The strongest part of our team is the adaptivity and robustness of the solution, which are very important characteristics in robotics. Our implementation can cope well with unexpected situations and misleading perception. We have proven this in the (unfortunately) not counted first simulation of the match against the team lampe. In this simulation a truck of each team was dead-locked in the middle of the map due to an unexpected server error. Our team was able to deal with the situation of one unusable agent and still made profit in the simulation. In contrast the lampe team suffered a lot from this event and experienced heavy loss.

ᅟ:

How viable were your chosen programming language, methodology, tools, and algorithms?

The unconventional approach of applying a framework and environment meant to be used in the robotics domain in such a contest worked well in general. However, we discovered some bottlenecks in the underlying ROS communication infrastructure, we have not been aware of before, as we never worked with so many robots in parallel. The particular problem is that fully-meshed communication channels, with agents listening and sending information to all other agents, lead to an exponential use of threads, because ROS purely relies on point-to-point communication with distinct sockets for each connection. Apart from this, our approach worked out well and showed that ROS and RHBP can also be successfully applied to problems with a large number of robots working together.

ᅟ:

Did you encounter new problems during the contest?

Preparing the contest contribution helped us to detect and fix several bugs as well as to improve the performance of the applied RHBP framework. During the contest we discovered several implementation bugs in the team coordination logic.

ᅟ:

Did playing against other agent teams bring about new insights on your own agents?

Due to the tight schedule and the limited time and men power, we could not make use of the competitive situation within the contest scenario before the actual contest. In consequence, we were confronted with new effects during the contest, like delivered items of incompletely solved jobs, stay/fill up storages.

ᅟ:

What would you improve if you wanted to participate in the same contest a week from now (or next year)?

The task coordination part of our implementation could be cleaned, improved and made more robust in general. Furthermore, we would run more tests in a competition setup in advance. Longer-term considerations could strive for more optimisation in completing the jobs, like applying smart caching strategies as the winning team. Our actual implementation was so far mainly focused on being able to fulfil the scenario in general, we did not yet explore many options of optimising the job execution and profit making.

ᅟ:

Which aspect of your team cost you the most time?

Most time was spent on debugging and bug fixing concurrency related problems.

ᅟ:

What can be improved regarding the contest/scenario for next year?

Most important is the provision of a complete and bug free documentation. Furthermore, it would be quite helpful to have a basic competitor team available that can be used as a sparring partner in own test matches. This does not have to be provided on code-level, a binary release would already be very beneficial. From the scenario and protocol point of view it would be great to avoid as much as possible the necessity of scenario specific implementations that have to be done, but are not interesting from the scientific point of view and more some kind of diligence work, e.g. the already mentioned job decomposition (figuring out which items, tools have to be used in which order etc.).

Moreover, it would be great to have the entire participation process, covering all tests, qualifications, dates etc., documented as early as possible.

ᅟ:

Why did your team perform as it did? Why did the other teams perform better/worse than you did?

Our entire approach was intended to be a proof-of-concept regarding the applicability of the RHBP framework in such kind of scenario. Of particular interest has been the scalability with many agents/robots and the performance in the limited time per simulation step. Moreover, it was also the first time we tested our RHBP framework with students in a project course, thus we were also required to teach the concepts and utilisation. Before it has only been used in more simplified scenarios.