Zoro: A robotic middleware combining high performance and high reliability

Highlights

• We propose a shared memory pool architecture for high performance transportation.

• We propose a socket based control algorithm to improve communication reliability.

• We propose a hierarchical memory protection to improve communication safety.

• We propose a novel high efficiency and high reliability service discovery method.

• We implement Zoro, in the middleware layer of ROS2 to validate its efficiency.

Abstract

With the significant advances of AI technology, robotic systems have achieved remarkable development and profound effects. The robotic middleware plays a critical role in robotic systems to provide message definition, data transmission, and service discovery functions. As the modern robotic systems are usually operated in safety critical environments, such as various autonomous driving scenarios, it requires the design of robotic middleware to combine both high performance and high reliability in order to achieve system reliability and product safety. However, conventional robotic middleware used in the majority of robotic systems is based on an inefficient socket communication mechanism and relies on a hasty service discovery design, which leads to system instability and high resource usage. In this work, we propose a sophisticated robotic middleware, Zoro, to fulfill both high performance and high reliability. For communication, we employ shared memory for performance improvement and propose a socket-based communication control algorithm to improve reliability during data transmission. Also, a hierarchical memory protection mechanism is proposed to address safety problems caused by shared memory. Furthermore, we design a light-weight service discovery, to achieve high performance and a weak centralized mechanism for high reliability. Experiments show the communication latency of Zoro significantly outperforms state-of-the-art robotic middleware such as ROS2 and CyberRT by up to 41%. In terms of service discovery, Zoro reduces CPU usage by up to 44% compared to ROS. Zoro achieves reliability with respect to communication and service discovery.

Introduction

Benefiting from the rising AI technology [20], [26], intelligent autonomous systems and their applications are growing on a rapid pace [33]. In particular, both performance and reliability are critical characteristics in autonomous navigation systems as they generate a large amount of data from their surrounding environment at runtime. In addition, they require underlying communication middleware to transmit and process data in real-time without any system failure data loss. As a core infrastructure, middleware provides message definition, data transmission, and service discovery functions. Message definition provides pre-defined message types valid in the robotic systems, which can be communicated between modules with data transmission function. To allow modules with the same topic to communicate each other, service discovery notifies the modules when the others start. We note that conventional middleware cannot provide both high performance and high reliability in terms of data transmission and service discovery functions.

Modules in robotic system are usually coarse-grained organized by processes. In such a function-independent and resource isolation system design, a process crash in one module generally should not impact other modules or the entire system. Thus, data transmission between different modules usually relies on Inter-Process Communication (IPC) mechanism. Taking a typical autonomous navigation as an example, a planning process receives messages from perception and localization processes, and further sends planning results to other processes such as vehicle control module.

To provide message passing support for autonomous navigation applications, robotic communication middleware [10], such as ROS [31], ROS2 [2], [23] and CyberRT [3], has been widely used due to its high compatibility and extensibility. Most of robotic communication middleware employs socket-based IPC methods [7] for its reliability, since it provides a loosely coupled communication model, where both sender (publisher) and receiver (subscriber) are decoupled and connected with socket communication. However, such a socket communication mechanism is not satisfied in scenarios of large message transmission between multiple receivers (or subscribers), where communication latency increases linearly with the growth of the message size [23], [21].

Therefore, shared memory based IPC methods are proposed to improve the performance. Shared memory allows two or more processes to share a given region of memory, which is the most efficient way of IPC, especially in scenarios of multiple receivers. It is because the data do not need to be copied. Furthermore, memory throughput can be largely reduced due to less memory copy.

Although the shared memory based methods provide high performance and less memory access, it often leads to system safety problems. First, to avoid race condition and control different processes to read and write the same data in shared memory, read-write locks are necessary to be utilized. In particular, if one process crashes while it is holding the read-write lock, the shared memory may be destroyed unexpectedly and the other processes that need to read or write that shared data are blocked due to the failure of lock acquiring. The tightly coupled communication model causes Crash Safety Issue. Second, the shared memory based methods typically adopt a ring buffer to manage data for communication. The sender (publisher) writes the data into the ring buffer in turn and the outdated data are replaced once new data are written in that ring buffer. Therefore, this model cannot guarantee that all the receivers (subscribers) can successfully read the data before they are replaced. In this case, it leads to Data Reliability Issue, especially in processes that require very high reliability in receiving the data in stable frequency. Third, as an efficient IPC method, shared memory allows peer processes to directly access and modify the shared memory buffer, leading to potential invalid memory access and data corruption. This problem is defined as Memory Protection Issue. The above issues hinder to achieve high reliability of robotic systems.

In addition to communication, service discovery is another crucial aspect to achieve high performance and high reliability. Service discovery allows modules updating their states and maintains the data-flow graph between modules. Although service discovery provides awareness between modules with the same topic with notifications, traditional middleware lacks either high performance or high reliability. For example, ROS implements service discovery with a centralized organization. This centralized method simplifies the module notifications and improves resource usage, but leads to the unreliability of the system, as the single crash of the service discovery incurs the crash of the whole systems. In comparison, ROS2 improves system reliability by switching from centralized notification to de-centralized approach. With de-centralized approach, however, the resource usage rises sharply with the increasing number of modules, because every module will broadcast its status to all other modules. This peer-to-peer notification significantly increases communication traffic and limits its applicability for robotic systems (such as autonomous system) consisting a large amount of modules. Thus, for service discovery, both efficiency and reliability should be taken into account.

We summarize the performance and reliability about communication and service discovery for different robotic middleware. As shown in Table 1, the conventional robotic middleware whether lacks performance/reliability for communication or lacks performance/reliability for service discovery. To the best of our knowledge, none of the conventional middleware provides both high performance, namely low communication latency, and high reliability that covers low data loss rate during communication and low interference between processes when some crash.

In this paper, we design a framework leveraging shared memory and reliable notification mechanism to provide high performance and high reliability data communication. And the middleware exploits light weight notification and decoupled design to reduce resource usage. We have four contributions in this work.

• We study performance and system safety problems in conventional robotic middleware, and propose a shared memory pool architecture for high performance transportation.

• We propose a socket based control algorithm and hierarchical memory access protection to improve the reliability and safety for data communication.

• We propose a novel service discovery, which provides both high efficiency and high reliability.

• Based on our proposed techniques, we implement a robotic middleware, Zoro, in the middleware layer of ROS2. The evaluation results in a realistic workflow show that (1) Zoro is able to gain up to 41% performance improvement on average over its counterparts. And the memory throughput can be reduced from 17 GB/s to 5 GB/s in a typical workflow. (2) Zoro achieved crash safety and reduces 5.2% data missing rate compared with CyberRT. (3) Zoro can reduce 44% CPU usage compared with ROS in service discovery.