RT-ROS: A real-time ROS architecture on multi-core processors
Introduction
ROS, an open-source robot operating system, has been being rapidly developed and widely used in the robotics community [1]. Based on the ROS framework, many researchers have developed their software for diverse robots such as Barrett WAM [2] and Raven-II [3]. However, ROS runs on Linux, and cannot provide real-time guarantees. This limits its usage, as real-time tasks are required in many robot applications such as robot motion control. Therefore, it becomes a key issue to make ROS be real-time. On the other hand, multi-core processors offer a promising platform for robot applications. Compared to the traditional robot computing platform with separated host and guest systems, a multi-core processor can provide more powerful computing capacity and less communication overhead. Thus, it is also vitally important to effectively run ROS on multi-core processors by exploring the multi-core architecture for robot applications. This paper focuses on solving the problem of making ROS be real-time with high performance on multi-core processors.
To make ROS be real-time, a common approach is to run real-time tasks on guest embedded systems and run non-real-time tasks on a host system such as in ROS Industrial and ROS Bridge [4]. However, by separating ROS tasks into different computing systems, it not only introduces big communication overhead but also increases the manufacturing cost. In fact, a multi-core processor such as Intel Pentium multi-core processors is powerful enough to run both real-time and non-real-time tasks of ROS. Implementing ROS based on a system with a multi-core processor can help reduce communication overhead by replacing inter-system communication with inter-core communication, decrease the system cost and simplify the system design. However, it is still an open issue for how to make ROS be real-time on multi-core processors.
There are challenges to make ROS be real-time on multi-core processors. First, considering the portability, a real-time OS environment should be provided to support the execution of real-time ROS tasks. It is challenging to run both a real-time OS and a general-purpose OS without interfering each other on multi-core processors. In particular, it is not trivial to provide a mechanism so interrupts, devices and other hardware resources can be separated and isolated to support real-time and non-real-time ROS tasks. Second, we also need to provide effective and efficient communication mechanisms between real-time and non-real-time tasks.
This paper for the first time presents a real-time ROS architecture called RT-ROS on multi-core processors. RT-ROS provides an integrated real-time/non-real-time task execution environment so real-time and non-real-time ROS nodes can be separately run on a real-time OS and Linux, respectively, with different processor cores. In such a way, real-time tasks can be supported by real-time ROS nodes on a real-time OS, while non-real-time ROS nodes on Linux can provide other functions of ROS. In RT-ROS, we develop a hybrid OS platform that can support the execution of real-time and non-real-time OSes. RT-ROS provides a mechanism in its hybrid OS so processor cores and other hardware resources such as interrupts and devices are divided and isolated, by which real-time ROS tasks on a real-time OS and non-real-time ROS tasks on a general-purpose OS can be run separately without interfering each other. Furthermore, in RT-ROS, we develop an efficient communication mechanism between real-time and non-real-time OSes.
We have implemented RT-ROS on an Intel dual-core processor. Nuttx [5], an open source real-time OS, is chosen as the real-time platform, and Linux is used as the general-purpose OS. We further implement the RT-ROS system in the industrial controller of a 6-DOF modular manipulator. The experimental results show that RT-ROS can effectively provide real-time support for the ROS platform with high performance by exploring the multi-core architecture.
The remainder of this paper is organized as follows. Section 2 introduces the background. Sections 3 RT-ROS, 4 System implementation present the design and implementation of RT-ROS, respectively. Evaluation is presented is Section 5. Section 6 introduces the related work. Finally, the conclusions are drawn in Section 7.
Section snippets
Background
In this section, we provide the background. We first introduce ROS and then Nuttx that is used as the real-time OS in the implementation of RT-ROS.
RT-ROS
The system architecture of RT-ROS is shown in Fig. 4. At the application layer, ROS nodes are divided into real-time and non-real-time nodes. ROS nodes can communicate with each other via either inner-core or inter-core communications. These cores do communications via shared memory. At the OS level, GPOS (General-Purpose Operating System) and RTOS reside on isolated hardware resources such as processor cores, interrupts, and devices. At the hardware level, hardware resources are divided so
System implementation
We have implemented RT-ROS on an Intel dual-core processor with the IA32 SMP architectures [9] as shown in Fig. 5. Intel dual-core processors are selected because they are with powerful computing capacity and complete software support. Note that RT-ROS is a general architecture and other multi-core processors can be adopted as well.
In the implementation, Linux is used as the GPOS and Nuttx as the RTOS. RT-ROS consists of two parts. One part is the non-real-time ROS system on Linux, and the
Evaluation
An X86-based industrial robot controller is employed to test the performance of RT-ROS, including the isolation between GPOS and RTOS, and the execution efficiency of the whole system. The testing platform consists of the following hardware: (1) Mainboard: AIMB-780; (2) CPU: Intel(R) Pentium (R) Dual CPU E2200 of 2.2 GHz; (3) Memory: DDR2 of 2 GB; (4) CAN device: Advantech PCI-1680U CAN; (5) VGA card: HD6670 DDR5 of 1 GB. The software of the testing platform includes: (1) Ubuntu 11.10 with
Related work
There are two major issues for the development of robotics. The first is about the enhancement of the development efficiency of the controlling software. Most existing industrial robots are designed to perform special tasks in specific environments [12], [13]. Because different robots generally have different controlling software, developers always have to spend a lot of time writing diverse programs with similar functions to meet new operational requirements. Thus, the complexity of
Conclusions and future work
In this paper, we proposed a real-time ROS architecture called RT-RTOS on multi-core processors. RT-ROS provides an integrated real-time/non-real-time task execution environment so real-time and non-real-time ROS nodes can be separately run on a real-time OS and Linux, respectively, with different processor cores. We have implemented RT-ROS on a dual-core processor and conducted various experiments with real robot applications. The experimental results show that RT-ROS can effectively provide
Acknowledgments
This work was supported by the National High Technology Research and Development Program of China (“863” Program) (2012AA041402 and 2012AA041405), National Natural Science Foundation of China (Grant No. 61175079 and No. 51105012), and National Key Technology Support Program (Grant No. 2013BAH45F01). A preliminary version of this work appears in the Proceedings of the 2011 International Conference on Robotics and Biomimetics [48].
Hongxing Wei was born in the Inner Mongolia Autonomous Region, China, in 1974. He received a Ph.D. from the College of Automation, Harbin Engineering University, Harbin, China, in 2001. Since 2004, he has been an Associate Professor in the School of Mechanical Engineering and Automation, Beihang University (formerly Beijing University of Aeronautics and Astronautics), Beijing, China. His current research interests include self-assembly swarm robots, modular robotics architecture, mobile sensor
References (48)
Architecture-based design and optimization of genetic algorithms on multi-and many-core systems
Future Gener. Comput. Syst.
(2014)- et al.
Recent progress on programming methods for industrial robots
Robot. Comput.-Integr. Manuf.
(2012) Minimizing accumulative memory load cost on multi-core DSPs with multi-level memory
J. Syst. Archit.
(2013)Exploiting process variation for write performance improvement on NAND flash memory storage systems
IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
(2015)- ROS, 2015....
- et al.
ROS open-source audio recognizer: ROAR environmental sound detection tools for robot programming
Auton. Robots
(2013) - et al.
Raven-II: an open platform for surgical robotics research
IEEE Trans. Biomed. Eng.
(2013) Embedded ROS
IEEE Robot. Autom. Mag.
(2013)- NuttX, 2015...
Exponential growth of ROS
IEEE Robot. Autom. Mag.
(2011)
Sharing software with ROS
IEEE Robot. Autom. Mag.
ROS on the PR2
IEEE Robot. Autom. Mag.
Cloud E-learning for Mechatronics: CLEM
Future Gener. Comput. Syst.
YARP: Yet another robot platform
Int. J. Adv. Robot. Syst.
Robotics middleware: A comprehensive literature survey and attribute-based bibliography
J. Robot.
Development environments for autonomous mobile robots: A survey
Auton. Robots
Message-based evaluation in scheme for high-level robot control
J. Intell. Robot. Syst.
Hybrid event based control architecture for tele-robotic systems controlled through Internet
J. Zhejiang Univ. Sci.
Cited by (74)
The XBot2 real-time middleware for robotics
2023, Robotics and Autonomous SystemsZoro: A robotic middleware combining high performance and high reliability
2022, Journal of Parallel and Distributed ComputingCitation Excerpt :For example, if one process publishes the same message to the other three processes, the entire communication path is conducted three times. To improve collective communication efficiency, several methods have been proposed, such as multi-path protocols [32,13,37] and DDS protocol [1,28,15,9]. The advantage for socket-based IPC methods is high reliability.
A review on communication protocols for autonomous unmanned aerial vehicles for inspection application
2021, Microprocessors and MicrosystemsCitation Excerpt :Sticha in [81] measured the real-time capabilities of the ROS communication middleware. To provide reliable and short latency, a real-time ROS architecture, RT-ROS [82], is proposed with a real-time node and a non-real-time node that can be respectively run on a real-time OS as well as Linux. Measurements of the latency and the diversity of the latency using ROS communication middleware are presented by [83].
ReTiF: A declarative real-time scheduling framework for POSIX systems
2021, Journal of Systems ArchitectureCitation Excerpt :Another example is HSF-FreeRTOS, proposed by Inam et al. [34], which implements a hierarchical scheduling framework for FreeRTOS, providing support to temporal isolation among applications running on a single processor. In 2016, Wei et al. [35] proposed RT-ROS, a real-time ROS architecture that provides an integrated task execution environment that is able to run real-time and non-real-time tasks in the same system. The real-time tasks run on top of a real-time OS while non-real-time ones run on Linux.
ROS2 Real-time Performance Optimization and Evaluation
2023, Chinese Journal of Mechanical Engineering (English Edition)
Hongxing Wei was born in the Inner Mongolia Autonomous Region, China, in 1974. He received a Ph.D. from the College of Automation, Harbin Engineering University, Harbin, China, in 2001. Since 2004, he has been an Associate Professor in the School of Mechanical Engineering and Automation, Beihang University (formerly Beijing University of Aeronautics and Astronautics), Beijing, China. His current research interests include self-assembly swarm robots, modular robotics architecture, mobile sensor networks, and embedded systems.
Zhenzhou Shao received the B.E. degree and M.E. degree in the Department of Information Engineering at Northeastern University, China, in 2007 and 2009, respectively, and the Ph.D. degree in The Department of Mechanical, Aerospace, and Biomedical Engineering at University of Tennessee, US, in 2013. He is currently working in the College of Information Engineering at Capital Normal University, China. His research interests include computer vision, machine learning and human-robot interaction.
Zhen Huang graduated from Beijing University of Aeronautics and Astronautics in 2010. He is a software engineer at Smokie Robotics, Inc., China.
Renhai Chen received the B.E. degree and M.E. degree in the Department of Computer Science and Technology, Shandong University, China, in 2009 and 2012, respectively. He is currently working toward the Ph.D. degree in the Department of Computing at the Hong Kong Polytechnic University. His research interests include embedded systems, mobile virtualization and hardware/software codesign, especially for secondary storage.
Yong Guan received the Ph.D. degree in College of Mechanical Electronic and Information Engineering from China University of Mining and Technology, China, in 2004. Currently, he is a Professor of Capital Normal University. His research interests include formal verification, PHM for power and embedded system design. Dr. Guan is a member of Chinese Institute of Electronics Embedded Expert Committee. He is also a member of Beijing Institute of Electronics Professional Education Committee and Standing Council Member of Beijing Society for Information Technology in Agriculture.
Jindong Tan received the PhD degree from Michigan State University, East Lansing, MI, in 2002, in Electrical and Computer Engineering. He is currently an associate professor in the Department of Electrical and Computer Engineering, Michigan Technological University. His research interests include distributed robotics, wireless sensor networks, human robot interaction, biosensing and signal processing, and surgical robots and navigation. Dr. Tan is a member of the IEEE, ACM and Sigma Xi.
Zili Shao received the B.E. degree in electronic mechanics from the University of Electronic Science and Technology of China, Sichuan, China, in 1995, and the M.S. and Ph.D. degrees from the Department of Computer Science, University of Texas at Dallas, in 2003 and 2005, respectively. He has been an Associate Professor with the Department of Computing, the Hong Kong Polytechnic University, since 2010. His research interests include embedded systems, real-time systems, compiler optimization and hardware/software co-design.
- 1
Member, IEEE.