RT-ROS: A real-time ROS architecture on multi-core processors

https://doi.org/10.1016/j.future.2015.05.008Get rights and content

Abstract

ROS, an open-source robot operating system, is widely used and rapidly developed in the robotics community. However, running on Linux, ROS does not provide real-time guarantees, while real-time tasks are required in many robot applications such as robot motion control. This paper for the first time presents a real-time ROS architecture called RT-RTOS on multi-core processors. RT-ROS provides an integrated real-time/non-real-time task execution environment so real-time and non-real-time ROS nodes can be separately run on a real-time OS and Linux, respectively, with different processor cores. In such a way, real-time tasks can be supported by real-time ROS nodes on a real-time OS, while non-real-time ROS nodes on Linux can provide other functions of ROS. Furthermore, high performance is achieved by executing real-time ROS nodes and non-real-time ROS nodes on different processor cores. We have implemented RT-ROS on a dual-core processor and conducted various experiments with real robot applications. The experimental results show that RT-ROS can effectively provide real-time support for the ROS platform with high performance by exploring the multi-core architecture.

Introduction

ROS, an open-source robot operating system, has been being rapidly developed and widely used in the robotics community  [1]. Based on the ROS framework, many researchers have developed their software for diverse robots such as Barrett WAM  [2] and Raven-II  [3]. However, ROS runs on Linux, and cannot provide real-time guarantees. This limits its usage, as real-time tasks are required in many robot applications such as robot motion control. Therefore, it becomes a key issue to make ROS be real-time. On the other hand, multi-core processors offer a promising platform for robot applications. Compared to the traditional robot computing platform with separated host and guest systems, a multi-core processor can provide more powerful computing capacity and less communication overhead. Thus, it is also vitally important to effectively run ROS on multi-core processors by exploring the multi-core architecture for robot applications. This paper focuses on solving the problem of making ROS be real-time with high performance on multi-core processors.

To make ROS be real-time, a common approach is to run real-time tasks on guest embedded systems and run non-real-time tasks on a host system such as in ROS Industrial and ROS Bridge   [4]. However, by separating ROS tasks into different computing systems, it not only introduces big communication overhead but also increases the manufacturing cost. In fact, a multi-core processor such as Intel Pentium multi-core processors is powerful enough to run both real-time and non-real-time tasks of ROS. Implementing ROS based on a system with a multi-core processor can help reduce communication overhead by replacing inter-system communication with inter-core communication, decrease the system cost and simplify the system design. However, it is still an open issue for how to make ROS be real-time on multi-core processors.

There are challenges to make ROS be real-time on multi-core processors. First, considering the portability, a real-time OS environment should be provided to support the execution of real-time ROS tasks. It is challenging to run both a real-time OS and a general-purpose OS without interfering each other on multi-core processors. In particular, it is not trivial to provide a mechanism so interrupts, devices and other hardware resources can be separated and isolated to support real-time and non-real-time ROS tasks. Second, we also need to provide effective and efficient communication mechanisms between real-time and non-real-time tasks.

This paper for the first time presents a real-time ROS architecture called RT-ROS on multi-core processors. RT-ROS provides an integrated real-time/non-real-time task execution environment so real-time and non-real-time ROS nodes can be separately run on a real-time OS and Linux, respectively, with different processor cores. In such a way, real-time tasks can be supported by real-time ROS nodes on a real-time OS, while non-real-time ROS nodes on Linux can provide other functions of ROS. In RT-ROS, we develop a hybrid OS platform that can support the execution of real-time and non-real-time OSes. RT-ROS provides a mechanism in its hybrid OS so processor cores and other hardware resources such as interrupts and devices are divided and isolated, by which real-time ROS tasks on a real-time OS and non-real-time ROS tasks on a general-purpose OS can be run separately without interfering each other. Furthermore, in RT-ROS, we develop an efficient communication mechanism between real-time and non-real-time OSes.

We have implemented RT-ROS on an Intel dual-core processor. Nuttx  [5], an open source real-time OS, is chosen as the real-time platform, and Linux is used as the general-purpose OS. We further implement the RT-ROS system in the industrial controller of a 6-DOF modular manipulator. The experimental results show that RT-ROS can effectively provide real-time support for the ROS platform with high performance by exploring the multi-core architecture.

The remainder of this paper is organized as follows. Section  2 introduces the background. Sections  3 RT-ROS, 4 System implementation present the design and implementation of RT-ROS, respectively. Evaluation is presented is Section  5. Section  6 introduces the related work. Finally, the conclusions are drawn in Section  7.

Section snippets

Background

In this section, we provide the background. We first introduce ROS and then Nuttx that is used as the real-time OS in the implementation of RT-ROS.

RT-ROS

The system architecture of RT-ROS is shown in Fig. 4. At the application layer, ROS nodes are divided into real-time and non-real-time nodes. ROS nodes can communicate with each other via either inner-core or inter-core communications. These cores do communications via shared memory. At the OS level, GPOS (General-Purpose Operating System) and RTOS reside on isolated hardware resources such as processor cores, interrupts, and devices. At the hardware level, hardware resources are divided so

System implementation

We have implemented RT-ROS on an Intel dual-core processor with the IA32 SMP architectures  [9] as shown in Fig. 5. Intel dual-core processors are selected because they are with powerful computing capacity and complete software support. Note that RT-ROS is a general architecture and other multi-core processors can be adopted as well.

In the implementation, Linux is used as the GPOS and Nuttx as the RTOS. RT-ROS consists of two parts. One part is the non-real-time ROS system on Linux, and the

Evaluation

An X86-based industrial robot controller is employed to test the performance of RT-ROS, including the isolation between GPOS and RTOS, and the execution efficiency of the whole system. The testing platform consists of the following hardware: (1) Mainboard: AIMB-780; (2) CPU: Intel(R) Pentium (R) Dual CPU E2200 of 2.2 GHz; (3) Memory: DDR2 of 2 GB; (4) CAN device: Advantech PCI-1680U CAN; (5) VGA card: HD6670 DDR5 of 1 GB. The software of the testing platform includes: (1) Ubuntu 11.10 with

Related work

There are two major issues for the development of robotics. The first is about the enhancement of the development efficiency of the controlling software. Most existing industrial robots are designed to perform special tasks in specific environments  [12], [13]. Because different robots generally have different controlling software, developers always have to spend a lot of time writing diverse programs with similar functions to meet new operational requirements. Thus, the complexity of

Conclusions and future work

In this paper, we proposed a real-time ROS architecture called RT-RTOS on multi-core processors. RT-ROS provides an integrated real-time/non-real-time task execution environment so real-time and non-real-time ROS nodes can be separately run on a real-time OS and Linux, respectively, with different processor cores. We have implemented RT-ROS on a dual-core processor and conducted various experiments with real robot applications. The experimental results show that RT-ROS can effectively provide

Acknowledgments

This work was supported by the National High Technology Research and Development Program of China (“863” Program) (2012AA041402 and 2012AA041405), National Natural Science Foundation of China (Grant No. 61175079 and No. 51105012), and National Key Technology Support Program (Grant No. 2013BAH45F01). A preliminary version of this work appears in the Proceedings of the 2011 International Conference on Robotics and Biomimetics  [48].

Hongxing Wei was born in the Inner Mongolia Autonomous Region, China, in 1974. He received a Ph.D. from the College of Automation, Harbin Engineering University, Harbin, China, in 2001. Since 2004, he has been an Associate Professor in the School of Mechanical Engineering and Automation, Beihang University (formerly Beijing University of Aeronautics and Astronautics), Beijing, China. His current research interests include self-assembly swarm robots, modular robotics architecture, mobile sensor

References (48)

  • S. Cousins et al.

    Sharing software with ROS

    IEEE Robot. Autom. Mag.

    (2010)
  • S. Cousins

    ROS on the PR2

    IEEE Robot. Autom. Mag.

    (2010)
  • Intel Corporation. Intel 64 and IA-32 architectures software developers manual,...
  • Z. Mwaikambo, A. Raj, Linux kernel hotplug CPU support, in: Proceedings of the Linux Symposium,...
  • PCI-SIG, PCI local bus specification revision 3.0,...
  • K.M. Chao

    Cloud E-learning for Mechatronics: CLEM

    Future Gener. Comput. Syst.

    (2014)
  • M. Montemerlo, N. Roy, S. Thrun, Perspectives on standardization in mobile robot programming: The carnegie mellon...
  • B.P. Gerkey, R.T. Vaughan, A. Howard, The player/stage project: tools for multi-robot and distributed sensor systems,...
  • A. Huang, E. Olson, D. Moore, LCM: Lightweight communications and marshalling, in: IEEE/RSJ International Conference on...
  • G. Metta et al.

    YARP: Yet another robot platform

    Int. J. Adv. Robot. Syst.

    (2006)
  • A. Elkady et al.

    Robotics middleware: A comprehensive literature survey and attribute-based bibliography

    J. Robot.

    (2012)
  • J. Kramer et al.

    Development environments for autonomous mobile robots: A survey

    Auton. Robots

    (2007)
  • C. Lee et al.

    Message-based evaluation in scheme for high-level robot control

    J. Intell. Robot. Syst.

    (1999)
  • X.M. Li et al.

    Hybrid event based control architecture for tele-robotic systems controlled through Internet

    J. Zhejiang Univ. Sci.

    (2004)
  • Cited by (74)

    • The XBot2 real-time middleware for robotics

      2023, Robotics and Autonomous Systems
    • Zoro: A robotic middleware combining high performance and high reliability

      2022, Journal of Parallel and Distributed Computing
      Citation Excerpt :

      For example, if one process publishes the same message to the other three processes, the entire communication path is conducted three times. To improve collective communication efficiency, several methods have been proposed, such as multi-path protocols [32,13,37] and DDS protocol [1,28,15,9]. The advantage for socket-based IPC methods is high reliability.

    • A review on communication protocols for autonomous unmanned aerial vehicles for inspection application

      2021, Microprocessors and Microsystems
      Citation Excerpt :

      Sticha in [81] measured the real-time capabilities of the ROS communication middleware. To provide reliable and short latency, a real-time ROS architecture, RT-ROS [82], is proposed with a real-time node and a non-real-time node that can be respectively run on a real-time OS as well as Linux. Measurements of the latency and the diversity of the latency using ROS communication middleware are presented by [83].

    • ReTiF: A declarative real-time scheduling framework for POSIX systems

      2021, Journal of Systems Architecture
      Citation Excerpt :

      Another example is HSF-FreeRTOS, proposed by Inam et al. [34], which implements a hierarchical scheduling framework for FreeRTOS, providing support to temporal isolation among applications running on a single processor. In 2016, Wei et al. [35] proposed RT-ROS, a real-time ROS architecture that provides an integrated task execution environment that is able to run real-time and non-real-time tasks in the same system. The real-time tasks run on top of a real-time OS while non-real-time ones run on Linux.

    • ROS2 Real-time Performance Optimization and Evaluation

      2023, Chinese Journal of Mechanical Engineering (English Edition)
    View all citing articles on Scopus

    Hongxing Wei was born in the Inner Mongolia Autonomous Region, China, in 1974. He received a Ph.D. from the College of Automation, Harbin Engineering University, Harbin, China, in 2001. Since 2004, he has been an Associate Professor in the School of Mechanical Engineering and Automation, Beihang University (formerly Beijing University of Aeronautics and Astronautics), Beijing, China. His current research interests include self-assembly swarm robots, modular robotics architecture, mobile sensor networks, and embedded systems.

    Zhenzhou Shao received the B.E. degree and M.E. degree in the Department of Information Engineering at Northeastern University, China, in 2007 and 2009, respectively, and the Ph.D. degree in The Department of Mechanical, Aerospace, and Biomedical Engineering at University of Tennessee, US, in 2013. He is currently working in the College of Information Engineering at Capital Normal University, China. His research interests include computer vision, machine learning and human-robot interaction.

    Zhen Huang graduated from Beijing University of Aeronautics and Astronautics in 2010. He is a software engineer at Smokie Robotics, Inc., China.

    Renhai Chen received the B.E. degree and M.E. degree in the Department of Computer Science and Technology, Shandong University, China, in 2009 and 2012, respectively. He is currently working toward the Ph.D. degree in the Department of Computing at the Hong Kong Polytechnic University. His research interests include embedded systems, mobile virtualization and hardware/software codesign, especially for secondary storage.

    Yong Guan received the Ph.D. degree in College of Mechanical Electronic and Information Engineering from China University of Mining and Technology, China, in 2004. Currently, he is a Professor of Capital Normal University. His research interests include formal verification, PHM for power and embedded system design. Dr. Guan is a member of Chinese Institute of Electronics Embedded Expert Committee. He is also a member of Beijing Institute of Electronics Professional Education Committee and Standing Council Member of Beijing Society for Information Technology in Agriculture.

    Jindong Tan received the PhD degree from Michigan State University, East Lansing, MI, in 2002, in Electrical and Computer Engineering. He is currently an associate professor in the Department of Electrical and Computer Engineering, Michigan Technological University. His research interests include distributed robotics, wireless sensor networks, human robot interaction, biosensing and signal processing, and surgical robots and navigation. Dr. Tan is a member of the IEEE, ACM and Sigma Xi.

    Zili Shao received the B.E. degree in electronic mechanics from the University of Electronic Science and Technology of China, Sichuan, China, in 1995, and the M.S. and Ph.D. degrees from the Department of Computer Science, University of Texas at Dallas, in 2003 and 2005, respectively. He has been an Associate Professor with the Department of Computing, the Hong Kong Polytechnic University, since 2010. His research interests include embedded systems, real-time systems, compiler optimization and hardware/software co-design.

    1

    Member, IEEE.

    View full text