A Machine-to-Machine protocol benchmark for eHealth applications – Use case: Respiratory rehabilitation

https://doi.org/10.1016/j.cmpb.2016.03.004Get rights and content

Highlights

  • The software tool presented allows the deployment of different benchmarking tests for M2M protocols.

  • The most relevant M2M protocols were evaluated considering different specific performance metrics.

  • Benchmark results allowed to select the most suitable M2M protocol a clinical case of use: respiratory rehabilitation.

Abstract

Background

M2M (Machine-to-Machine) communications represent one of the main pillars of the new paradigm of the Internet of Things (IoT), and is making possible new opportunities for the eHealth business. Nevertheless, the large number of M2M protocols currently available hinders the election of a suitable solution that satisfies the requirements that can demand eHealth applications.

Objectives

In the first place, to develop a tool that provides a benchmarking analysis in order to objectively select among the most relevant M2M protocols for eHealth solutions. In the second place, to validate the tool with a particular use case: the respiratory rehabilitation.

Methods

A software tool, called Distributed Computing Framework (DFC), has been designed and developed to execute the benchmarking tests and facilitate the deployment in environments with a large number of machines, with independence of the protocol and performance metrics selected.

Results

DDS, MQTT, CoAP, JMS, AMQP and XMPP protocols were evaluated considering different specific performance metrics, including CPU usage, memory usage, bandwidth consumption, latency and jitter. The results obtained allowed to validate a case of use: respiratory rehabilitation of chronic obstructive pulmonary disease (COPD) patients in two scenarios with different types of requirement: Home-Based and Ambulatory.

Conclusions

The results of the benchmark comparison can guide eHealth developers in the choice of M2M technologies. In this regard, the framework presented is a simple and powerful tool for the deployment of benchmark tests under specific environments and conditions.

Introduction

Advances in Information and Communication Technologies (ICT) have become an indispensable tool in most sectors of modern society, including healthcare [1]. These technological progresses are transforming the traditional healthcare landscape through a better optimization of the available resources and a more efficient administrative and clinical management. In response to these new opportunities the concept of eHealth emerges [2], which is presented as an alternative to traditional telemedicine systems where the patient was considered as a passive element in the healthcare practice [3]. In this sense, the eHealth concept comprises not only the care of the patient, but also includes the citizen her/himself, who is more responsible for and involved in her/his treatment and healthcare [4]. Furthermore, the concept of citizen empowerment is part of this new health ecosystem, extending the care process, which has traditionally been located exclusively in hospital facilities [5], toward a scenario of ubiquitous computing [6].

In this context, the paradigm of IoT (Internet of Things) [7], [8], “promises to create a world where all the objects around us are connected to the Internet and communicate with each other with minimum human intervention” (sic). IoT is also defined as the interconnection of communications devices using a variety of devices and communication Machine to Machine (M2M) protocols [9]. However, despite some remarkable interoperability efforts [10], there is currently a lack of standardization [11], although some recent initiatives have already headed in that direction [12]. In the meantime, given the extensive fragmentation of M2M technologies for IoT purposes [13], different papers have addressed comparative analysis from several perspectives [14] – architectures, standards, media access interfaces –, whilst others are focused on light communication protocols for portable and embedded devices [15], [16], [17]. In [18], a comparison between the generic architectural models of request/reply and publish/subscribe is presented, and it is shown the optimal performance of the latter. Besides, under the premise of providing the best quality of service (QoS), the M2M publisher/subscriber model stands out showing significant advantages in terms of latency [19] and energy [20]. This way, different communication protocols have emerged in the last years that implement the publish/subscribe pattern and that are specifically designed for IoT [21]. However, most of these comparative studies are preliminary [14], focusing in most cases in only two specific protocols that are tested with few hosts, which is far away from a real massive device-connected scenario like the IoT [22]. To this end, recent studies [5], [14], [23] highlight the current need of more extensive benchmarking and performance analysis from realistic working conditions. In particular, and to the best of the authors’ knowledge, there are no studies that address the specific requirements demanded by eHealth environments.

In order to address the above needs, the present paper describes a benchmarking of a variety of M2M protocols oriented to IoT communication based on the publish/subscribe pattern. Taking into account the level of thoroughness that implies comparing different technologies not interoperable with each other in the same conditions, as well as the existing variety of features and configurations, it was necessary to implement in the first place a software tool, referred to as Distributed Computing Framework (DCF), in order to implement the different benchmarking tests. The flexibility, modular design and ease of use of DCF allow to obtain in real-time the most relevant performance metrics whilst hiding to the users the complexity of the different communication technologies and topologies evaluated. Thus, the results obtained can serve as a general guide to help developers of eHealth solutions for the selection of M2M protocols based on the publish/subscribe model, according to their particular needs. For this purpose, it is presented as case of use an eHealth platform aimed at the supervised monitoring of a training program for respiratory rehabilitation (RR) of COPD patients (from now on called e-RR platform), which has been developed in the context of several a R&D project being carried out by the authors [24]. The platform encompasses many of the M2M challenges detailed as it requires the real-time communication of the data gathered by sensing devices attached to patients when performing their training routines indoor and in mobility. In both training cases, ambulatory and home-based, the supervised RR program is similar, and mainly consists in the development of moderate-intensity aerobic physical activity (PA) (SPWT, 6MWT test [25], [26]) that can be assessed through the monitoring of heart rate and blood oxygen saturation [27], as well as caloric consumption [28]. Finally, it is also relevant to add contextual information for home-based training: for instance, patient's location is vital regarding the occurrence of episodes of panic attacks and dyspnea, which are common in COPD [29].

The paper is divided into the following sections: Section 2.1 describes the framework developed for the implementation of the benchmarking, Section 2.2 presents an analysis of the technologies chosen for the development of the performance testing and a methodological description of the benchmarking itself is presented in Section 2.3. Then, in Section 3.1 the use case of the e-RR platform is presented for the two scenarios considered, which demand different technological needs. In Section 3.2, the results of the benchmarking are presented for the most important performance metrics. Considering the results obtained and the requirements for scenarios of the case of use presented, Section 3.3 chooses the M2M protocols more suitable for each situation and the data model is presented hierarchically following good practices in the use of the communication technologies used. Finally, the discussion and conclusions analyze the results obtained by breaking down each protocol in terms of its strengths and weaknesses.

Section snippets

Distributed Computing Framework description

The rapid emergence of new technologies and the need of performance evaluation require general purpose benchmarking tools characterized by a modular, scalable and portable architecture. Thus, the present landscape motivates the development of the Distributed Computing Framework (DFC), a tool which allows evaluating different communication technologies in terms of performance, considering specific environments and conditions. The DCF has been designed in order to ease the measurement of

The RR eHealth platform (e-RR)

Despite the strong evidence of the clinical benefits of ambulatory RR [27], it is a clinical practice with relatively low adherence [44]. This drawback has promoted the research and adoption of the domiciliary RR as a supplementary alternative in order to minimize the drop rate [45]. The development of exercises in the domiciliary RR requires the remote supervision by medical staff, paving the way for the deployment of an eHealth platform, called e-RR and shown in Fig. 2, which is currently

Discussion

In a simple communication using publish/subscribe paradigm between only two machines, performance is almost similar for all M2M protocols [54]. However, an increase in the number of machines can reduce general performance, mainly in those protocols that use centralized brokers, adding an additional overhead. Some brokers may be unable to handle the load of incoming messages when traffic is very intense and the broker may fail. At this point, machines lost communication because in a centralized

Conclusions

This paper presents a comparative study of different M2M protocols in terms of several performance metrics. For the execution of tests, it was designed and developed a Distributed Computing Framework based on master–slave communication architecture that supports the deployment of the benchmark. The Framework stands out for its flexibility, modular design, and ease of use, which makes it an excellent tool to aid developers to choose among M2M protocols for eHealth solutions. The results obtained

Acknowledgments

This work was supported in part by the Fondo de Investigaciones Sanitarias under Grant PI11/00111, in part by the Dirección General de Investigación, Tecnología y Empresa, Government of Andalucía, under Grants TIC-6214, P08-TIC-04069 and PI-0010-2013, and in part by the CIBER de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN) under intramural project PERSONA. CIBER-BBN is an initiative funded by the VI National R&D&i Plan 2008–2011, Iniciativa Ingenio 2010, Consolider Program, CIBER

References (58)

  • A. Solanas et al.

    Smart health: a context-aware health paradigm within smart cities

    IEEE Commun. Mag.

    (2014)
  • L. Atzori et al.

    From “smart objects” to “social objects”: the next evolutionary step of the Internet of Things

    IEEE Commun. Mag.

    (2014)
  • P. Upadhyay et al.

    Modeling agility in Internet of Things (IoT) architecture

  • E. Cavalcante et al.

    An analysis of reference architectures for the Internet of Things

  • M. Serrano et al.

    Defining the stack for service delivery models and interoperability in the Internet of Things: a practical case with OpenIoT-VDK

    IEEE J. Sel. Areas Commun.

    (2015)
  • F. Khodadadi et al.

    Simurgh: a framework for effective discovery, programming, and integration of services exposed in IoT

  • Industrial Internet Reference Architecture | Industrial Internet Consortium, 2016. http://www.iiconsortium.org/IIRA.htm...
  • A. Elmangoush et al.

    Application-derived communication protocol selection in M2M platforms for smart cities

  • S. Bandyopadhyay et al.

    Performance of a publish/subscribe middleware for the real-time distributed control systems

    JCSNS Int. J. Comput. Sci. Netw. Secur.

    (2007)
  • P. Bellavista et al.

    Quality of service in wide scale publish

    IEEE Commun. Surv. Tutor.

    (2014)
  • M. Korostelev et al.

    M2-PASS: SMS-based mobile patient support and responding to challenges of transitional care

  • C. Pereira et al.

    Towards efficient mobile M2M communications: survey and open challenges

    Sensors

    (2014)
  • C. Doukas et al.

    Providing generic support for IoT and M2M for mobile devices

  • Y.-C. Tseng

    Smarter IoT: a situated computing perspective

  • V. Karagiannis et al.

    A survey on application layer protocols for the Internet of Things

    Trans. IoT Cloud Comput.

    (2015)
  • PI-0010-2013

    Research, development and technical and clinical validation of a comprehensive platform for e-rehabilitation, Research project application report

    (2014)
  • D.B.S. Solway

    A qualitative systematic overview of the measurement properties of functional walk tests used in the cardiorespiratory domain

    Chest

    (2001)
  • K. Bennell et al.

    Measures of physical performance assessments: Self-Paced Walk Test (SPWT), Stair Climb Test (SCT), Six-Minute Walk Test (6MWT), Chair Stand Test (CST), Timed Up & Go (TUG), Sock Test, Lift and Carry Test (LCT), and Car Task

    Arthritis Care Res.

    (2011)
  • J.C.M. de Oliveira et al.

    Outpatient vs. home-based pulmonary rehabilitation in COPD: a randomized controlled trial

    Multidiscip. Respir. Med.

    (2010)
  • Cited by (48)

    • A distributed communication framework for smart Grid control applications based on data distribution service

      2021, Electric Power Systems Research
      Citation Excerpt :

      It is worthwhile mentioning that the results obtained in this case study are closely aligned with another benchmarking evaluation done in [42], which also compares DDS, AMQP, and MQTT. In [42], benchmarks conducted on a computer with 1 GB RAM indicate that DDS has superior message latency at 1000 messages/second (20 ms) as compared to AMQP (40 ms), and MQTT (100 ms), with MQTT performing significantly worse when moving from 50 messages/second (10 ms) to 1000 messages/second (100 ms), which was also seen in this case study. When comparing the results in this case study to [42], it is noteworthy that the DCF-PC significantly outperforms the DDS benchmark by a factor of 10 (2 ms to 20 ms), while the DCF-RIO is slower by a factor of 1.8 (36ms to 20ms), even though the DCF-RIO has considerably less RAM than the computer used in [42] (256MB to 1GB).

    • Communication model in the internet of things

      2020, Procedia Computer Science
    • Automated synthesis of mediators for middleware-layer protocol interoperability in the IoT

      2019, Future Generation Computer Systems
      Citation Excerpt :

      Finally, as part of the HTML 5 initiative, WebSockets [3] was introduced to support real-time full duplex (streaming) interactions, using only two bytes of overhead in message payloads. The authors in [9,12,13] compare the most promising IoT middleware protocols: DPWS, CoAP, MQTT, Websockets, XMPP, REST and AMQP. They recommend combining one or more protocols in an IoT application to better exploit the underlying communication infrastructure.

    • Investigations in Security Challenges and Solutions for M2M Communications—A Review

      2024, International Journal of Electrical and Electronic Engineering and Telecommunications
    View all citing articles on Scopus
    View full text