Technical note
When assistive eye tracking fails: Communicating with a brainstem-stroke patient through the pupillary accommodative response – A case study

https://doi.org/10.1016/j.bspc.2021.102515Get rights and content

Highlights

  • Communication without movement remains an open research and clinical challenge.

  • Voluntary control of pupil size is a novel approach to serve this purpose.

  • A low-cost communication system, based on the pupil accommodative response, is presented.

  • For the first time this approach is successfully tested in a clinical condition.

Abstract

Purpose

Poor control of eye movement and coordination may impair the use of eye-trackers for communication in patients affected by severe motor diseases. Recently, the “voluntary” pupil accommodative response (PAR) was suggested as a possible alternative to traditional assistive technology. Aim of this study is to provide a proof of concept of this methodology in a clinical setting.

Materials and methods

A low-cost communication system was implemented, which detects the accommodative pupillary constrictions in real time and generates trigger events to drive a commercial scanning-selection interface. As a first implementation, a simple binary yes/no selection interface was designed to be tested with a brainstem stroke patient, unable to use standard communicators based on eye tracking. The patient was instructed to operate the intended selection by switching the focus of attention from a far to a near target, and was then presented with 10 questions with obvious answer.

Results

The patient easily understood how to perform the accommodative task. The pupillary constrictions were marked and clearly detectable in spite of the disturbing action of persistent nystagmus. On the first presentation of the device, the patient managed to correctly answer 8 out of 10 questions.

Conclusions

The present results provide a proof-of-concept for PAR-based communication in a clinical setting and support its usefulness with patients who, due to impaired control of eye movements, may be unable to use tracking-based devices.

Introduction

Patients suffering from stroke, spinal traumas, brain lesions, and progressive motor diseases such as multiple sclerosis and amyotrophic lateral sclerosis (ALS) may eventually develop a condition of complete muscular paralysis, in which consciousness and awareness are retained, known as locked-in state (LIS) [[1], [2], [3], [4]]. For many of these patients, the ability to control eye movements is spared, and, therefore, they often rely on augmentative and alternative communication (AAC) devices, based on gaze tracking as the main aid for communicating [5,6].

However, the use of gaze tracking devices is hindered when oculomotor control deteriorates, as eventually happens in the progression of ALS, or if brain lesions affect ocular mobility. In the condition known as completely locked-in state (CLIS) [7], eye movements can be completely lost. For these patients, the only chance to maintain communication is to rely on other systems, for example based on EEG signals to control AAC devices. These methods are commonly referred to as brain computer interfaces (BCIs) [8]. Partial success in communicating with CLIS patients has been achieved with some BCIs, especially those based on event-related potentials [9,10]. However, these systems require a relatively long preparation, as well as the presence of specialized AAC facilitators, and have difficult learning curves because the patient must learn the proper control of specific physiological signals [11]. Moreover, they are often quite expensive. Thus, simpler and more patient-friendly methods are desirable.

We have recently described a method to establish reliable and rather fast binary communication in healthy individuals based on a very simple and natural act, namely, shifting the gaze from a far to a near target. Such gaze shift in depth is associated to pupillary constriction, the pupillary accommodation response (PAR), which is controlled by the autonomic nervous system and which is normally easily measurable, even with an ordinary webcam [[12], [13], [14], [15]]. Therefore, responding “yes” to a question may be achieved by simply shifting visuospatial attention from a far to a near target. We suggested that this approach could be exploited to communicate with patients in whom the skeletomotor and oculomotor systems are impaired, but with spared autonomic control.

We here present a simple implementation of this approach designed to integrate a widely used commercial AAC interface. This software interface can be easily configured to offer different possibilities ranging from text writing, to surfing the internet to control other devices (domotics) and is normally driven with eye-movements detected by eye-tracking devices. As a proof-of-concept, this interface is here configured to implement a simple binary YES/NO communication and integrated in a pupil-driven communication prototype. The effectiveness of the system is tested in a brainstem-stroke patient, unable to control AAC devices by means of eye movements.

Section snippets

The communication prototype

A system was devised to implement a pupil-controlled scanning-selection interface, as illustrated in Fig. 1a. Pupil size and eye movements are binocularly and continuously detected by a low-cost eye-tracking device (EyeTribe, Denmark) and transmitted to a personal computer via USB. A custom program (Matlab, Natick MA, USA) manages the data acquisition (sampling frequency: 30 Hz) and recording of all signals and the real-time processing of pupil size (monocular). In particular, signal processing

First visit: PLR and PAR characterization results

The patient’s pupil exhibited a clear response to the light stimulus (Fig. 2b), presenting a mean magnitude of constriction of 30.96 ± 5.0 % of the basal value (corresponding to the dark condition), a mean latency of 510 ± 63.5 ms, and a speed of constriction of 58 ± 20.4 %/s.

The accommodative task was easily performed by the patient and the PARs were present and clearly identifiable (Fig. 1S), in spite of ocular instability due to nystagmus. On average, mean PAR amplitude was 26 ± 6 % of basal

Discussion

We have here presented a device prototype which implements the detection of pupillary constrictions to drive a commercial and customizable scanning-selection AAC interface. As a proof of concept, the device was tested on a brainstem stroke patient unable to autonomously use other AAC devices. The patient immediately learned how to use the device and, on the same session of device presentation, correctly answered 8 out of 10 questions with obvious answer. While the probability of randomly

CRediT authorship contribution statement

Andres Eduardo Lorenzo Villalobos: Software, Investigation, Data curation, Visualization, Writing - original draft. Silvia Giusiano: Investigation, Data curation, Visualization. Luca Musso: Investigation. Claudio de’Sperati: Validation, Writing - review & editing. Alessandra Riberi: Resources. Piotr Spalek: Resources. Andrea Calvo: Funding acquisition. Cristina Moglia: Supervision, Writing - review & editing, Funding acquisition. Silvestro Roatta: Conceptualization, Methodology, Writing -

Declaration of Competing Interest

The authors report no declarations of interest.

Acknowledgments

This research was supported by San Paolo Foundation (CSTO164044) and by the Dept of Neuroscience, University of Torino (ROAS_AUTOF_18_01).

Some of the authors (CdS and SR) hold a patent that partly covers the described prototype

References (31)

  • U. Chaudhary et al.

    Brain-computer interfaces for communication and rehabilitation

    Nat. Rev. Neurol.

    (2016)
  • U. Chaudhary et al.

    Brain–Computer interface–based communication in the completely locked-in state

    PLoS Biol.

    (2017)
  • M. Marchetti et al.

    Effectiveness of the P3-speller in brain-computer interfaces for amyotrophic lateral sclerosis patients: a systematic review and meta-analysis

    Front. Neuroeng.

    (2014)
  • E. Chirre et al.

    Dynamics of the near response under natural viewing conditions with an open-view sensor

    Biomed. Opt. Express

    (2015)
  • E.C. Lee et al.

    Experimental investigations of pupil accommodation factors

    Investig. Ophthalmol. Vis. Sci.

    (2011)
  • Cited by (0)

    View full text