Technical noteWhen assistive eye tracking fails: Communicating with a brainstem-stroke patient through the pupillary accommodative response – A case study
Graphical abstract
Introduction
Patients suffering from stroke, spinal traumas, brain lesions, and progressive motor diseases such as multiple sclerosis and amyotrophic lateral sclerosis (ALS) may eventually develop a condition of complete muscular paralysis, in which consciousness and awareness are retained, known as locked-in state (LIS) [[1], [2], [3], [4]]. For many of these patients, the ability to control eye movements is spared, and, therefore, they often rely on augmentative and alternative communication (AAC) devices, based on gaze tracking as the main aid for communicating [5,6].
However, the use of gaze tracking devices is hindered when oculomotor control deteriorates, as eventually happens in the progression of ALS, or if brain lesions affect ocular mobility. In the condition known as completely locked-in state (CLIS) [7], eye movements can be completely lost. For these patients, the only chance to maintain communication is to rely on other systems, for example based on EEG signals to control AAC devices. These methods are commonly referred to as brain computer interfaces (BCIs) [8]. Partial success in communicating with CLIS patients has been achieved with some BCIs, especially those based on event-related potentials [9,10]. However, these systems require a relatively long preparation, as well as the presence of specialized AAC facilitators, and have difficult learning curves because the patient must learn the proper control of specific physiological signals [11]. Moreover, they are often quite expensive. Thus, simpler and more patient-friendly methods are desirable.
We have recently described a method to establish reliable and rather fast binary communication in healthy individuals based on a very simple and natural act, namely, shifting the gaze from a far to a near target. Such gaze shift in depth is associated to pupillary constriction, the pupillary accommodation response (PAR), which is controlled by the autonomic nervous system and which is normally easily measurable, even with an ordinary webcam [[12], [13], [14], [15]]. Therefore, responding “yes” to a question may be achieved by simply shifting visuospatial attention from a far to a near target. We suggested that this approach could be exploited to communicate with patients in whom the skeletomotor and oculomotor systems are impaired, but with spared autonomic control.
We here present a simple implementation of this approach designed to integrate a widely used commercial AAC interface. This software interface can be easily configured to offer different possibilities ranging from text writing, to surfing the internet to control other devices (domotics) and is normally driven with eye-movements detected by eye-tracking devices. As a proof-of-concept, this interface is here configured to implement a simple binary YES/NO communication and integrated in a pupil-driven communication prototype. The effectiveness of the system is tested in a brainstem-stroke patient, unable to control AAC devices by means of eye movements.
Section snippets
The communication prototype
A system was devised to implement a pupil-controlled scanning-selection interface, as illustrated in Fig. 1a. Pupil size and eye movements are binocularly and continuously detected by a low-cost eye-tracking device (EyeTribe, Denmark) and transmitted to a personal computer via USB. A custom program (Matlab, Natick MA, USA) manages the data acquisition (sampling frequency: 30 Hz) and recording of all signals and the real-time processing of pupil size (monocular). In particular, signal processing
First visit: PLR and PAR characterization results
The patient’s pupil exhibited a clear response to the light stimulus (Fig. 2b), presenting a mean magnitude of constriction of 30.96 ± 5.0 % of the basal value (corresponding to the dark condition), a mean latency of 510 ± 63.5 ms, and a speed of constriction of 58 ± 20.4 %/s.
The accommodative task was easily performed by the patient and the PARs were present and clearly identifiable (Fig. 1S), in spite of ocular instability due to nystagmus. On average, mean PAR amplitude was 26 ± 6 % of basal
Discussion
We have here presented a device prototype which implements the detection of pupillary constrictions to drive a commercial and customizable scanning-selection AAC interface. As a proof of concept, the device was tested on a brainstem stroke patient unable to autonomously use other AAC devices. The patient immediately learned how to use the device and, on the same session of device presentation, correctly answered 8 out of 10 questions with obvious answer. While the probability of randomly
CRediT authorship contribution statement
Andres Eduardo Lorenzo Villalobos: Software, Investigation, Data curation, Visualization, Writing - original draft. Silvia Giusiano: Investigation, Data curation, Visualization. Luca Musso: Investigation. Claudio de’Sperati: Validation, Writing - review & editing. Alessandra Riberi: Resources. Piotr Spalek: Resources. Andrea Calvo: Funding acquisition. Cristina Moglia: Supervision, Writing - review & editing, Funding acquisition. Silvestro Roatta: Conceptualization, Methodology, Writing -
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgments
This research was supported by San Paolo Foundation (CSTO164044) and by the Dept of Neuroscience, University of Torino (ROAS_AUTOF_18_01).
Some of the authors (CdS and SR) hold a patent that partly covers the described prototype
References (31)
- et al.
Transition from the locked in to the completely locked-in state: a physiological analysis
Clin. Neurophysiol.
(2011) - et al.
Brain-computer interfaces for communication and control
Clin. Neurophysiol.
(2002) - et al.
A human-computer interface based on the “voluntary” pupil accommodative response
Int. J. Hum. Stud.
(2019) - et al.
Defining brain-machine interface applications by matching interface performance with device requirements
J. Neurosci. Methods
(2008) Breaking the silence: brain-computer interfaces (BCI) for communication and motor control
Psychophysiology
(2006)- et al.
Communication support for people with ALS
Neurol. Res. Int.
(2011) - et al.
Locked-in syndrome: a review of 139 cases
Stroke
(1986) - et al.
Varieties of the locked-in syndrome
J. Neurol.
(1979) - et al.
Differential susceptibility of the ocular motor system to disease
Ann. N. Y. Acad. Sci.
(2002) - et al.
AAC for adults with acquired neurological conditions: a review
Augment. Altern. Commun.
(2007)