Position Paper
STQS: Interpretable multi-modal Spatial-Temporal-seQuential model for automatic Sleep scoring

https://doi.org/10.1016/j.artmed.2021.102038Get rights and content
Under a Creative Commons license
open access

Highlights

  • Deep learning model for multimodal multichannel signals for predicting sleep stages.

  • Spatio-temporal and sequential information are promising for sleep stage prediction.

  • Post-hoc interpretability shows the model’s conformance with sleep scoring rules.

  • Multimodal input signals (EEG, EOG, EMG) are important for sleep stage prediction.

Abstract

Sleep scoring is an important step for the detection of sleep disorders and usually performed by visual analysis. Since manual sleep scoring is time consuming, machine-learning based approaches have been proposed. Though efficient, these algorithms are black-box in nature and difficult to interpret by clinicians. In this paper, we propose a deep learning architecture for multi-modal sleep scoring, investigate the model's decision making process, and compare the model's reasoning with the annotation guidelines in the AASM manual. Our architecture, called STQS, uses convolutional neural networks (CNN) to automatically extract spatio-temporal features from 3 modalities (EEG, EOG and EMG), a bidirectional long short-term memory (Bi-LSTM) to extract sequential information, and residual connections to combine spatio-temporal and sequential features. We evaluated our model on two large datasets, obtaining an accuracy of 85% and 77% and a macro F1 score of 79% and 73% on SHHS and an in-house dataset, respectively. We further quantify the contribution of various architectural components and conclude that adding LSTM layers improves performance over a spatio-temporal CNN, while adding residual connections does not. Our interpretability results show that the output of the model is well aligned with AASM guidelines, and therefore, the model's decisions correspond to domain knowledge. We also compare multi-modal models and single-channel models and suggest that future research should focus on improving multi-modal models.

Keywords

Sleep scoring
Sleep stage annotation
Deep learning
EEG, EOG, EMG signals
Post-hoc interpretability
Explainable AI

Cited by (0)