Skip to main content
SpringerLink
Log in

NapWell: An EOG-based Sleep Assistant Exploring the Effects of Virtual Reality on Sleep Onset

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

We present NapWell, a Sleep Assistant using virtual reality (VR) to decrease sleep onset latency by providing a realistic imagery distraction prior to sleep onset. Our proposed prototype was built using commercial hardware and with relatively low cost, making it replicable for future works as well as paving the way for more low cost EOG-VR devices for sleep assistance. We conducted a user study (\(n= 20\)) by comparing different sleep conditions; no devices, sleeping mask, VR environment of the study room and preferred VR environment by the participant. During this period, we recorded the electrooculography (EOG) signal and sleep onset time using a finger tapping task (FTT). We found that VR was able to significantly decrease sleep onset latency. We also developed a machine learning model based on EOG signals that can predict sleep onset with a cross-validated accuracy of 70.03%. The presented study demonstrates the feasibility of VR to be used as a tool to decrease sleep onset latency, as well as the use of embedded EOG sensors with VR for automatic sleep detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://jins-meme.com/en/.

  2. https://vr.google.com/daydream/.

  3. https://www.asus.com/Phone/ZenFone-AR-ZS571KL/.

  4. https://jins-meme.com/en/.

  5. https://www.apple.com/nz/macbook-air/.

  6. https://www.gtracingchair.com/.

  7. https://www.samsung.com/global/galaxy/galaxy-s8/.

  8. https://www.sony.co.nz/electronics/headband-headphones/wh-1000xm3.

  9. https://www.apple.com/apple-watch-series-5.

  10. https://www.mi.com/global/mi-smart-band-4.

  11. https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA.fit_transform.

  12. https://lynx-r.com/.

References

  • Adib F, Mao H, Kabelac Z, Katabi D, Miller RC (2015) Smart homes that monitor breathing and heart rate. pp 837–846

  • Akert K, Koella W, Hess R Jr (1951) Sleep produced by electrical stimulation of the thalamus. Am J Physiol Legacy Content 168(1):260–267

    Article  Google Scholar 

  • Amores J, Benavides X, Maes P (2016) Psychicvr: increasing mindfulness by using virtual reality and brain computer interfaces. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM, pp 2

  • Bait M (2019) Sleep survey. [online] google docs 28, https://docs.google.com/forms/d/1Nso725KJgYKm1KjODsosnBh2FBuzAIejCga4m4vNodk/viewanalytics

  • Bernardino C, Ferreira HA, Chambel T (2016) Towards media for wellbeing. In: Proceedings of the ACM international conference on interactive experiences for TV and Online Video, ACM, pp 171–177

  • Berry RB, Brooks R, Gamaldo CE, Harding SM, Marcus CL, Vaughn BV et al (2012) The aasm manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine, p 176

  • Blake H, Gerard RW, Kleitman N (1939) Factors influencing brain potentials during sleep

  • Borazio M, Van Laerhoven K (2012) Combining wearable and environmental sensing into an unobtrusive tool for long-term sleep studies. In: Proceedings of the 2nd ACM SIGHIT international health informatics symposium, ACM, pp 71–80

  • Boukadoum A, Ktonas P (1986) Eog-based recording and automated detection of sleep rapid eye movements: a critical review, and some recommendations. Psychophysiology 23(5):598–611

    Article  Google Scholar 

  • Brown FC, Buboltz WC Jr, Soper B (2002) Relationship of sleep hygiene awareness, sleep hygiene practices, and sleep quality in university students. Behav Med 28(1):33–38

    Article  Google Scholar 

  • Casagrande M, De Gennaro L, Violani C, Braibanti P, Bertini M (1997) A finger-tapping task and a reaction time task as behavioral measures of the transition from wakefulness to sleep: Which task interferes less with the sleep onset process? Sleep 20(4):301–312

    Article  Google Scholar 

  • Choe EK, Lee B, Kay M, Pratt W, Kientz JA (2015) Sleeptight: low-burden, self-monitoring technology for capturing and reflecting on sleep behaviors. In: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing, ACM, pp 121–132

  • Connelly L (2004) Behavioural measurements of sleep onset: A comparison of two devices. PhD thesis

  • Daneshmandi M, Neiseh F, SadeghiShermeh M, Ebadi A (2012) Effect of eye mask on sleep quality in patients with acute coronary syndrome. J Caring Sci 1(3):135

    Google Scholar 

  • Davis H, Davis PA, Loomis AL, Harvey EN, Hobart G (1939) Electrical reactions of the human brain to auditory stimulation during sleep. J Neurophysiol 2(6):500–514

    Article  Google Scholar 

  • Demoule A, Carreira S, Lavault S, Pallanca O, Morawiec E, Mayaux J, Arnulf I, Similowski T (2017) Impact of earplugs and eye mask on sleep in critically ill patients: a prospective randomized study. Critical Care 21(1):1–9

    Article  Google Scholar 

  • Ehleringer EH, Kim SJ (2013) The wearable lullaby: Improving sleep quality of caregivers of dementia patients. In: CHI ’13 Extended Abstracts on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI EA ’13, pp 409–414, https://doi.org/10.1145/2468356.2468429, http://doi.acm.org/10.1145/2468356.2468429

  • Felsten G (2009) Where to take a study break on the college campus: an attention restoration theory perspective. J Environ Psychol 29(1):160–167

    Article  Google Scholar 

  • Ferrer-Garcia M, Gutiérrez-Maldonado J, Riva G (2013) Virtual reality based treatments in eating disorders and obesity: a review. J Contemp Psychother 43(4):207–221

    Article  Google Scholar 

  • Green A, Cohen-Zion M, Haim A, Dagan Y (2017) Evening light exposure to computer screens disrupts human sleep, biological rhythms, and attention abilities. Chronobiol Int 34(7):855–865

    Article  Google Scholar 

  • Gromala D, Tong X, Choo A, Karamnejad M, Shaw CD (2015) The virtual meditative walk: Virtual reality therapy for chronic pain management. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI ’15, pp 521–524, https://doi.org/10.1145/2702123.2702344, http://doi.acm.org/10.1145/2702123.2702344

  • Gupta K, Hajika R, Pai YS, Duenser A, Lochner M, Billinghurst M (2020) Measuring human trust in a virtual assistant using physiological sensing in virtual reality. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), IEEE, pp 756–765

  • Harvey AG, Payne S (2002) The management of unwanted pre-sleep thoughts in insomnia: distraction with imagery versus general distraction. Behav Res Ther 40(3):267–277

    Article  Google Scholar 

  • Rf Hu, Jiang Xy, Zeng Ym, Chen Xy, Zhang Yh (2010) Effects of earplugs and eye masks on nocturnal sleep, melatonin and cortisol in a simulated intensive care unit environment. Critical Care 14(2):1–9

    Google Scholar 

  • Ibáñez V, Silva J, Cauli O (2018) A survey on sleep assessment methods. PeerJ 6:e4849

    Article  Google Scholar 

  • Ishimaru S, Kunze K, Uema Y, Kise K, Inami M, Tanaka K (2014) Smarter eyewear: using commercial eog glasses for activity recognition. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp 239–242

  • Jerald J (2015) The VR book: Human-centered design for virtual reality. Morgan & Claypool

  • Jirakittayakorn N, Wongsawat Y (2018) A novel insight of effects of a 3-hz binaural beat on sleep stages during sleep. Front Human Neurosci 12:387

    Article  Google Scholar 

  • Johns MW (1991) A new method for measuring daytime sleepiness: the epworth sleepiness scale. Sleep 14(6):540–545

    Article  Google Scholar 

  • Kanady JC, Drummond SP, Mednick SC (2011) Actigraphic assessment of a polysomnographic-recorded nap: a validation study. J Sleep Res 20(1pt2):214–222

    Article  Google Scholar 

  • Katsumata K, Noda Y, Isokawa N, Katayama S, Okoshi T, Nakazawa J (2019) Sleepthermo: The affect of in-cloth monitored body temperature change during sleep on human well-being. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, ACM, New York, NY, USA, UbiComp/ISWC ’19 Adjunct, pp 1174–1177, https://doi.org/10.1145/3341162.3347079, http://doi.acm.org/10.1145/3341162.3347079

  • Kitson A, Schiphorst T, Riecke BE (2018) Are you dreaming?: A phenomenological study on understanding lucid dreams as a tool for introspection in virtual reality. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI ’18, pp 343:1–343:12, https://doi.org/10.1145/3173574.3173917, http://doi.acm.org/10.1145/3173574.3173917

  • Kosunen I, Salminen M, Järvelä S, Ruonala A, Ravaja N, Jacucci G (2016) Relaworld: neuroadaptive and immersive virtual reality meditation system. In: Proceedings of the 21st international conference on intelligent user interfaces, ACM, pp 208–217

  • Krishnan A, Kumar R, Venkatesh P, Kelly S, Grover P (2018) Low-cost carbon fiber-based conductive silicone sponge eeg electrodes. In: 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE, pp 1287–1290

  • Lack L, Mair A (1995) The relationship between eeg and a behavioral measure of sleep onset. Sleep Res 24:218

    Google Scholar 

  • Lee M, Song CB, Shin GH, Lee SW (2019) Possible effect of binaural beat combined with autonomous sensory meridian response for inducing sleep. Front Human Neurosci 13:425

    Article  Google Scholar 

  • Liang SF, Kuo CE, Lee YC, Lin WC, Liu YC, Chen PY, Cherng FY, Shaw FZ (2015) Development of an EOG-based automatic sleep-monitoring eye mask. IEEE Trans Instrum Meas 64(11):2977–2985

    Article  Google Scholar 

  • Liang Z, Ploderer B (2016) Sleep tracking in the real world: A qualitative study into barriers for improving sleep. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction, ACM, New York, NY, USA, OzCHI ’16, pp 537–541, https://doi.org/10.1145/3010915.3010988, http://doi.acm.org/10.1145/3010915.3010988

  • Liberson W, Liberson CW (1966) Eeg records, reaction times, eye movements, respiration, and mental content during drowsiness. In: Recent advances in biological psychiatry, Springer, pp 295–302

  • Masai K, Sugiura Y, Sugimoto M (2018) Facerubbing: Input technique by rubbing face using optical sensors on smart eyewear for facial expression recognition. In: Proceedings of the 9th Augmented Human International Conference, pp 1–5

  • Matsui S, Terada T, Tsukamoto M (2017) Smart eye mask: eye-mask shaped sleep monitoring device. In: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, pp 265–268

  • Mednick SC, Nakayama K, Cantero JL, Atienza M, Levin AA, Pathak N, Stickgold R (2002) The restorative effect of naps on perceptual deterioration. Nature Neurosci 5(7):677

    Article  Google Scholar 

  • Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Personal Ubiquitous Comput 18(1):19–26

    Article  Google Scholar 

  • Milner CE, Cote KA (2009) Benefits of napping in healthy adults: impact of nap length, time of day, age, and experience with napping. J Sleep Res 18(2):272–281

    Article  Google Scholar 

  • Min JK, Doryab A, Wiese J, Amini S, Zimmerman J, Hong JI (2014) Toss’n’turn: smartphone as sleep and sleep quality detector. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, pp 477–486

  • Morales-Cobas G, Ferreira MI, Velluti RA (1995) Firing of inferior colliculus neurons in response to low-frequency sound stimulation during sleep and waking. J Sleep Res 4(4):242–251

    Article  Google Scholar 

  • Muzet A (2007) Environmental noise, sleep and health. Sleep Med Rev 11(2):135–142

    Article  Google Scholar 

  • Nicholson A, Stone BM (1987) Influence of back angle on the quality of sleep in seats. Ergonomics 30(7):1033–1041

    Article  Google Scholar 

  • Ogilvie RD (2001) The process of falling asleep. Sleep Med Rev 5(3):247–270

    Article  Google Scholar 

  • Ogilvie RD, Wilkinson RT (1984) The detection of sleep onset: behavioral and physiological convergence. Psychophysiology 21(5):510–520

    Article  Google Scholar 

  • Ogilvie RD, Wilkinson RT (1988) Behavioral versus eeg-based monitoring of all-night sleep/wake patterns. Sleep 11(2):139–155

    Article  Google Scholar 

  • Ogilvie RD, Wilkinson RT, Allison S (1989) The detection of sleep onset: behavioral, physiological, and subjective convergence. Sleep 12(5):458–474

    Article  Google Scholar 

  • Okamura T, Isoyama N, Lopez G (2016) A method to detect accurately falling asleep and awakening time. In: Adjunct Proceedings of the 13th international conference on mobile and ubiquitous systems: computing networking and services, ACM, pp 47–52

  • Pedemonte M, Testa M, Díaz M, Suarez-Bagnasco D (2014) The impact of sound on electroencephalographic waves during sleep in patients suffering from tinnitus. Sleep Sci 7(3):143–151

    Article  Google Scholar 

  • Peña JL, Pérez-Perera L, Bouvier M, Velluti RA (1999) Sleep and wakefulness modulation of the neuronal firing in the auditory cortex of the guinea pig. Brain Res 816(2):463–470

    Article  Google Scholar 

  • Powers MB, Emmelkamp PM (2008) Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J Anxiety Disord 22(3):561–569

    Article  Google Scholar 

  • Rahman MM, Bhuiyan MIH, Hassan AR (2018) Sleep stage classification using single-channel EOG. Comput Biol Med 102:211–220

    Article  Google Scholar 

  • Roo JS, Gervais R, Frey J, Hachet M (2017) Inner garden: Connecting inner states to a mixed reality sandbox for mindfulness. In: Proceedings of the 2017 CHI conference on human factors in computing systems, ACM, pp 1459–1470

  • Rothbaum BO, Hodges L, Alarcon R, Ready D, Shahar F, Graap K, Pair J, Hebert P, Gotz D, Wills B et al (1999) Virtual reality exposure therapy for ptsd vietnam veterans: a case study. J Traum Stress Official Pub Int Soc Traum Stress Stud 12(2):263–271

    Article  Google Scholar 

  • Rothkrantz L (2016) Automatic detection system of micro sleeps of car drivers based on eeg analysis. In: Proceedings of the 17th international conference on computer systems and technologies 2016, ACM, pp 214–221

  • Sanchez-Vives MV, Slater M (2005) From presence to consciousness through virtual reality. Nature Rev Neurosci 6(4):332

    Article  Google Scholar 

  • Schmidt A, Shirazi AS, Van Laerhoven K (2012) Are you in bed with technology? IEEE Pervasive Comput 11(4):4–7

    Article  Google Scholar 

  • Scott H, Lack L (2017) The revival of active behavioural devices for measuring sleep latency. SM J Sleep Disord 3(3):1015

    Google Scholar 

  • Scott H, Lack L, Lovato N (2018) A pilot study of a novel smartphone application for the estimation of sleep onset. J Sleep Res 27(1):90–97

    Article  Google Scholar 

  • Semertzidis NA, Sargeant B, Dwyer J, Mueller FF, Zambetta F (2019) Towards understanding the design of positive pre-sleep through a neurofeedback artistic experience. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI ’19, pp 574:1–574:14, https://doi.org/10.1145/3290605.3300804, http://doi.acm.org/10.1145/3290605.3300804

  • Shaw C, Gromala D, Song M (2011) The meditation chamber: towards self-modulation. In: Metaplasticity in virtual worlds: Aesthetics and semantic concepts, IGI Global, pp 121–133

  • Shirazi AS, Clawson J, Hassanpour Y, Tourian MJ, Schmidt A, Chi EH, Borazio M, Van Laerhoven K (2013) Already up? Using mobile phones to track & share sleep behavior. Int J Human Comput Stud 71(9):878–888

    Article  Google Scholar 

  • Uema Y, Inoue K (2017) Jins meme algorithm for estimation and tracking of concentration of users. In: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, ACM, New York, NY, USA, UbiComp ’17, pp 297–300, https://doi.org/10.1145/3123024.3123189, http://doi.acm.org/10.1145/3123024.3123189

  • Wells AS, Read N, Uvnas-Moberg K, Alster P (1997) Influences of fat and carbohydrate on postprandial sleepiness, mood, and hormones. Physiol Behav 61(5):679–686

    Article  Google Scholar 

  • Yazdannik AR, Zareie A, Hasanpour M, Kashefi P (2014) The effect of earplugs and eye mask on patients perceived sleep quality in intensive care unit. Iranian J Nursing Midwifery Res 19(6):673

    Google Scholar 

  • Zhang Z, Guan C, Chan TE, Yu J, Ng AK, Zhang H, Kwoh CK (2014) Automatic sleep onset detection using single eeg sensor. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 2265–2268, https://doi.org/10.1109/EMBC.2014.6944071

  • Zhao N, Azaria A, Paradiso JA (2017) Mediated atmospheres: A multimodal mediated work environment. Proc ACM Interact Mob Wearable Ubiquitous Technol 1(2):31:1–31:23, https://doi.org/10.1145/3090096, http://doi.acm.org/10.1145/3090096

Download references

Acknowledgements

This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. , WISE AR UI/UX Platform Development for Smartglasses), as well as the Japan Society for the Promotion of Science (JSPS) Kakenhi Grant Number 18H03278.

Author information

Authors and Affiliations

  1. Keio University Graduate School of Media Design, Yokohama, Japan

    Yun Suen Pai, Marsel L. Bait, Jingjing Xu & Kai Kunze

  2. Korea Advanced Institute of Science and Technology, Daejeon, Korea

    Juyoung Lee & Woontack Woo

  3. Rochester Institute of Technology, New York, USA

    Roshan L Peiris

  4. The University of Auckland, Auckland, New Zealand

    Mark Billinghurst

Authors
  1. Yun Suen Pai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marsel L. Bait
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingjing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Roshan L Peiris
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Woontack Woo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mark Billinghurst
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kai Kunze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun Suen Pai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pai, Y.S., Bait, M.L., Lee, J. et al. NapWell: An EOG-based Sleep Assistant Exploring the Effects of Virtual Reality on Sleep Onset. Virtual Reality 26, 437–451 (2022). https://doi.org/10.1007/s10055-021-00571-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-021-00571-w

Keywords

Search

Navigation